FOOD FOR THE HUNGRY INTERNATIONAL
IMPROVING FOOD SECURITY PROGRAMMING AND RESOURCE 
MANAGEMENT
PL 480 Title II Institutional Support Agreement (FY1999-FY2003)
FINAL EVALUATION 
 
Evaluation Team: Buck Deines
 Keith Wright
 Isabelle Schmidt (Team leader) December 2002 
 
i
TABLE OF CONTENTS
 
CHAPTER 1: INTRODUCTION 
1.1 Introduction 4
1.2 Goals and objectives of ISA (FY1999-FY2003) 4
1.3 Objective of the final evaluation 5
CHAPTER 2: METHODOLOGY
2.1 Introduction 6
2.2 Objectives of the field visits 6
2.3 Methodology 7
2.4 Groups and resources 9
2.5 Field visit schedule 10
CHAPTER 3: MONITORING AND INDICATOR TRACKING TABLE
3.1 Introduction 11
3.2 Program performance indicator tracking table 11
CHAPTER 4: OVERVIEW OF PROGRAM RESOURCES AND ACTIVITIES
4.1 Introduction 16
4.2 Staffing 16
4.3 Budgets and expenditure 19
4.4 Modus operandi 20
CHAPTER 5: ACTIVITIES, ACHIEVEMENTS AND IMPACT
5.1 Introduction 23
5.2 Tools for the design and implementation of food security programs 23
5.2.1 Project proposals 23
5.2.2 Educational Messages 25
5.2.3 Positive Deviance 28
5.2.4 Barrier analysis/TIPS 30
5.2.5 Monitoring and Evaluation 32
5.3 Commodity management 34
5.4 Emergency response preparedness 38
5.5 Food security needs assessments 43
5.6 FAM cooperating partnership and working groups 44
5.7 FAM knowledge of and proficiency in suing information technology 47
CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS
 
ii
TABLE OF CONTENTS (continued)
ANNEX A: SCOPE OF WORK
ANNEX B: DETAILED METHODOLOGY
ANNEX C: FIELD VISIT SCHEDULE
ANNEX D: DETAILED TABLES 
 
iii
LIST OF TABLES
 
 PAGE
Table 1: Summary of the groups consulted during the field visits 7
Table 2: Sampling of community based staff and their supervisors….. 8
Table 3: ISA effect/impact indicator performance 12
Table 4: ISA Activity/output indicator performance 13
Table 5: Staff composition and changes in staffing 17
Table 6: ISA team member outputs and activities for the period FY1999-Fy2002 18
Table 7: USAID perceptions about the most recently submitted Title II DAP 24
 proposal
Table 8: Educational message delivery: a comparison of the situation …….. 26
Table 9: Comparative ranking of community level trainings attended ………. 27
Table 10: Knowledge about and use of barrier analysis and TIPS 31
Table 11: Use of Quality Improvement checklists at regional and community level 33
Table 12: External assessments of changes in monetization and CM 37
Table 13: Emergency response preparedness 39
Table 14: Emergency responses by FHI in the four years preceding ISA……. 41
Table 15: Activities related to FAM information technology 48
Table 15: Number of subscribers to the FAM listserves 50
LIST OF GRAPHS
 
Graph 1: Categorization of program expenditures FY1999-FY2002 19
Graph 2: Visits to the FAM website FY1999-FY2002 49
 
iv
LIST OF ACRONYMS
CD Country Director
CDP Child Development Program
CHW Community Health Worker
CIP Children in Program
CM Commodity management
CS Cooperating Sponsors
DPRP Disaster Preparedness Response Plan 
EO Extension Officer
ER Emergency response
FAM Food Aid Management
FFP Food for Peace
FHI Food for the Hungry International
FHI/B Food for the Hungry Bolivia
FHI/K Food for the Hungry Kenya
FHI/M Food for the Hungry Mozambique
FSRC Food Security Resource Centre
FY USAID Financial Year (September to August)
ISA Institutional Support Agreement
ISG Institutional Support Grant 
i.t.o in terms of
KPC Knowledge, Practice and Coverage 
LCB Local Capacity Building
LOA Life of Activity
MCHN Maternal and Child Health
M&E Monitoring and Evaluation
MYOP Multi-year Operational Plan
N/a Not available
QICL Quality Improvement Checklists
RDA Rapid Disaster Assessment 
TA Technical Advisor
TIS Technical Information Specialist
US United States
USAID United States Agency for International Development
VOC Voice of community
WFP World Food Program
WHO World Health Organization
1
EXECUTIVE SUMMARY
Introduction
USAID awarded its first Institutional Support Grant (ISG) to FHI for the period January 
1997 to August 1998. The ISG’s main objective was to strengthen the monitoring and 
evaluation (M&E) systems and tools used by FHI’s Title II fields. Upon successfully 
meeting this objective, FHI was awarded a second grant for the period FY1999 to 
FY2003. The main goal of the FHI’s ISA is to increase the impact of its Title II food 
security programs in Bolivia, Ethiopia, Kenya and Mozambique via the improvement of 
its technical, programmatic and managerial capability. 
This goal was to be achieved through the following objectives: 
1) Select, promote and train staff in the use of standard, high-quality tools 
for Title II program design and implementation as a follow up to the 
accomplishments achieved under the current ISG program in M&E 
system standardization;
2) Improve FHI's capacity to respond to emergencies and facilitate a rapid 
transition to development activities in Sub-Saharan Africa;
3) Conduct needs assessments in the West African Sahel (Mali, Niger, 
Burkina Faso) and Haiti to determine rationale for and feasibility of 
initiating activities in those countries;
4) Improve FHI's capacity to efficiently and effectively manage commodities; 
5) Collectively improve, with the other cooperating sponsors (CS): a) 
program monitoring and evaluation, b) monetization activities and 
Bellmon analyses, and c) local capacity building via substantive 
collaborative efforts with other Title II cooperating sponsors.
6) Contribute toward the improvement of FAM knowledge and proficiency in 
using information technology to enhance communication and information 
flow between the PVO members of FAM (mentoring partnership).
This report summarizes the findings of the final evaluation of the FHI ISA. This 
evaluation took place in October/November 2002 and its purpose was to provide an 
assessment of the results achieved, reasons for levels of achievement or non￾achievement, and lessons learned from the ISA program. Its focus is on an external 
review of the ISA’s contribution towards food security achievements through FHI Title II 
programs.
The evaluation methodology consisted of two distinct parts. The first included a thorough 
review of existing information and documentation, whilst the second focused on 
collecting new information via questionnaires and group interviews. The following groups 
and individuals were consulted: FFP; CS members based in the US; USAID missions in 
Bolivia, Kenya, Mozambique; FHI managerial staff, regional supervisors and community 
based staff in Bolivia, Mozambique and Kenya. 
2
General findings
The FHI ISA successfully completed most of its targeted activities and outputs. With one 
year of the program still to be completed, thirty-eight of the forty-two output/activity 
targets of the FHI ISA, were fully met or exceeded. Three of the four targets that were 
not met were only slightly below target and adequate explanations could be given for 
underperformance. An analysis of staff outputs and budget and expenditure patterns 
shows that activities largely correspond with planning done in the original project 
proposal. 
Training materials were generally comprehensive and well prepared. It is important that 
enough time and resources be allocated during the next ISA, to consolidate and compile 
all the training materials of this ISA. Post test scores and comments by the course 
participants, suggest that there has been a significant increase in knowledge amongst 
those who attended the training. Several suggestions as to how the format of 
workshops can be improved have been included in the report (pages 22-23).
Objective specific findings and impact
In addition to increasing staff knowledge, the ISA also succeeded in improving 
implementation of various aspects of its Title II programs. In so doing it positively 
impacted the ability of its Title II fields to meet their food security targets. However, the 
team found that most of the ISA trainings are only likely to make significant impacts on 
food security during the next DAP cycle.
Objective 1: Select, promote and train staff in the use of standard, high-quality tools for 
Title II program design and implementation
Several training workshops were conducted to impact this objective. These include: a 
program design and analysis workshop (17 participants), workshops on educational 
messages and methods messages and methods (73 participants), Positive Deviance 
and Hearth Method (100 participants), Barrier-analysis/TIPS (76 participants) and 
remedial M&E (81 participants). 
The evaluation team noted the following impacts that may be attributed in large part 
to these workshops: 
· The quality of recent project proposals has improved significantly and staff 
participation in the process has increased. 
· The evaluation team found that all fields have developed training curricula and 
lesson plans as a result of the ISA assessment and training. Where lesson plans 
already existed before the ISA, their quality has also improved. In most fields, 
training has become more participatory and the use of appropriate visual aids 
and other techniques to facilitate learning has increased.
· Awareness and use of Quality improvement checklists (QICs) is high in 
Mozambique and Kenya. Use of QICs has probably contributed to improved 
performance by community-based workers and has strengthened FHI’s M&E 
capacity. On the other hand, the evaluation team also noted that turnover of key 
management staff during the past two years has weakened FHI’s M&E capacity 
in some countries.
3
The following program improvement tools have only been partially implemented in 
the fields and it is still too early to note an impact on program implementation and 
food security:
· Positive Deviance has been implemented in Mozambique, but not yet in Kenya
and Bolivia. Both Kenya and Bolivia plan to implement it in 2003.
· The use of Barrier analysis and TIPS was very limited in all three fields although 
the main reason for this may be that training was done quite recently. 
Objective 2: Improve FHI's capacity to respond to emergencies and facilitate a 
rapid transition to development activities in Sub-Saharan Africa;
A total of eight training workshops, related to emergency relief, were presented 
during the first four years of the ISA program. Topics covered in these workshops 
include: Emergency Relief preparedness (9 participants); rapid disaster assessment 
workshop(15 participants); Emergency program design and implementation 
workshop (9 participants); Improving capacity to respond to emergencies and rapid 
transition to development (35 participants) and emergency program M&E workshop 
(14 participants) 
The evaluation team found that less than 20% of the managers who attended the 
first three courses are still working in FHI country programs. The Title II fields in 
Kenya and Mozambique have applied, albeit on a small scale and in a modified form, 
the knowledge that was gained during the ISA training. There is some evidence that 
the training had an impact outside FHI’s Title II fields e.g. the DRC, Kosovo, 
Nicaragua and India. 
Objective 3: Conduct needs assessments in the West African Sahel (Mali, 
Niger, Burkina Faso) and Haiti to determine rationale for and feasibility of 
initiating activities in those countries;
The evaluation team concluded that this objective has been completed successfully. 
The assessments were based on careful consideration of all the factors that could 
have an influence on a decision about FHI’s Title II expansion.
Objective 4: Improve FHI's capacity to efficiently and effectively manage 
commodities;
This program component included a significant monitoring and support component 
and focused on the two FHI fields that are directly involved in handling Title II 
commodities (Bolivia and Ethiopia). Seven training workshops were presented. 
These workshops were mainly aimed at explaining the monetization process and 
introducing/refining commodity management systems. In general, the team found 
evidence of the standardization and streamlining of commodity management 
systems in Bolivia and Ethiopia and increased knowledge at FHI’s headquarter level 
in respect to all aspects of commodity management. 
Objective 5: Collectively improve, with the other cooperating sponsors (CS): a) 
program monitoring and evaluation, b) monetization activities and Bellmon 
analyses, and c) local capacity building via substantive collaborative efforts 
with other Title II cooperating sponsors.
FHI had a significant impact on the M&E working group in terms of its 
contributions as chair of the working group and in terms of research and 
other support that went into the development of the M&E toolkit. Its 
involvement in the monetization and local capacity working groups were 
less significant.
4
Objective 6: Contribute toward the improvement of FAM knowledge and 
proficiency in using information technology to enhance communication and 
information flow between the PVO members of FAM (mentoring partnership).
FHI met the overall objective and the three sub-objectives that it set out to achieve in 
relation to FAM information technology capabilities. It also assisted with the creation 
of the FSRC database, which is an activity that was not included in the original 
project proposal. The evaluation team concluded that this has been a very 
successful mentoring partnership. FHI’s support enabled FAM to achieve most of its 
technology related objectives.
Recommendations
The evaluation team had the following recommendations:
· The loss of managerial level staff was high in all fields. Since the capacity built 
during this ISA was lost with them, FHI should assess the underlying reasons for 
what appears to be a higher than normal turnover rate and then take appropriate 
action to resolve the problem. 
· The ISA team, in conjunction with FHI Regional Directors and Country Directors 
(CD’s), should develop a basic orientation package of documents, tools and 
methods that should be given to new food security managers in order to improve 
program continuity. 
· FHI should continue to think strategically about how to improve its capacity to 
respond to emergencies and facilitate a rapid transition to development. The 
practice of training development staff to become “emergency experts” and then 
expecting them to assume leadership of emergency relief activities in times of a 
disaster appears to be less than ideal. Additionally, high turnover rates of staff 
trained in Emergency and Relief management have negated to some degree the 
benefits of the ISA Emergency and Relief Management training. 
· Future ISA training programs should consider putting increased emphasis on 
training of trainers, to improve the ability of in-country managers who receive ISA 
training to effectively replicate that training with their subordinates
· The mentoring component of future ISA programs should be increased.
· The next ISA should consolidate the training materials and strengthen the 
implementation of the new tools and methods introduced during the current ISA 
and previous ISG. If fields are expected to implement Barrier Analysis, TIPS and 
Positive Deviance effectively, additional mentoring will be required.
In terms of the content of a future ISA program, various possible themes emerged from 
this evaluation. The most important of these are:
· Consolidating the training materials of the current ISA into manual format
· Developing and expanding CD-ROM and general reference library capabilities in 
Title II fields
· Linking monitoring to management information systems with the objective of 
strengthening the relationship between monitoring information and changes in 
program implementation. Training in data analysis: basic EPI-Info and SPSS; 
statistical methods for more advanced data analysis 
· Technical support in the areas of income generating activities, marketing and 
livestock
· Continued support in terms of Commodity management and FAM information
technology
· Identifying, adapting and sharing food security innovations used by Non-Title II 
development programs. 
5
CHAPTER1: INTRODUCTION
1.1 Introduction
Food for the Hungry International (FHI) implements and facilitates a variety of 
development programs in Asia, Africa, Latin America, and Eastern Europe. The 
worldwide goal of FHI is to reduce hunger and poverty. One of its key objectives 
is to increase food security via increased agricultural production and improved 
health and nutrition. FHI focuses on building local capacity in order to enable 
communities to progress beyond meeting their basic needs. By strengthening 
leadership and community cohesion, the organization strives to empower 
communities to continue to grow and develop even after FHI is no longer present 
(FHI 1998) One or more of the following components are usually included in the 
organization’s integrated community development programs: 
· Agricultural production
· Water and sanitation
· Natural resources management
· Health and nutrition 
· Income generation
· Child education
In addition to receiving private donations from individuals and organizations, FHI 
also receives funding from a variety of International Donors such as EEU, USAID 
and ODA amongst others. FHI’s first Title II funded activities began during the 
mid-1980’s, when Bolivia and Ethiopia initiated their first Multi-Year Operational 
Plans (MYOP1
). Kenya followed shortly afterwards in 1989 whilst FHI
Mozambique received its first Title II grant in 1997. 
USAID awarded its first Institutional Support Grant (ISG) to FHI for the period 
January 1997 to August 1998. The ISG’s main objective was to strengthen the 
monitoring and evaluation (M&E) systems and tools used by FHI’s, Title II fields 
in Bolivia, Ethiopia, Kenya and Mozambique. Its related purposes were to 
improve the following components of the food security M&E continuum: 1) 
macro-targeting, 2) micro-targeting, 3) indicator development, 4) baseline data 
collection and analysis, 5) monitoring, and 6) evaluation. Upon successfully 
meeting the objectives of this ISG2
, FHI was awarded a second grant for the 
period FY1999 to FY2003. The next section briefly describes the main objectives 
of this agreement. 
1.2 Goals and objectives of ISA (FY1999-FY2003)
The main goal of FHI’s Institutional Support Agreement (ISA) is to increase the 
impact of its Title II food security programs via the improvement of its technical, 
programmatic and managerial capability. 
 
1
These were later changed into Development Activity Proposals (DAP)
2 USAID changed the name of the Institutional Support Grant (ISG) to Institutional Support 
Agreement (ISA) in 1998.
6
This goal was to be achieved through the following objectives: 
1) Select, promote and train staff in the use of standard, high-quality 
tools for Title II program design and implementation as a follow up 
to the accomplishments achieved under the current ISG program 
in M&E system standardization;
2) Improve FHI's capacity to respond to emergencies and facilitate a 
rapid transition to development activities in Sub-Saharan Africa;
3) Conduct needs assessments in the West African Sahel (Mali, 
Niger, Burkina Faso) and Haiti to determine rationale for and 
feasibility of initiating activities in those countries;
4) Improve FHI's capacity to efficiently and effectively manage 
commodities; 
5) Collectively improve, with the other cooperating sponsors (CS): a) 
program monitoring and evaluation, b) monetization activities and 
Bellmon analyses, and c) local capacity building via substantive 
collaborative efforts with other Title II cooperating sponsors.
6) Contribute toward the improvement of FAM knowledge and 
proficiency in using information technology to enhance 
communication and information flow between the PVO members 
of FAM (mentoring partnership).
The main beneficiaries of the ISA program have been: 
· FHI Title II programs in Bolivia, Ethiopia3
, Kenya and Mozambique, 
· Potential future FHI Title II programs 
· Other FAM-member CS’s 
Food For Peace (FFP) requested all CS’s who benefited from the current ISA 
grant period to do their final evaluations at the beginning of FY2003 so that the 
findings could be used as the basis for the proposals for the next grant period. 
This report summarizes the findings of this final evaluation.
1.3 Objective of the final evaluation
The purpose of the final evaluation was to provide an assessment of the results 
achieved, reasons for levels of achievement or non-achievement, and lessons 
learned from the ISA program. Its focus is on an external review of the ISA’s 
contribution towards impacts on food security through FHI Title II programs. 
 
3
FHI Ethiopia decided to suspend its Title II funded activities at the end of FY 2002. Ethiopia will 
therefore not be included in the planned ISA activities for FY 2003. 
7
CHAPTER 2: METHODOLOGY
2.1 Introduction 
The evaluation methodology consisted of two distinct parts. The first included a 
thorough review of existing information and documentation. Details about the 
nature of these documents are contained in the Scope of Work (Annex A). The 
second part focused on collecting new information via questionnaires and group 
interviews. 
Most of the questionnaires, especially those aimed at FFP, and CS members 
based in the United States (US), were dispatched electronically. Follow-up 
questions were also sent in this manner. In order to conduct the group interviews, 
three of the four Title II programs supported by this grant were visited. The 
programs included in the field visits were: Kenya, Mozambique and Bolivia. The 
main reasons for selecting these fields were that Bolivia and Kenya were not 
visited during the Mid-Term Evaluation. Ethiopia was visited during the Mid-term 
Evaluation, but phased out its Title II activities in FY2002. ISA program activities 
were re-focused towards the other three fields once it became clear that FHI 
Ethiopia would not be continuing with Title II activities. 
A combination of questionnaires, semi-structured group interviews and focus 
groups were used during the field visits. The remainder of this document contains 
details about the nature of these interviews and the people who were 
interviewed.
2.2 Objectives of the field visits
Within the framework of the overall objective of the evaluation4
 the in-field 
information collection was aimed at:
· Collecting information about the ISA program that is not contained in 
existing documentation. 
· Measuring the impact indicators as defined in the Monitoring and 
Evaluation Plan.
· Determining the extent to which ISA training messages are being used 
and applied in the field.
· Determining the extent to which ISA messages have contributed towards 
improving the quality of its food security programs.
· Identifying the factors that may have influenced the adoption of ISA 
training messages.
· Setting a process in motion (through the use of participatory interviewing 
methods) whereby field staff actively identify and remove barriers to the 
adoption (by FHI staff) of alternative or improved practices. 
 
4
The purpose of the final evaluation was to provide an assessment of the results 
achieved, reasons for levels of achievement or non-achievement, and lessons learned 
from the ISA program. The evaluation centers around an external review of impact-level 
results on the ISA’s contribution to impacts on food security achievements through FHI 
Title II programs. 
8
2.3 Methodology
A combination of questionnaires, semi-structured group interviews and focus 
groups were used during the field visit. These interviews were conducted with a 
wide range of FHI staff, counterparts and donor representatives. Details of the 
people who were interviewed, the questionnaires and interview schedules are 
provided in Annex B. The table below summarizes the composition and size of 
the groups interviewed in each field.
Table 1: Summary of the groups consulted during the field visits
Number of people interviewed
Nature of focus group Bolivia Mozambique Kenya
USAID mission 2 2 1
FHI Commodity management 4 4 3
FHI Emergency response - 4 3
FHI National management 8 4 4
FHI Regional management 8 19 *
FHI Community based staff 17 32 17
* This group interview was done jointly with the community based staff
As can be seen in the preceding table, most group interviews were with focus 
groups, consisting of between 2 and 10 members. However, one interview – the 
semi-structured group interview with community-based staff and their 
supervisors, sometimes had up to 41 staff members present. The semi￾structured interview was preceded by a questionnaire survey that was completed 
by a random sample of 30 staff members5
 who work at community level. The 
data of the survey was analyzed immediately after the completion of the 
questionnaire and the findings were integrated in the semi-structured group 
discussion. During the group interview, the Community Level Staff and their 
direct supervisors were present. The purpose of this big group interview was to 
illicit discussions between staff members and their supervisors about reasons 
why ISA promoted practices are being used or not. Thus, this particular group 
discussion was not only aimed at providing information to the evaluators, but its 
participatory nature set a process in motion that could enhance the 
implementation of the country programs and the next ISA program. 
The in-field assessment also included a community based assessment of the 
educational practices6
 used in each country, because the success of most of 
FHI’s food security programs rely heavily on community training and influencing 
behavior change at community level. As a result of limited time and limited 
 
5
The number thirty was selected as it is the biggest possible group that can be managed in a 
group interview of this nature. In some cases such as Kenya, the group included all staff 
members and their supervisors and the total was less than 30. In the case of Mozambique 30 
represented 43% of the total community level staff. Combined with their supervisors 41 staff 
members were interviewed in Mozambique. 
6 Objective one of this program focused on promoting and training staff in the use of high quality 
tools for program implementation. One of the training courses that was developed centered 
around improved methods that can be used to train rural communities. 
9
existing information on this 7
, the evaluation team had to base their conclusions 
on a random selection of six community trainers per country. 
The six trainers were randomly chosen from the already randomly selected list of 
thirty health and agricultural workers, who participated in the semi-structured 
group interview. To save time and enable the evaluators to complete this 
exercise in one day, the selected trainers worked in three non-randomly selected 
villages. One evaluation team member was allocated to each village. The 
evaluators had to evaluate the training methods of one health and one 
agricultural trainer in the village allocated to them. These villages were all 
located in the same operational area (region). In order to extrapolate 
observations about community training methods to the country as a whole, the 
evaluators also asked the groups, who were being trained, whether their training 
normally is like the one presented to them on that day. The evaluators also had a 
short meeting afterwards, during which they compared notes and discussed their 
observations during the training.
Table 2:Sampling of community based staff and their supervisors in each field
Health sector
Agriculture and Natural 
Country and resource management 
regions
Total 
number 
of staff
Number 
of staff 
inter￾viewed
% of 
staff 
inter￾viewed
Total 
number 
of staff 
Number 
of staff 
inter￾viewed
% of 
staff 
inter￾viewed
Bolivia8
 Cochabamba 8 4 50 10 3 30
 Sucre 4 3 75 10 4 40
 TOTAL 12 7 58 20 7 35
Mozambique
 Gorongosa 12 4 33 15 7 47
 Caia 10 4 40 11 5 45
 Nhamatanda 11 5 45 14 6 43
 Marromeu 10 6 60 8 4 50
 TOTAL 43 19 44 48 22 46
Kenya
 Marsabit 8 8 100 9 9 100
 TOTAL 8 8 100 9 9 100
The randomly selected community level trainers were told of their selection on 
the afternoon preceding the training. They could cover any topic during the field 
visit. The topic choice largely depended on community needs and trainings that 
preceded the visits of the evaluators. The community workers, who normally 
work in the three selected communities, were also present during the training in 
order to introduce the group and help smooth over possible problems related to 
having a ‘different’ trainer present in the community. Translators also
accompanied the three evaluators during these visits. After training, the 
evaluators gave individual feedback to the trainers. 
 
7
The final evaluations of most of these country programs did not include a detailed assessment 
of community level training.
8
 Nineteen people were interviewed and completed the questionnaires; five were from other 
departments not listed here.
10
2.4 Groups and resources
The groups and individuals that were consulted during the evaluation consisted 
of two distinct groups. Those based outside the countries where FHI Title II 
programs are being implemented and those who live and work in those countries. 
The former group mainly supplied documentation and information about the 
project, but some also completed short questionnaires about the ISA program. 
Even though two of the ISA team members formed part of the evaluation team, 
none of the other team members were visited personally. However, a range of 
individuals and groups were interviewed and visited in the country programs. 
 
 Prior to the field visit, the following sources and individuals were consulted: 
1) FAM working group members 
2) The FAM Information technology representative and the FAM Executive 
Director
3) Information obtained from ISA staff members/consultants about 
implementation
4) Final impact evaluation of field projects: Kenya, Ethiopia, Bolivia and 
Mozambique
5) ISA Project documentation 
6) Electronic Interviews/questionnaire for ex-ISA team members and ISA 
staff members who are not evaluation team members 
The visits to the country programs included discussions with USAID 
representatives and FHI staff members involved in the implementation of Title II 
programs. All the USAID delegations are based in the country capitals. The 
three programs that were visited are organized differently and for this reason a 
short description of each is given below:
Mozambique:
The Country Director and national level management are based in Beira. 
The Title II program has activities in three districts in Sofala province: 
Nhamatanda, Marromeu and Gorongosa. In these regions there are 
supervisory staff members and Community Level Staff who work directly 
with communities.
Kenya:
In Kenya, the Title II program is focused in one geographical area, 
Marsabit. The Title II program manager and his management team are 
based in Marsabit, whilst the Country Director and his senior management 
team are based in Nairobi.
Bolivia:
The country director and national level management of the FHI Bolivia 
program are based in La Paz. Title II program activities are restricted to two 
geographical areas namely the Cochabamba and Potosi departments9
. In 
each geographic area, there are some regional or middle management staff 
and Community Level Staff who work directly with communities. 
 
9
The water and sanitation program was also implemented in La Paz and Oruro departments, but 
since most of the ISA activities focused on capacity building in the areas of health, agriculture and 
commodity management, only the Head Office and the two geographical areas where agricultural 
and health activities were implemented will be considered for the evaluation. 
11
In all three programs, staff that work directly with the implementation of Food 
Security programs, were regarded as operational staff. This included all technical 
specialists in the fields of health and agriculture. Staff members, who work with 
logistics, administration and finance are referred to as support staff. 
Summary of the interviews conducted during country program visits: 
1) A random sample of 30 Community Level Staff (operations) participated 
in a questionnaire survey and group interview. Sampling was based on a 
complete list of health and agricultural workers working at community 
level in all regions 10
.
2) Regional level staff. The supervisors and coordinators of Community 
Level Staff (operations) from all regions: one group interview 
3) Regional and National Level Staff (operations): one focus group 
discussion
4) Staff involved in emergency response: focus group discussion – only for 
Kenya and Mozambique 
5) Staff involved in commodity management: Inspection of the commodity 
management process and focus group interview. 
6) USAID local missions– One Title II representative and one M&E 
representative.
 
2.5 Filed visit schedules
The following schedule for the visits to each country is a generic example of the 
field activities. In some countries this was changed slightly as a result of logistical 
considerations. A detailed field visit schedule is attached in Annex C.
Day one: Interviews in capital with USAID 
Day Two: Community based staff complete the questionnaire (1/2-1 hour max)
 Analysis of their responses while monetization/emergency response
 interview takes place (1 hour)
 Monetization/emergency response interview (2 hours) 
 Group interviews with community staff & their supervisors (2 hrs)
Day Three: Field visit: team split : one per community; attend training of two 
 groups in each village; have short interview with attendees and give 
 feedback to trainers. Whole day.
Day Four: Management interviews/outstanding matters (2 to 4 hours)
 
 
10 In Kenya and Bolivia, the total number of field staff was less than 30. All the Kenya staff 
attended the group interviews. In Bolivia, not all of the staff could participate in these interviews 
as a result of an evaluation and training program that were running concurrently with the ISA 
evaluation. However, all staff members that were available at the time were interviewed. 
12
CHAPTER 3: MONITORING AND INDICATOR TRACKING TABLE
3.1 Introduction
The ISA’s program performance and monitoring plan (FHI1999b) was approved 
at the beginning of FY2000. The original indicator tracking table that formed part 
of the project proposal was included in this document, as well as program impact 
indicators. The activity/output indicators had specific annual targets and were 
measured annually. These indicators mainly reflected the number of workshop 
presented, number of participants and pre- and post-test scores. Most of this 
information was obtained from workshop attendance records. 
The effect and impact indicators were only to be measured during the final 
evaluation that was originally scheduled for FY2003. These indicators were 
reflective of the actual impact that the program would have on food security 
program planning and implementation at field level.
3.2 Program performance indicator tracking table 
FHI fully complied with the monitoring and evaluation activities that it set out to 
do according to its monitoring and evaluation plan (FHI 1999b). A mid-term 
evaluation was conducted in April-June 2001 (FHI 2001b). The annual reports 
always included a detailed indicator performance tracking table which contained 
all the indicator information that was associated with planned activities for that 
year. In some cases, where program activities were modified as a result of new 
advances or other factors, this was also reported on and the absence of 
performance indicators values was justified. Table Two on the next page 
summarizes the program’s performance for the period FY1999 to FY2002. 
With one year of the program still to be completed, thirty eight of the forty two 
output/activity targets that the FHI ISA set out to achieve, were fully met or 
exceeded. The four targets that were not met were:
· Educational methods and messages post-test score (70% instead of 
80%)
· Number of staff trained on how to adequately prepare for an emergency 
(9 instead of 10)
· Number of staff trained in designing and implementing an emergency 
program (9 instead of 10)
· Average post test score for emergency transition workshop participants
(65% instead of 80%)
In the case of educational messages most fields were close to the target, but 
Mozambique’s score of 58% was very low. Several reasons for the lower-than￾planned post-test scores in the educational messages workshops were given 
(FHI 2000:4):
1) The workshops covered a lot of topics and adding a day or two may have 
helped the staff to fully absorb the material;
2) One of the test questions was answered incorrectly by almost all the 
participants, which is generally a sign that the question was poorly worded. 
13
Without that question test scores would have been close to or greater than 
80% for three of the four fields in question;
3) In the case of Mozambique (post test –score 58% vs. 80% planned), the 
workshop was conducted by second-generation trainers as the original 
workshop was postponed due to the flooding in the program area in February 
2000.
In the case of lower than planned test scores for the workshop on transitioning 
from emergencies into development, one of the main reasons may again be that 
a lot of materials were covered within a very short time period. It is also important 
to note that average pre-test scores increased from a low 31% to 65%. Thus 
staff, who attended the workshop, probably started from a much lower knowledge 
base than expected (FHI 2001a:6). The number of staff trained in emergency 
preparedness and the design and implementation of emergency programs was 
only one short of the target. Considering that ISA only recommends who should 
attend training and the fields make the actual decision on who and how many 
staff members will be trained, this difference is not significant from the ISA 
perspective. 
Table 3: ISA effect/impact indicator performance (FY1999-FY2002) 
ACTIVITY/OUTPUT INDICATORS
Black Font = Goal was met
 Bold Font = Goal was exceeded
 ( ) = Goal was not met Planned Achieved11
90% of Bolivia and Ethiopia Title II managers will be 
correctly using standardized commodity management 
procedures established by the FHI ISA team
90% Bolivia 
Percentage of FHI Title II staff (educators and trainers) 
who will be using participatory and learner centered 
educational methods and messages 12
75% 90%
Percentage of FHI Title II field programs that will have at 
least one staff person who is correctly using Factor 
Analysis to discover barriers to behavioral change 
90% Mozambique
Bolivia
Percentage of FHI title II fields that will be implementing 
the Hearth methods in its health/nutrition programs 50% Mozambique
Percentage of FHI Title II fields that will be implementing 
a Census -based, impact oriented approach to food 
security programming
25% Not implemented13
Percentage of FHI Title II fields that have responded to 
an emergency using ISA recommended response plan 100%14 Kenya
Mozambique
Percentage of FHI fields that have responded to a recent 
disaster in the country using the FHI ISA standard set of 
methods/tools selected for designing, monitoring and 
evaluating emergency programs
75%14 Kenya
Mozambique
Number of FAM staff persons who will be successfully 
managing FAM’s web site listservs and chats 1 1
 
11 Ethiopia was not included in the evaluation due to the phasing out of Title II projects. Original 
targets for indicators expressed as ‘percentage of Title II fields’ included Ethiopia. 
12 This calculation is based on the percentage of non-supervisory staff members who reported using three or 
more of the eight educational methods promoted by ISA. Weights were applied to adjust for sampling bias. 
13 This approach was included in the original project proposal, but FHI since then decided not to implement. 
14 Bolivia was never targeted for emergency training. The 100% therefore originally referred to Ethiopia, 
Mozambique and Kenya.
14
Table 4: ISA Activity/output indicator performance(FY1999-FY2002) 
ACTIVITY/OUTPUT INDICATORS
Black Font = Goal was met
 Bold Font = Goal was exceeded
 ( ) = Goal was not Planned Achieved
Objective 1: Program Design and Implementation
Number of ISA team staff oriented to program
4 5
Number of surveys conducted to determine remaining 
M&E problems 1 1
Number of field practicums conducted on 3 most 
problematic M&E tools 4 4
Number of staff trained in remedial M&E 60 81
Number of reviews of tools for problem analysis and a 
review to identify the best practices in program design. 1 1
Number of training workshops conducted for Title II 
program managers from all fields in generic tools for 
problem identification, analysis and solving and review of 
best-practice program design.
1 1
Number of staff trained in problem analysis and program 
design. 12 17 
Average post-test scores for problem analysis and 
program design workshop 80% 85%
Number of field assessments conducted in educational 
methods and messages being used in Title II programs. 4 4
Number of Epi-Info workshops conducted 1 1
Number of staff trained in Epi-Info 12 14
Average post-test scores for Epi-workshop 80% 87%
Number of field training exercises conducted in 
educational methods and messages. 4 4
Number of staff trained in educational methods and 
messages. 60 73
Average Post-test score for educational methods and 
messages workshop participants 80% (70%)
Number of training workshops and field exercises on the 
Hearth method in nutrition and positive deviance models 
for agriculture 4 4
Number of Title II staff trained on the Hearth method in 
nutrition and positive deviance models for agriculture 60 100
Average Post-test score for Hearth workshop participants 80% 82%
15
ACTIVITY/OUTPUT INDICATORS
Black Font = Goal was met
 Bold Font = Goal was exceeded
 ( ) = Goal was not Planned Achieved
Number of workshop/practicums conducted in barrier 
analysis of food security-related behavior change. 4 3 (Eth not done)15
Number of Title II staff trained and having completed 
practicum in factor (barrier) analysis. 60 76
Objective 2: Disaster Relief
Number of visits to review of two emergency/transition 
programs in SSA that have successfully transitioned from 
an emergency to development program in the quickest 
possible time
1 1
Core sets of tools selected for design and M&E of 
emergency and transition programs 1 1
Number of workshops conducted for IRO/Title II staff in 
how to prepare for an emergency 1 1
Number of IRO/Title II staff trained in how to adequately 
prepare for an emergency. 10 (9)
Number of training workshops conducted for IRO/Title II 
staff in how to conduct rapid disaster assessments 1 1
Number of IRO/Title II staff trained in how to conduct 
rapid disaster assessments 10 17
Average Post-test score for rapid disaster assessment 
workshop participants 80% 93%
Number of training workshops conducted for IRO/Title II 
staff in designing and implementing an emergency 
program
1 1
Number of IRO/Title II staff trained in designing and 
implementing an emergency program 10 (9)
Nu\mber of Title II staff trained in how to transition quickly 
from emergency to development programming 15 35
Average post test score for emergency transition 
workshop participants 80% (65%)
Number of training workshops conducted for Title II staff 
in how to monitor and evaluate emergency programs. 2 2
Number of Title II staff trained in how to monitor and 
evaluate emergency programs. 15 15
Objective 3: Food security needs assessments
Number of food security needs assessments conduced 
in the West African Sahel and in Haiti 2 2
Objective 4: Commodity Management
Number of assessments of current commodity 
management system and FHI capacity 1 1
 
15 Dropped due to the phasing out of Title II in that field as approved for the 02 Work Plan
16
ACTIVITY/OUTPUT INDICATORS
Black Font = Goal was met
 Bold Font = Goal was exceeded
 ( ) = Goal was not Planned Achieved
Number of reviews of commodity management 
methodologies, procedures, and tools in current use by 
other CSs and FFP 1 1
Number of training workshops conducted on 
standardized commodity management procedures. 6 7
Number of Title II commodity staff trained in standardized 
commodity management procedures. 60 97
Average Post-test score for commodity management 
procedures workshop participants 80% 85%
Objective 5: Collaboration with Cooperating Sponsors
Number of FAM workshops conducted on statistical 
sampling 1
1
Collaborative efforts with other FAM members in M&E, 
monetization, and local capacity-building (ongoing, not 
quantified)
yes Yes
Objective 6: FAM Mentorship
Number of FAM information system mentoring outputs 
achieved. 3 4
17
CHAPTER 4: OVERVIEW OF PROGRAM RESOURCES, ACTIVITIES 
AND CURRENT STAFF CAPACITY
4.1 Introduction
The primary aim of this ISA was capacity building of FHI staff working in Title II fields. 
Themes covered by the training were tools for the design and implementation of food 
security programs (objective 1); commodity management (objective 4) and emergency 
programs (objective 2). Other aspects, such as support to FAM in terms of information 
technology (objective 6) and close cooperation with cooperating sponsors in 
developing common methods (objective 5) were ongoing and may be regarded as 
secondary activities in terms of the time and resources spent on them. Objective three 
namely ‘food security needs assessments in Haiti and the Sahel’, was a once off 
activity that was completed in FY 1999. 
This chapter briefly reviews the general inputs and resources used for the 
implementation of the program. Short descriptions and an analysis of the staff and 
financial resources used are provided. This is followed by a description of the methods 
used to implement the program and suggested changes and opportunities for future 
programs. 
4.2 Staffing
All the team members spent only a part of their time on ISA activities. At any one time, 
most of them also had other responsibilities within FHI. With the exception of the team 
leader, who was busy for 90% of his time with ISA activities, the relative contribution of 
the other team members varied between 25 and 75%. The MCHN Trainer is the only 
one of the five team-members, who was not a full-time employee of FHI. He was called 
in as a consultant when needed. Generally, the outputs of the ISA team corresponded 
well with the activities contained in the original project proposal (FHI 1998). The 
discussion of the program results in Chapter 5, indicate that ISA achieved most of its 
output objectives. During the past four years the ISA team developed a total of 
fourteen training courses. Approximately 550 people16 were trained. In the case of 
commodity management, a considerable amount of time was devoted to mentoring, 
training and providing head office support to aspects related to monetization, 
commodity management and the management of Title II resources in general. 
With regards to the team leader, one would suspect that the ISA program activities 
took place in spite of a very heavy workload. The first team leader doubled as 
Regional Director for Africa. In that position he had several difficult situations to resolve 
and several large country programs to supervise, in addition to keeping up with the 
demands of the ISA program. Currently the team leader doubles as the Regional 
Director for Latin America. The fact that the program was largely executed as planned, 
in spite of heavy additional demands on the team leaders, is an indication of their 
commitment and dedication to the work of FHI and the objectives of ISA. 
 
16 Some people attended more than one training course.
18
The table below illustrates the composition of the team that implemented the ISA 
program as well as changes in staffing that took place between FY1999 and FY2002. 
Table 5: Staff composition and changes in staffing (FY1999-FY2002)
FY1999 FY2000 FY2001 FY2002
Position Main Technical
Responsibilities
Time 
devoted 
to ISA
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
Team 
Leader
General Program 
Management
Objectives 1, 3, 5
Tech. support 
agric.
90%
MCHN 
TA 
&Trainer
Objectives 1, 3, 5
Technical support 
Mother and Child 
Health
50%
Info. 
Systems
TA & 
Trainer
Objective 6
Technical support 
Information 
systems
25%
CM
TA & 
Trainer
Objective 1, 4 
Technical 
Commodity 
Management
75%
ER
TA & 
Trainer
Objective 2&4
Technical 
Emergency 
response
30%
 Key:
 Original incumbent
 New staff member
There have been some changes in the ISA team composition during the past four 
years. The preceding table shows that three of the five positions in the ISA team had 
to be filled by new incumbents during the past four years. Two of the three positions 
became vacant as a result of internal restructuring and promotions within FHI. There is 
some evidence to suggest that the changes in staffing did not affect the functioning of 
the team negatively. For example, in the case of information technology support, FAM 
specifically mentioned in their FY 2000 evaluation that these changes did not affect the 
support provided to them, because other staff members stepped in to ensure continuity 
(FHI 2000). Table 5 and the analysis of expenditure on especially travel, shows that 
there was a slight decrease in activity during FY2002. This can probably be attributed 
to the need for the new team leader to find his feet, but it is also a reflection of the fact 
that by the beginning of FY2002 most of ISA’s training objectives have already been 
met. The only training scheduled so far for FY2003 is on Pocket PCs that will 
substitute training on the Census-based Impact Oriented Methodology (CBIO). 
19
Table 6: ISA team member outputs and activities for the period FY1999-FY2002
FY1999 FY2000 FY2001 FY2002
Position
(% time)
Technical aspects related to the 
Main Program Objectives
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
Q
1
Q
2
Q
3
Q
4
O1:Tools for program design & implementation 
 Reviews/assessments/orientation
 Training/workshops 17 1 2 2 1 1 1 2 1 1 1 1 2
O3:Needs assessments in Sahel and Haiti
 Reviews/assessments/orientation
O5:FAM: M&E, monetization, capacity building
 Development of FAM tools
O6:FAM and FHI Information technology 
 Mentoring
Team 
Leader 
(90%)
General Program Management and Coordination
O1:Tools for program design & implementation
 Reviews/assessments/orientation
 Mentoring
 Training/workshops9
2 2 1 2 1 2 1 1 1 1 1
O3:Needs assessments in Sahel and Haiti
MCHN 
TA 
&Trainer
(50%)
 Reviews/assessments/orientation
O3:Needs assessments in Haiti
 Reviews/assessments/orientation
O4: Improved commodity management
 Training/workshop 1
O6:FAM and FHI Information technology 
 Systems developed
Info 
systems
TA & 
Trainer
(25%)
 Mentoring/training FAM staff 
O4: Improved commodity management
 Reviews/assessments/orientation
 Mentoring/preparation of manual/general support
CM
TA & 
Trainer
(75%) Training/workshop 1 1 2 1
O2:Emergency response and transition to dev. 
 Reviews/assessments/orientation
 Mentoring
ER
TA & 
Trainer
(30%) Training/workshop 1 1 2 1 1 2
 
17 The team leader, who is an agricultural specialist, normally presented training courses jointly with the MCHN trainer. Thus, the number of courses 
reported for the team leader and MCHN TA, are similar, because they normally taught as a team. 
20
4.3 Budgets and expenditure
The total budget for this ISA was 1,399,432 USD for the period FY1999 to FY2003. 
FFP provided approximately 71% of these funds and FHI the remainder. The FHI ISA 
financial data is audited according to A-133 regulations. 
To date, program expenditure closely matched annual budgets, with approximately 
25% of the total budget remaining for FY2003. The Graph below shows the program’s 
expenditure on the various budget categories during the past four years. In order to 
achieve their outputs, the FHI ISA spent nearly 50% of its budget on personnel and 
16% respectively on travel and procurement (procurement includes the cost of 
consultants such as MCHN TA and mid-term evaluation). Other less significant 
expenditures included: indirect costs, training, communication and supplies.
Graph 1: Categorization of program expenditures FY1999-FY2002 
Personnel
45%
Travel
16%
Training
8%
Procurement
16%
Indirect costs
9%
Communicat.
& supplies
6%
Expenditure on personnel showed a 23% increase during the four-year period under 
review. This probably reflects FHI’s efforts to bring their salaries closer to industry 
standards. Expenditure on travel varied significantly, but followed activity patterns 
closely (especially the presentation of workshops). 
21
Actual training costs were kept relatively low, partly because of contributions made by 
the Title II fields. In instances where workshops for various countries were presented 
in one place, fields had to contribute towards flight costs and in some cases 
accommodation. Even though it did not appear as if the expenditures in themselves 
were a problem, one field did mention that they do not get sufficient warning to include 
additional expenditures related to the ISA into their budgets. This problem could be 
avoided if annual training plans were developed and agreed upon with fields prior to 
the start of FHI’s annual budget process in July. Given that ISA fiscal year beginning 
in September, this should be possible. 
4.4 Modus operandi
4.4.1 General description
At the onset of the ISA, the team leader made contact with the Country Directors 
(CD’s) of Title II fields and asked for their inputs in relation to program composition. 
Thereafter they were briefed about the main objectives and contents of the program. 
Since then, new country directors have been appointed in two of the four fields 
originally targeted by the ISA. During the ISA final evaluation, the Mozambique CD 
expressed a need for a more thorough briefing/explanation about the objectives of ISA 
and other aspects of the organization. Mozambique in particular, lost several of its 
senior managers during the past two years and it appears as if this program has been 
affected by a lack of continuity and gaps in its institutional memory.
Generally, ISA team members precede the development of training materials by 
informal or formal needs assessments and situation analysis. Tools for food security 
program design and implementation were selected on the basis of its ability to meet 
specific needs within the organization, but also in terms of proven effectiveness 
elsewhere. The spirit of the program was from the beginning to expose project staff to 
a set of tools/methods that they can use as is or adapt for their own programs. One of 
the comments made in Bolivia about the ISA training was:
’The courses that we received before ISA were mostly technical and were not of 
such great benefit to us. There has been a definite move within FHI towards working 
in a more systematic and integrated manner’.
Before the onset of training, a course outline is usually sent to the CD’s for their 
comments. During the evaluation field visit some community level staff and supervisors 
said that they would like to participate more actively in the process of determining the 
contents of courses. This probably suggests that some CD’s do not involve all their 
staff members in discussions about the draft course contents.
Most of the training materials were adapted and refined during the course of 
presenting the training courses repeatedly. The materials were nearly always sent a 
few weeks in advance to especially Bolivia and Mozambique where it was translated 
into Spanish and Portuguese by FHI staff. This was mainly done to save costs, but the 
country programs (specifically Mozambique) expressed their reservations about the 
effectiveness of this method and suggested that professional translators be used in the 
future. 
22
The training materials are now ready for consolidation and the preparation of more 
formal handbooks or manuals are being planned for the next ISA. In the case of 
commodity management, an excellent and comprehensive manual18 containing 
standard procedures has already been compiled. 
The mid-term evaluation recommended that an action plan be developed during 
trainings and that materials be developed that could be used for follow-up training to 
train the staff members who were not included in the original training (FHI 2001b). This 
was done for the trainings that took place during FY2002 (barrier analysis and TIPS). 
4.4.2 Field Suggestions for future ISA programs
Community level staff and management suggested the following general 
additions/changes to the way in which ISA training courses are presented:
· A general comment across all fields was that most of the training courses 
covered too much material in too short a time period. Courses should either be 
extended or less material should be included in each course.
· In countries such as Mozambique, where replication has to be done, groups 
should be small so that trainees can receive a lot of attention and get a better 
understanding of the subject matter. 
· Most fields suggested that courses should include even more opportunities for 
practical exercises and using the techniques being taught.
· A request for the consolidation of training materials in the form of manuals was 
made in Bolivia
· Most fields requested to be provided with additional resource materials. For 
example the FANTA documentation proved to be very useful reference 
materials during the baseline survey. They would like to get similar backup 
materials, books, bibliographies or CD ROMS related to the other training 
courses that were presented by ISA. 
· Videos are a great help when trying to replicate materials and training 
sessions. Such tools will help fields to orientate new staff members quickly. 
· Field staff prefer being trained in mixed groups – health combined with 
agriculture and/or groups from different regions/countries and even 
organizations. The opportunity to share practical experiences with others, who 
work under different conditions or with slightly different target groups, increase 
motivation and enrich workshops. 
· Exchange programs that give staff the opportunity to visit other fields and see 
how some of the tools promoted by ISA are being used will help them broaden 
their experience.
· There may be some instances in which ISA should consider involving specialist 
consultants. However, fields are also aware that one of the disadvantages of 
this would be that the consultant may not know the organization and its needs 
well.
 
18 Parts of this manual are based on the CARE Commodity Manual, which was made available to the 
CS’s as part of FAM working group activities. 
23
· Bolivia and Mozambique expressed a need to appoint specific people in each 
field to be responsible for training and the replication of training. 
· The trainers need to visits fields to follow up training and provide some 
mentoring
· Management commitment is essential if training is to be translated into 
practice. One of the suggestions would be to work the implementation of these 
methods into the monitoring and evaluation plans of each DAP. 
· There is a need for a better road map or mechanism that will help identify who 
should attend the training.
· An annual training plan, which is developed jointly with the fields at the time 
when they do their annual planning, will be useful.
· The possibility of linking fields with Universities for backstopping purposes 
should also be investigated. 
Field staff suggested the following new themes for future ISA training courses:
· New ideas/systems in the area of income generation and marketing. Indicator 
development should also be included for this focus area. 
· New ideas/systems in animal husbandry support (Kenya)
· New strategies and approaches for the implementation of health/nutrition 
programs
· Management skills, team management and coordination
· Best practices for integrating nutrition, health and agriculture
· FFP Washington training on regulations/procedures related to Title II for CS’s 
and USAID missions to improve understanding and clarify appropriate exercise 
of roles and authority. 
· Environmental mitigation techniques (Kenya)
· Integrated pest management (Mozambique)
· Local capacity building and the training of counterparts at various levels
· Organizational audits are needed where one looks at resource flows, levels of 
authority, responsibility etc. 
24
CHAPTER 5: ACTIVITIES, ACHIEVEMENTS AND IMPACT
5.1 Introduction
This chapter focuses on the activities and achievements of the FHI ISA during the first 
four years of program implementation. It is structured according to the six objectives of 
the program beginning with the design and implementation of food security programs. 
This will be followed by commodity management, emergency response preparedness, 
food security needs assessments, FAM cooperating partnership and working groups 
and finally FAM knowledge of and proficiency in using information technology. For 
each objective, a brief description of activities and outputs will be followed by a 
summary of the achievements as well as an assessment of the impact of that specific 
program component on FHI’s Title II fields and food security. Where applicable, this is 
then followed by a section that lists the suggestions made by the FHI country 
programs for future ISA programs. 
5.2 Tools for the design and implementation of food security programs (Objective 1)
5.2.1 Project proposals (Objective 1)
5.2.1.1Activities and inputs
A program design and analysis workshop was held in Zimbabwe in September 1999. 
Sixteen participants (all in managerial/supervisory positions) from all four fields 
attended the training. The most important objectives of this workshop were to:
· Discuss food security problem analysis within the context of achieving the 
Vision of a Community 
· Learn some tools/techniques for problem identification
· Apply those tools to various simulated food security problems
· Learn basic steps in program design
· Review and discuss gold standard program designs
· Conduct critical analysis of Title II program designs in FHI and suggest 
improvements
5.2.1.2 Achievements and impact
Summary of key findings: 
The evaluation team found evidence to suggest that, at least in some countries, ISA 
support has lead to improved quality of project proposals. Improvements were noted 
in the areas of better problem identification/analysis, broader staff participation in 
project design, increased staff ownership of project plans, improved M&E plans. 
However, the evaluation notes with concern a lack of continuity of key managerial staff 
in some fields which threatens to erode improved institutional capacity in this area
Discussion of key findings: Upon completion of the five day course, the average 
post- test score of the participants was 85% compared to a pre-workshop test-score of 
44%. Considering the content of the above training program, one would expect 
changes in the process of compiling Title II project proposals, as well as changes in 
their content and presentation. During group interviews with FHI managerial, 
25
supervisory and field staff the team asked them to compare the process of developing 
the present DAP proposal with the previous one. The following findings were common 
to all three fields:
· The process was much more participatory and included staff at least down to 
regional supervisory level. Community groups and other stakeholders were 
consulted and included in the process to a much greater degree than before.
· Staff in general are more familiar with the contents of the DAP proposal and 
feel a greater degree of ownership than during the previous DAP. 
The evaluation team leader also studied the previous DAP project proposals of 
Mozambique, Bolivia and Kenya. These were compared with the recently submitted 
proposals in terms of the structure of the proposal, the degree to which it was based 
on thorough problem identification and analysis, and the fit between problem analysis 
and proposed activities. It was found that in all three cases, the quality of the most 
recently submitted DAP proposals were higher than the previous proposals. Of the 
three proposals, the Bolivia proposal stood out as exceptionally well written and 
presented. These observations are confirmed to some extent by the ratings given to 
the current FHI DAP proposals by the various USAID/FFP representatives that work 
with FHI. Their views are summarized in table 7.
Table 7: USAID perceptions about the most recently submitted Title II DAP proposal
FHI Bolivia FHI Mozambique FHI Kenya
PROPOSAL 
CHARACTERISTICS
FFP
Washing
-ton
USAID
Local 
Mission
FFP
Washing
-ton
USAID
Local 
Mission
FFP
Washing
-ton
USAID
Local 
Mission
Degree to which 
proposal is based on 
good problem 
identification and 
analysis
Above
Average
Average Above
Average
No 
response
Above
Average Average
Degree to which goals 
and objectives have 
been clearly written and 
defined
Above
Average
Above
Average Average
No 
response
Above
Average Average
Degree to which the 
proposed interventions 
are appropriate to the 
identified problems and 
needs
Above
Average
Above
Average Average
No 
response
Above
Average 
Below 
Average
Degree of fit between 
the objectives of the 
proposal and the 
strategic objectives of 
the USAID local mission
Above
Average Average Average
No 
response
Above
Average Average
Degree to which a 
balanced mix of impact, 
effect and output 
indicators have been 
included in the M&E 
plan
Above
Average
Above
Average Average
No 
response
Above
Average Average
 Source: Questionnaire sent to Food Aid Officers (FFP Washington) responsible for the three
 countries and representatives of the USAID missions in Kenya, Mozambique and Bolivia. 
26
It is heartening to see that so many staff members have been involved in the process 
of preparing the DAP proposal and that the general knowledge and experience base 
has increased within the organization. However, it is important to note that less than 
35% of the people who attended this training were still working in the FHI Field offices
at the time of the ISA final evaluation. In the case of Mozambique and Kenya, none of 
the managerial or supervisory staff, who were interviewed during the evaluation, 
attended this specific course. The possible negative consequences that this may have 
on the next cycle of proposal writing may be mitigated by the fact that many of the 
current managerial staff participated actively in the preparation of these proposals. 
Furthermore, the general capacity of FHUS/International to support proposal writing 
activities has increased significantly, partly because some of the staff who used to be 
in managerial positions in the field, have been promoted. 
In Bolivia, the problem analysis concepts that formed part of this course, are used 
extensively in communities to identify strengths, weaknesses, opportunities and 
threats.
5.2.1.3 Suggestions for future ISA programs
The following suggestions related to report and proposal writing were given by some of 
the country programs: 
· Some fields expressed a need for more training and mentoring in report writing. 
Even though they admit that some progress have been made, they feel that 
there is a need for more growth in this regard
· Additional training that focuses on proposal writing in general, providing 
practical examples and different formats for different donors will be helpful. 
5.2.2 Educational messages (Objective 1)
5.2.2.1 Activities and inputs
 The main activity related to educational messages was the Educational messages and 
methods workshop. The aim of the workshop was to improve the quality of educational 
messages, make them more appropriate in order to affect behavior change and to 
improve the way in which educational messages are being delivered.
This training course was presented during FY 2000 in all four fields:
 Kenya: April 2000, 25 trainees
 Mozambique: June 2000, 20 trainees
 Bolivia: August 2000, 26 trainees
 Ethiopia: September 2000, 23 trainees
The original workshop in Mozambique was cancelled due to flooding. Once the crises 
passed, the course was presented by two FHI Mozambique staff members. Before the 
educational messages and methods training course was developed, a field 
assessment was done to review educational messages and their relationship to key 
behavioral changes being promoted. This assessment covered both the health and 
agricultural sectors in all four Title II fields. Its purpose was to uncover key factors that 
would lead to the improvement of the capacity of FHI Title II program educators and 
27
trainers to successfully promote positive behavior change of Title II program 
participants. 
5.2.2.2 Achievements and impact
Summary of key findings:
The evaluation team found that all fields have developed training curricula and lesson 
plans as a result of the ISA assessment and training. In most fields, training has 
become more participatory and the use of visual aids and other techniques to facilitate 
learning amongst non-literates has increased. This is probably the one area in which 
ISA has already impacted on food security in its Title II fields. 
Discussion of key findings:
Average workshop pre-test scores were 39%. This increased to 70% after the 
workshop was completed. The evaluation team found a lot of evidence that several 
significant changes took place in all three Title II fields after the workshop was 
conducted. 
 
Table 8: Educational messages and message delivery: a comparison of the situation before 
and after the workshop 
Bolivia Mozambique Kenya
Educational aspect 1999 2002 1999 2002 1999 2002
Agriculture
Extension curriculum in place No Yes No Yes No Yes
Holistic messages in place Few Yes No Some No Yes
Standardized lesson plans in 
place Few Yes Few Yes No Yes
Use of non-formal training 
techniques No Some No Some No Yes
Use of visual aids No No No Some No Some
Review of existing knowledge; 
evidence of flexibility and 
message renewal
No Some No Some No Some
Health
Health curriculum in place No Yes No Yes No Yes
Holistic messages in place Some Yes No Some No Yes
Standardized lesson plans in 
place No Yes Yes Yes No Yes
Use of non-formal training 
techniques Some Yes Yes Yes Yes Yes
Use of visual aids Some Yes Yes Yes Yes Yes
Review of existing knowledge; 
evidence of flexibility and 
delivery of new/appropriate 
messages
No Some No Some No Some
28
Table 8 summarizes the present situation in so far as educational messages and 
message delivery are concerned (Field visits ISA Final evaluation 2002). This is 
compared with the findings of the assessment of educational messages that was done 
prior to the development of the training course (FHI 1999c). The table shows that most 
fields made significant progress in developing training curricula, lesson plans and 
using non-formal educational techniques and visual aids. Countries who had lesson 
plans before the training also improved significantly on them. 
There is room for improvement in the use of visual aids and non-formal educational 
methods in the agricultural sector in Bolivia and Mozambique. In all three fields, but a 
little more so in Mozambique, attention should be given to more message innovation. 
The use of tools such as barrier analysis and TIPS may be very useful in this process. 
Mozambique also needs to do some more work in the area of holistic message 
development for both health and agriculture. The health program in Bolivia has made 
considerable progress in introducing non-formal training techniques and visual aids in 
their trainings. Health Curricula reviewed in all three countries were very good. 
Although significant improvement has been made, there is room for innovation in 
curricula in both health and agricultural sectors. Agriculture also needs to work a little 
harder on using non-formal education techniques and developing appropriate visual 
aids.
 
Table 9: Comparative ranking19 of community level trainings attended by the evaluation team
Table 9 shows the comparative ranking allocated by the evaluation team to the training 
sessions they attended during the evaluation. In general, health trainings were good 
and similar in all three fields. In the case of agriculture, the team felt that Kenya had an 
exceptionally good agricultural training program. Mozambique and Bolivia had similar 
 
19 The comparative ranking was done independently by the three evaluators. Each evaluator attended 
one health and one agricultural training session per country. A total of six training sessions were 
evaluated per country. Ranking was based on a scale of 1-3; with 1 being the worst and 3 the best. 
Similar ranks could be allocated to two different countries for the same aspect. The maximum score 
that any country could achieve in any category was 9. The maximum total score that any country could 
achieve was 36.
COUNTRY
Degree of 
involvement, 
interest and 
participation 
of audience
Appropria￾teness of 
message 
i.t.o. farmer/
mother 
character￾istics
Use of 
visual aids 
and demos
Use of 
holistic 
messages
TOTAL
HEALTH
Bolivia 5 6 8 6 25
Mozambique 7 8 8 5 28
Kenya 7 9 4 7 27
AGRICULTURE
Bolivia 6 5 4 5 20
Mozambique 3 7 5 4 19
Kenya 9 9 9 8 35
29
overall scores. The low degree of involvement and interest of the farmers in 
Mozambique is probably largely the result of a lack of message renewal and to some 
extent the limited use of visual aids and demos. 
In terms of staff capacity for the new DAP cycle, the evaluation team found that 84% of 
the staff in Bolivia said that they received training in educational message delivery, 
whilst all staff in Mozambique and Kenya have undergone training. All of the Bolivian 
staff, who have not been trained are non-supervisory staff. Most have been working for 
FHI for less than one year and are working for the health program.
Across all fields, the tools most widely used by non-supervisory staff are:
 Lesson plans: 86%
 Stories: 78%
Development related bible studies: 68%
Cultural proverbs: 64%
Songs: 54%
 
Supervision and guidance of community-based staff in terms of community based 
training is the strongest in Mozambique, where all staff reported having had a visit from 
their supervisor while training community members. In Kenya 82% received such a 
visit and in Bolivia only 40%. In the areas of giving guidance with lesson plans and 
developing stories and ideas to be used in training, Mozambique once again came out 
tops at 67%, followed by Kenya 27% and Bolivia (20% for guidance with lesson plans 
and 7% in developing stories and ideas for training). 
It is worthwhile to note that the supervisors of both the health and agricultural sectors 
in Mozambique call meetings of all their staff every two weeks. During these meetings 
the current situation in the communities is discussed and community education plans 
for the coming two weeks are developed. 
5.2.3 Positive Deviance (Objective 1)
5.2.3.1 Activities and inputs
Positive deviance workshops were scheduled and held during FY2001. A total of 100 
participants took part in the workshops that were held in Ethiopia, Kenya, Bolivia and 
Mozambique. The objectives of these workshops were to (FHI2001a:4):
· Introduce Title II staff to the concept of Positive Deviance and its use in food 
security programming 
· Train staff to conduct a positive deviance study in one or more communities 
where FHI conducts food securities activities
· Train staff to organize and conduct nutritional rehabilitation workshop for 
malnourished children using information gained during the positive deviance 
study
· Work with agriculture staff to collectively brainstorm possible methods for 
conducting agricultural rehabilitation workshops and plan extension visits for 
negative deviant farmers using information gained during positive deviance 
study
30
Participants from World Vision Peru and Bolivia and Save the Children Bolivia also 
participated in the training. Participants from World Relief participated in the PD 
workshop in Mozambique. According to the trainer the participation of people from 
these organizations significantly enriched the workshop discussions and practicum 
activities (FHI2001a:4). 
5.2.3.2 Achievements and impact
Summary of key findings:
In the case of Positive Deviance, the activity impact would have been an improved 
capacity of FHI field staff to successfully use positive deviance in nutrition and 
agriculture leading to less malnutrition and increased technology adoption on the part 
of the beneficiaries. The evaluation team found that Mozambique is the only program 
that is implementing the Hearth Method. Kenya and Bolivia plan to implement it in 
2003. 
Discussion of findings:
Mean post-test scores for this course was 82%, which exceeded the target of 80%. 
Even though agricultural staff was included in the training on an experimental basis, it 
was found that positive deviance will need substantial adjustments if it has to be 
applied to agriculture. This discussion will therefore largely focus on the perceptions 
and knowledge of the health staff about positive deviance.
During the evaluation it was found that 86% of the health staff in Bolivia and all the 
Mozambican and Kenyan staff have heard about Positive deviance. Of the health staff 
in Bolivia, 20% could identify the correct definition for Positive Deviance. In 
Mozambique, 78% of the respondents marked the correct option and in Kenya 83%. 
In so far as the technique impacted on the work of the field programs and per 
implication food security, Mozambique is the only country where we could find 
evidence that Positive Deviance is being implemented. The health manager is 
enthusiastic about it and they have received significant mentoring from the MCHN TA 
who is assisting them with its implementation. Reports on mentoring visits and 
interviews with the health department’s manager and staff indicate that staff at all 
levels have bought into the method. It is also being adjusted and modified according to 
the needs and characteristics of target communities. According to the health manager 
they have been able to follow through with the implementation of the Hearth method 
because they developed an action plan after training. The action plan attached specific 
responsibilities to specific people. 
In Kenya, all staff attended the ISA training course and awareness about the method is 
high. To date, positive deviance has not been extensively used in Kenya, although 
FHI/K Title II plans for 2003 outline its use. The FHI Food Security Team will want to 
monitor this situation, as some FHI/K health staff have expressed the opinion that 
Positive Deviance may create community conflict within their cultural context.
Some of the health staff in Bolivia are enthusiastic about the potential of Positive 
Deviance in their communities, however, to date it has not been used widely. New staff 
members have not received training in the method. According to the health manager, 
all the materials needed have been prepared and FHI/B plans to orient all staff and 
begin implementation in January 2003. Their policy is to introduce new concepts 
31
gradually, rather than give training in all tools at once. Another factor that may have 
influenced implementation may have been the fact that FHI/B spent the first nine 
months of their new DAP with an acting CD, before the current CD was appointed. 
5.2.4 Barrier analysis/TIPS (Objective 1)
5.2.4.1 Activities and inputs
Barrier Analysis is designed to help community workers discover and overcome 
obstacles to food security-related behavior change. Between May and July 2002 
barrier analysis and TIPS workshops were held in Kenya, Mozambique and Bolivia. A 
total of 76 staff members attended the training. 
Several of the recommendations of the mid-term evaluation (FHI 2001b) were 
implemented during this course. The most important being:
· To conduct a training needs assessment for FY2003
· In Mozambique, a training of trainers was conducted – this was done because 
there are so many field staff and also because some ISA trainers are not fluent 
in Portuguese.
· Development of in-country action plans after the training
· Staff who attended the training were requested to conduct a second 
generation training 
5.2.4.2 Achievements and impact
Summary of key findings:
The evaluation team found that the use of Barrier analysis and TIPS was very limited 
in all three fields and the main reason for this may be that training was done quite 
recently. Replication has taken place in Mozambique, but it has not been used widely 
and could therefore not have had an impact on food security programs. The replication 
training may also not have been of high quality as less than 60% of the staff who knew 
about these methods could identify their correct definitions. The lack of replication in 
Bolivia probably reflects a lack of management commitment to the implementation plan 
that was developed after completing the training course. 
Discussion of findings:
The activity impact will be an improved capacity of FHI field staff to successfully use 
this method to help Title II program beneficiaries to overcome their obstacles to 
behavior change. It is hoped that this turn will have a positive impact on their food 
security.
The table on the next page summarizes the findings of the questionnaire survey that 
was conducted amongst community based staff and their supervisors. The table 
confirmed the findings of the group discussions in so far as replication of the training is 
concerned. It is important to note that even though Mozambique had a very high 
awareness of these concepts, a large percentage of them could not identify the most 
appropriate definition, possibly putting the quality of the replications that were done 
into question. 
32
Table 10: Knowledge about and use of Barrier Analysis and TIPs
Variable Bolivia Mozambique Kenya
% of staff who heard of 
barrier analysis
63 100 100
% of those who heard of 
barrier analysis who could 
identify the definition 
correctly
73 59 94
% of staff who heard of 
TIPS
47 100 100
% of those who heard of 
TIPS who could identify the 
definition correctly
78 22 63
This training course was presented six months before the final evaluation, thus it is 
unrealistic to expect an impact on food security at this stage. Field visits confirmed the 
following:
· All fields made action plans. However, the only country where the action plan 
was fully implemented was Mozambique.
· Bolivia and Mozambique prepared training materials that can be used for 
follow-up training
· Mozambique conducted its follow-up training in October. This training 
condensed the original four-day ISA training into two days. Bolivia Title II 
funded programs have not yet done any follow-up training. Even though they 
say that they did not have time, they are in exactly the same cycle as 
Mozambique. In that sense both countries had to hire new staff and conduct 
their baseline studies during FY2002. It is worth noting however that FHI 
Bolivia’s Child Development Program, that sent staff to participate in the ISA 
training, has already replicated the barrier analysis training and has plans to 
replicate TIPS training in 2003. The Kenya program temporarily suspended 
operations shortly after the ISA training and hence has not yet replicated the 
barrier analysis or TIPS training. However, all of the Kenya Title II staff were 
included in the ISA training, thus reducing the need for further replication of 
that training within FHI Kenya.
· Kenya has just restarted with their new DAP cycle and may soon start 
implementing barrier analysis and TIPS. One agricultural supervisor in 
Mozambique reported having used TIPS in six communities in his area. The 
health section in Mozambique used barrier analysis to identify barriers that 
prevent mothers from attending their HEARTH workshops. A full-scale barrier 
analysis was done in Bolivia in two communities as a follow-up of the findings 
of the baseline survey. The agricultural sector used barrier analysis in one 
province to determine the factors that prevent women from accepting 
leadership positions. The Child Development Program in Bolivia (non-Title II 
funding) has been using Barrier Analysis extensively. They are planning to 
start implementing TIPS in 2003.
33
5.2.5 Monitoring and Evaluation (Objective 1)
5.2.5.1 Activities and inputs
The ISG20 that preceded the ISA was mainly aimed at establishing a robust monitoring 
and evaluation system in support of FHI’s Title II programs. Under this program, 394 
Title II staff received training in M&E methodologies and tools. Building on these 
workshops, the current ISA team conducted a review to evaluate food security staff’s 
post ISG knowledge and practice of various M&E methods and tools. The team also 
reviewed recent field monitoring and KPC reports. Four remedial M&E workshops 
(including a lot of practical work) were held during FY1999 and a total of 81 staff 
members were trained. 
This workshop focused on the following areas:
· Review of food security definitions
· Development and use of factor analysis tool
· Development of a good system to monitoring annual progress towards the 
achievement of impact indicators
· Use of focus groups to fill in baseline data gaps
· Development and use of Quality Improvement Checklists
· Indicator Development and Operationalization
5.2.5.2 Achievements and impact
Summary of key findings:
Awareness and use of Quality improvement checklists is high in Mozambique and 
Kenya. This has probably contributed to improved performance by community-based 
workers and as such had an impact on improved food security in the target 
communities. Monitoring and evaluation capacity was strengthened, but losses of key 
management staff during the past two years has left gaps in Mozambique and Kenya 
for their new DAPs. 
Discussion of findings:
Considerable progress has been made within FHI in regards to monitoring and 
evaluation. In Bolivia (Interview with USAID 2002), FHI has been used as the role 
model in a recent effort to develop common indicators and a common baseline survey 
methodology amongst PVO’s. Most of the FHI managers interviewed felt confident 
about their combined ability to develop and manage their monitoring and evaluation 
system. In the past they used to hire consultants and now they can largely do it on 
their own. During the FY2002 baseline study, FHI staff used the focus group interview 
techniques that they learned during ISA M&E training courses. 
The M&E system in Mozambique has likewise been robust and well implemented 
during the previous DAP. This is evident from the final evaluation report of its FY1997 
to FY2001 DAP (FHI 2001d). However, during the ISA final evaluation team visit, 
USAID expressed concern about M&E in FHI. Several managers and the M&E officer 
left the organization at approximately the same time. The position of M&E officer was 
vacant for more than a year before it was filled. According to USAID, they expected 
 
20 The ISG covered the period January 1997 to August 1998.
34
FHI to play a leadership role in the process of developing common indicators, but 
instead noted that FHI staff contributed little to the discussions. According to USAID, 
there is a problem with continuity within FHI Mozambique. For example the FHI person 
trained on INCPROX has left and now USAID has to repeat the training with another 
staff member. 
In Kenya the situation is more complex. A robust system for collecting monitoring 
information was established in the program. In general, good baseline studies were 
conducted. However, before the mid-term evaluation was due, both the agricultural 
and health managers left the program. The staff who replaced them as managers did 
not have all the M&E training that their predecessors received. Although new staff 
continued with routine information collection, some gaps developed in the 
implementation of the monitoring and evaluation plan of the DAP (FHI 2002b). These 
problems were already evident in the annual reporting from FY2000 onwards, but the 
situation was not rectified until it was time for the final evaluation. One of the reasons 
why this situation developed, is the fact that there were three different country directors 
during the previous DAP. A lack of continuity in this respect led to a lack of the 
necessary backup for the new health and agricultural managers. 
Another aspect that was covered during this training was the monitoring of work quality 
through quality improvement checklists. During the field visits groups of health and 
agricultural community based staff and their supervisors completed questionnaires 
aimed at establishing the extent of use of some of the tools introduced during ISA 
trainings. The table below summarizes the finding for Quality Improvement checklists.
Table 11: Use of Quality Improvement Checklists (QICL) at regional and community 
level
 % of supervisors and community staff
Variable Bolivia Mozambique Kenya
% of staff who knows about 
QICL 79 98 94
Of those who know about the method:
% who could select the right 
definition for QICL
60 67 94
% whose supervisor has used 
the checklists with them
60 87 81
% whose supervisor explained 
what is expected of them while 
using the checklists
60 66 67
The above data shows that QICL are known to most staff in Mozambique and Kenya. 
Kenya was the only country where a significant percentage of the staff, who knows 
about QICL, could select the exact definition21 (94%). The use of QICL is also more 
widespread in Mozambique and Kenya than in Bolivia. More than 80% in the latter two 
countries reported that their supervisor has used it with them, whilst only 60% in 
Bolivia could say the same. Even though Bolivia has a lot of new staff, who started at 
 
21 The definitions used in the questionnaire was phrased to also capture the spirit of a particular tool. 
Other options were also supplied that were similar, but definition recognition was only possible if you 
were really familiar with the whole concept. 
35
the beginning of FY2002, this does not provide an explanation for the low recognition 
and usage rates of QICL as the staffing situation is very similar in Mozambique. 
Amongst the staff who completed the questionnaire, 47% of the Bolivian and 46% of 
the Mozambican staff reported having worked for FHI for less than two years. 
5.2.5.3 Suggestions for future ISA programs
· Bolivia expressed a need for training in more detailed data analysis. They have 
a lot of baseline survey data that could help them a lot if they knew how to 
analyze it in more detail.
· Both Kenya and Mozambique expressed a need to learn how to use the 
software used to analyze monitoring and evaluation data i.e. Epi Info and 
SPSS. 
· Continued support to establish FHI Kenya capacity to establish and implement 
a robust monitoring and evaluation system.
· USAID in Bolivia and Mozambique were also concerned about M&E. Bolivia 
specifically mentioned developing systems that link monitoring information 
more closely to program modifications – thus moving monitoring systems 
closer to the realm of management information systems. The main concern in 
Mozambique was continuity as a result of staff changes.
5.3 Commodity management (Objective 4)
5.3.1 Activities and inputs
The commodity management (CM) component of the ISA was mainly aimed at helping 
FHI to improve its institutional capacity to manage commodities effectively. The main 
objectives of this activity was to:
· Improve and standardize FHI’s Title II commodity management system based 
on best practice models
· Train and provide technical assistance to FHI Title II staff in all aspects of the 
standardized procedures
· Improve the capacity of FHI headquarters commodity management staff to 
successfully move towards best-practice commodity management
During FY 1999 an assessment was made of commodity management systems and 
capacity in Ethiopia and Bolivia (FHI 1999:22). The methods, procedures and tools 
used by the CS’s and FFP were also reviewed. On the basis of this, the ISA 
Commodity management TA developed workshop materials and an FHI Commodity 
management manual. She sourced materials from the CARE manual and also used 
aspects of the FAM monetization manual. This manual was translated into Spanish 
and was distributed to the Title II fields during FY2002. The commodity management 
training materials were subdivided into three parts and a total of seven workshops 
were held between June 2000 and July 2002. An average of 16 people were trained 
per workshop. Even though most of the workshop attendees were from Bolivia and 
Ethiopia, staff from Mozambique and Kenya also participated. A considerable amount 
of time also went into mentoring and acting as a link between field offices and FFP in 
Washington in so far as resource requests and other aspects related to monetization is 
concerned.
36
FAM and or FFP have been conducting on-going workshops to keep CS’s informed of 
new developments in the area of commodity management and transportation issues. 
The ISA CM TA attended most of these meetings throughout the four years of project 
implementation. 
5.3.2 Achievements and impact
Summary of key findings:
In general, the team found evidence of the standardization and streamlining of 
commodity management systems in Bolivia and Ethiopia and increased knowledge at 
FHI’s headquarter level in respect to all aspects of commodity management. 
The management of warehouses and commodities was good in Bolivia even before 
the ISA. However, the ISA mentoring and training support led to further refinement and 
improvements in their systems. Ethiopia benefited significantly from the ISA 
Commodity Management training, especially in the areas of warehouse management 
and loss control. 
Discussion of findings:
During the final evaluation, the field visit to Bolivia was the main source of information 
of the impact of the ISA CM component on Commodity Management within FHI Title II 
fields. The evaluation team found many positive aspects about commodity 
management in Bolivia. This perception was confirmed by USAID Bolivia who rated 
them as slightly more advanced than the other CS’s in this respect (USAID group 
interview 2002). However, it is difficult to attribute these strengths to ISA, as many 
were there even before the ISA was implemented. The most important of these 
strengths are:
· Warehouses are well managed and maintained; computerized inventory 
systems contribute to greater efficiency and better quality information 
· They had a commodity management procedures manual even before the ISA. 
This manual was used as a tool to develop the ISA manual
· FHI/Bolivia hired consultants in 1998 to develop a computerized inventory 
system. USAID Bolivia was so impressed with the system that they bought the 
rights to the program and encouraged other CS’s to also use it. 
The evaluation team reviewed the commodity management training materials and 
found them to be comprehensive, well researched and well written. Staff in Bolivia 
(final evaluation group interview 2002) and Ethiopia (mid-term evaluation FHI 
2001b:32) were generally very positive about the quality of the commodity 
management workshops. Comments made by staff in Ethiopia and Bolivia were that 
the training was participatory and well organized. The average commodity 
management workshop’s post-test score was 85%. The FHI commodity management 
manual is well written and it is evident that a lot of time and effort was spent on 
researching and finding the most appropriate materials to include. The draft manual 
was circulated extensively before finalization and the field offices in Ethiopia and 
Bolivia contributed towards its final version. The manual was described as useful by 
the Bolivia and Ethiopia staff and as such it succeeded in standardizing procedures in 
the two fields where FHI actively manages Title II commodities. A measure of its 
appropriateness and success is the fact that other CS’s have been requesting copies 
of the FHI Bolivia ISA commodity management manual. 
37
Staff in Mozambique and Kenya also received copies of these manuals, but felt that it 
is of limited use to them as their program resources are 100% from monetization 
proceeds22. One specific addition to the work of ISA in terms of the workshops and 
manual, might have been a section on warehousing that is not just aimed at the needs 
of Ethiopia and Bolivia, but also Mozambique and Kenya. The latter two deal with 
commodity management within an emergency context. Within the present ISA period, 
both Mozambique and Kenya had to deal with disasters – in the case of Mozambique 
serious flooding and in Kenya droughts. The Kenya program and to some extent 
Mozambique, experienced problems with commodity losses as a result of faulty 
storage and loading procedures. Kenya eventually managed to improve their systems 
with the help of a World Food Program (WFP) training course. They also largely 
adopted WFP systems for commodity management in their warehouse. 
Staff in Bolivia and Ethiopia23 felt that the ISA workshops and manual had the following 
benefits:
· Improved knowledge of the whole system and chain related to Title II 
commodities
· Improved knowledge and use of commodity survey reports, improved 
procedures for the inspection and off-loading of goods and better preparation 
of call forwards
· Increased cooperation between staff and improved motivation to continue to 
improve commodity management procedures24. The evaluation team found a 
very strong spirit of wanting to reduce losses even more and wanting to perfect 
systems as much as possible. 
· Ethiopia reported significant improvements in warehouse management and 
procedures. Storage inspection lists were introduced and deemed of great 
benefit. 
· Bolivia has used the video about Food Aid25 extensively in their communities. 
It has helped significantly to dispel negative perceptions about food donations. 
Other CS’s have asked them for copies of this video.
· The implementation of the ‘delivery survey’, with surveyors present in the 
warehouses, resulted in a reduction of losses. USAID Bolivia was so 
impressed with this that they requested all he other PVO’s to also use this 
system. 
One of the commodity management challenges faced by Title II fields, is frequent 
change in administrative regulations. The ISA CM TA was also responsible for keeping 
 
22 FHI in Kenya and Mozambique are not directly involved in monetization and the distribution of Title II 
commodities. Lead agencies (CRS in Kenya and WV in Mozambique) are responsible for all these 
activities and then transfer the money into the other CS’s accounts. 
23 Even though Ethiopia was not formally included in the final evaluation, it was considered important to 
include them in the section on commodity management as Bolivia and Ethiopia were the two main fields 
that benefited from this intervention. References to Ethiopia in this section therefore refer to the findings 
of the mid-term evaluation, as well as to personal communications with Thomas Stocker, the present 
Country Director of Ethiopia. 
24 In Bolivia the evaluation team sensed a very strong spirit of wanting to perfect systems even more. 
One example of this is losses. Even though commodity losses is less than 1% at the moment, the 
Bolivia team wants to look at ways of helping to reduce losses between the port of embarkation and the 
destination port. 
25 The CM TA gave FHI/B a video aimed at rural communities that explains the motivation for and the 
process of providing Food Aid. 
38
the fields informed about these changes. An additional benefit of this position has 
been improved CM related communication with USAID (FHI 2001b:33). In Bolivia, CM 
staff mentioned that they regularly contact ISA’s commodity management TA for 
information and assistance (Group interview with commodity management staff 2002). 
In July 2001, FHI was one of four PVO’s selected by USAID to do their own surveys as 
it was felt that they have sufficient capacity to do so (Memo, USAID 2001). All other 
documentation and information e.g. commodity status reports and internal audit 
reports, show that the capacity of FHI CM headquarter staff to respond to the 
requirements of USAID and the needs of the field offices has increased considerably 
during the past four years. According to Sylvia Greeves of FFP Washington, FHI’s 
performance has improved during the past four years. She regards the organization’s 
ability to meet USAID commodity management requirements as above average. As 
can be seen in the table below, her observations are to some extent confirmed by her 
colleagues, who have noted slight improvements in certain aspects of FHI’s CM. 
Table 12: External assessments26 of changes in monetization and commodity 
management
Factor USAID 
Bolivia
USAID 
Kenya
FFP 
Washing￾ton:
Kenya
FFP 
Washing￾ton:
Mozam￾bique
Changes during the 
past four years to 
making timely and 
appropriate resource 
requests
Improved 
slightly
Improved 
slightly
Lack 
sufficient 
time in 
position to 
judge 
Improved 
slightly
Present ability to make 
timely and appropriate 
resource requests
Average Average Above 
average
Average
Changes in FHI’s 
ability to overcome 
short-term cash flow 
problems
Not 
applicable
Improved 
slightly
Lack 
sufficient 
time in 
position to 
judge
Lack 
sufficient 
time in 
position to 
judge
FHI’s ability to deal 
with short term cash 
flow problems
Not 
applicable
Above 
average
Average Average
FHI’s ability to 
maintain approved 
commodity levels
Average Average Average Average
Changes during the 
past four years in 
regards to expenditure 
and commodity reports
Improved a 
lot
No change
Lack 
sufficient 
time in 
position to 
judge
Improved 
slightly
Ability to supply quality 
and appropriate 
information in its 
CSR4 reports
Above 
average
Average Average Average
 
26 USAID Mozambique did not complete a questionnaire, whilst the FFP representative for Bolivia has 
not been in his present position for long enough to judge. 
39
The table on the previous page summarizes the assessments of USAID local missions 
and field officers responsible for the country programs included in the evaluation. 
Slight improvements have been noted during the past four years in FHI’s ability to 
make timely and appropriate resource requests and its expenditure and commodity 
reports. However, despite these improvements, FHI’s performance is being judged as 
average in most instances covered by the questionnaire27
. 
5.3.3 Suggestions for future ISA programs
· The FHI field staff expressed a need for continued backstopping in the area of 
commodity management. This discipline has so many variables and there are 
so many different things that can go wrong that the CM teams are faced by 
new situations all the time. In addition to this, FFP and US Government 
regulations change from time to time and it is nearly impossible for the field 
offices to remain informed or abreast of these changes without permanent 
support in Washington. 
· In Bolivia USAID wants the CS’s to accept more responsibilities, such as for 
example contracting surveyors. This implies new and more responsibilities and 
more administrative work. They are busy developing new systems and finding 
appropriate ways of dealing with the new challenges and would like support 
from the ISA team.
· FHI Bolivia would like to observe what happens between the port of departure 
and the receiving port. They felt that they can make a contribution towards 
improving systems and reducing losses that are incurred before commodities 
reach their destinations. 
5.4 Emergency response preparedness (Objective 2)
5.4.1 Activities and inputs
A total of eight training workshops, related to emergency relief, were presented during 
the first four years of the ISA program. Field staff members from Ethiopia, Kenya and 
Mozambique were trained. Less than 20% of the managers trained during the first 
three courses are still working in FHI.
Staff who attended the emergency preparedness training, ranked it between very good 
and excellent. One of the strengths of the training as seen from the Kenyan staff’s 
viewpoint was that the trainer was from FHI, knew the organization well and could 
share practical experiences, based on her work elsewhere in Africa. The evaluation 
team looked at the training materials and felt that it was comprehensive, concise and 
professionally prepared. English, Spanish or Portuguese versions were made available 
to the training participants. 
 
27 At this stage it is important to note that the kind of assessment used in this evaluation is dependent 
on a strong institutional memory. In the case of FFP in Washington, frequent staff changes make it 
difficult to obtain judgments that relate to changes over time. This problem is less pronounced at the 
USAID mission level as the questionnaires were generally completed by a group of people. 
40
The table below summarizes the training courses that were presented as part of the 
ISA emergency preparedness component. 
Table 13: Training courses presented as part of the emergency preparedness component 
Name of Training 
Course
Dates Presen￾ters
Participating
Countries
# of 
trainees
# of Ma￾nager
trainees
% Mana￾gers still 
in FHI
Emergency Relief 
Preparedness 
Workshop 
Aug. 1999 Walsh Kenya
Ethiopia
Mozambique
9 9 22
Rapid disaster 
assessment 
workshop
April 2000
June 2000
Walsh
Fitzpat.
Ethiopia
Kenya
Mozambique
15 10 20
Emergency Program 
Design and 
Implementation 
Workshop
Aug.2000 Fitzpat. Kenya 9 9 11
Improving capacity to 
respond to 
emergencies and 
rapid transition to 
development
April 2002
Aug. 2002
Fitzpat.
Ethiopia
Kenya
Mozambique
DRC
Rwanda
35 Unknown Unknown
Emergency Program 
M&E Workshop
Aug.2002 Fitzpat. Kenya
Mozambique
14 6 100
 
5.4.2 Achievements and impact
Summary of key findings:
There is evidence that some aspects of the training have been useful and have had a 
positive impact on FHI relief programs in the DRC and relief efforts in Kosovo, 
Nicaragua and India. There is also some evidence that Kenya (in Meru) and 
Mozambique (in Malawi) have used the ISA training knowledge, albeit in a modified 
form, in their most recent response to emergencies. 
Discussion of findings:
The disaster relief training was mainly aimed at Africa and had two planned effects. 
These were to: 
· Increase capacity to respond efficiently and effectively to emergencies and 
transitional situations
· Successfully monitor and evaluate the results of FHI’s assistance which will lead 
to more lives saved and an increase in the number of regions that are equipped 
to transition to development activities. 
An analysis of the kind of staff trained in emergency response and whether they are 
still in the organization, shows that staff turnover is one of the factors that seriously 
affects the impact of this program component. An assessment of the actual impact of 
the ISA training on emergency preparedness within FHI also needs to take the specific 
characteristics of each field into consideration. Mozambique does not have full-time 
41
staff members devoted to emergencies, as their whole program and its resources is 
geared towards development. They tend to believe in training various staff members, 
who would normally be working in the areas of health or agriculture, in emergency 
preparedness. If there is a sudden disaster, such as for example a flood, these staff 
members stop with their normal activities and work jointly with an external consultant 
who is hired for the duration of the emergency. However, as a result of recent staff 
losses, only two ISA trained staff members remain (Interviews with management 
2002). 
The last emergency experienced in Mozambique was the floods of 2000/2001. The 
second ISA workshop was presented during that time. The Rapid Disaster 
Assessment (RDA) tools that they developed during the workshop were not applied to 
their flood response as the Mozambican Government preferred that NGO’s use their 
own and the UN’s tools (Mid-term evaluation 2001:30). At the time of the mid-term 
evaluation, Mozambique had a Disaster Preparedness Response Plan (DPRP), but 
this was aimed at slow onset disasters such as droughts and inappropriate for the 
2000/2001 floods. A more appropriate plan was developed with the help of Merry 
Fitzpatrick in 2001. The logistics director28 felt during the mid-term evaluation that he 
was not given the ‘time nor supervision’ to enable him to implement what he learnt at 
the emergency preparedness response workshop (Mid-term evaluation 2001:30). 
Since then most of the senior management of FHI Mozambique has left the 
organization and it appears as if there are some gaps in terms of institutional memory 
and access to documents such as the DPRP. During the ISA final evaluation 
Mozambique was busy with a small seeds and tools distribution in Malawi as part of a 
drought relief effort. According to the coordinator of this effort, they have used a 
modified form of the RDA promoted by ISA to assess the situation before they went 
there. During the implementation of their disaster relief plan, they used most of the 
programmatic aspects promoted by ISA, but in a slightly modified form. 
In the case of Ethiopia29, emergency response is institutionalized, mainly because of 
endemic droughts. They have a full-time Relief Project Manager and are very 
experienced in handling emergencies/disasters. Their familiarity with emergencies and 
the procedures they developed, is at such a level that some team members felt that 
the ISA training may have been too basic for them. The DPRP that was developed as 
a result of the ISA training was written too late for their drought response of 2000, but 
they have used it to develop new proposals for disaster relief programs. They also 
used their RDA tools and knowledge to do three post workshop assessments (Mid￾term evaluation 2001:30-31). 
Kenya has an altogether different situation. Even though the Title II program area is 
also hit by periodic droughts (like Ethiopia), the geographical area covered by the 
program is relatively small. Like Mozambique, FHI Kenya does not have a full-time 
emergency response team. However, the Marsabit Financial and Administrative 
Manager has received some training and knows that in the event of a disaster he will 
 
28 At the time of the final evaluation the logistics director was busy in Malawi with a Drought Relief 
Emergency response and was not available for the interviews. 
29 Even though Ethiopia has been excluded from the field visits as they will not continue with Title II 
programs, they are included in the discussion on Emergency Response as this program component 
was targeted at the organization in general and not just Title II fields. Thus in order to comprehensively 
evaluate the impact of this ISA program component, one needs to also look at Ethiopia. 
42
become responsible for coordinating relief activities. Several other national level staff 
members have been trained in all aspects of emergency preparedness. They can for 
example be mobilized to write proposals and do other support tasks at short notice. 
FHI has trained community members in the DAP target area to work as food monitors 
and according to the Marsabit Finance and Admin manager they can be mobilized 
within one week to start with food distributions. More recently, knowledge that they 
gained through the ISA training was used to assess the situation in Meru after a 
landslide took place. The Vice Chancellor of the Kenya Methodist University asked FHI 
Kenya to train 50 pastors in the Meru area on emergency assessment methods. This 
training was scheduled for the last week of November 2002. According to the trainer 
he was basing his training largely on the training and training materials that he 
received from ISA (Interview with management 2002). 
Table 14: Emergency responses by FHI in the four years preceding ISA and the four years 
after ISA*
Value of cash component 
Country Donors Nature of in USD
intervention
Transition into 
development
1994-1998 1998-2002
Kenya WFP, DFID,
SIDA, FH
Commodity and food 
distribution; drought 
relief
Run concurrently 
with development 
programs
1,938,100 1,194,378
Mozambique EEC, ODA, WFP, 
FH, OFDA, FH
OFDA,DFID,
WFP, FH, 
Commodity and food 
distribution, war 
relief, flood relief.
Transitioned in FY 
1995
4,510,48630 1,229,160
Angola UCAH,USAID,
FH
Rehabilitation of 
disarmed soldiers, 
seed, tools and food 
distributions
Did not transition 357,525 0
Ethiopia FH, 
USAID,CFGB,CR
WRC
To meet immediate 
food needs of 
drought affected 
people in different 
parts of the country
Run concurrently 
with development 
programs
2,984,440 4,807,438
Rwanda FH, UNHCR, 
UNICEF,USAID
Child related 
programs; housing 
and agricultural 
rehabilitation
Transitioned in FY 
1999
6,892,845 409,941
Congo USAID,UNHCR,
WHO,EU, 
Tearfund, FH, 
Local
War relief, 
commodity and food 
distributions
In transition at 
present
2,343,000 4,386,000
TOTAL 19,026,396 12,026,857
* Countries where emergency responses involved less than 100,000 $ have been excluded from this 
table
 
30 This amount includes 1,120,049 ECU converted with a 1$:1ECU exchange rate and an amount of 
951,202 BP converted with a 1$:1.5BP exchange rate. 
43
The FHI Regional Finance Director for Africa, Tobias Mahiri, feels that the overall 
capacity of FHI to deal with emergencies inside and outside Africa, has increased 
significantly during the past four years. He felt that the ongoing relief operations in 
Africa as well as the ISA training, enabled FHI to respond to emergencies in Kosovo, 
Nicaragua and India31. For example, during the Kosovo emergency, he and several 
FHI staff members from Ethiopia and the great lakes region went to assist. In the case 
of the Congo, the program has evolved significantly over time, especially since the 
year 2000. It has become much more effective and has become more recognizable in 
terms of its coverage, nature and number of beneficiaries. The ISA trainer on 
emergency response, Merry Fitzpatrick, has been one of the main forces that shaped 
this particular program. 
Table 14, on the previous page, summarizes the value of the monetary component of 
the ISA emergency responses in Africa for the period before and during the current 
ISA. According to the table, the amount of money donated towards FHI’s relief 
activities has not increased significantly during the ISA period, despite several conflict 
and other emergency situations on the continent. FHI also did not initiate any 
significant emergency relief activities in new fields during the ISA period. A small 
emergency response effort was underway in Malawi during the evaluation field visit 
and a food security assessment is currently being conducted in Sudan. It is planned 
that both Sudan and Malawi will result in OFDA funded food security programs in early 
2003.
An additional spin-off of the emergency response component was that some of the 
workshop materials that were developed for the ISA, were also presented at a training 
workshop for staff of the Global Hope Network (FHI 2001a:7).
5.4.3 Suggestions for future ISA programs
· The Kenyan management team felt that they are strategically well placed to 
house a regional emergency response team. They would like to continue to 
participate in ISA capacity building programs to enable them to play a stronger 
regional role in this respect.
· Trainees need more time during training for interacting and sharing ideas with 
each other. They benefit a lot from mutual sharing. 
· Training formats that include learners from different countries are more 
beneficial as the trainees exchange ideas and experiences.
· Including blueprints of emergency proposals, indicating the needs, 
requirements and definitions of them all, will be very useful to staff involved in 
Emergency preparedness.
· There is a need for more resource materials such as reference books and 
CD’s. 
· Country programs want to know more about coping mechanisms used by 
farmers and slow-onset emergencies. There is a need to establish cut-off 
points that distinguish emergencies from non-emergency situations. 
 
31
 The cash component of the relief effort in Honduras amounted to 454,000 USD, in India 293, 000 
USD and 2,355,980 USD in Kosovo.
44
5.5 Food security needs assessments (Objective 3) 
5.5.1 Activities and inputs
One of the objectives of the ISA was to conduct needs assessments in Mail, Niger, 
Burkina Faso and Haiti to determine a rationale for and feasibility of starting with Title II 
activities in those four countries 32. Niger was dropped from the list with the approval of 
FFP, as it became clear before the assessment, that USAID in Niger did not see room 
for including more CS’s in its Title II program.
Two ISA team members
33 visited the three countries targeted for this assessment 
during FY 1999. The team reviewed secondary data and met with Government 
officials, USAD/FFP and NGO’s during their visits (FHI 1999:8). Their assessments 
looked at regional differences within each country and were primarily based on the 
following parameters:
· Current food availability and access from a smallholder producer perspective
· Demand for monetized products in each region as per Bellmon analysis and other 
indicators 
· Food utilization as reflected in malnutrition rates; breastfeeding and weaning 
practices; under-five child mortality rates; incidence of diseases such as diarrhea 
· Financial implications for FHI (availability of private funding) 
In the case of Haiti and Burkina Faso it was found that these countries have food 
security problems, but that they generally do not have room for expansion in the area 
of monetization and Title II programming. Burkina Faso had less serious food security 
problems than Haiti and was also generally well covered by NGO’s. In Mali, food 
security problems were found to be mainly behavioral and thus linked to utilization. It 
was also the only country amongst the three visited which was identified by the team 
as having the biggest potential as an FHI field for the future (FHI 1999:16). During 
FY2000, FHI had several meetings with CRS and OICI with the view of doing a micro￾level assessment in Mali. Finally the idea was abandoned as it became clear that 
USAID in Mali was opposed to Food Aid and would not welcome proposals for Title II 
programs (Personal communication with Dave Evans, November 2002).
5.5.2 Achievements and conclusion 
A review of the reports compiled during the assessments showed them to be 
comprehensive and well researched. The conclusions reached by the assessment 
teams were based on careful consideration of all the factors that could have an 
influence on a decision about FHI’s Title II expansion. The evaluation team therefore 
has to conclude that this objective has been completed successfully and in line with 
the original project proposal.
 
32 Copies of the assessment is available on FHI’s Food security Extranet: www.fhi.net/gme/fse
33 In the case of Haiti the team consisted of Dave Evans and Ted Okada, whilst Mali and Burkina Faso 
were covered by Dave Evans and Tom Davis.
45
5.6 FAM cooperating partnership and working groups (Objective 5)
5.6.1 Introduction
Four FAM working groups were established in order to increase cooperation between 
CS’s and to create forums where common needs can be addressed. These working 
groups are: the monetization working group, environmental working group, local 
capacity building working group and the monitoring and evaluation working group. 
Within the context of the present ISA, FHI committed itself to work actively with three 
working groups: Monitoring and evaluation, monetization and local capacity building. 
FHI served as the chair of the FAM steering committee during FY2001. Other FAM 
activities/resources which the FHI ISA team used and which they deem valuable are:
· Food Security Resource center
· FAM web site
· FAM list serves
· Food forum newsletter
· ISA managers meetings
Generally, FAM regards FHI’s representatives as having exceptional team skills. They 
are very committed to FAM’s work and their strong leadership and facilitation skills 
make a significant contribution towards the spirit of collaboration within FAM. They are 
described as very active, making good comments and presenting excellent ideas (FHI 
2001b:36). These comments were confirmed by some of the CS’s representatives 
who participated in the peer review and described FHI’s involvement in FAM working 
groups as ‘dynamic’. 
5.6.2 Monitoring and evaluation working group
Summary of key findings:
FHI had a significant impact on the M&E working group in terms of its 
contributions as chair of the working group and in terms of research and 
other support that went into the development of the M&E toolkit. 
Discussion of findings:
In its ISA project proposal, FHI proposed to commit ISA and other resources to a five￾year FAM collaborative effort with other CS’s in M&E. The objectives of this joint effort 
were to: 
1) review and recommend a set of robust tools to design, monitor and evaluate Title II 
programs, and
2) produce instructions on how to use those tools in various situations 
A variety of inputs and reviews were necessary in order to achieve these objectives. 
FHI played an active role in this working group throughout the period under review. 
During FY1999 and part of FY2000, the ISA team leader was the chairperson for this 
working group. The working group completed reviews of baseline research and survey 
methods for agricultural projects and health and nutrition programs, as well as a review 
of health and agriculture project monitoring tools for Title II funded programs during 
that time. Subsequently a complete toolkit for the monitoring and evaluation of Title II 
programs was also developed. 
46
Three workshops were organized by the working group: 
· Sampling for Title II managers (FY1999): attended by three FHI staff members
· Data analysis (FY2000): attended by two FHI staff members 
· Monitoring tools and methods (FY2002): attended by four FHI staff members. 
Dave Evans, the former ISA team leader and Tom Davis, the ISA MCHN TA, 
presented the majority of this workshop. 
It was anticipated that the above outputs would lead to the following effects on FHI:
· Improved program design and results measurement
· Monitoring and evaluation toolkit developed cost effectively for all PVO staff 
· Monitoring and evaluation toolkit produced through PVO staff collaboration 
resulting in greater field use and improved monitoring and evaluation systems. 
As the Monitoring Tools and Methods workshop was only presented at the beginning 
of the year 2002, it is still too early to determine its possible impact on results 
achievement. However, staff who attended the workshop, rated it as highly relevant 
and the quality of the toolkit is certainly high. Time will show to what extent it will be 
integrated in the monitoring and evaluation activities of the field programs. 
A short questionnaire, aimed at assessing FHI’s role in this working group was 
circulated to six monitoring and evaluation working group members. Five of them 
responded. The average time of involvement of the members who responded to the 
survey varied between one and two years. They agreed that the working group 
achieved most of its objectives and is still on track in terms of what needs to be 
achieved during FY2003. According to them the FHI representative was present at 
most meetings and the quality of his contributions was rated as very valuable by four 
of the five respondents and valuable by the remaining one. The role played by FHI has 
been described as providing comments on work done by other members, involvement 
in the writing and production of materials, driving force in the production of 
tools/materials, as well as involvement in the organization and presentation of FAM 
workshops. The following additional comments about FHI were made: 
· FHI is a valuable member of working groups as they are quick to accept 
responsibility for organizing meetings and programs and also to facilitate them.
· I would appreciate it, if FHI can make a bigger contribution in terms of 
writing/sharing/presenting its experiences.
· FHI’s participation in the working group is valuable. Comments, suggestions or 
thoughts about the working group’s activities have been useful.
5.6.3 Monetization working group
Summary of key findings:
FHI made a contribution towards the development of the monetization 
manual, actively participated in working group meetings and the general 
exchange of information and expertise between members. However, 
participation was not nearly as significant as in the case of M&E. 
47
Discussion of findings:
In its ISA project proposal, FHI agreed to collaborate in the following joint activities as 
part of joint CS’s efforts:
1) Development of a Cooperating Sponsor Monetization Manual. This practical 
manual was to reconcile existing materials, building on the USAID Food for Peace 
"Monetization Field Manual" and incorporating best practices and lessons learned 
from Title II Country Programs. According to the CM TA, FHI participated in the 
development of this manual. It was published during FY1999 and was also used by 
FHI as a reference, while FHI was developing its own Commodity Management 
manual.
2) Participate in the organization of Monetization Training Workshops aimed at 
building capacity in the area of monetization. Four regional workshops were held 
between 1998 and 2001. Three FHI staff members participated in monetization 
training courses. These were regarded as highly relevant and useful to FHI’s 
programs (FAM final evaluation, FHI response 2002). 
3) Maintain ongoing Collaboration Mechanisms, which facilitate information sharing 
and distance learning among Cooperation Sponsors and USAID/FFP had to be set 
up by the working group. The aim of these efforts would have been to promote 
interactive communication with field offices, increase learning and improve the 
dissemination of new trends and best practices in monetization programs. To some 
extent there is evidence that this aspect has been implemented. For example, 
when the FHI commodity management manual was developed, the CM TA relied 
heavily on manuals and materials used by other CS’s. These would have been 
obtained through working group contacts. 
According to the CM AT, they have noticed an improvement in the quality of 
monetization plans. There has also been some reduction in commodity losses, 
especially in Ethiopia34 (FHI 2001b:32). Some of this can be attributed to the outputs of 
this working group. A short, one page questionnaire aimed at verifying FHI’s 
participation in this working group was sent to six working group members. Only two of 
them responded. The respondents confirmed that FHI attended most of the meetings 
and made valuable to very valuable contributions towards the work of the group. They 
felt that FHI contributed in terms of commenting on the work done by other members, 
produced some of the tools/materials and were also involved in the organization and or 
presentation of FAM workshops. 
5.6.4 Local capacity building (LCB)
Summary of key findings:
The local capacity working group was less active than anticipated and FHI’s 
involvement has been restricted to commenting on documents generated by 
the group as a result of a lack of staff. 
 
34 Bolivia seemed to have had a negligible loss percentage even before the initiation of the working 
group activities.
48
Discussion of findings:
Several objectives were formulated for FHI’s participation in this working group. Most 
of the activities envisaged for this specific working group were: 
· The development of position papers
· Development of a best practices manual 
· Regional meetings to share best practices and consolidate positions 
· Ongoing collaboration between CS’s. 
· The development of food security monitoring and evaluation indicators related 
to LCB.
In practice, FHI’s involvement in the LCB working group was limited to providing 
feedback as part of the larger FAM annual review and planning process. The main 
reason for this scaled down involvement is a lack of staff (FHI 2001c:5). Whilst FHI 
was chair of the FAM steering committee (FY 2001), indirect support was given 
through a funding strategy for this working group’s activities. By the end of FY2002, 
the ISA team rated this working group as having relevance to their program of two (on 
a five point scale with five the highest relevance) (FAM questionnaire 2002:7). A 
review of available documentation suggests that with or without FHI’s participation this 
working group was not very active. To date, no best practices manual has been 
developed and limited references could be found about workshops and position 
papers. 
5.7 FAM information technology capabilities (Objective 6)
5.7.1 Activities and inputs
Under FHI’s proposed ISA activities, FAM and FHI agreed to pursue a mentoring 
partnership to improve the information technology capabilities of FAM. This 
partnership had two goals. Firstly FAM would learn and become proficient at 
current/new information technology capabilities. Secondly that the FAM consortium will 
receive the ultimate benefit and become stronger as a result of the technical 
leadership of FHI and FAM. The original project proposal (FHI 1998) outlines three 
objectives related to its FAM mentoring effort: Improvement of the FAM website with 
basic maintenance by FAM; establishment of listserv capability and management skills 
by FAM; and the establishment of internet relay chat capability and encouragement of 
increased usage by the FAM consortium.
The table on the next page summarizes the role that FHI played in the development of 
FAM information technology capabilities between FY1999 and FY2002. The last row 
shows the creation of a FSRC database. This activity was not part of the original FHI 
ISA proposal, but they were asked by FAM to assist with it. They provided technical 
support to the consultant who was hired by FAM to establish the database. 
49
Table 15: Activities related to the FAM information technology for the period FY1999-FY2002
Main objectives 
of FAM 
mentoring
FY1999 FY2000 FY2001 FY2002
Improve FAM 
website with basic 
maintenance by 
FAM
FAM taught to 
manage and 
update website 
independently; 
general mentoring 
on website 
management; FHI 
server used as 
host for website. 
FAM TIS leaves, 
but is self-sufficient 
enough to train 
new TIS in web 
maintenance from 
Mexico. FHI 
provides 
NeTracker 
software to FAM
Website 
completely 
redesigned; 
provided technical 
mentoring for 
search engine and 
website survey
Amount of links 
and documents 
nearly doubled; 
domain name 
changed
Establish listserv 
capability and 
management skills
FHI developed 
template on FHI 
server to host FAM 
listserves; FHI 
provide training 
and mentoring on 
listserve 
management
FHI mentors and 
assists FAM 
through episode 
with hackers; 
FHI provided new 
listserv software; 
number of 
listserves 
expanded from 5 
to 7
Assisted FAM to 
maintain seven list￾serves
Establish internet 
relay chat 
capability and 
encouragement of 
increased usage 
by the FAM 
consortium
Advantages were 
presented to FAM 
members, 
unwilling/unable to 
fully pursue this 
technology within 
their organizations
Not explored as 
option as a result 
of the departure of 
FAM TIS
FAM and FHI 
tested chat 
capabilities 
Research indicate 
a limited 
application for this, 
especially outside 
the US. Trial voice 
chats may be done 
in 2003 
Development of 
on-line FSRC 
database35 (not 
part of the original 
objectives) 
Started with the 
assistance of FHI 
and pro-bono 
programmer
Continued Prototype 
completed by 
consultant FHI 
provides technical 
support to the 
consultant.
Completed;
FHI provides the 
physical location
Source: FHI annual reports FY1999 to FY2002
5.7.2 Achievements and impact
Summary of the key findings:
This has been a very successful mentoring partnership. FAM regards the training and 
support that they received as excellent and oriented towards their needs. FHI’s support 
enabled FAM to achieve most of its objectives that relied on technology for its 
execution. In addition to this, FHI also supported an additional activity that was not 
contained in the original project proposal - the development of a FSRC database.
 
35 This objective was not included in the original objectives of FHI’s ISA. However, once FAM identified 
the need for this, FHI provided assistance to both the consultant and FAM. 
50
Discussion of the findings:
FHI met the three sub-objectives that it set out to achieve under this ISA objective36. In 
addition to this, the organization also assisted with an activity that was not included in 
the original proposal, namely the creation of the FSRC database. FHI provided 
technical support to a pro-bono consultant who developed the database. FAM rates 
FHI’s support as going ‘above and beyond the call of duty’ (FAM 2002). Examples 
include additional support in the form of making their server available to FAM free of 
charge and providing software such as web tracking software exceeded the basic 
needs of FAM and FHI’s original objective. According to FAM (2002), they would not 
have been able to function effectively without FHI’s assistance in the areas of website 
development and maintenance, provision of FHI server capabilities, hardware 
backstopping, setting up and management of listserves. They also rated all FHI’s 
services, with the exception of training in software use, as excellent. However, 
software training was still rated as very good.
It’s support in the development and maintenance of the FAM website has been fruitful 
to the whole partnership and beyond. The FAM website has been singled out by both 
FAM members and non-members as the most important service of FAM. Visits to the 
site nearly doubled during FY2000 and increased by 150% during FY2001. The graph 
below shows the extent to which visits to the FAM website has increased during the 
period FY1999-FY2002. 
 Graph 2: Visits to the FAM website FY1999-FY2002 
 
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
50000
FY99 FY00 FY01 FY02
Financial Year 
Number of visits
 
 Source: NetTracker 4.5 (internet website tracking software), FAM 2002.
 
36 The objective of establishing internet relay chat capabilities, was met in the sense that it was tested 
by FAM and FHI. However, it has not yet been adopted widely by the FAM consortium for a variety of 
reasons, most of which relate to constraints within those organizations. Further work is planned on this 
objective for FY2003. 
51
Unfortunately, this resource has not yet found its way to the field offices. On average 
four staff members per FHI field know about the FAM website. They are usually 
national or regional staff, who speak English. Some fields, for example Kenya 
(Marsabit) and Mozambique (Beira) have some problems getting and maintaining 
internet connections and even though they may know about the website, few of them 
are actually able to use it. In Bolivia, where the connections are generally quite good, 
staff who do not speak English generally do not know about the website and if they do, 
find it less accessible and useful as a result of language barrier. The three staff 
members who reported having used it, are all expatriates. Another factor that has to be 
considered is that even if internet connections are good in a particular field, only a 
limited number of managerial staff are authorized to use the internet as a result of the 
cost implications. One of the alternatives presented to the trainees who attended the 
FAM M&E Toolkit workshop, is a selection of website materials on CD-Rom. This was 
very well received and some staff reported having already used this information. 
Listserves were created for the first time in FY1999. Five were initially formed and two 
were added during FY2001. The table below shows that most listserves doubled their 
number of subscribers during the past financial year. Their main purpose is to improve 
communication and contact between FAM working group members and also non￾members interested in the specific focus of the listserv. Some listserv members 
request and receive technical assistance through the listserv. 
Table 16: Number of subscribers to the FAM listserves
Number of subscribers
 Name of Listserve FY01 FY02
 Commodity Management 67 134
 Environment 146 292
 Local Capacity Building 145 290
 Monitoring & Eval 217 434
 Monetization 176 352
 Nutrition 41 82
 FAM members listserve - 70
 Source: FAM 2002
Over time, FAM’s needs changed from direct training to requesting help from FHI as 
required by their new initiatives. According to them, FHI has been flexible enough to 
adjust to their changing needs. They regard FHI as consistently client oriented and 
considerate of FAM’s needs. Advice and/or suggestions related to new technologies 
were based on a thorough understanding of FAM’s needs. Their advice and training 
was also always given in such a manner that FAM was able to continue working 
independently (FAM 2002). One example that shows the extent to which the FAM TIS 
became self-sufficient as a result of FHI’s training, is the transition during FY2000 from 
one TIS to the present occupant of the position. The first TIS was sufficiently well 
versed in website management (as a result of FHI mentoring) that she could provide 
training and support from Mexico to the new TIS. 
52
[This page was intentionally left blank]
53
CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS 
6.1 Introduction
The FHI ISA successfully completed most of the activities and outputs 
that it set out to achieve. With one year of the program still to be 
completed, thirty-eight of the forty-two output/activity targets were fully 
met or exceeded. Three of the four targets that were not met were only 
slightly below target and adequate explanations could be given for 
underperformance. 
Training materials were in general comprehensive and well prepared. 
There is some evidence that the ISA trainers were continually striving to 
improve the quality of their training materials and presentation. Handouts 
given during the workshops can be improved with more detailed and in￾depth information. It is important that enough time and resources be 
allocated during the next ISA, to consolidate and compile all the training 
materials of this ISA. 
Post test scores and comments by the people who participated in the 
courses, suggest that there has been a significant increase in knowledge 
amongst those who attended the training. However, the main goal of the 
ISA was not only to increase knowledge, but also to increase the 
technical, programmatic and managerial capability of FHI to ultimately 
increase the impact of its Title II food security programs. 
The next section summarizes the conclusions of the evaluation team on 
program impact. The discussion leaves no doubt that the program has 
succeeded in improving several aspects related to the functioning of its 
Title II fields. This impact is expected to grow as implementation of the 
new DAPs progress.
6.2 Conclusions on impact
Objective 1: Select, promote and train staff in the use of standard, high￾quality tools for Title II program design and implementation
Several training workshops were conducted to impact on this objective. 
The evaluation team found ample evidence that the ISA program 
impacted on certain program aspects in such a way that it contributed 
towards an increased ability of the fields to meet food security targets. 
· The process of preparing and writing project proposals has become 
more participatory and feelings of ownership of the process and the 
new DAP has increased significantly. 
· The quality of food security problem analysis and the general 
structure and presentation of project proposals have improved in all 
fields.
· The activity impact of the educational messages and methods 
workshop was to be an improved capacity of FHI field staff to 
successfully promote behavior change on the part of Title II program 
54
beneficiaries. The evaluation team found that all fields have 
developed training curricula and lesson plans as a result of the ISA 
assessment and training. In most fields, training has become more 
participatory and the use of visual aids and other techniques to 
facilitate learning amongst non-literates has increased. This is 
probably the one area in which ISA has already impacted on food 
security in its Title II fields. 
· In the case of Positive Deviance, the activity impact would have been 
an improved capacity of FHI field staff to successfully use positive 
deviance in nutrition and agriculture leading to less malnutrition and 
increased technology adoption on the part of the beneficiaries. The 
evaluation team found that Mozambique is the only program that is 
implementing the Hearth Method. Kenya and Bolivia plan to 
implement it in 2003. 
· The evaluation team found that the use of Barrier analysis and TIPS 
was very limited in all three fields and the main reason for this may be 
that training was done quite recently. Replication has taken place in 
Mozambique, but it has not been used widely and could therefore not 
have had an impact on food security programs. The replication 
training may also not have been of high quality as less than 60% of 
the people who knew about these methods could identify their correct 
definitions. The lack of replication in Bolivia probably reflects a lack of 
management commitment to the implementation plan that was 
developed after completing the training course. 
· Awareness and use of Quality improvement checklists is high in 
Mozambique and Kenya. This has probably contributed to improved 
performance by community-based workers and as such had an 
impact on the ability of Title II fields to improve food security. 
· Monitoring and evaluation capacity was strengthened, but losses of 
key management staff during the past two years, has left a gap in 
Mozambique and Kenya for the new DAP. 
Objective 2: Improve FHI's capacity to respond to emergencies 
and facilitate a rapid transition to development activities in Sub￾Saharan Africa;
There is some evidence that aspects of the training has been useful 
and has impacted on programs such as the DRC and relief efforts in 
Kosovo, Nicaragua and India. There is some evidence that Kenya (in 
Meru) and Mozambique (in Malawi) have applied, albeit in a modified 
form, the knowledge that was gained during the ISA training. 
Objective 3: Conduct needs assessments in the West African 
Sahel (Mali, Niger, Burkina Faso) and Haiti to determine 
rationale for and feasibility of initiating activities in those 
countries;
The conclusions reached by the assessment teams were based on 
careful consideration of all the factors that could have an influence on 
a decision about FHI’s Title II expansion. The evaluation team 
therefore has to conclude that this objective has been completed 
successfully and in line with the original project proposal.
55
Objective 4: Improve FHI's capacity to efficiently and effectively 
manage commodities;
The management of warehouses and commodities was good in 
Bolivia even before the ISA. However, the ISA mentoring and training 
support led to further refinement and improvements in their systems. 
Ethiopia benefited significantly from the ISA Commodity Management 
training, especially in the areas of warehouse management and loss 
control. In general, the team found evidence of the standardization 
and streamlining of systems and increased knowledge at FHI 
headquarter level as regards to all aspects of commodity 
management. 
Objective 5: Collectively improve, with the other cooperating 
sponsors (CS): a) program monitoring and evaluation, b) 
monetization activities and Bellmon analyses, and c) local 
capacity building via substantive collaborative efforts with other 
Title II cooperating sponsors.
FHI had a significant impact on the M&E working group in 
terms of its contributions as chair of the working group and in 
terms of research and other support that went into the 
development of the M&E toolkit. There is also some 
evidence that FHI made a contribution towards the activities 
of the monetization working group, but it was not nearly as 
significant as in the case of M&E. The local capacity working 
group was less active than anticipated and FHI’s 
involvement has been restricted to commenting on 
documents generated by the group. 
Objective 6: Contribute toward the improvement of FAM 
knowledge and proficiency in using information technology to 
enhance communication and information flow between the PVO 
members of FAM (mentoring partnership).
This has been a very successful mentoring partnership. FAM regards 
the training and support that they received as excellent and oriented 
towards their needs. FHI’s support enabled FAM to achieve most of 
its technology related objectives.
6.3 Recommendations
All indications are that this particular ISA has been implemented 
successfully. However, there is also a definite need to continue with this 
kind of capacity building program in FHI’s Title II fields. The following 
recommendations are aimed at highlighting the most important aspects of 
program management and implementation that could be modified to 
increase the impact of future ISA’s capacity building efforts. 
I. The loss of managerial level staff that participated in the ISA program 
was high in all fields. No program aimed at building capacity within an 
organization can be successful if there is a high staff turnover and lack 
of continuity. It would appear that FHI needs to assess the underlying 
reasons for what appears to be a higher than normal turnover rate and 
then take appropriate action to resolve the problem. 
56
II. The ISA team, in conjunction with the Regional Directors and CD’s, 
should develop a basic orientation package of documents, tools and 
methods that should be given to new managers. Changes in 
management should be accompanied by an assessment of their training 
and mentoring needs, so that program continuity can be enhanced. The 
next ISA may need to contain a specific component that focuses on this 
aspect.
III. There is a need to do some more thinking about the emergency 
component of FHI. Besides losing most of the staff trained in 
Emergency and Relief Management, the present practice of using 
development staff to become ‘emergency experts’ when relief 
operations are necessary, may not be the ideal. Not only does it 
withdraw resources away from development programs, but it also 
makes it very difficult to develop a corps of well trained and motivated 
emergency experts. Having specialized regional units can also make a 
significant contribution towards the expansion of emergency programs 
in Africa.
IV. Several excellent recommendations have been made during the mid￾term review about the format of the training. The most important of 
these being, greater consultation with the fields about the nature and 
content of training courses, the development of implementation action 
plans after a training course is presented and the development of a 
handbook to include all topics covered by the present ISA. The ISA 
team needs to continue building on those. 
V. The replication of trainings have had mixed results. Part of the problem 
is a lack of management commitment in some countries to do the 
replications. However, in Mozambique, where training has been 
replicated faithfully, test scores and present knowledge of definitions of 
the main interventions were low. Even though relying on replications is 
the most sustainable and practical for large fields such as Mozambique, 
more attention should perhaps be given to mechanisms to improve the 
quality of replications. This could perhaps be done by for example 
training smaller groups of trainers over a longer period of time. Also if 
the replications are shorter than the original training, there should be a 
commitment towards several follow-up trainings/sharing of experiences 
whilst implementation takes place. 
VI. Mentoring is normally a task of regional and program management. 
However, when completely new tools and methods are introduced, a 
good case can be made for the active mentoring of implementation for 
at least a year after training was completed. This can serve three 
purposes: increase the quality of implementation; increase innovation 
and improve the sharing of innovative experiences between fields. The 
frequent requests of field staff for more practical sessions during the 
training will probably be better addressed by a mentor system, as a 
significant amount of training time is already devoted to practical 
exercises. 
57
The ISA should consider building time and resources into the next ISA to 
mentor the implementation of Barrier Analysis, TIPS, Positive Deviance 
and the new themes that will be introduced. 
VII. Even though there is a need to introduce new themes during the next 
ISA, it may be advisable to focus a large part of the resources on the 
consolidation of the training materials and the strengthening of 
implementation of the new tools and methods introduced during the 
current ISA and previous ISG. 
VIII. In terms of the content of a future ISA program, various possible themes 
emerged from this evaluation. These are:
Tools for Title II program design and implementation
· Consolidating the training materials of the current ISA into manual 
format
· Providing mentoring assistance to fields for the implementation of 
Barrier Analysis/TIPS within the context of refining agricultural and 
health training curricula. The mentoring activities of Positive 
Deviance in Mozambique should be continued and expanded to the 
other two fields.
· Developing and expanding CD-ROM and general reference library 
capabilities in Title II fields
· Linking monitoring to management information systems with the 
objective of strengthening the relationship between monitoring 
information and changes in program implementation 
· Training in data analysis: basic EPI-Info and SPSS; statistical 
methods for more advanced data analysis 
· Management skills, team management and coordination
· Technical areas that could be considered for inclusion are: program 
innovations/techniques that can be used income generation 
activities and marketing programs
· Establishing mentoring relationships between Universities and FHI 
for specific program areas. For example, FHI/K will be involved with 
livestock development activities for the first time. Are there 
resources/institutions that can enter into a formal relationship with 
them to develop and test new innovations?
Commodity management support
There is a need for continued support in the area of commodity 
management given the demands of monetization and continual changes 
in the administrative requirements. 
FAM information technology support
FAM expressed a need for continued cooperation and support with FHI. 
Various CS’s commented that FHI has played a pivotal role in the past. 
Continuing in some sort of a support and mentoring role, may lead the 
partnership to new heights.
58
Innovation and renewal
Our ability to effectively impact on food security largely depends on 
merging new ideas with existing, but ever-changing realities. The FAM 
working groups have been successful in spreading innovation and 
promote sharing of experiences between Title II CS’s. However, there is a 
wealth of other development agencies, who by virtue of not being Title II 
participants, have different experiences, systems and approaches. It may 
be in the interest of FHI and the partnership if a specific component is 
added to the ISA that focuses on learning from non-Title II agencies; 
identifying, adapting and sharing food security innovations. 
REFERENCES
FAM 2002 Statistics provided by FAM. Responses to questionnaire sent to
them. FAM: Washington.
FAM question. 2002 Questionnaire sent to FAM members as part of the FAM Final
Evaluation. FAM: Washington. 
FHI 1998 Program proposal: Improving Food Security Programming
and Resource Management. FHI: Gettysburg
FHI 1999a FY 1999 Annual Report. FHI: Gettysburg.
FHI 1999b Program Performance Monitoring and Evaluation Plan. August 1999. 
FHI: Gettysburg.
FHI 1999c Food security education messages and methods assessment report. 
December 1999. FHI: Gettysburg.
FHI 2000 FY 2000 Annual Report. FHI: Gettysburg.
FHI 2001a FY 2001 Annual Report. FHI: Gettysburg.
FHI 2001b Mid-term review report: improving food security program and 
resource management report. FHI: Gettysburg. 
FHI 2001c Annual work plan for FY2002. FHI: Gettysburg.
FHI 2002b DAP Final Evaluation, Marsabit. FHI: Nairobi
USAID 2001 Memo from director of FFP, USAID to USDA. July 3, 2001. FFP: 
Washington.
ANNEX A: EVALUATION SCOPE OF WORK
FOOD FOR THE HUNGRY INTERNATIONAL
P.L. 480 TITLE II 
INSTITUTIONAL SUPPORT ASSISTANCE PROGRAM
Award #: FAO-A-00-98-00032-00
“IMPROVING FOOD SECURITY PROGRAMMING AND 
RESOURCE MANAGEMENT”
SCOPE OF WORK FOR FINAL EVALUATION
Submitted to USAID/DCHA/FFP/CTO on 7 June 2002
FHI Headquarters Office Contact: 
Buck Deines
ISA Team Leader
7807 E. Greenway Rd. Suite 3
Scottsdale AZ 85260
Tel: (480) 609-7791 Fax: (480) 951-9035
Email: bdeines@fhi.net
8 i
TABLE OF CONTENTS
1. BACKGROUND 1
2. PURPOSE OF THE REVIEW 2
3. EXISTING PERFORMANCE INFORMATION 
SOURCES
3
4. REVIEW QUESTIONS 4
5. REVIEW METHODOLOGY 5
6. REVIEW TEAM COMPOSITION 6
7. TIMEFRAME 7
8. REPORTING 8
8 2
1. BACKGROUND 
Food for the Hungry International (FHI) has been implementing Title II food 
security programs in Africa and Latin America since 1985. Despite that long 
history and contrary to many other cooperating sponsors who have been 
receiving ISG/As since the beginning, FHI received its first institutional support 
grant in 1997. The grant period for that ISG was 20 months with the goal being 
to achieve significant impact in food security via the establishment of a robust 
monitoring and evaluation (M&E) system in FHI’s Title II programs. The focus of 
the ISG activities was to design and develop methodologies and systems, and 
train and provide technical assistance to FHI Title II-related field staff in the 
following components of the food security M&E continuum: 1) macro-targeting, 
2) micro-targeting, 3) indicator development, 4) baseline data collection and 
analysis, 5) monitoring, and 6) evaluation. FHI met all and surpassed many of its 
targets that it set for the ISG outputs and activities.
As a follow-on to the ISG, FHI is currently implementing a five-year ISA program 
that began in September 1998 and is scheduled to end in August 2003. The 
program is addressing six major headquarters’ and field priority areas: 
1. program design and implementation of development programs, 
2. emergency and transition programs, 
3. new country program initiation, 
4. commodity management, 
5. collaborative efforts in M&E, monetization and local partner facilitation, 
and 
6. information systems.
The goal of the ISA activities is to increase the impact of FHI’s Title II food 
security programs via the improvement of its technical, programmatic and 
managerial capability. This is being accomplished by way of the following 
objectives: 
A. Select, promote and train staff in the use of standard, high-quality 
tools for Title II program design and implementation as a follow up to 
the accomplishments achieved under the former ISG program in 
M&E system standardization;
B. Improve FHI's capacity to respond to emergencies and facilitate a 
rapid transition to development activities in Sub-Saharan Africa;
C. Conduct needs assessments in Mali/Burkina Faso and Haiti to 
determine rationale for and feasibility of initiating activities in those 
countries;
D. Improve FHI's capacity to efficiently and effectively manage 
commodities;
E. Collectively improve a) program monitoring and evaluation, b) 
monetization activities and Bellmon analyses, and c) local capacity 
building via substantive collaborative efforts with other Title II 
cooperating sponsors; and
8 3
F. Via a mentoring agreement, contribute toward the improvement of 
FAM knowledge of and proficiency in using information technology to 
enhance communication and information flow between the PVO 
members of FAM.
FHI’s ISA program is targeted to impact three distinct sets of beneficiaries in the 
following order of importance: 1) current FHI Title II programs in Bolivia, 
Ethiopia, Kenya and Mozambique, 2) potential future FHI Title II programs, and 
3) other FAM-member Title II Cooperating Sponsors. The program is 
implemented by a five-member, multi-disciplinary team composed of 1) a team 
leader (who also serve as the technical assistant in agriculture and training 
facilitator), 2) a technical assistant and trainer in maternal-child health and 
nutrition, 3) a technical assistant and trainer in commodity management, 4) a 
technical assistant and trainer in information systems, and 5) a technical 
assistant and trainer in emergency response programming. In addition, FAM 
staff and other Title II Cooperating Sponsors provide indirect support to the 
program via the collaborative efforts described above in objective E.
FHI is now completing the fourth year of its ISA program having accomplished 
the great majority of its activity and output objectives to date. USAID/FFP’s 
comments on FHI’s Mid-Term Evaluation included the following:
· With two minor exceptions, all the planned activities and outputs have 
occurred. Achievement of targets has been very strong. 
· Trainings are high quality and have resulted in improved field capacity and 
tools, including the adaptation of tools from other PVOs.
· While it is difficult at this stage to link improvements in food security 
directly to the ISA, field staff do perceive the ISA to be having a positive 
impact on food security through higher quality programs, more efficient 
use of resources, and improved techniques learned from ISA training.
As FHI is completing implementation of this ISA program we will now conduct the 
planned external review to assess achievement of planned objectives in activities 
and outputs as well as effects and impact. 
2. PURPOSE OF THE REVIEW
The purpose of this final evaluation will be to provide an assessment of the 
results achieved, reasons for levels of achievement or non-achievement, and 
lessons learned from the ISA program. An emphasis of the evaluation will be an 
external review of impact-level results on the ISA’s contribution to impacts on 
food security achievements through FHI Title II programs. 
Other considerations for the final evaluation:
· Need for statistically valid quantitative data collection
· Externally oriented assessment
· Indicator Performance Tracking Table will be updated for all indicators
8 4
· Recommendations can suggest where the need may or may not exist for 
further institutional strengthening activities.
3. EXISTING PERFORMANCE INFORMATION 
SOURCES
In order for the review team to successfully conduct the mid-term review, they will 
need to conduct a thorough review of existing ISG/ISA documentation. The 
following list of documents is comprehensive, but not necessarily exhaustive. All 
of these documents can be obtained through FHI’s ISA team leader and on FHI’s 
Food Security Extranet at: 
http://www.fhi.net/gme/fse/isapr/index.htm#isadocumentreview 
· FHI’s corporate identity (including Vision of Community);
· 1997-98 ISG final proposal;
· Quarterly and final reports for the ISG;
· 1999-2003 ISA final proposal; 
· ISA program performance M&E plan;
· 2000, 2001, 2002, and 2003 ISA annual work plans; 
· 1999, 2000, 2001, and 2002 annual results reports; 
· FHI Food Security Extranet website;
· ISA team orientation notes;
· Mali/Burkina Faso and Haiti food security needs assessments;
· FHI Title II commodity management procedures manual;
· Educational messages and methods assessment report;
· Workshop notes and handouts for the following ISA workshops:
· M&E remedial; 
· Food security problem analysis and program design; 
· Epi-Info
· HEARTH methodology (positive deviance) 
· Food security education messages and methods; 
· Barrier Analysis (Factor Analysis)
· Emergency program preparation; 
· Rapid disaster assessments; 
· Emergency program design;
· Emergency program monitoring and evaluation; and 
· Commodity management procedures – part 1 and 2.
· Pre/post test scores for the workshops above; 
· Participant evaluation summaries of the workshops above;
· Quality improvement checklist scores from Title II fields;
· Food Aid Management (FAM) website 
· FAM annual evaluations of FAM mentoring activities; and
· FHI ISA Mid-Term Evaluation
8 5
4. REVIEW QUESTIONS
Several key questions need to be answered in order to fully review FHI’s ISA 
performance. The questions below should form the bulk of the review. However, 
it is likely that additional questions will arise as a result of going through the 
review process. The review team should include these additional questions—
and their answers—in the review report.
1. To what extent were the planned objectives achieved for the program? 
Specifically, were the annual monitoring indicators (activities and outputs) 
and final impact indicators (effects and impacts) successfully reached?
2. In what ways and to what extent has the ISA program made a positive 
impact on FHI’s ability to increase food security at the field level?
3. Which ISA program components where the most effective in strengthening 
FHI’s food security program capacity? Why? Which program components 
were the least effective in strengthening FHI’s food security program 
capacity? Why?
4. To what extent were the recommendations from the Mid-Term evaluation 
implemented?
5. What were the most significant constraints and/or difficulties in 
implementing the program and, where appropriate, how did FHI overcome 
them? What lessons learned does the review team identify that have 
implications for future capacity building programs?
6. Given that a large part of FHI’s ISA focuses on capacity building of Title II 
staff, what has been the change in the related knowledge, skills and 
practices of those staff? 
7. Are the ISA training materials appropriate- tailored to the user and, 
accurate, state of the art? Which materials need strengthening, if any, 
and how?
8. What is the perspective of FHI Title II field staff with regards to the 
services provided under the ISA?
9. How did the best practices identified in the CS collaborative efforts in 
M&E, monetization and local capacity building effect FHI’s ISA program?
10.What are lessons learned from this program? What implications for future 
institutional support activities can be extracted from those lessons 
learned?
8 6
11.What specific future needs can the review team recommend be prioritized 
for future institutional support activities? Of activities in the current ISA, 
which areas would benefit from further support in the future?
8 7
5. REVIEW METHODOLOGY
The final evaluation will determine the effects and impact that the ISA grant 
activities and outputs have had on FHI's Title II programs. The evaluation will be 
both quantitative and qualitative and will be implemented during the first quarter 
of the last program year (October-December 2002).
The evaluation will rely primarily on qualitative methods including, but not limited 
to, semi-structured interviews, direct observation, focus groups, and secondary 
data review. Additional sources of information for the evaluation will include 
monitoring project data and recommendations made in the mid-term review. 
Through participatory methods a multi-disciplinary team composed of an external 
consultant (team leader) and FHI headquarters staff will examine FHI’s ISA 
program results. A visit to three Title II fields will allow field staff and 
beneficiaries to provide their inputs to the review process. The final evaluation 
will be conducted in three stages: 
Stage 1: Review of Existing Documentation
Time Frame: 01 September – 15 October 2002
The review team will conduct a thorough review of existing data and information 
from the documents cited above in Section 3. In addition, the team leader 
(external consultant) may decide to consult additional documentation from the 
headquarters office, Title II field offices, or other sources. He/she will also 
evaluate the FAM mentoring activities and results during this stage. Finally, the 
team leader will outline preliminary field visit plans. 
Stage 2: Refinement of Review Methods
Time Frame: 1 - 15 October 2002
FHI’s ISA activities are heavily weighted toward building the capacity of field staff 
in order to increase their effectiveness in improving food security. In order to 
determine whether capacity has been built and, more importantly, that this 
increased capacity is being used on a routine basis by the trained staff and 
impacting FHI Title II programs, the team will need to combine both quantitative 
and qualitative approaches to data collection. Rather than stating the exact mix 
in this scope of work, we feel that it is crucial for the team leader to be 
instrumental in the method selection process. For some performance indicators, 
we will attempt to gather statistically valid quantitative data. For other indicators, 
qualitative methods will be a better way to gather more useful information. At a 
minimum the following data collection methods will be utilized: focus group 
interviews, key informant interviews, document review, observation, random spot 
checks (visits to field offices and target population homes/fields), and surveys. In 
addition, decisions will need to be made on choosing a sample of staff and target 
population to be interviewed/surveyed. Thus, during this stage, the team leader 
will decide on final selection of the methods and instruments to be used during 
the field visit and prepare for the data collection exercise in the field. 
8 8
Stage 3: Field Data Collection
Time Frame: 08 – 25 November 2002 
The team leader will plan and coordinate all the necessary logistics for the 
qualitative and quantitative collection of data at the field level. The FHI 
evaluation team members will assist the team leader as requested in this stage. 
FHI proposes conducting the field review in three of its four Title II fields—
Bolivia, Kenya and Mozambique. The reasons for selecting these fields are: 
Bolivia and Kenya were not visited in the Mid-Term Evaluation. Ethiopia is 
phasing out its Title II activities in FY2002 and as a result will not be a focus of 
this Final Evaluation as program activities have been re-focused in the other 
three fields during the last two years of the ISA. The team will spend a maximum 
of four days in each of these three fields collecting data. 
Stage 4: Write Evaluation Report
Time Frame: 20 November – 15 December 2002
Upon completion of the field data collection, the team leader will draft the 
evaluation report with conclusions and recommendations. The team leader 
will hold a meeting (in person or virtual) with FHI ISA staff to present 
findings, lessons learned, and recommendations. The final report will be 
submitted to USAID/DCHA/FFP no later than 31 December 2002.
6. REVIEW TEAM COMPOSITION
The final evaluation team will include an external technical consultant who will 
serve as the evaluation team leader and two selected FHI ISA team members. 
The final evaluation will be conducted over a ten-week period at an estimated 
cost of $28,159.
The final evaluation team will be composed of:
1. Team Leader
The team leader will be an external technical consultant who will be 
responsible for structuring and designing all review activities and 
methodologies, assigning evaluation tasks among the other team members, 
conducting interviews, meeting the specified objectives, collaborating with 
USAID and/or FANTA, and writing the report according to the defined 
timeline. 
2. Two FHI ISA team members (from headquarters)
Two members of FHI’s ISA team will assist the external consultant in 
providing requested background and organizational (HQ and field) 
information, arranging evaluation logistics for field data collection, and 
generally facilitating requested information to the team leader. The 
presence of these FHI ISA team members will be beneficial by enabling FHI 
8 9
to experience greater insight into the lessons learned of the program. One 
member will be the ISA Team Leader. The other member will be the trainer 
in information systems. 
7. TIMEFRAME
The following time frame and deliverables reflect the management plan for this 
evaluation and, as such, each set of dates is the expected time of completion for 
each evaluation component and set of deliverables.
Stage 1: Review of Existing Documentation -- Time Frame: 1 September -15 October 
2002
Total Person/Days = 9 (3 days x 3 team members)
Deliverables: None.
Stage 2: Refinement of Review Methods --Time Frame:1-15 October 2002
Total Person/Days = 12 (8 days x 1 team leader + 2 days x 2 team 
members)
Deliverables: Field visit schedule and itinerary, respondent 
selection, and data collection tools.
Stage 3: Field Data Collection -- Time Frame: 8 – 25 November 2002
Total Person/Days = 57 (19 days x 3 team members)
Deliverables: Completed survey tools, data.
Stage 4: Write Report -- Time Frame: 20 November – 15 December 2002
Total Person/Days = 13 (8 days x 1 team member + 2.5 days x 2 team 
members)
Deliverables: Draft to FHI by 6 December 2002 for comment; 18 
December 2002 final report to FHI; Final Report submitted by 31 
December 2002 by FHI to USAID/DCHA/FFP.
Total Time Frame: 1 September – 31 December 2002 
Total Person/Days = 90 (38 days for team leader (external consultant), 26 days for two ISA 
team members)
8 10
8. REPORTING
The final report must be submitted to USAID/DCHA/COTR on or before 31 
December 2002. The report must contain at least the following sections. 
Additional sections may be recommended by the review team.
1. Executive Summary 
2. Introduction 
a. Objective of SOW
b. Brief description of project
3. Methodology 
4. Updated Indicator Performance Tracking Table
5. Discussion of Performance Results
a. Brief description of interventions
b. Achievement of results
i. Meeting targets (annual and impact indicators)
ii. Other achievements 
c. Discussion of general evaluation questions
6. Cross-Cutting Issues
7. Lessons Learned
8. Recommendations for further institutional strengthening activities
9. Annexes
a. Evaluation SOW
b. Composition of the team
c. Methods
d. List of sites visited
e. List of key informants
f. References
g. Indicator Performance Tracking Tables (IPTT)
h. Survey tools
i. List of acronyms
8 11
ANNEX B: DETAILED METHODOLOGY
8 12
FOOD FOR THE HUNGRY INTERNATIONAL
FAM WORKING GROUP PEER REVIEW 
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
We would appreciate your contribution towards the Final Evaluation of our Institutional Support 
Agreement for FY1999-FY2003. The aim of this questionnaire is to capture your views about the 
role played by FHI representatives in working group activities. Thank You.
NO QUESTIONS OPTIONS ANSWER
1 Please make a cross next to the working group in Monitoring and evaluation 1
which you participate Monetization 2
2 Please indicate how many years you have been participating in this working group. 
(If less than one year, specify the number of months and write the word months, 
e.g. 10 months).
The group achieved all its 
objectives 1
The group achieved most of 
it objectives 2
The group achieved some 
of its objectives 3
3 How would you classify the effectiveness of the 
working group 
The group achieved none of 
its objectives 4
Always 1
Most of the time 2
Sometimes 3
4 How would you classify the attendance of working 
group meetings by the FHI representative(s). He/she 
attends………
Never 4
Nothing 1
Comments on work done by 
other members 2
Writing/production of some 
of the tools/materials 3
Driving force in the 
writing/production of 
tools/materials
4
Involved in the organization 
and presentation of FAM 
workshops
5
5 What kind of contributions did the FHI 
representative(s) make to working group activities? 
Other (specify)
6
Very valuable 1
Valuable 2
Of some value 3
6 What do you think of the quality of the FHI 
representatives’ contributions? The contributions have 
been…….. 
Of no value 4
7 Do you have any other comments about FHI’s 
participation in FAM working group activities that may 
be of value in the evaluation? If yes, please write 
them in the next column 
8 13
FOOD FOR THE HUNGRY INTERNATIONAL
FAM INFORMATION SYSTEMS REVIEW 
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
We would appreciate your contribution towards the Final Evaluation of our Institutional Support 
Agreement for FY1999-FY2003. The aim of this questionnaire is to capture your views about the 
role played by FHI in mentoring FAM in information systems development. Please bear the two 
distinct purposes that FHI set out to achieve in mind when thinking about these questions. 
The main purposes of the FHI support program were:
FAM will learn and become proficient at current/new information technology capabilities through 
the existing knowledge base of FHI; and the FAM consortium will receive the ultimate benefit and 
become stronger through the technical leadership of FHI and FAM. 
Thank you for your cooperation.
NO QUESTIONS OPTIONS ANSWER
Very satisfied 1
Satisfied 2
1 How would you rate your level of satisfaction with the 
work that FHI has been doing with FAM during the 
past four years in the area of information systems 
development
Not satisfied 3
2 Please give a reason for your answer in the previous question:
They responded quickly to 
assist and solve problems 
when needed
They were client orientated 
and considered our needs 
before giving advice and/or 
suggesting new 
technologies/methods
The technologies/methods 
they promoted were the 
most appropriate/best 
available at the time
3 How would you rate the following aspects (next 
column) of FHI’s support to FAM in relation to 
information systems development (please use five 
point scale below):
1- Always
2- Most of the time
3- Usually
4- Sometimes
5- Never
Their advice and training 
enabled us to continue 
working independently 
Website development and 
maintenance
Provision of FHI server 
capabilities
Training in the use of 
software e.g. NetTracker; 
Dream weaver
Hardware backstopping
Setting up and management 
of listservs.
4 When considering the overall work of FAM, how 
would you rate the importance of the FHI 
contributions in the next column (please use the five 
point scale below):
1- We could not function effectively 
without it
2- Very important
3- Important
4- Some contribution
5- No contribution
General mentoring/advice
8 14
Website development and 
maintenance
Provision of FHI server 
capabilities
Training in the use of 
software e.g. NetTracker; 
Dream weaver
Hardware backstopping
Setting up and management 
of listservs.
5 How would you rate the quality of the FHI 
contributions listed in the next column (please use the 
five point scale below): 
1- Excellent
2- Very good
3- Good
4- Poor
5- Very Poor
General mentoring/advice
6 Do you have any other comments about FHI’s contribution towards FAM’s information 
technology capabilities? 
8 15
FOOD FOR THE HUNGRY INTERNATIONAL
USAID REVIEW OF PROPOSALS, M&E and COMMODITY 
MANAGEMENT
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
We would appreciate your contribution towards the Final Evaluation of our Institutional Support 
Agreement for FY1999-FY2003. The aim of this questionnaire is to capture the views of USAID 
officials about changes that may have taken place in relation to selected aspects of FHI’s Title II 
programs during the past four years. Your responses will form the basis of a brief focus group 
discussion that will take place when the evaluation team visits your country. Thank you.
PROPOSAL WRITING 
NO QUESTIONS ASPECTS
ANSWER
(use codes as 
indicated in 
the question 
block) 
Degree to which the proposal is based on 
good problem identification and analysis
Degree to which the goals and objectives 
have been clearly written and defined
Degree to which the proposed 
interventions are appropriate to the 
identified problems and needs
Degree of fit between the objectives of the 
proposal and the strategic objectives of the 
USAID local mission
1 We would like you to evaluate the most 
recently submitted FHI Title II DAP 
proposal. Please use the three point scale 
below to rate each of the aspects listed in 
the next column. 
6- Above average
7- Average
8- Below average
Degree to which a balanced mix of impact, 
effect and output indicators have been 
included in the M&E plan
Degree to which the proposal is based on 
good problem identification and analysis
Degree to which the goals and objectives 
have been clearly written and defined
Degree to which the proposed 
interventions are appropriate to the 
identified problems and needs
Degree of fit between the objectives of the 
proposal and the strategic objectives of the 
USAID local mission
2 Please compare the most recently 
submitted DAP proposal with the previous 
one submitted by FHI in respect to the 
various aspects listed in the next column.
Use the three point scale below:
The most recent project proposal is:
 1- Better
2- The same 
3- Worse
Degree to which a balanced mix of impact, 
effect and output indicators have been 
included in the M&E plan
3 Do you have any other comments about 
the most recent FHI proposal submitted to 
your mission?
8 16
MONITORING AND EVALUATION: 
PLEASE REVIEW THE MOST RECENTLY COMPLETED OR ALMOST COMPLETED DAP 
NO QUESTIONS OPTIONS ANSWER
4 The quality of the original monitoring and evaluation 
plan was …….
1- Above average
2- Average
3- Below average
5 The execution of the monitoring and evaluation plan 
was …….
1- Above average
2- Average
3- Below average
6 The selected indicators generally gave a ………. 
reflection of program achievements 
1- Good
2- Acceptable
 3- Poor
7 The selected impact indicators gave a ……… 
reflection of program impact
1- Good
2- Acceptable
 3- Poor
8 The quality of indicator measurement was….. 1- Above average
2- Average
 3- Below average
9 The indicator information provided in annual reports 
were …………..complete and corresponded with 
commitments made in the monitoring and evaluation 
plan
1- Always
2- Mostly
3- Sometimes
4- Never
10 The original targets that were set for this program 
were generally
1- Too high
2- Realistic
3- Too low
11 The quality of the mid-term evaluation was 1- Above average 
2- Average
3- Below average
4- Not done
12 The quality of the final project evaluation was 1- Above average 
2- Average
3- Below average
4- Not done
13 Please compare the Monitoring and Evaluation plan of 
the most recently completed/almost completed DAP 
with the plan in the new DAP proposal. The quality of 
the M&E plan in the new DAP is …….. 
 1- Better
2- The same
3- Worse
14 When considering the past four years, has there been 
any change in the way FHI project management has 
dealt with USAID Title II monitoring and evaluation 
requirements? If yes, qualify the change that took 
place. 
1- No change
2- Improved a lot
3- Improved slightly 
4- Deteriorated slightly
5- Deteriorated a lot
15 Please give a reason for your answer in question 14:
8 17
MONETIZATION AND COMMODITY MANAGEMENT
 
NO QUESTIONS OPTIONS ANSWER
16 Have you noticed any changes during the past four 
years in regards to FHI’s ability to make timely and 
appropriate resource requests. If yes, please specify 
the nature of the change.
1- No change
2- Improved a lot
3- Improved slightly
4- Deteriorated slightly
5- Deteriorated a lot 
17 How would you rate FHI’s ability at present to make 
timely and appropriate resource requests…….
1- Above average
2- Average
3- Below average
18 If for example a ship arrives much later than planned, 
and there are delays in monetizing the commodities, 
have you noticed any changes during the past four 
years in regards to FHI’s ability to overcome short￾term cash flow problems. If yes, please specify the 
nature of the change.
1- No change
2- Improved a lot
3- Improved slightly 
4- Deteriorated slightly
5- Deteriorated a lot
19 How would you rate FHI’s ability to deal with short￾term cash flow problems such as in the example 
mentioned in Question 18. 
1- Above average
2- Average
 3- Below average
20 Do you have any other comments about the way FHI deals with the whole process of 
monetization and commodity management? If yes, specify.
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR USAID STAFF 
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
This in-depth discussion will build on responses given in the USAID questionnaire that 
will be sent and completed before the field visit. Ideally two USAID staff members will be 
present: the person responsible for Title II programs and the M&E staff member. The 
discussion will explore responses related to above average and below average 
performance – clarifying why FHI was rated the way it was. Comparative questions 
where responses were better or worse will also be explored further. The main reasons 
why a questionnaire will be sent out beforehand are:
· It will give the USAID guys time to think (and perhaps look at project 
documentation) before responding – thus avoiding ‘top of my head’ judgments
· It will shorten the interview time and allow us to zoom in on the essence
8 18
FOOD FOR THE HUNGRY INTERNATIONAL
QUESTIONNAIRE FOR COMMUNITY LEVEL STAFF 
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
We would appreciate your contribution towards the Final Evaluation of our Institutional Support 
Agreement for FY1999-FY2003. Your responses will be used in the development of a new 
training strategy for ISA. Thank you.
NO QUESTIONS OPTIONS ANSWER
1a Do you supervise other staff members? 0) No
 1) Yes
1b Please indicate for how long you have 
been working for FHI. 
1c Do you work in the health or agricultural 
sectors?
 0) No
 1) Yes
EDUCATIONAL MESSAGES
2 Have you ever attended training (or have 
been briefed) on educational methods?
 0) No
 1) Yes
3 If yes, who trained or briefed you? 0) Never been trained or briefed
1) My supervisor 
2) Other FHI Manager
3) Other FHI colleague
4) ISA staff member
5) Other (specify)
1
2
3
4
5
6
7
8
4 Have you used any of the following 
educational methods in your work?
1) Stories
2) Songs
3) Poems
4) Puppet shows
5) Drama
6) Cultural proverbs
7) Lesson plans
8) Development related bible studies
9) Other (specify) 9
5 If you have used these techniques, how 
often do you use these techniques
0) Have never used these techniques
1) Every time
2) Most of the time when teaching
3) About half the time
4) Sometimes when teaching
1
2
6 Please indicate whether your supervisors 
have done any of the following …………. 
1) Attended when I was training the
 community and gave me feedback on 
 how to improve
2)Gave me lesson plans and 
 showed me how to use it 
3)They developed stories or gave me
 ideas that I could use in my training 
3
8 19
QUALITY IMPROVEMENT CHECKLISTS 
 
NO QUESTIONS OPTIONS ANSWER
7 Have you ever heard of quality checklists 0- No
1- Yes
IF YOUR ANSWER TO QUESTION 7 WAS NO, GO TO THE SECTION ON BARRIER ANALYSIS
1
2
3
4
8 From whom did you hear about quality 
checklists
1) My supervisor 
2) Other FHI Manager
3) Other FHI colleague
4) ISA staff member
5) Other (specify) 5
9 Which ONE of the following options 
describe quality checklists the best?
1) Tool that is used to determine who are 
 promoted 
2) Method that is used to see what we do
 wrong
3) Tool that is used to encourage, monitor
 and improve our performance 
4) Method used by supervisors to punish
 us when we disagree 
10 Do you think the use of quality checklists 
has a positive or negative impact on staff 
member’s performance. Please explain 
your answer.
11 Has your supervisor used these checklists 
with you?
0) No
 1) Yes
12 Did your supervisor involve you in the 
development of checklists?
0) No
 1) Yes
13 Did your supervisor explain to you what is 
expected of you (in terms of your work 
performance) before he/she started using 
the checklists
0) No
1) Yes
14 If he/she has used it, how many times has 
it been used during the past two years?
Please write the number of times in the 
answer block
8 20
BARRIER ANALYSIS
15 Have you ever heard of barrier analysis 0) No
1) Yes
IF YOUR ANSWER TO QUESTION 15 WAS NO, GO TO THE SECTION ON TIPS
1
2
3
4
16 From whom did you hear about barrier 
analysis
1) My supervisor 
2) Other FHI Manager
3) Other FHI colleague
4) ISA staff member
5) Other (specify) 5
17 Which ONE of the following options 
describe barrier analysis the best?
1) Group interviews aimed at identifying 
reasons why communities do not want to 
listen to us
2) It is a research method that helps us 
identify key obstacles to achieving desired 
changes in behavior
3) It investigates various behaviors in a 
group at once in order to identify obstacles
4) It is a research method that identifies the 
barriers to change
18 Do you think barrier analysis is a tool that 
can be used to improve the quality of your 
work. Please explain your answer.
19 Have you ever done or been involved in 
barrier analysis?
0) No
1) Yes
20 Has your supervisor ever done barrier 
analysis?
0) No
1) Yes
2) I do not know
TIPS 
NO QUESTIONS OPTIONS ANSWER
21 Have you ever heard of TIPS 0) No
1) Yes
IF YOUR ANSWER TO QUESTION 21 WAS NO, GO TO THE SECTION ON POSITIVE 
DEVIANCE/HEARTH METHODOLOGY
1
2
3
4
22 From whom did you hear about TIPS 1) My supervisor 
2) Other FHI Manager
3) Other FHI colleague
4) ISA staff member
5) Other (specify) 5
8 21
TIPS (continued)
23 Which ONE of the following options 
describe the TIPS method best?
1) Through a series of interviews actual 
practices are compared with ideal practices. 
Recommendations are developed jointly 
and chosen through negotiation. 
2) It is a research method that helps us 
identify key obstacles to achieving desired 
changes in behavior – recommendations 
are then discussed with farmers
3) It investigates various behaviors in a 
group and then recommends to the group 
what should be changed
4) Research into beliefs and attitudes are 
analyzed and specific recommendations are 
presented to the respondents 
24 Have you ever used TIPS in the 
community where you work?
0) No
1) Yes
25 Do you think TIPS is a tool that can be 
used to improve the quality of your work. 
Please explain your answer.
POSITIVE DEVIANCE/HEARTH METHODOLOGY
NO QUESTIONS OPTIONS ANSWER
26 Have you ever heard of positive deviance 0) No
1) Yes
IF YOUR ANSWER TO QUESTION 26 WAS NO, GO TO THE END
1
2
3
4
27 From whom did you hear about positive 
deviance
1) My supervisor 
2) Other FHI Manager
3) Other FHI colleague
4) ISA staff member
5) Other (specify) 5
8 22
POSITIVE DEVIANCE (continued)
28 Which ONE of the following options 
describe positive deviance best?
1) It is a research method that helps us
 identify key obstacles to that hinder
 farmers/mothers to achieve desired 
 changes in behavior
2) It identifies best practices of farmers/ 
 mothers who live under the same
 conditions as their neighbors, but have
 higher yields/healthier children 
3) It investigates various behaviors of 
 farmers/mothers and then
 identify the strongest and best ones
4) Through a series of interviews with
 farmers/mothers actual practices are
 compared with ideal practices. 
 Recommendations are developed jointly 
 and chosen through negotiations. 
29 Have you ever used the positive deviance 
technique?
0) No
1) Yes
30 Do you think positive deviance is a tool 
that can be used to improve the quality of 
your work. Please explain your answer.
END
8 23
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR COMMUNITY LEVEL STAFF AND 
THEIR SUPERVISORS/COORDINATORS 
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
This open ended group discussion starts with an introduction of the findings of the 
questionnaire that was completed by the community level staff. 
For each of the main methodologies (education methods, barrier analysis, quality 
improvement checklists, TIPS and positive deviance) the group discusses the following:
· Why has it been used/not been used to the extent that it has? Try to gain a better 
understanding of the dynamics within the program. 
· Follow up on the positive and negative experiences/perceptions raised about the 
methodology. Get a better understanding of the weaknesses/strengths/problems/ 
opportunities. 
· The future in relation to these methodologies. Should it be used as is, modified or 
replaced with others? How can we increase the use/implementation? 
· Vision of community: do they know about it. To what extent has it been 
implemented? Have they seen progress in this respect? Are they consciously 
incorporating it in their work? Cover three aspects: families meetings each others 
needs; leaders solving problems in the community; churches reaching out to 
meet people’s needs. 
· Get photocopies of barrier analysis, Quality Improvement Checklists, Positive 
deviance and TIPS done by the program. 
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR NATIONAL AND PROJECT 
MANAGEMENT (only operational staff)
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
This focus group discussion will involve all key national and project management staff 
who are responsible for project implementation at the operational level.
The discussion starts with a summary of the group interview that was held with the 
community level staff and their supervisors/coordinators. The discussion will touch on 
this but also on programs design, M&E and food security related training. Also cover 
VOC.
The following areas are explored during the discussion:
· Identify who have been trained on what
· Their views on barriers/challenges towards the use of these methods
· Follow up on the positive and negative experiences/perceptions raised about the 
methodology in the community level/supervisor interview
· The future in relation to these methodologies
· The extent of implementation of the VOC philosophy. Challenges and successes.
· Comments about the nature and quality of ISA training/services received. 
8 24
· Has ISA made an impact on FHI’s ability to increase food security at field level. 
Expand.
· Do they know about the CS collaborative efforts and best practices exercise –
has it had any effect on them
· Do they know about it or have any of them consulted/used any of the following 
resources: Food security Resource Center; FAM web site, FAM listservs; Food 
forum Newsletter. 
· Implementation of the recommendations of the mid-term evaluation.
· Which other information /training needs should be built into the next support 
program
PS : Need to identify a volunteer who would look at training participant lists and indicate who are 
still in the program and whether they have a management position or not. 
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR EMERGENCY STAFF & CD
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
This interview will only be done in Kenya and Mozambique. Will include staff and 
managers involved in emergency response as well as the CD.
· How many have been trained, on what and when
· Were there any changes that took place in regards to the way they respond to 
emergencies after the training took place. If yes, what.
· Discussion on the kinds of emergencies that they have been dealing with during 
the past 8 years. Nature of problem, nature of response, length of intervention 
and transition into development (discussion of the responses to the table 
circulated before the field visits). 
· Look at M&E systems used – have they implemented changes after the M&E 
training
· Current challenges/problems in relation to emergency response
· Additional training and support needed 
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR COMMODITY MANAGEMENT
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
The commodity management inspection and interview will only be done in Bolivia. 
FHI standard inspection form/system etc. will be used. Interview takes place after the 
inspection and includes managers and key staff involved in commodity management.
The following points are explored during the discussion:
· Identify who have been trained and when
· Identify strengths and weaknesses of their implementation of the commodity 
management procedures 
· Talk about the constraints they faced in implementing these procedures
· Get their ideas on suggested changes to the system
· Additional information and training needs
8 25
FOOD FOR THE HUNGRY INTERNATIONAL
INTERVIEW SCHEDULE FOR COMMUNITY GROUPS THAT 
WERE TRAINED
FINAL EVALUATION PL 480 Title II ISA: (FY1999-FY2003)
This interview will be done with three health groups and three agricultural groups in each 
field. After participating in a training session conducted by a FHI staff member, they will 
be given a short break before this discussion starts. Suggested areas of discussion are:
· What did they like most about the training
· What did they like least
· Compare this training with other trainings that they had before:
What are the differences, similarities
· What is their normal interaction with their trainers like – do they get a chance to 
give feedback, to participate with examples etc. 
· Have they ever been involved in the development of training materials
.
8 26
ANNEX C: DETAILED FIELD VISIT SCHEDULES
8 27
BOLIVIA
Day one 8/11 (La Paz): 
· Interviews with USAID (2 hours) : 14h00-16h00 (Isabelle&Keith) 
Day two 9/11 (La Paz): 
· Commodity inspection (3 hours) (Keith & Isabelle) 
· Focus group Interview with commodity management staff (2 hours) 
(Buck, Keith & Isabelle)
Day three 10/11 (La Paz)
· Focus group interview with National and regional management (Keith, 
Isabelle & Buck)
· Collection of other outstanding information (Isabelle)
Day four 11/11 (Cochabamba): 
· A sample of 30 community based staff (all areas health and agric.) 
complete the questionnaire and their supervisors (1/2-1 hour max) 
(Isabelle)
 Analysis of their responses (1 hour) (Keith & Isabelle)
· Semi-structured group interviews with supervisors 
· Semi-structured group interviews with sample of 30 community Level 
Staff & their supervisors (2 hrs) (Isabelle, Keith & Buck)
Day five 12/11 (Cochabamba):
· Field visit: team split : three communities; attend training of two groups in 
each village; have short interview with attendees and give feedback to 
trainers. (Buck, Keith and Isabelle)
MOZAMBIQUE
Day one 14/11 (Maputo) : 
· Interviews with USAID (16h00-18h00) (Keith, Buck & Isabelle)
Day Two 15/11 (Gorongosa):
· A sample of 30 community based staff (all areas; health and agriculture) 
complete the questionnaire (1/2-1 hour max) (Isabelle)
· Analysis of their responses while emergency response
 interview takes place (1 hour) (Keith)
· Emergency response focus group interview (2 hours) (Buck and Isabelle) 
· Semi-structured group interviews with sample of 30 community based 
staff & their supervisors (2 hrs) (Isabelle, Keith and Buck)
Day Three 16/11 (Gorongosa)
· Field visit: team split : three communities; attend training of two groups in 
each village; have short interview with attendees and give feedback to 
trainers. Whole day. (Keith to Nhamatanda; Isabelle & Buck to 
Marromeu)
8 28
Day Four: 18/11 (Beira)
· Commodity management inspection (3 hours) (Buck, Isabelle & Keith)
· Focus group interview with Commodity Management team (2 hours)
 (Buck, Isabelle & Keith)
Day Five: 19/11 (Beira)
· National and regional management focus group interview /outstanding 
matters (2 to 4 hours) (Buck, Keith & Isabelle)
· Collect outstanding Information (Isabelle)
KENYA
Day one 21/11 (Nairobi): 
· Focus group interview with USAID (Keith, Isabelle & Buck) (2 hours)
· National and Regional Management team focus group interview (2-4 
hours) (Buck, Isabelle & Keith) 
· Focus group interview with Commodity Management team (2 hours)
 (Buck, Isabelle & Keith)
Day Two 22/11: (Marsabit) 
· A ll community based staff and their supervisors complete the 
questionnaire (1/2-1 hour max) (Isabelle)
· Analysis of their responses (1 hour) (Keith & Isabelle)
· Semi-structured group interview with community staff & their supervisors 
(2 hours)
Day Three 23/11: (Marsabit) 
· Field visit: team split : three communities; attend training of two groups in 
each village; have short interview with attendees and give feedback to 
trainers. Whole day. (Buck, Isabelle & Keith)
8 29
LIST OF PEOPLE CONSULTED
USAID/FFP Washington
Food Aid Officers
 Michelle Cachaper
 Catherine Brown
 Rudy Vigil
 Carolyn Hughes
 Sylvia Graves
Emergency Food Aid Officer
 Lesley Peterson
USAID Mozambique
 Sydney Bliss
 Suzanne Poland
USAID Bolivia
 Walter Sheperd
 Angel Vasquez
 Daniel Sanchez Bustamante
USAID Kenya
 George Mugo
FAM
 Trisha Schmirler
Cooperating Sponsors/FAM working group members
Anthony Koomson (CRS) 
Mugo Muita (CARE) 
Patricia Bonnard (FANTA) 
Kristin Turra (ACDI-VOCA) 
Charles Owubah (WV) 
Bob Bell (CARE) 
Sabinus Anaele (Technoserve) 
8 30
ANNEX D: ADDITIONAL TABLES
8 31
Training courses presented throughout the life of ISA
Name of Training Course Characteristic Ethiopia Kenya Mozam￾bique
Bolivia Interna￾tional
Facilitator: Evans
Date 
presented:
Sep. 1998 
ISA Orientation Workshop 
(Washington D.C.)
# Participants: 5
Presenters: Evans/
Davis
Evans/
Davis
Evans/
Davis
Evans/
Davis
Date 
presented:
May 1999 Dec. 1998 March 
1999
Feb. 1999
M&E Review Workshop
# Participants: 19 21 16 25
Presenters: Walsh
Date 
presented:
Aug. 1999
Emergency Relief Preparedness 
Workshop (Held in Kenya, but 
had reps from Ethiopia and 
Mozambique as well as Kenyans # Participants: 9
Presenters: Evans/
Davis
Date 
presented:
Sep. 1999 
Problem Analysis and Program 
Design Workshop (One 
workshop with reps sent from all 
four fields-held in Zimbabwe)
# Participants: 16
Presenters: Evans/
Davis
Evans/Mc 
Ewen/Davis
Clark/
Junior
Evans/
Davis
Date 
presented:
Sep 2000 April 2000 June 2000 Aug. 2000
Educational Messages and 
Methods Workshop
# Participants: 23 25 20 26
Presenters: Davis
Date 
presented:
May 2000
EPI Info Workshop (Held in 
Kenya in cooperation with 
World Vision)
# Participants: 14
Presenters: Fitzpatrick Walsh/Fitzp
atrick
Date 
presented:
April 2000 June 2000
Rapid Disaster Assessment 
Workshop
# Participants: 9 6
Presenters: Fitzpatrick
Date 
presented:
Aug. 2000 
(3 days)
Emergency Program Design and 
Implementation Workshop
# Participants: 9
Presenters: McCulley McCulley
Date 
presented:
June 2000 May 2000
Commodity Management 
Workshop: Part 1
# Participants: 18 15
Presenters: McCulley McCulley
Date 
presented:
July2001 June 2001
Commodity Management 
Workshop: Part 2
# Participants: 19 16
Presenters: Davis/
Evans
Davis/
Evans
Davis/
Evans
Davis/
Deines
Date 
presented:
May 2001 Feb. 2001 Nov. 2000 June 2001
Positive Deviance in Food 
Security
# Participants: 23 25 25 27
8 32
Name of Training Course Characteris
tic
Ethiopia Kenya Mozam￾bique
Bolivia Interna￾tional
Presenters: Deines/
Davis
Deines/
Moses
Deines/
Davis
Date 
presented:
July 2002 June 2002 May 2002
Barrier Analysis and TIPS
# Participants: 23 25 28
Presenters: Wright McCulley
Date 
presented:
July 2002 July 2002
Commodity Management 
Workshop: Part 3
# Participants: 13 16
Presenters: Fitzpatrick Fitzpatrick
Date 
presented:
Aug. 2002 Aug. 2002
Emergency Program M&E 
Workshop
# Participants: 7 7