IDEAL Mid-Term Evaluation: Findings
Evaluation Period: Oct. 1, 2018 – Mar. 31, 2021
By Nancy Mock
ABOUT IDEAL
IDEAL is an activity funded by the USAID Bureau for Humanitarian Assistance (BHA) that works to support the United 
States Government’s goal of improving food and nutrition security among the world’s most vulnerable households and 
communities. IDEAL addresses knowledge and capacity gaps expressed by the food and nutrition security implementing 
community to support them in the design and implementation of effective emergency and non-emergency food security 
activities. 
SUGGESTED CITATION
Mock, N. (2021). IDEAL Mid-Term Evaluation: Findings. Washington, D.C.: Implementer-led Design, Evidence,
Analysis and Learning (IDEAL) Activity.
DISCLAIMER
This mid-term evaluation is made possible by the generous support of the American people through the United States 
Agency for International Development (USAID). The contents are the responsibility of the evaluator and do not necessarily 
reflect the views of IDEAL or USAID or the United States Government.
CONTACT INFORMATION
IDEAL Activity
c/o Save the Children
899 North Capitol Street NE, Suite #900
Washington, DC 20002
www.fsnnetwork.org
info@fsnnetwork.org
IDEAL Mid-Term Evaluation: Findings
iii
Table of Contents
List of Tables and Figures ...............................................................................................................................iii
Abbreviations ....................................................................................................................................................iv
Brief Summary and Background ...................................................................................................................1
Objectives and Scope of Evaluation.............................................................................................................2
Evaluation Methodology.................................................................................................................................3
Findings ................................................................................................................................................................5
Recommendations ......................................................................................................................................... 22
Annex 1: Scope of Work for MTE............................................................................................................... 27
Annex 2: List of Key Informants Interviewed ........................................................................................ 30
Annex 3: Focus Group Discussions/Group Discussions and Observations:.................................. 31
Annex 4: IDEAL Timeline ............................................................................................................................. 32
Annex 5: IDEAL’s Theory of Change (Overview) .................................................................................. 33
Annex 6: References...................................................................................................................................... 34
Annex 7: Additional Tables and Figures .................................................................................................. 35
List of Tables and Figures
Table 1: Evidence Utilized for MTE ...................................................................................................................................3
Table 2: Key Informant Distribution According to Type of Stakeholder (n=60 completed interviews) .............4
Table 3: Basic Structure of the Qualitative KIIs..............................................................................................................4
Table 4: IDEAL Small Grants .............................................................................................................................................14
Table 5: Level of Effort......................................................................................................................................................19
Table 6: Consortium roles and rationale (Revised Proposal, August 16, 2018).....................................................35
Table 7: Partner Roles .......................................................................................................................................................36
Table 8: Consortium Selection Rationale, and Roles and Responsibilities .............................................................36
Table 9: MTE SWOT Summary ..........................................................................................................................................38
Table 10: IDEAL Revised Monitoring and Evaluation Plan, March 2021..................................................................39
Figure 1: Results of a group poll done via Mentimeter at an IDEAL all staff meeting...........................................8
Figure 2: Frequency of Participation in IDEAL Events (CRM database, events = 75, participants = 2,264 .......37
Figure 3: Logic Model Direction ......................................................................................................................................38
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
iv
Abbreviations
AA Associate Award
AAR After-action review
AOR Agreement Officer’s Representative
BHA Bureau for Humanitarian Assistance
CCP Johns Hopkins Center for Communications Programs
CFA Content Focus Area
CLA Collaborating, learning and adapting
CoP Community of Practice
CRM Customer Relationship Management
DFSA Development Food Security Activity
DRR Disaster risk reduction
FFP USAID’s Office of Food for Peace
FGD Focus group discussion
FSTT Food Security Technical Team
HDP Humanitarian, Development, Peace
HQ Headquarters
HuMEL Humanitarian Monitoring, Evaluation, and Learning
IDEAL Implementer-Led Design, Evidence, Analysis and Learning
INGO International non-governmental organization
IO International organization
IP Implementing partner
KI Key informant
KII Key informant interview
KM Knowledge management
KYC Know your client
LOA Life of Award
LOE Level of Effort
LWA Leader with Associate Award
M&E Monitoring and evaluation
MEL Monitoring, evaluation, and learning
MTE Mid-term evaluation
NGO Non-governmental organization
NRM Natural resource management
OFDA Office of Foreign Disaster Assistance
OGG Operationalizing Global Guidance
OSTP Organizational Self-Assessment and Transformation Program
P2P Peer-to-peer
PAC Program Advisory Committee
PAL Pre-Award Letter
PCS Program Cycle Support Associate Award
PIA Program Implementation Award
PREP Pipeline and Resource Estimate Proposal
R&I Refine and Implement
IDEAL Mid-Term Evaluation: Findings
v
RFA Request for Applications
RFS USAID’s Bureau for Resilience and Food Security
RFSA Resilience Food Security Activity
SBC Social and Behavioral Change
SGP Small Grants Program
SLKS Strategic learning and knowledge sharing
SMT Senior Management Team
SLCD Strategic Learning and Capacity Development
SWOT Strengths, Weaknesses, Opportunities, Threats
TAP Technical Advisory Panels
TOC Theory of Change
TOPS Technical and Operational Performance Support
USAID United States Agency for International Aid
WASH Water, Sanitation, and Hygiene
WFP World Food Program
This page is intentionally left blank.
IDEAL Mid-Term Evaluation: Findings
1
Brief Summary and Background
For many years, the legacy Office of Food for Peace (FFP) funded mechanisms to improve the capacity of 
implementing partners (IPs) to design and implement quality food security programs. The Implementer-Led 
Design, Evidence, Analysis and Learning activity (referred to in this report as "IDEAL" or "the Activity") is 
the most recent of these initiatives.
IDEAL is a Leader with Associate (LWA) Award led by Save the Children. Consortium members include 
Mercy Corps, TANGO International (TANGO), and The Kaizen Company (Kaizen). Acute Incite was brought in 
as a subcontractor to support work in adaptive management/collaborating, learning, and adapting (CLA)
starting in Y3. Other resource institutions include Johns Hopkins Center for Communications Programs
(CCP), ICF, Tufts University, and Tulane University. The IDEAL award amount is 30 million USD over the five￾year life of award (LOA), while the Associate Awards (AAs) had an initial ceiling of 65 million USD, which was 
recently revised to 104 million USD. These AAs may continue up to five years after the end date of the 
IDEAL prime award. IDEAL was a successor to The Technical and Operational Performance Support (TOPS) 
Program (also an LWA cooperative agreement). After the transition between TOPS and IDEAL, five AAs 
from the TOPS LWA continue today. Three new AAs have been developed under the IDEAL LWA, and three 
more are currently in process. This constellation of separate activities is the IDEAL immediate 
“ecosystem” managed by Save the Children.
TOPS was judged as a relevant, effective, and efficient program. It had a lean staffing structure but 
provided technical and operational support to improve the quality of primarily FFP’s 
resilience/development-focused food security activities. TOPS contributed to major improvements in FFP 
guidance documents and IP capabilities in technical areas such as nutrition, WASH (water, sanitation, and 
hygiene), agriculture/natural resource management (NRM), and commodity management. In addition, TOPS 
was instrumental in influencing Resilience Food Security Activity (RFSA) (formerly Development Food 
Security Activity [DFSA]) design approaches. For example, TOPS promoted the Theory of Change (TOC) 
and Refine and Implement (R&I) approaches to activity design, strengthened monitoring and evaluation, 
and emphasized social and behavioral change (SBC) strategies.
IDEAL was designed to pivot from the more sectoral and development activity-focused capacity support 
provided through TOPS to a broader approach described as Content Focus Areas (CFAs) in the IDEAL 
request for applications (RFA). These CFAs included: 
● Strengthening the quality and impact of emergency programming
● Data collection and analysis for improved program design and impact
● Adaptive management and application of learning
● Integrated activity design and implementation
● Sustainability and planning for exit
● Social and behavior change
● Equity, empowerment, social cohesion and social accountability
Areas of overlap between TOPS and IDEAL include monitoring and evaluation/data collection and use as 
well as social and behavioral change.
Similarly to TOPS, IDEAL’s pathways to food security activity improvement included a small grants program 
(SGP), stakeholder consultations, peer-to-peer (P2P) learning, and capacity strengthening initiatives. 
IDEAL’s structure included a Program Advisory Committee (PAC), Technical Advisory Panels (TAPs) initially 
organized around CFAs, the SGP team, regional advisors, and several functional groups:
● Strategic Learning and Capacity Development (SLCD)
● Monitoring and Evaluation (M&E)
● Collaborating, Learning, and Adapting (CLA) – organized in 2021
● Communications and Knowledge Management
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
2
● Management and Operations
● Food Security Technical Team (FSTT)
IDEAL’s major activities include training/mentoring/tools development, knowledge sharing events/P2P 
learning activities, stakeholder consultations, awarding and managing small grants, building/nurturing 
communities of practice (CoPs), monitoring and evaluation, and technical work around the CFAs.
As discussed in more detail below, IDEAL experienced several contextual and internal challenges during the 
first two-and-a-half years of its implementation. Because of these exceptional challenges, IDEAL had only 
spent one-third of its budget by the activity mid-term and had not achieved many of its initial targets.
In response to this landscape, IDEAL has adapted a great deal since its inception. This mid-term evaluation
(MTE) focuses on understanding how IDEAL is now relevant, effective, and adaptive, especially considering
its recent structure and function. The MTE is primarily a learning evaluation, focusing on what is working 
now, not working, and the implications for the final two years of programming.
This report includes the following sections: an overview of the scope and methodology of the evaluation, 
topline findings, findings specific to the evaluation questions, and recommendations for the way forward.
Data collection and analysis for this mid-term evaluation took place between May and August, 2021. The 
draft report was submitted in September and finalized in December of 2021.
Objectives and Scope of Evaluation
The purpose of this mid-term evaluation, presented in the Evaluation Scope of Work (see Annex 1), is to 
take stock of the IDEAL activities implemented to date and compare to its original intent, with an emphasis 
on recent adjustments (including the comprehensive Theory of Change (TOC) developed in Year 2 as well as 
its “pivot and propel” work related to the COVID-19 pandemic). The mid-term evaluation aims to:
● Inform activity leadership on effective efforts to date to improve food security programming,
which IDEAL can continue to catalyze; and
● Provide recommendations for the IDEAL team structure and programming to better meet the 
needs of the food security implementing community.
This mid-term evaluation has four broad objectives, listed below. Each objective has suggested lines of 
research that guided the MTE.
RELEVANCE
Assess the degree to which IDEAL's functional and organizational design enables it to meet its goal and 
purposes. Specifically:
● Are IDEAL's strategic direction and programming around the initial CFAs relevant and appropriate 
to meet BHA and IP needs in Year 4 and Year 5?
● Has IDEAL sufficiently adapted to COVID-19 and other contextual changes to date? What further 
adaptations are required to ensure the Activity remains relevant to BHA and IP stakeholders?
EFFECTIVENESS AND EFFICIENCY
Assess if IDEAL is being implemented effectively and efficiently. Identify barriers the Activity encountered 
and assess if the Activity adapted appropriately. Specifically:
● Is IDEAL being implemented according to its Theory of Change? What is working and what is not 
working?
● Are IDEAL's internal processes and ways of working effective and efficient in achieving their 
desired outputs and outcomes?
● The merger of FFP and the Office of Foreign Disaster Assistance (OFDA) into the Bureau for 
Humanitarian Assistance (BHA) and the onset of the COVID-19 pandemic changed the operating 
IDEAL Mid-Term Evaluation: Findings
3
environment for IDEAL and its stakeholders in significant ways. How effective were IDEAL's pivot 
and adaptation to these factors?
● What changes can strengthen and enhance IDEAL's performance through the remaining LOA?
● To what extent has IDEAL successfully achieved its stated objectives/outcomes and met its 
reporting targets?
● To what extent are IDEAL's Theory of Change and reporting indicators in the most recent M&E plan 
appropriate? Is the internal monitoring system sufficient to capture and report on progress 
towards the Theory of Change and indicators?
● Is IDEAL's current size and structure appropriate to meet the continuing IP and BHA needs and to 
achieve its purposes and goal? The structure includes the functional teams within IDEAL and the 
working relationships with its Associate Awards, consortium members, Program Advisory 
Committee, and Technical Advisory Panels.
EXIT STRATEGY PREPAREDNESS
Assess IDEAL's current preparedness for an exit. Specifically:
● How has the sustainability plan been put into practice? What are the necessary changes to best 
prepare the Activity for successful exit?
● Have IDEAL IP stakeholders begun to establish their own capacity to do capacity strengthening?
Evaluation Methodology
The MTE was a multi-method developmental evaluation style activity. It was also iterative and involved 
document review, analysis of secondary data, key informant interviews (KIIs), focus group discussions 
(FGDs), and group interviews and polls. A description of the evidence base can be found in Tables 1 and 2.
Table 1: Evidence Utilized for MTE
Evidence sources Relevance Effectiveness/Efficiency Sustainability
KIIs X X X
FGDs/GDs X
CRM X X X
Website analytics X X X
Administrative data X
Event evaluations X X
USAID policy and procurement documents X X
PAC meeting notes and agendas X X
TAP meeting notes and agendas X X
Activity output reports and products X X
Functional group meeting notes X X
TOPS final evaluation X X
Activity quarterly and annual reports X
M&E Plans X
COVID-19 event series reports, meeting 
recordings and evaluations
X X
Consultant reports X
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
4
Table 2: Key Informant Distribution According to Type of Stakeholder (n=60 completed interviews)
Type of Organization Represented Number of Interviews 
Represented
IPs (not IDEAL consortium members) 10
Field-based IP staff 9
BHA Agreement Officer’s Representatives (AORs) and senior staff, 
USAID’s Bureau for Resilience and Food Security (RFS) staff
8
IDEAL staff 23
IDEAL senior backstop executives 4
PAC members 17
IDEAL consultants 3
Associate Awards (Save the Children and BHA staff overseeing AAs) 10
Documents reviewed include activity documents, consultant reports, the final evaluation of TOPS, activity 
meeting notes, IDEAL outputs (including tools, reports, and recordings), and USAID policy and procurement 
documents. Secondary data includes website usage analytics, the Customer Relationship Management 
(CRM) database, SGP tracking database, and activity administrative data.
Key informants (KIs) were selected to reflect the various stakeholders of IDEAL (see Table 2). Group 
interviews included IDEAL functional teams, the PAC, the TAPs, All Staff meetings, IDEAL Senior 
Management Team (SMT) meetings, and BHA check-ins. Sixty KIs were interviewed in sessions ranging from 
30 minutes to 1.5 hours. Several KIs were interviewed multiple times for clarifications and consultations. 
The KIIs were semi-structured. All informants were asked to identify IDEAL’s successful activities that have 
made an impact on the quality of food security programming or show great promise to do so. They also 
were asked to identify weaknesses or barriers to achieving IDEAL’s mission. In-depth probing into aspects 
of IDEAL’s structure and function were asked of appropriate informants. Table 3 shows the basic structure 
of the qualitative KIIs. Questioning varied, however, depending on the knowledge and experience of 
individual informants. For example, many informants had extensive knowledge of IDEAL and its 
predecessor (TOPS), even though they may hold a position external to IDEAL. In that case, they may have 
been interviewed about the range of IDEAL components. Another example is that many of the IP staff, 
though currently in HQ positions, had recently held field project management posts. In that case, they were 
asked specific questions relative to the field needs and relevance of IDEAL work to food security staff in 
the field.
Table 3: Basic Structure of the Qualitative KIIs
Component KI PAC KI TAPS KI All Staff Ki BHA KI IPs
PAC x x x x
TAPS x x x x
SGP x x x x
KSM/CoP x x x x
IDEAL regional roles x x x x
Stakeholder engagement x x x x
Capacity strengthening x x x x
TOC x x x x
The evaluation activity manager was Lloyd Banwart, and Nancy Mock led the MTE. An IDEAL MTE Steering 
Committee consisted of Karen Romano, Luisa Angelsmith, Gitau Mbure, Lloyd Banwart, Laura Zillmer, and 
Dasha Kosmynina. The evaluator met weekly during June, July, and August with the Steering Committee to 
IDEAL Mid-Term Evaluation: Findings
5
present new findings and discuss implications for IDEAL strategy and any MTE adaptations needed. These 
regular meetings also made available rolling findings of the MTE for Year 4 work planning purposes. 
Limitations of the evaluation: Primary data for this evaluation were not collected using probability 
sampling methods. However, this evaluation used multiple sources of evidence to triangulate findings. 
Where discrepancies between data sources occurred, follow-up interviews were conducted to expand the 
sample of KIs.
A second limitation was that a single interviewer, the evaluator conducted all KIIs. The decision to use a 
single interviewer not identified as IDEAL staff or a staff of one of its consortium members was made to 
decrease respondent bias and to better ensure confidentiality of the interviews.
Analysis: This multiple method qualitative evaluation analyzed the information in an iterative way. The 
evaluator triangulated findings among the data sources and reported findings each week to the MTE 
Steering Committee. Based on input from the MTE Steering Committee, additional analysis and data 
collection led to the iteration of key findings until saturation was achieved; that is, no additional change in 
findings was achieved through new information. In the presentation of findings below, these key findings 
resulted from saturation. Where applicable, quantitative evidence from IDEAL’s monitoring systems is 
presented to illustrate findings.
Findings
Topline Findings
IDEAL EXPERIENCED A DIFFICULT BIRTH IN AN EXTREMELY DISRUPTED CONTEXT
IDEAL experienced a difficult birth in a very disrupted context. First, IDEAL was a radical departure from its 
predecessor program, TOPS. While the design of the IDEAL RFA included similar pathways to change, the 
dramatic shift away from sectoral approaches to program improvement was a dramatic shift in the vision 
for this new activity. This shift in emphasis was challenging for IDEAL and many of its stakeholders.
The LWA ecosystem was managed differently by Save the Children in IDEAL, unlike in TOPS, causing undue 
burden on IDEAL senior staff. The development of the AAs was largely undertaken by IDEAL senior staff. 
Because of the high ceiling of the AAs, this led to a drain on IDEAL staff as they struggled to manage the 
development of the AAs. In addition, key senior staff migrated to AAs, leaving gaps in technical expertise 
on the IDEAL team.
The context in which IDEAL finds itself has changed profoundly since the start of the Activity. COVID-19 
was a global disruptor that started to impact programming in early 2020. The scale and scope of impacts to 
food security programs required immediate and adaptive support from IDEAL.
USAID’s recent reorganization was another profound change in IDEAL’s environment and one that is 
continuing to evolve. Perhaps the most profound change within USAID is the merger of FFP with OFDA into 
the Bureau for Humanitarian Assistance (BHA), which officially occurred in June 2020, though the process 
of the merger began before IDEAL’s launch at the very end of 2018. These two organizations within USAID 
each had large program portfolios and strong cultures. The merger process continues to this day. While FFP 
included both emergency and non-emergency food security activities, OFDA was primarily focused on 
emergency response and non-food components of emergency relief, having a very small resource 
allocation to disaster risk reduction (DRR). The dynamics of this merger resulted in more ambiguity in 
Activity management within BHA.
The expansion of IDEAL’s stakeholders to include emergency program implementers, other sectoral actors 
within BHA, and local organizations has added to the diversity of stakeholder interests and needs. While 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
6
integrating these stakeholders is important for achieving high-impact and sustainable improvements in 
food security, this diversity also presents great challenges to IDEAL programming.
Internal changes in IDEAL leadership and senior staff also affected the trajectory of the Activity. IDEAL did 
not have a stable senior leadership team until October 2020. IDEAL’s inaugural Activity Director (Save the 
Children) stepped down in February 2020 after BHA and Save the Children senior staff concluded that she
had not succeeded in reaching a common understanding with BHA of the vision of the Activity. Other 
senior staff departures affected the Activity’s progress. IDEAL’s Strategic Learning and Capacity 
Development Advisor (The Kaizen Company) also departed early on, again a problem of fit with the 
Activity’s needs. IDEAL’s Deputy Director (Save the Children) moved to support the Program Cycle Support 
(PCS) AA, as did one of its senior technical leads (Save the Children). Notably, only one of the five key 
personnel listed in the Save the Children IDEAL proposal is still in place today (see Annex 4).
As a result, IDEAL could not reach a common vision with BHA for IDEAL during the critical first year of the 
Activity. Opportunities were missed. For example, IDEAL could not build on the successes of its 
predecessor project, and a systematic IP needs assessment did not happen. This was very unfortunate for 
IDEAL’s forward progress, and while largely resolved after the appointment of the new Save the Children
Activity Director in late April 2020, many important foundational activities (such as the CFA baseline 
capacity assessment) did not occur. The TOPS Summative Evaluation (2019) concluded that delegation and 
trust were among the three most prominent reasons TOPS was successful. IDEAL was unable to attain BHA 
trust during its early days resulting in missed opportunities to operationalize a vision and strategy.
STRENGTHS OF IDEAL
Team spirit and activity leadership: Virtually all KIIs mentioned the new leadership of IDEAL as a great 
improvement of the Activity. Staff felt that the new Activity Director had largely repaired and built a more 
productive relationship with BHA, especially at the AOR level and with direct BHA stakeholders (those 
overseeing specific aspects of the Activity). She was judged to have strong leadership qualities and an 
inclusive and highly adaptive leadership style. She created an environment for collaborative engagement. 
Continuity of organizations and some staff from TOPS: Save the Children, Mercy Corps, and TANGO were 
all consortium partners in The TOPS Program. Several staff members also transitioned from TOPS to IDEAL. 
This was viewed to have facilitated organizational learning between the two activities.
Monitoring/evaluation/data collection and use workstream: This workstream was mentioned by nearly 
all stakeholders as one the most successful in terms of relevance, effectiveness, and efficiency. Reasons for 
this include the seamless transition from TOPS to IDEAL of organizations and technical leadership, 
excellent engagement with BHA counterparts, the clear linkage between the workstream and food security 
activity staff responsibilities, and a professionally run workstream. Notably, to be discussed later, this team 
is also the leanest team given its mandate.
Knowledge management and communications workstream: This workstream also transitioned 
seamlessly from TOPS. The team is highly productive, technically experienced/trained, and has an excellent 
esprit de corps. The linkage of IDEAL to AAs through the joint use of the KM/Communications team staff 
and FSN Network website are viewed as strengths that facilitate better leveraging of AAs in service of 
IDEAL’s mandate. This team also is lean, given its workload.
COVID-19 pivot: Most stakeholders felt that IDEAL had done an exemplary job in pivoting quickly to focus 
resources on the pandemic crisis while demonstrating new (to the food security community) and existing 
approaches to adaptive management (scrum/sprint, scenario planning), collaboration (through virtual 
modalities), and capacity strengthening (application labs). IDEAL was given especially high marks for what 
was considered by many to be “best practice” in the design and implementation of virtual P2P learning 
events and meetings.
IDEAL Mid-Term Evaluation: Findings
7
However, most IPs interviewed judged that IDEAL should have accelerated the speed of its efforts. The 
first publicly facing event, offered by the monitoring and evaluation workstream, occurred on April 1, 2020. 
This event was early enough to have an impact on monitoring and evaluation in the field, especially for 
introducing remote technologies and methods for monitoring during the pandemic. However, most events 
occurred in July 2020 or later. IPs indicated that the utility of IDEAL COVID-19 workstreams would have
been greater if IDEAL had rolled out workstreams earlier. By July, IPs noted that they were bombarded by 
invitations to webinars and virtual events related to COVID-19 programming advice/tools.
Strong understanding of and connectivity to RFSAs and to nexus topics: IDEAL consortium partner 
organizations and staff brought much of the capacity and experience of the legacy TOPS Program. Senior 
consortium staff had a wealth of food security design and evaluation experience. 
Stakeholder Consultation: All sources of evidence converged on the significance of the stakeholder 
consultation that was conducted in April of 2021. This consultation engaged BHA and IPs on issues related 
to the RFSA RFA design and solicitation process. All stakeholders concluded that BHA and IP engagement 
was critical at this particular time of USAID organizational changes and when the context for humanitarian 
and development projects is changing so dramatically.
BHA voiced strong support for the value of stakeholder consultations and provided numerous examples of 
how BHA policies and guidance had been informed by these activities. IPs, though, expressed some 
skepticism about the impact of stakeholder consultations. This underlines the importance of tracking and 
disseminating changes to BHA guidance policy and management resulting from stakeholder engagement.
Knowledge sharing events: Executive and technical staff of IPs appreciated knowledge sharing events 
where P2P exchange involved expertise beyond BHA and its stakeholder IP staff to include technical 
experts with global experience. Great appreciation was expressed to include the wider network of 
expertise and experience in learning events. The Nepal Resilience Learning Event was cited as an example. 
Many respondents had high hopes for the Middle East and North Africa (MENA) regional knowledge sharing 
event that was canceled due to COVID-19.
Through the COVID-19 virtual events, IDEAL was able to better reach and activate local partners than was 
the predecessor TOPS. The careful design and implementation of these learning events led to high levels of 
participation of local staff and organizations. This has enabled IDEAL to progress on the localization 
agenda.
CRM: This data system tracks the engagement of individual stakeholders in IDEAL’s externally facing 
activities (consultations, learning events, CoPs, webinars). The CRM now consists of a substantial database 
tracking individual stakeholder contacts. It provides a valuable resource for assessing IDEAL’s stakeholder 
engagement, though its full potential is yet to be realized. The evaluator, however, was able to analyze 
engagement patterns among stakeholders. The expansion of more frequent analysis of this database can 
generate useful performance monitoring indicators. The data system currently tracks only IDEAL externally 
facing events, not including AAs and internal IDEAL activities.
Technical assistance for program design and refinement: Stakeholders unanimously appreciated the
support that IDEAL provided to develop TOCs, undertake R&I activities, and design monitoring and 
evaluation strategies.
PROMISING DIRECTIONS
Recent staffing and technical leadership improvements: While the effects of the changes are yet to be 
translated into food security capacity improvement outcomes, recent recruitments related to CFAs
(sustainability, integration, social cohesion, social accountability, social and behavioral change), and small 
grants management, were viewed by most relevant KIs as important improvements.
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
8
Developing adaptive management capacity strengthening tools: The demand for improved capacity in 
scenario planning from IPs and resulting IDEAL activities led to the development (ongoing) of an online 
course aimed at IP field staff. IPs expressed great appreciation for the support to develop scenario 
planning expertise and expressed the need for the growth of products and services that strengthen 
adaptive management.
Nascent but well-appreciated emergency program improvement workstream. KIs overwhelmingly 
judged the emerging workstreams through the Emergency Practitioners’ Forum (EP Forum) and the 
Humanitarian Monitoring, Evaluation, and Learning (HuMEL) group as important first steps towards
improving emergency programs. Informants identified programming in conflict contexts/protracted crises, 
nexus issues, and monitoring and evaluation as critical issues for improving program results.
Leveraging AAs: The immediate Activity ecosystem of IDEAL and its affiliated AAs provides a large pool of 
resources that have as a goal to improve food security programming. IDEAL and Save the Children have 
begun to strategically leverage these resources by hiring a senior staff member whose job is to manage 
AAs. Joint programming between IDEAL and AAs is beginning to be more common.
Emergence of functional CoPs: During the past year, IDEAL has established four CoPs that are beginning 
to gain traction. The strategic learning and knowledge sharing (SLKS) CoP is the most mature, focusing on 
field-based RFSA/backbone learning advisors. While small and targeted, it has developed an engagement 
platform for learning advisors of RFSAs and backbone projects. All KIs felt that mechanisms to engage 
project staff interest groups were important work for IDEAL and that CoPs/interest groups developed by 
the Activity to date were helpful. However, KIs warned that CoPs should be organic and functional as they 
have been to date. They warned to avoid the proliferation of mechanisms that are not relevant to the 
stakeholder communities. 
Revision and support of the FSN Network website: The FSN Network website is a legacy tool developed 
by TOPS. Its revision and continuation are critical to IDEAL’s knowledge management approach, though 
continued improvements will be required to ensure that FSNNetwork.org can facilitate field programming 
capacity improvements in a dynamic context.
AREAS OF CONTINUING CHALLENGE FOR IDEAL
Absence of a clear vision and logic: During the early days of the MTE, the evaluator conducted a group 
poll querying the fit for purpose of various components of IDEAL. The poll reflected staff concern that 
work still needed to be done to make IDEAL fulfill its mandate. Figure 1 summarizes the findings from the 
group poll. 
Figure 1: Results of a group poll done via Mentimeter at an IDEAL all staff meeting
IDEAL Mid-Term Evaluation: Findings
9
KIIs with staff and key stakeholders revealed that the vision and logic framework linking IDEAL activities to 
its mission were not yet sufficiently focused and clear. Staff felt that they were quite busy but “not sure to 
what end,” which reflected a general feeling that the Activity was still overly process-oriented (as opposed 
to outcome-oriented). Supporting evidence included the low level of engaged stakeholders (see below).
IDEAL’s consortium structure did not work as expected: Kaizen was not able to effectively lead the 
capacity strengthening component of the Activity to the satisfaction of IDEAL and BHA. Though Kaizen 
devoted significant effort to the development of a capacity assessment tool, insufficient engagement with 
key stakeholders resulted in a tool that was not judged useful. A mature capacity strengthening strategy 
for the content focus areas was not developed. Mercy Corps has not yet delivered the senior food security 
leadership that it is capable of providing and that is needed to ensure that substantive food security 
programming improvement strategies are developed by IDEAL.
Several of IDEAL’s activity components need to be strengthened: These include the PAC, the SGP, the 
organization of work to achieve gains in CFAs, and staffing configurations.
Monitoring and evaluation is the strongest workstream but needs more resources: Though monitoring 
and evaluation has been a strength, IDEAL needs greater emphasis on this workstream to support adaptive 
management and ensure that IDEAL works effectively towards its mandate to improve food security 
programming.
Increase prioritization of emergency programs: While IDEAL has made progress in reaching the 
emergency food security community, greater focus of resources on this stakeholder group is needed.
Integration is insufficiently incorporated into IDEAL’s conceptual/logic framework: BHA and other 
stakeholders consistently identified integrated activity design and implementation as one of the highest 
priority CFAs, citing its importance for graduating vulnerable populations from poverty and for protecting 
them from transitory food insecurity due to shocks and stresses; however, IDEAL has not yet articulated a 
strategy for strengthening this capacity among its stakeholders.
Below, a more detailed analysis of these findings is presented following the flow of the original evaluation 
questions.
Findings to the Evaluation Questions
RELEVANCE: ASSESS THE DEGREE TO WHICH IDEAL’S FUNCTIONAL AND ORGANIZATIONAL 
DESIGN ENABLES IT TO MEET ITS GOAL AND PURPOSES
Specifically: Are IDEAL's strategic direction and programming around the initial CFAs relevant and 
appropriate to meet BHA and IP needs in Year 4 and Year 5?
The CFAs did not provide a logical framework for organizing IDEAL’s work. They are a mix of programming 
types (emergencies), management strategies (integration, adaptive management, data/evidence/ learning), 
and program thematic areas (SBC, empowerment of youth, women and marginalized groups, building social 
cohesion and social accountability, sustainability). These CFAs reflected BHA’s priorities for results over the 
course of the Activity. The creation of the TAP structure was an attempt by IDEAL to systematically address 
the CFAs, though this was not successful for a variety of reasons, in large part the absence of a logic 
framework. BHA did not require a TOC as part of the Activity proposal, and the logical framework provided 
was high level, providing little guidance for the structure and organization of the Activity.
Another issue repeatedly mentioned by many stakeholders was that IDEAL never clearly defined what 
implementer-led meant in practical terms. Early misalignment between the inaugural Activity Director and 
BHA staff resulted in a lack of meaningful engagement and the creation of a mutually endorsed vision for 
IDEAL. A TOC for IDEAL was finally developed in the later part of 2020, but its organization focused heavily 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
10
on mechanisms for learning and adaptation without clear pathways to the intended results (improved food 
security programming). 
Operating in the absence of a clear vision of tangible intended results, IDEAL established TAPs that 
grouped and regrouped over time the CFAs prioritized by BHA. This approach failed to achieve results 
because:
a) The scope of themes in the CFAs were very broad;
b) TAPs were composed of consortium member staff who self-selected (not necessarily with the right 
level of expertise related to the CFA)—they excluded the broader array of expertise in the IP 
community, and they had very little level of effort (LOE) devoted to the workstream around the 
CFA;
c) TAPs did not have sufficiently strong and dedicated leadership with adequate LOE devoted to the 
CFA work;
d) TAPs did not go through a strategic planning process connecting the content focus area to results, 
a problem definition related to the content focus areas, and a plan as to how the larger IDEAL 
activity components would be mobilized in service of the identified results.
One exception was the Data collection and analysis for improved program design and impact TAP, an 
aberration for several reasons. First, this was one area of work that moved relatively seamlessly from TOPS 
to IDEAL, maintaining continuity of TANGO leadership, staff, and excellent rapport between project staff 
and BHA counterparts. The Data TAP also had clear program counterparts (both at BHA and among IPs) in 
food security activities and was, therefore, more easily engaged with their target stakeholder groups.
The identification of thematic and Activity priorities in the RFA was important and relevant but covered a 
broad spectrum of concerns that were not operationally defined, prioritized, and inserted into a logic 
framework connecting them with project resources and activities (though attempts were made). For 
example, emergency programs represent the majority of BHA food security programming. Improvement of 
the effectiveness and efficiency of this activity type is a high priority for BHA. However, IDEAL did not 
identify capacities that needed to be improved and a strategy to move forward. Rather, IDEAL went back 
and forth about whether emergency programming should have its own TAP or be integrated into the other 
TAPs, missing the point that emergency program needs/gaps needed to be identified, strategies developed 
to meet these needs, and activity resources devoted to implementing the strategies.
In its work approach, IDEAL did not sufficiently elevate the improvement of the two types of food security 
activities in its logic framework. Emergency programs and RFSAs are the two types of activities that must 
be improved on the pathway to reaching IDEAL’s goal. Improving the design and management of these two 
types of activities and better integrating them (with each other and other sectors/projects) to adaptively 
manage in a shifting landscape is a more concrete approach. In contrast to TOPS and prior BHA-funded IP 
food security capacity strengthening projects, IDEAL was charged with elevating emergency programs in its 
work. This was due both to the preponderance of BHA food security resources being spent on these 
activities (approximately 75%) and the fact that most of these activities were taking place in multi-year 
protracted crises that present special challenges to activity design and adaptive management. 
IDEAL has not yet given emergency programs the high level of focus and investment that is needed. This is 
reflected in staffing, activities, and participation data. Staffing expertise will be discussed below. Few 
IDEAL activities directly targeted emergency program themes or staff. Activity data monitoring external 
participation in IDEAL events reflect much greater participation of RFSA stakeholders. Additionally, 
emergency program field practitioners and HQ backstop staff felt that important topics, such as the 
nexus/resilience, programming in protracted crises/conflict, and third party monitoring, were not yet 
receiving sufficient prominence in IDEAL’s work plan. While Save the Children recruited a Deputy Director 
with considerable emergency program experience, IDEAL has tried unsuccessfully to recruit a senior 
emergency specialist to complement its team. 
IDEAL Mid-Term Evaluation: Findings
11
A substantive CFA, integrated activity design and implementation, is of great importance because no single 
activity will result in high impact and sustainable change. However, up until now, IDEAL does not have a 
coherent definition of integration and a plan for strengthening integration. Integration is needed across 
sectors, activities, interventions, programs, and organizations. USAID is highlighting humanitarian, 
development, peace (HDP) coherence across the triple nexus. IDEAL should more practically address this 
concern. An important opportunity for IDEAL, given its COVID-19 pivot, is strategic integration of 
emergency and non-emergency food security activities (integrating crisis modifiers, layering 
emergency/non-emergency food security activities, and appropriately sequencing humanitarian and 
developmental assistance). This is particularly important because of the economic impacts of the COVID-19 
pandemic on vulnerable populations and the fact that this work is in the manageable interests of BHA. 
Numerous sources of evidence reflect increased food insecurity among RFSA and other program 
participants and targeted populations benefiting from resilience investments. There is an immediate need
to prevent further stress on these populations benefiting from resilience investments by USAID.
While adaptive management was a purpose-level component of the TOC and is a critical requirement of 
improved food security programming, pathways to improvement were never clearly identified. This 
includes the lack of key competencies needed to adaptively manage, such as having real-time and relevant 
information about the context and program processes; scenario and contingency planning; agile 
agreement/contracting arrangements; and ability to flexibly surge/mobilize relevant expertise/experience. 
Also important for IDEAL, but not fully implemented, is the know your client (KYC) approach. Efforts to 
better understand client stakeholders and their needs are fragmented. While the CRM contains useful 
information about those who attend IDEAL events, a broader roster of the population of clients or client 
organizations that IDEAL is trying to reach is not a part of that database. This is especially problematic 
among practitioners of emergency programs. IDEAL could use the “population” of intended stakeholders to 
target communications and better monitor engagement and change over time. 
Improvements IDEAL made to address Activity weaknesses include the recruitment of senior and 
experienced leadership in the areas of SBC, Social Cohesion/Social Accountability, Integration and 
Sustainability, Adaptive Management, Small Grants management, and PAC management. 
Has IDEAL sufficiently adapted to COVID-19 and other contextual changes to date? What further 
adaptations are required to ensure the Activity remains relevant to BHA and IP stakeholders?
IDEAL faced enormous contextual challenges, as discussed above. Due to improved engagement between 
Save the Children and BHA, IDEAL was able to quickly pivot to provide support to IPs who were struggling 
with the new field realities resulting from the COVID-19 pandemic. IDEAL developed and implemented a 
strategic approach of sequenced activities identified through a process of consultations, webinars, tools 
development/dissemination, establishment of knowledge sharing workstreams, and mentored capacity
strengthening strategies. Particularly in monitoring, evaluation, and learning (MEL), this integrated and 
strategic approach bore results. The MEL workstream sustained the highest level of engagement of the 
five COVID-19 learning streams (as reflected by CRM data, KIIs, and as yet anecdotal information about 
impacts—the first three events had participation levels of 569, 239, and 167 participants, respectively). 
IDEAL developed a logical and sequenced approach, through its P2P learning events and applications labs, 
resulting in tangible impacts on monitoring and targeting activities in the field. The M&E workstream is 
used to illustrate a well-articulated approach to IP engagement and follow-up with a set of sequenced 
activities that lead to tangible changes in specific areas of program practice. 
IDEAL exerted successful leadership to support IPs in a meaningful way by: 
1. Early engagement with stakeholders to identify priority needs,
2. Following through with defined workstreams, especially P2P learning activities, development of 
guidance/tools for data collection and use, and hands-on support through its application lab, and
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
12
3. Leveraging IDEAL’s Operationalization of Global Guidance (OGG) workstream to access expertise 
outside the BHA immediate stakeholder community. 
This experience illustrated how IDEAL could structure its work to support adaptive management in the 
field. However, IP stakeholders felt they would have benefited more concretely if IDEAL had responded 
more rapidly to COVID-19. For example, the monitoring and evaluation workstream was the first to launch 
events (April 1, 2020), and its interventions were recognized by IP KIs and participant evaluation data to 
have helped them adapt MEL approaches in response to the pandemic. Other workstreams launched in May 
and June. The OGG consultation occurred in July due to delays in approval. IP stakeholders felt that by late 
spring, they were overwhelmed by webinars and events related to coping with COVID-19. 
The second contextual change, the merging of former FFP and OFDA and other USAID reorganizations, 
however, was an enormous contextual change not adequately taken into account by IDEAL or BHA. The 
various reorganizations are among the most significant in USAID history. This context presented several
challenges and opportunities for IDEAL. 
All stakeholder groups lament the impact of the merger on bureaucratic processes and the resulting 
timeliness and relevance of procurement processes. Approval processes are opaque and protracted (three 
times longer than before the merger according to several BHA informants), confusion exists about the 
legal use of different funding streams for project activities, and BHA colleagues are under great stress 
(BHA has approximately 50% staff vacancy). 
IDEAL has not responded sufficiently to these challenges. Both the majority of BHA staff interviews and IP 
KIs indicated that stakeholder engagement should be prioritized as a part of IDEAL’s agenda to become 
helpers in building new connective tissue between a rapidly evolving BHA (and USAID) and the IP 
community. KIIs also felt that mainstreaming stakeholder engagement should be IDEAL’s strategy as 
opposed to relying on one-off stakeholder consultations. IDEAL’s engagement with InterAction, whose 
mandate includes policy-level engagement with BHA, was also identified as a priority.
These two major contextual changes also offer opportunities for IDEAL to accelerate and scale its impact. 
COVID-19 will result in more than two billion dollars’ worth of additional supplemental funding to be 
managed and released by BHA. IDEAL may be able to assist BHA and IPs to improve the impact and 
sustainability of outcomes from the use of these funds, in part based upon IDEAL’s desk review and 
evaluation of first and second rounds of COVID-19 supplemental funding. The merger offers potential 
openings to enhance an important BHA priority, integration of activity design and implementation. IDEAL 
can leverage a greater impact of activities by bringing together food assistance and other emergency 
program sectors. Most immediately, IDEAL can facilitate improved layering of emergency response into 
longer-term development of resilience capacities. 
A final “contextual change” relates to the great interest and increased proliferation of AAs. The 
repercussions for IDEAL work are noted above. Many informants also felt that much of the technical work 
migrated out of IDEAL to the AAs, questioning IDEAL’s continued compelling role in supporting food 
security program improvement. On the other hand, strengthened linkages between and among the IDEAL 
AAs were felt to be an enormous opportunity for improved food security program impact.
EFFECTIVENESS AND EFFICIENCY: ASSESS IF IDEAL IS BEING IMPLEMENTED EFFECTIVELY AND
EFFICIENTLY. IDENTIFY BARRIERS THE ACTIVITY ENCOUNTERED AND ASSESS IF THE 
ACTIVITY ADAPTED APPROPRIATELY.
Specifically: Is IDEAL being implemented according to its Theory of Change? What is working and what is 
not working?
IDEAL’s TOC was not yet one year old when the evaluation was conducted. IDEAL activities are consistent 
with the TOC, which is broad and process-oriented, providing scope for a proliferation of activities and 
forums. This is exactly what has occurred. IDEAL’s charge to be implementer-led (many implementers with 
IDEAL Mid-Term Evaluation: Findings
13
differing agendas) and its three broad purposes do not provide a framework for prioritizing activities 
towards high impact improvements of Activity quality. In fact, the TOC easily accommodates all activities 
conducted by IDEAL regardless of their potential impact.
The three implied intermediate outcomes include improved design and management of RFSAs, emergency 
programs, and integration of these activities to adaptively manage their programming landscape. However, 
the pathways to achieve these outcomes are not explicitly mapped out in the TOC. A systematic
assessment of stakeholder capacity gaps and priorities related to the CFAs did not occur. 
Though the initial project design included an assessment (Organizational Self-Assessment and 
Transformation Program—OSTP) to be led by Kaizen, this did not happen, and an alternative assessment 
was not developed. A systematic assessment of IP practice, gaps, and priorities is needed to guide IDEAL’s 
agenda. Identifying and prioritizing strategic pathways to change is critical to focusing IDEAL’s agenda and 
thus ensuring sustained progress towards its mandate.
On the other hand, IDEAL is very active, with many activities (79 externally facing events or CoP meetings 
since the start of IDEAL recorded in the CRM database), all of which fit within its current TOC. The problem,
though, is the focus on prioritizing key improvements in food security programming and then using its 
pathways of capacity strengthening, P2P learning, stakeholder engagement, and small grants to achieve 
these.
IDEAL’s inputs include tools/training, knowledge sharing events, small grants, knowledge management 
platforms, regional coordination activities, and the establishment/maintenance of peer-to-peer learning 
forums (CoPs). To this should be added leveraging Associates Awards. Leveraging AAs is particularly 
important given the magnitude of resources flowing through the AAs.
To date, IDEAL’s most visible activities, in terms of reach, are knowledge sharing events. These registered 
the greatest number of stakeholders and are mentioned most widely by informants as recognizable 
contributions to learning for program improvement. However, these are resource-intensive activities and 
have limited tangible impact on program quality.
In terms of activities that will have or have potential impact on program improvement, IPs at the HQ level 
and a number of BHA staff identified the stakeholder consultation that took place in April as particularly 
useful, especially in this dynamic context of change. Some informants referred to the merger as the 
“elephant in the room.” The merger was cited by KIIs as a setback for the field relevance of the 
procurement process and content of food security activities. Because of this, the stakeholder consultation 
in April surfaced many of the concerns of IPs, and BHA staff also felt that the consultation was useful for 
revising guidance and procurement. Stakeholder engagement also was viewed to be particularly important 
to facilitate adaptive management and integration.
Both IP HQ and field staff also mentioned skills gaps related to a number of program design and 
implementation competencies. Scenario planning, targeting, TOC development, conducting R&I phases of 
activities, developing Learning Agendas/Learning Plans, and MEL competencies were among those most 
often cited. In terms of best practice identification and sharing, HDP nexus/programming in protracted 
crises (including how to improve social cohesion, social accountability) is a high priority among all 
stakeholders. RFSA HQ and field staff also cited sustainability model approaches to be important. But 
importantly, priority competencies cited varied by both respondent and context, again begging the need 
for a systematic needs assessment by IDEAL.
Cited competency needs currently are being addressed by IDEAL to varying degrees. IDEAL has prioritized 
scenario planning. In December 2020, IDEAL conducted a popular webinar on scenario planning and 
launched the development of the online course and a capacity support strategy. TOC capacity 
strengthening has largely migrated to the PCS AA, though a gap remains in capturing and analyzing lessons 
learned and best practices across activities (meta-review). The TOC office hours activity was popular among 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
14
IPs, particularly in the appropriate time of the procurement cycle. Office hours reach a small number of 
participants and have both scalability and sustainability issues. As part of the COVID-19 workstream, the 
targeting application lab was conducted; however, to date, data on the impact of this workstream is 
lacking. Improved capacity to undertake meaningful R&I remains an area of felt need among IPs. IDEAL has 
teamed up with PCS to support regional culmination workshops and will conduct a joint stakeholder 
consultation next year. Again, however, concrete impact data on R&I competencies is not available.
The MEL team’s workstream stresses capacity strengthening work. This includes support to R&I culmination 
activities and the COVID-19 MEL workstream that builds capacities in targeting and remote monitoring and 
qualitative data collection. Qualitative data collection is being developed into an online course and will 
include other mentoring support. Course evaluations for these activities were very positive though 
concrete impact data is not yet available.
Other IDEAL interventions were judged to have high potential, but have not been implemented well, such 
as the SGP. The SGP was identified to be of great interest among IPs outside the consortium. Thus, it has a 
very high potential for increasing IP engagement and learning around the CFAs. IP KIIs and the 
overwhelming IP response to the calls for application (the last round was an exception) are evidence of the 
potential importance placed on this component by IPs. Table 4 shows that through four distinct grant 
rounds, 351 grant applications were submitted. 
IPs were enthusiastic about the small grants despite what was reported to be an agonizing grant 
management process for IPs, IDEAL staff, and BHA. To date, IDEAL has made 24 awards. Table 4 illustrates 
that the time between submission of grants and grant awards was variable but lengthy in most cases. The 
first round of Micro Grants proceeded most rapidly, typically requiring five to seven months between 
submission of proposals and award signatures. Round two of the Micro Grants and the Program 
Implementation Award (PIA) proposal review and award process typically lasted in excess of one year (after 
proposal submission). If the RFA approval process is included, these grant cycles are at or over 1.5 years.
Table 4: IDEAL Small Grants
RFA # of 
months for 
RFA 
approval 
(date 
IDEAL sent 
to BHA and 
approval 
received)
Date RFA 
Released
Date RFA
Closed
Number 
of 
Appli￾cations
Number 
of Grants 
Awarded
Date First 
Proposal in 
Round Funded
(BHA 
Approval 
Received)
Date of Last 
Proposal in 
Round 
Funded 
(BHA 
Approval 
Received)
Time to 
signature 
for first 
award 
approved
(from 
approval)
Time to 
signature
for last
award
awarded
(from 
approval)
Average 
Number 
of Issues 
Letters
Micro Grant
(Rd 1)
1.5
months
7/2/2019 8/30/2019 101 13 12/18/2019 2/28/2020 1.5 months 1
month
3
Micro Grant
(Rd 2)
7/2/2019 11/1/2019 97 8/13/2020 2/9/2021 3
Program 
Improvement 
Award (Rd 1)
1.5
months
9/23/2019 10/25/2019 34 7 8/14/2020 3/23/2021 1 month 1.5 
months
2
Program 
Improvement 
Award (Rd 2)
9/23/2019 12/13/2019 54 8/14/2020 2/9/2021 2
Applied 
Research Award
6
months
8/17/2020 10/16/2020 57 3 3/10/2021 6/2/2021 2.5 months 2.5 
months
2
Qualitative M&E 
Program 
Improvement 
Award
4
months
1/25/2021 3/5/2021 8 n/a n/a n/a n/a n/a n/a
IDEAL Mid-Term Evaluation: Findings
15
The excellent grant tracking system set up by IDEAL’s small grant manager tracks the process of grant 
management, illustrating well the bottlenecks in the process. For example, for the PIAs, the average 
lengths of time were as follows: screening the concept notes took 31.5 days; scoring 15.5 days; writing 
invitation letters to submit full proposals, 7.5 days; full application development 34.5 days; application 
screening, 7.5 days; internal technical review 32 days; development of IDEAL issues letter 27 days; response 
to the issues letter 35.5 days; issues letter response review, 40 days; award package preparation, 10 days; 
USAID initial review, 32 days; USAID issue identification, 2.5 days; issue letter response 10 days; USAID final 
review 18.5 days; agreement preparation 37.5 days.
Also important to consider is that a large number of grant applications were not relevant to the call for
proposals, so those requiring technical review could have been greatly reduced in number. 
More recently, even the creation of Requests for Applications (RFAs) is time consuming. Completion of the 
RFA for the Applied Research Award and qualitative monitoring and evaluation PIA took 6 and 4 months,
respectively. In addition, two additional grant rounds were anticipated in Year 3, though neither has or will 
be conducted.
There are many management challenges with the grant program that can be traced back to inadequate 
grant program design; insufficient staffing and shortcomings in the selection of application reviewers; 
differing perspectives between IDEAL and BHA on the expectations from the SGP; and lack of 
understanding of BHA’s review and approval process by IDEAL SGP managers. The root causes of this 
problem are numerous. On Save the Children’s side, the SGP did not receive the resource focus and 
leadership it needed (technical). BHA and IDEAL did not have a common understanding of the basic 
purpose of the SGP. While IDEAL viewed the SGP as a mechanism for IP engagement and capacity 
strengthening, USAID had high expectations for the direct impact of grant funded activities on food 
security programming and on the quality of grants, in part, because the SGP was identified in the grant 
agreement as an area of “substantial involvement.” During the review process, IDEAL relied heavily on 
review committees drawing from internal IDEAL staff, often members of the TAPs. Those selected often 
did not have the expertise needed to review the grant application. As a result, grants applications were 
frequently forwarded to BHA with deficiencies in substance. As a consequence, BHA received grant 
applications from IDEAL that it judged to be of low quality or to be misaligned with BHA intent. This led to 
greater scrutiny and involvement of BHA in the review process resulting in a longer process.
The SGP vision was not developed, so the grant program structure was never clear. It was unclear why a mix 
of Micro Grants, PIAs, and the Applied Research Award were used and why they were sequenced in that 
order. Meaningful applied research might take more time than other types of grant activities, yet it was 
among the later grant rounds.
From recent after-action reviews (AARs) by BHA and IDEAL and KIIs, streamlining the SGP management is 
necessary. This includes earlier emphasis on reaching agreement with BHA about priority grant themes and 
focus; streamlining the solicitation process; upgrading the quality and speed of IDEAL application review 
(including especially screening) and reviewing the sequencing of activities so that the process does not 
encounter unnecessary bottlenecks.
In addition, the SGP also does not have a clear structure for post-award support to small grants. This is 
essential to ensure they achieve their “learning” objectives for the BHA food security programming 
community and so that dissemination and learning activities are rolled out in a meaningful way. Because of 
this, opportunities to improve the grant performance and dissemination of results may be jeopardized.
In recognition of these problems, Save the Children took a major step to improve its SGP management by 
hiring an Associate Director for SGP. Having this level of senior focus on the SGP should permit many of the 
shortcomings to be addressed if Save the Children also ensures that the Associate Director is adequately
supported. Support staff will be required to monitor grant compliance; provide periodic and punctual 
check-ins to review grant quality and any required adaptations; and to prepare for production and 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
16
dissemination activities. Given the Associate Director’s planned absence for the period of October 2021 to 
April 2022, Save the Children will need to ensure continued leadership of the SGP. Now that grant projects 
are beginning to close, it is particularly important to provide adequate support.
This MTE concludes that the SGP is an important component of IDEAL and it should continue to be 
supported. However, priority emphasis should be given to post-award support of grants already awarded or 
in existing proposal pipelines. Though curtailing new calls for applications is a missed opportunity for 
IDEAL, current constraints, including IDEAL’s remaining LOA as well as senior staff family leave, suggest 
this to be the correct course of action.
Specifically, are IDEAL's internal processes and ways of working effective and efficient in achieving 
their desired outputs and outcomes?
All stakeholders share the concern that IDEAL must reach a better balance between process and results, 
where process is a means to results. The MTE planning matrix identified 27 different regular meetings held 
at various frequencies that staff attend. MTE observation, combined with review of notes and KIIs 
converged on the finding that the processes were not efficient nor guided by a clear understanding of how 
they were linked to results. Staff are appreciative of more recent and strong IDEAL leadership, however
they did not feel that the legacy emphasis on process over results has yet been overcome. This has led to 
morale problems among IDEAL staff despite the supportive atmosphere created by current leadership. 
Nearly all stakeholders consulted (across stakeholder groups) identified the lack of a clear vision linking 
IDEAL activities to results as the major barrier to a successful Activity; as one IDEAL staff noted, “We are 
working hard but to what end?”.
To what extent has IDEAL successfully achieved its stated objectives/outcomes and met its reporting 
targets?
Because of internal changes within IDEAL and dramatic changes in context, IDEAL’s reporting targets have 
changed and many of the indicator targets were never established in the first place. For example, there are 
no targets for 13 of 27 indicators in the second annual results report. No data is available for six key 
indicators and only two indicators reported achieving or surpassing targets: the number of organizations 
participating in IDEAL events and the number trained (interpreted liberally) in content focus skill areas. 
However, in March of 2021, the performance monitoring plan underwent another revision, slimming down 
the indicators to 21 and focusing on self-reported Kirkpatrick Level 3 and 4 indicators (self-reported 
capacity improvement and reported use of IDEAL information tools, changes in connectivity among 
community members), which is an improvement in the M&E plan. However, indicators are still relatively 
general self-reported improvements. An instrument that collects concrete information on specific 
behavioral change, for example in the quality of programming in CFAs, IP capacities in the CFAs, actual 
changes in individual and organizational practices (targeting, monitoring and evaluation) is needed. 
However, this level of specificity requires a greater Activity focus on specific capacities to improve. Until 
IDEAL’s program focus is sharpened, and because of program delays caused by contextual and internal 
factors, discussion of progress against targets may be premature.
To what extent are IDEAL's Theory of Change and reporting indicators in the most recent M&E plan 
appropriate? Is the internal monitoring system sufficient to capture and report on progress towards the 
Theory of Change and indicators?
As discussed earlier, the TOC may not be appropriate for reaching the Activity goal and purposes. The 
purposes and sub-purposes are heavily process-oriented (see Annex 5) challenging a performance 
measurement framework that logically traces improved food security activities. While the performance 
measurement framework appropriately adopted Kirkpatrick’s Model for measuring organizational capacity 
change, the TOC does not identify which capacities should be prioritized to make the measurement 
framework useful beyond the basic levels of Kirkpatrick’s scale focused on Levels 1 and 2 (appreciation of 
capacity strengthening events and self-reported changes in knowledge). Levels 3 and 4 (individual and 
IDEAL Mid-Term Evaluation: Findings
17
organizational changes) are only measured by general questions reflecting the respondents’ self￾assessment of any changes in individual or organizational behaviors influenced by participating in a specific 
IDEAL activity or activity stream. While appropriate to the TOC, it is difficult to know what changes in 
programming have actually occurred.
Another deficiency in the TOC that spills over to measurement and metrics is around the development of 
robust collaborative engagement, such as network metrics (how much and why IDEAL stakeholders 
interact, what types of joint activities they undertake, how much their networks grow), and engagement 
metrics capturing how workstreams lead to individual/organizational change (for example, the number and 
percent of participants who participate consistently in workstreams; the percent and number of 
participants in a workstream that actually improve targeting methods).
Sustained engagement of IDEAL stakeholders remains a challenge for IDEAL. An analysis of the CRM 
database shows that only nine individuals attended 10 or more of IDEAL’s outward-facing events. Among 
these nine, four were USAID staff, eight were US-based, and only two were Strategic Learning Advisors of 
RFSAs. The majority of CRM activity attendees had only attended one of the 75 events (78%). The program 
affiliation for the majority of event attendees is unknown, due to incomplete event registration data. 
Participants predominantly attended COVID-related events. Thirty-six participants were affiliated with 
learning mechanisms/backbones. Sixty-seven percent of these attended only one event. Two hundred and 
seventy nine unique participants from RFSAs attended one or more of the 75 events. Of these, 42%
attended only one event, 42% attended two to five events, and only 16% (44 participants) attended more 
than five events. Emergency practitioners were particularly under-represented. Only 95 attended any 
event, 61% of these attended only a single event.
The CRM is an excellent tool for monitoring external engagement. However, no similar system exists for 
measuring and monitoring IDEAL internal workings. Such a tool would probably flag problems such as the 
burden of meetings shouldered by staff members; PAC participation and gaps; staff satisfaction and 
challenges. 
Is IDEAL's current size and structure appropriate to meet the continuing IP and BHA needs and to 
achieve its purposes and goal? 
Structure includes the functional teams within IDEAL as well as the working relationships with its Associate 
Awards, consortium members, Program Advisory Committee, and Technical Advisory Panels.
The size and structure of IDEAL is not appropriate to achieving its intended results, though incremental 
progress has been made towards developing a results-oriented project. The central problem, as discussed 
above, is the lack of a clear vision guiding the TOC and the attendant activities and intermediate outcomes 
that lead to these results.
The day-to-day work of IDEAL is to be undertaken by functional teams and the TAPs. The functional teams 
include Knowledge Management/Communications, Strategic Learning and Capacity Development, 
Monitoring and Evaluation, CLA, Management and Operations, and the Food Security Technical Team. The 
TAPs include Integration and Sustainability, Social Cohesion and Social Accountability, Social Behavioral 
Change, Data, and Adaptive Management. The themes of the TAPs have been regrouped over the course of 
the IDEAL Activity. Stakeholders questioned the wisdom of grouping integration and sustainability in to 
one work group. 
Because program content resides in the TAPs, which have not made much progress, the functional teams 
are often working largely in a vacuum. They are able to identify and design processes but without enough 
of the “to what end” content that is needed. The functional teams also were designed, staffed, and set up 
to improve important processes such as adaptive management, CLA, managing CoPs, capacity 
strengthening, and M&E. They have set up, with the exception of capacity strengthening, solid approaches 
and tools. A major limitation, however, is that until recently, most of the functional teams did not have 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
18
sufficient resident experience/expertise in food security programming, which made them dependent on 
TAPs for content. This is in part a result of the original design of the consortium where content and 
function were divided around organizational lines, Mercy Corps: food security and resilience; TANGO: 
monitoring and evaluation; Save the Children: overall management and coordination, and contribution to 
limited technical areas; Kaizen: capacity and organizational development, strategic learning, knowledge 
management and managing the FSN Network).
The consortium arrangement has a number of deficiencies. The roles and responsibilities of different 
consortium partners were not clear from the start, have changed over time, and still result in uneven 
distribution of work among the consortium members. Consortium member responsibilities shifted over 
time. For example, while in the proposal Kaizen had responsibility for knowledge management and the 
management of the FSN Network, these were shifted to Save the Children prior to startup, based on 
implementation experience under the TOPS Bridge award. Capacity strengthening was originally a role 
tasked to Kaizen but is now less clearly linked to a single institution (appropriately). But importantly, 
Kaizen has yet to demonstrate its ability to lead the capacity strengthening activities.
Until last year, consortium members did not have clear scopes of work written into their agreements. 
Rather, the sub-awards refer to the Save the Children proposal-based award document, which outlined 
roles in general terms. This has led to a dysfunctional consortium structure where Mercy Corps implicitly 
leads food security (an area of what should be cross-cutting competence), while other consortium members 
have leadership over functional areas of the Activity. This structure has not facilitated achievement of 
results because Mercy Corps is responsible for the lion’s share of content development through the TAPs. 
Mercy Corps has not engaged its capable senior leadership sufficiently to lead and better rationalize 
substantive strategies to move along quality improvements in food security programming. Only the TANGO 
role, a legacy of The TOPS Program, is clear and its boundaries understood by Activity staff.
The staffing structure itself emphasizes process over content and LOE is fragmented or low in some 
technical areas (see Table 5). Of Save the Children’s 11 full-time or close to full-time positions, only three 
relate substantially to program content areas though these are senior positions with many other duties. 
Save the Children only leads one CFA, SBC, with a 20% lead and 30% co-lead LOE. However, Save the 
Children is clearly in a position to lead the emergency program workstream, which also would be logical, 
given the Deputy Director’s qualifications in the emergency programming area. Mercy Corps has six full￾time and one 50% technical staff devoted to food security program-related work (two of these are new), it 
has an additional three advisors at the 25% level, and 12 staff supported at 10% or less. It i s unclear how 
this LOE is distributed to support food security substantive work.
The delegation of roles to support the CFAs is fragmented. For example, about 70% LOE to support 
Integration and Sustainability, a key area of work, is split among four staff. TANGO and Kaizen have similar 
levels of LOE, though the workloads of these two organizations appear to be quite different. TANGO has 
only two full-time staff and two at 50% LOE, 1 at 30% and 1 at 20%, though it also includes small amounts 
of LOE for 10 staff that support CFAs. Given the importance of the monitoring and evaluation work to 
IDEAL, together with the important external support to the data and analysis CFA, the level of dedicated 
LOE is quite low. Kaizen is charged with developing the capacity strengthening plan for IDEAL with its four
full-time staff even though the capacity strengthening aspects of work are also supported by Save the 
Children and Acute Incite, a temporary subcontractor. Acute Incite provides 2.5 full-time staff total, 
supporting the adaptive management workstream and contributing to capacity strengthening and CLA.
The relatively low levels across the team of food security expertise and senior-level staff with the ability to 
lead workstreams creates inefficiencies that have a negative effect on IDEAL’s work, with staff spending 
disproportionate amounts of time trying to find their way forward on designing and executing work plan 
activities. This deficiency has especially affected the ability of IDEAL to successfully address capacity 
strengthening in the CFAs. Given Mercy Corps’ large role in developing the substantive components of the 
Activity, lack of engagement of Mercy Corps senior leadership in the Activity also has contributed to the 
IDEAL Mid-Term Evaluation: Findings
19
limited progress in developing the CFAs. Also problematic has been the limited relevant capacity of Kaizen
to lead in the important areas of capacity strengthening and strategic learning. 
Another irregularity in staffing is the allocation of the regional positions to two different consortium 
members. Regional staff communicated and the MTE lead agrees that scopes of work and reporting 
structures differed and that advisors should be hired and report through the Save the Children agreement.
IDEAL has made strides to strengthen the food security expertise of the team and allocate increased LOE 
to TANGO. The recruitment of three senior specialists to lead CFA content areas are examples. Two new 
TANGO positions were approved prior to the MTE. Through hiring of a Deputy Director, Save the Children
has a more substantive involvement in emergency programming, though this position has a very broad 
mandate.
Table 5: Level of Effort
LOE Description* Save the Children Mercy 
Corps
TANGO Kaizen Acute 
Incite
Total
Total LOE (%) 1,415 880(+35) 425 405 250(+25) 3,375 
(33.75 
full-time)
Total number of 
staff listed
19
# less than 20%(3)
20
(8)
16
(10)
5
(0)
4
(0)
64
21
Full-time staff 11 7 (2 new) 2 4 0 24
Staff LOE with food 
security technical/
program focus
450 880 425 100 0 1,855
# Full-time staff 
with food security 
program 
experience
4 (all but 1 with
substantial 
administrative 
responsibilities—Karen
Romano, Jenny Haddle, 
Suzanne Ammari, 
Beatrice Scheuermann)
7 2 1 0 14
*These LOE levels are accurate as of the end of July 2021
IDEAL has not yet leveraged the expertise/experience of its resource partners. The IDEAL university-based 
resource partners might have been particularly helpful in supporting analyses related to the CFAs, bringing 
emerging evidence from scientific literature to knowledge sharing events, adding an additional perspective 
on the PAC and assisting with the design and implementation of the SGP. Particularly in the case of Tufts 
University, its on-going work on sustainability should be leveraged by IDEAL.
The Program Advisory Committee is an important component of IDEAL in the sense that it provides a 
consistent venue for engagement among BHA, IPs, and IDEAL. In the words of some KIs, “this is where 
implementer-led” is supposed to be realized. Though implementer-led is a more mainstreamed 
consideration, the PAC is an important platform that regularly brings together BHA and implementers 
outside of the consortium. PAC members see the forum as one mechanism for inclusion of non-consortium 
members (in addition to the SGP). Although an early poll conducted by the MTE revealed that PAC 
members did not view PAC participation to be useful for their work, most believed (KIIs and MTE poll) that 
the PAC was important to the success of IDEAL. Many PAC members had a limited understanding of IDEAL 
though.
PAC members voiced many concerns about the structure and functioning of the PAC. The PAC has 27 
“official” members and another 10 ad hoc members. Official members represent 12 organizations, only 
seven of which are outside the IDEAL consortium or USAID. These seven organizations are all large 
international NGOs (INGOs). The number of representatives from each organization varies. One NGO has 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
20
four representatives and only three have a single slot. Membership gaps include a wider range of INGOs, 
local IPs, InterAction, regional knowledge backbone organizations, and academics. Engagement of an 
important member, World Food Program (WFP), is weak.
Attendance of PAC quarterly meetings reflects low commitment to the PAC. Only 10 of 27 members missed 
one or less of the last five meetings. Meeting attendance fluctuated between 11 and 20 regular members, 
but the trend has been downward. This seems in part due to the perception by PAC members that the PAC 
is primarily a venue to share with IPs what IDEAL is doing as opposed to a consultative or advisory body.
IDEAL has taken steps to improve the functioning of the PAC by designating a PAC lead among IDEAL staff 
and increasingly engaging the PAC to provide more systematic input to IDEAL strategies. Most PAC KIs 
indicated that the PAC functionality had improved since April 2020 when they were more systematically 
engaged by IDEAL in its COVID-19 strategy development, including on the identification of priorities for 
the pivot and the learning streams. The May 2021 PAC that led to the decision by IDEAL to reject a PAC 
regional structure is another example.
While the PAC is an important mechanism for engagement and consultation, IDEAL could benefit from 
regular strategic support to its executive team. This type of strategic support can best be provided by a 
smaller and senior team more familiar with IDEAL workings. IDEAL can better leverage senior expertise 
among its consortium members and a small number of senior external advisors (no more than six total
members). Their engagement might be linked to the pause and reflect mechanism of the Activity.
The Associate Awards comprise the bulk of financial resources among the components of the LWA 
ecosystem. The number of awards has proliferated, causing concern on the part of senior leadership at 
IDEAL and BHA about management inefficiencies, fragmentation of effort, and the loss of IDEAL’s technical 
roles. Some have mentioned that AAs have sucked the substance from IDEAL. Given the global impact of 
COVID-19, the high ceiling for AAs, the diversity of funding sources available to address food insecurity and 
the desire of some USAID staff to control resources, the pressure to increase the number of AAs is likely to 
continue.
Harnessing AAs in service of IDEAL’s goal is inherently challenging with the LWA structure, however, a 
worthwhile strategy for Save the Children due to the size of potential available resources. Save the 
Children did not adequately leverage these resources at the start of the Activity. In fact, IDEAL senior staff 
were overburdened with the development of the AAs. This situation has been partially rectified. The most 
important step was to hire a full-time and highly experienced (FFP, IP, and private sector) manager of AAs. 
The new manager developed a number of strategies to complement Save the Children’s earlier efforts to 
increase overlap of staff on activities and to promote the use of a common knowledge management 
platform. A lesson learned is that development and management of AAs should have been adequately 
staffed from the start of the Activity. 
Finally, the FSN Network knowledge management web platform provides continuity of institutional 
knowledge and cements connectivity among IDEAL and AAs. The website was modernized recently and has 
a moderate level of traffic though its users are U.S. centric and often one-time users. The moderate use of 
the site and KIIs suggest that the website is still not serving up easily accessible information that answers 
the needs across its wide range of stakeholders. IDEAL’s ability to continually upgrade and optimize the 
website has been constrained by not having on the KM/Communications team a dedicated website 
manager with high-level KM/website skills. 
EXIT STRATEGY PREPAREDNESS: ASSESS IDEAL'S CURRENT PREPAREDNESS FOR EXIT
Specifically: How has the sustainability plan been put into practice? What changes are needed to best 
prepare the Activity for successful exit?
Borrowing from the TOPS Summative Evaluation (2019), sustainability can be assessed in four different 
ways:
IDEAL Mid-Term Evaluation: Findings
21
1. All products and services continue without BHA funding
2. Certain aspects of IDEAL continue without BHA funding while others do not
3. Some outcomes of IDEAL can continue to replicate themselves
4. BHA makes a successor of IDEAL a core part of its business model and continues to fund it at some 
level
First and importantly, consistent with the TOPS Summative Evaluation conclusion, for the foreseeable 
future BHA will need to support mechanisms like IDEAL for continuing quality improvement of its 
programs. The need is much more acute because of the merger of FFP and OFDA, other USAID 
reorganizations, and the dramatically changing context of climate change, conflict, and COVID-19.
IDEAL has not yet made substantial progress towards sustainable change because of its slow start and lack 
of a clear vision and strategy for capacity strengthening and mainstreaming capacity strengthening into all 
IDEAL activities. This serious limitation should be addressed by IDEAL. While capacity strengthening is 
ostensibly part of the role of the SLCD team, their efforts focused more on adapting the OSTP capacity 
assessment methodology tool for food security activities rather than developing an IDEAL-wide, 
comprehensive, operationalized approach to capacity strengthening for the activity. The OSTP capacity￾assessment tool application did not meet IDEAL or BHA expectations, perhaps because the tool was 
developed in isolation of a broader program capacity strengthening strategy. Some workstreams such as 
the M&E and Data workstreams have developed a way of working that engenders capacity strengthening. 
The example above from the COVID-19 workstream is a case in point.
On the other hand, IDEAL is now prioritizing the development of mechanisms that will improve 
sustainability of its efforts. For example: focusing on establishing demand-driven CoPs and emphasizing 
the requirements for effectiveness and sustainability of these communities; designing knowledge sharing 
events that increase interpersonal connectivity among affinity groups; prioritizing development of online 
courses; creating fellowships for IPs to upgrade their impact evaluation skills; emphasizing the shared 
commons of a FSN Network knowledge management system. Although none of these initiatives will result 
in self-sustaining mechanisms, they should move the needle towards the sustainability of IDEAL’s products 
and services.
Finally, IDEAL’s more recent focus on localization is an important step towards improving the relevance, 
impact, efficiency, and sustainability of IDEAL efforts. Localization was not among TOPS’ or IDEAL’s 
strategies as “implementer” was implicitly defined as international organizations and INGOs. IDEAL’s 
inclusion of localization on its agenda is a good step; however, developing a strategy to promote 
localization is of urgent importance.
Strategies such as localization will be implemented more aggressively by IPs if required by BHA in 
solicitations. Given the Administrator’s commitment to funding localization, IDEAL should leverage this 
opportunity by elevating localization on its agenda. 
Sustainability will also be facilitated by greater progress to improve Activity results in the areas of social 
inclusion, social accountability, and integrated programs. 
Have IDEAL IP stakeholders begun to establish their own capacity to do capacity strengthening?
IPs are using a variety of approaches to build their own capacity in priority IDEAL CFAs, though typically 
following BHA’s lead. Many RFSAs now include a staff member devoted to strategic learning or CLA. RFAs 
are now increasingly identifying an SBC expert as key personnel; some are requiring gender/social inclusion 
personnel. Increasingly, IPs are conducting their own sustainability studies to identify what is sustained 
after a RFSA ends. At this time, IDEAL is not keeping track of these changes in staffing and organization nor 
whether the changes are leading to improved sustainability of food security program outcomes.
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
22
Recommendations
IDEAL Should Consider:
REVISION OF THE VISION AND LOGIC OF IDEAL AS WELL AS A PRIORITIZED AGENDA
IDEAL and BHA should consider elaborating a results framework to identify and prioritize strategic 
pathways for improving RFSAs, emergency programs, and integration in the design and management of 
programs, with clear and prioritized outcomes and capacities.
● Stress the pathways through three intermediate outcomes: 1) improved RFSAs, 2) improved 
emergency programs, 3) improved integration of programs.
● Stress continued improvement of the relevance of BHA program guidance to field needs.
● Increase the emphasis on stakeholder engagement, including with a broader landscape of 
stakeholders.
● Stress the difference between design and implementation management activities (TOC, R&I, 
adaptive management, scenario planning, integration, M&E, learning agendas) and programmatic 
themes (sustainability, social cohesion, social accountability, SBC, the nexus).
REORGANIZING ITS THEMATIC AND FUNCTIONAL WORK ACTIVITIES AROUND THESE THREE 
INTERMEDIATE OUTCOMES
IDEAL cannot make even progress across the outcomes to build competencies in all of these areas. Instead, 
IDEAL should consider prioritizing them in terms of IP and BHA priorities, IDEAL’s comparative advantage, 
and a recognition that CFAs also should address priority needs of emergency program implementers. At a 
minimum, however, IDEAL can conduct baseline assessments (as part of the intended capacity 
assessments) to determine how IPs in RFSAs and emergency programs are currently defining and 
addressing these issues, in order to prioritize more effective workstreams and create baseline measures for 
tracking change. 
DISCONTINUE THE TAP STRUCTURE 
The TAP structure should be abandoned in favor of workstreams around the key intermediate outcomes 
and the capacities required to reach these. This requires that the capacity assessments be undertaken 
immediately to guide relevant workstream development. 
CONSIDER DELINKING CONTENT IN SOME OF THE CFAS AND REORGANIZING WORK TEAMS
Integration and sustainability are two very different content areas. Integration could be elevated as a 
content focus area and stressed as an intermediate outcome in IDEAL’s logic diagram. Sustainability is a 
particularly important consideration for RFSAs. Sustainability will be improved by advances in social 
cohesion, social integration, integrated programming, localization, and a greater emphasis on capacity 
strengthening that includes the broader intended implementing partners. More analysis and learning on 
the nexus issues to improve integration of humanitarian, development, and conflict programming is a key 
need for improved integration. 
FUND AND CONDUCT SYSTEMATIC ANALYSIS AND RESEARCH AS PART OF IDEAL’S CORE 
ACTIVITIES
Due to the dramatically changing context of food security programming, analysis and research should be 
elevated on IDEAL’s work plan. While the SGP cannot fully address this need, remaining funding from the 
SGP should be devoted to non-competitive research and analysis related to priority information gaps. 
IDEAL is well placed, given its resource partners, the constellation of AAs, and core expertise to conduct 
research and analyses that identify emerging best practices in the design and management of food security 
activities. For example, while PCS is field focused, IDEAL can conduct meta-reviews of program 
improvements related to PCS interventions. Sustainability is a high priority consideration for RFSAs. Now 
that there are numerous closed DFSA/RFSAs, this opens an important opportunity to analyze sustainability 
IDEAL Mid-Term Evaluation: Findings
23
factors. Another target of opportunity is to identify adaptive management promising practices. Certain 
hotspots of challenges to food security programming provide opportunities for IDEAL to facilitate 
collective learning. For example, social cohesion and social accountability have become high profile 
concerns in Ethiopia. Studies aimed at identifying HDP programming improvements also are a high priority. 
More work is also needed to understand and resolve the bureaucratic constraints to integration. An 
integration workstream might consider a broader consultation around thinking and best practices for 
integration. Indicators should be developed to monitor progress in integration. 
INCREASE RESOURCES AND DEEPEN THE STRATEGY TO STRENGTHEN ADAPTIVE
MANAGEMENT
Adaptive management is central to improved programming and it logically subsumes many topics that will 
improve the processes of activity design and implementation. IDEAL should, in collaboration with IPs, 
analyze priority needs to improve adaptive management in the field, possibly through a series of 
stakeholder consultations. As discussed previously, a competency-based approach should prioritize and 
address key barriers to adaptive management. Meta-reviews of emerging promising practices in adaptive 
management should also be conducted. This area of work can especially leverage the resources of the PCS 
AA through closer collaboration and selected joint programming between IDEAL and PCS, for example, to 
develop meta-reviews of issues and lessons learned in applying TOC, R&I, learning agendas/plans, adaptive 
management, and novel approaches to monitoring, evaluation, analysis, and learning. IDEAL also should 
take better advantage of learnings for adaptive management around best practices in the annual Pipeline 
and Resource Estimate Proposal (PREP) process.
IDEAL has a strategic opportunity to prioritize and systematically build scenario planning capacity in the 
food security community. Scenario planning is a high demand area for emergency programs and RFSAs. 
Scenario planning capacity could be an important “tracer” of the effectiveness of IDEAL’s efforts to focus 
and integrate its interventions (training/mentoring, P2P, stakeholder consultations, SGP, and performance 
measures). Scenario planning capacity strengthening could be a model pilot for IDEAL. This model should 
be scalable (cascades, master mentors) and monitored closely.
HIGHLIGHT SUSTAINABILITY AS A PRIORITY FOCUS AREA
Sustainability is cross-cutting and has implications not only for what but how programming is done. 
Sustainability is a particularly acute concern for RFSAs. A stocktaking of how these activities have 
approached sustainability and studies of sustainability of outcomes for closed activities would provide an 
important evidence base for quality programming. The analysis should be part of and financed by core 
IDEAL support. Social cohesion and social accountability are key determinants of sustainability. For 
emergency programming, building social cohesion and accountability is both important and challenging but 
of great concern in the context of protracted crises where the bulk of emergency resources are spent. 
RFSAs are implemented in countries where lack of social cohesion and accountability are major constraints 
to sustainability. Solving these problems is context dependent, so promoting Mission-level events around 
these issues should be considered. Ethiopia is one good case in point. It is both a large Mission and the 
topical areas are particularly timely.
RE-ALLOCATE AND INCREASE STAFFING FOR THE DATA COLLECTION AND ANALYSIS FOR
IMPROVED PROGRAM DESIGN AND IMPACT WORKSTREAM
This area of work can leverage both externally facing and internally facing improvements in IDEAL 
performance. From an externally facing perspective, M&E is a critical tool for adaptive management. 
Externally facing needs include more supple M&E strategies and tools as well as more effective use of 
qualitative methods. IDEAL should build on its strengths and make sure it has an impact on the capacity of 
food security programs to gather and utilize data for decisions in near real-time. This is a precondition for 
adaptive management.
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
24
Internally facing needs include strengthened measurement of Kirkpatrick’s Level 3 and 4 outcomes broken 
down by different categories of users (Activity leadership, learning advisors, M&E advisors, and according 
to type and intensity of exposure to IDEAL activities), and review of fit-for-purpose of the CRM system. 
Review the CRM, including content, software and user interface, and include consideration of internal 
activity monitoring. The CRM should then be updated and mainstreamed into IDEAL staff and AA routine 
use. This will require additional staff support.
In addition, the M&E workstream will increasingly need to assist IPs to better monitor and evaluate 
programs in urban settings, as COVID-19 recovery will address the needs of these populations in addition 
to its historical rural vulnerable populations.
DE-EMPHASIZE SBC AS AN AREA OF WORK FOR IDEAL
SBC is a critical determinant of the impact and sustainability of food security programs. This topi c covers a 
wide range of consideration, and there are many centers of excellence engaged in research and tools 
development related to this topic. IDEAL has very little expertise to take a deep-dive into SBC. The baseline 
assessment of current status/capacities and the tool design currently under construction should be 
complemented by populating a resource section of the FSN Network for SBC. The SBC team can also 
consider engaging with leadership of food security activities to identify priority needs in this area. 
PLACE GREATER EMPHASIS ON STAKEHOLDER ENGAGEMENT AS A COMPONENT OF 
PROGRAM LOGIC
Stakeholder engagement is a broad and inclusive program concept that should be better articulated and 
strategically targeted by IDEAL. Improved stakeholder engagement, on the one hand, focuses on increased 
connectivity between IDEAL, BHA, IPs, and the broader relevant expert communities. This strategy should 
emphasize both formal and informal activities to improve BHA guidance/procurement processes and IP 
understanding of BHA’s priorities and bureaucratic constraints. Stakeholder consultations, such as the one 
that occurred in April of 2021, should result in a roadmap of engagement activities that lead to better 
guidance and procurement processes as well as improved quality of applications and their review. An IDEAL 
staff member should be tasked with overseeing stakeholder engagement. 
Stakeholder engagement will also be improved if IDEAL increases its work to “know your clients” (KYC). 
This can begin by developing a database of all current RFSAs and emergency programs together with 
contact information, location, organizations, including local partners with their contacts, and staff 
responsible for different programming areas together with their contact information. Similar information 
can be collected for BHA staff. Then the CRM information can be matched against the population IDEAL is 
trying to reach to better target activities and monitor engagement over time. Emphasis on emergency 
programs/staff is especially needed.
CONTINUE TO IMPROVE THE PAC AS A KEY COMPONENT OF IDEAL STAKEHOLDER 
ENGAGEMENT
The PAC’s purpose should be expanded to hear the voices of BHA, IP, and community in support of an 
expanded view of stakeholders. For example, agendas might include an update from these groups of 
emerging issues that will have relevance to IDEAL work (this would require preparation before meetings). 
The PAC might include dedicated space on agendas for identifying critical topics for future stakeholder 
consultations. The conversation around localization should be continued as part of the PAC dialogue to 
both help define a localization agenda for IDEAL but also to identify and recruit important community 
voices to include in the PAC.
PAC activities should not be limited to the quarterly meetings. PAC interests in engagement beyond the 
meetings should be explored. The localization agenda is one area of potential work. However, other topics, 
such as integration and sustainability, may also be relevant to the PAC work agenda.
IDEAL Mid-Term Evaluation: Findings
25
PAC membership should be expanded to include representation of regional backbones such as the Sahel 
Collaboration and Communication (SCC) Activity in the Sahel and the Horn of Africa backbone, as well as 
representatives that are closer to the ultimate clients of BHA. This might include direct representation by a 
sample of local organization leaders or tasking of PAC members to more systematically poll local 
organizations in preparation for PAC meetings and activities. The topic of improved local voice on the PAC 
should be included as a PAC agenda item.
The assignment of seats on the PAC should be revised. IPs should have one seat (they can assign an 
alternate) and the number of IPs represented should be increased. Recruit a high-level InterAction 
representative and strengthen engagement with WFP. Assign a seat on the PAC for each of the regional 
backbone organizations. Assign a seat to RFS and some legacy OFDA leads.
LEVERAGE STRATEGIC SUPPORT FOR IDEAL
IDEAL’s consortium members, particularly Mercy Corps, TANGO, and Save the Children have seasoned and 
senior advisors who could support IDEAL more effectively. These individuals bring a wealth of expertise 
and experience from BHA and other related programs. A strategic support team should be integrated into 
the pause and reflect sessions of IDEAL. The strategic support team should include one senior leadership 
representative each of the consortium partners (not IDEAL staff) with at most three experts external to 
IDEAL’s consortium members (senior and devoted PAC members).
RE-VISIT THE CONSORTIUM STRUCTURE AND STAFFING
Organizational leadership in the areas of capacity strengthening and strategic learning should be 
reassessed and perhaps led by Save the Children, putting into question the continuing need for Kai zen’s 
participation in the consortium. Recruitment of staff to assume leadership roles is a high priority. Given 
Mercy Corps’ critical role in providing substantive expertise in food security, engagement of Mercy Corps
senior staff is essential. This should include devotion of meaningful LOE by Mercy Corps from its leadership 
cadre.
Because some staff on other teams may be inwardly focused on process issues, especially among functional 
groups, staff reallocation from these areas to more external delivery-based roles should be considered. In 
addition, new staff members should be recruited to support innovations in M&E methods/approaches , CRM 
and CLA monitoring, and better definition and assessment of Kirkpatrick Levels 3 and 4 progress for more 
robust capacity strengthening monitoring.
A proper time allocation study should be conducted by IDEAL for all staff to guide the reallocation of staff 
and rationalization of a staffing structure.
UPGRADE SENIOR STAFF SUPPORT AND ACCOUNTABILITY FOR IMPROVEMENTS IN 
EMERGENCY PROGRAMS, RFSAS, AND INTEGRATION OF THESE ACTIVITIES
Designate a deputy for emergency programs (under Save the Children). Also, consider appointing a second 
deputy director for RFSAs and integration. These individuals would be tasked with leading these 
workstreams (not implementing them). Leadership will ensure accountability and improve coherence of 
activities around these workstreams.
CONTINUE TO SUPPORT THE MATURITY OF THE SMALL GRANT PROGRAM COMPONENT, 
FOCUSED ON POST-AWARD MANAGEMENT OF GRANTS
Given personnel shortages and continued challenges with the granting process, IDEAL should focus on 
post-award grant management, emphasizing the quality and utility of funded activities. In the absence of 
the Associate Director for SGP, Save the Children should ensure that post-award management of grants 
proceeds smoothly, especially because many of the grants will be closing during the later quarter of 2021. 
Support is needed for compliance monitoring, quality assurance, and review/modification of dissemination 
plans.
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
26
Based on the SGP AAR, BHA and IDEAL should develop a lessons learned document aimed at improving 
future grant-management programs.
CONTINUE TO LEVERAGE THE COVID-19 WORK AND CAPTURE LESSONS LEARNED
Through its COVID-19 workstreams and its AA spin-offs and desk review of COVID-19 supplemental 
programs, IDEAL has made major strides to improve food security programming in the COVID-19 context as 
well as to build model strategies for program improvement. Additional activities might include a meta￾review of adaptive management approaches taken by RFSA and emergency program IPs. Especially 
important is to analyze whether or not layering of humanitarian on development/resilience investments 
occurred. This work could include the identification of promising practices and barriers to layering. This is a 
particularly critical leaning question for BHA and its stakeholders, given the dramatic economic impacts of 
the COVID-19 response. IDEAL also should conduct an evaluation of the impact on program quality of 
IDEAL’s COVID-19 investments (model strategies).
CONTINUE TO STRENGTHEN AND SUPPORT FSN NETWORK
Analyze web analytics to better understand use patterns. Conduct user surveys to better understand user 
needs according to different categories of users. Develop community and resource links corresponding to 
CFAs.
SUBSTANTIALLY INCREASE FOCUS ON DEVELOPING AN ACTIVITY-WIDE CAPACITY 
STRENGTHENING STRATEGY
IDEAL can differentiate itself from predecessor projects by developing a comprehensive and operational 
plan for capacity strengthening across prioritized capacity gaps. Strategy development will require focused 
senior leadership with proven systems capacity strengthening competencies. The strategy should benefit 
from an initial assessment of priority capacities, a plan of action to mobilize IDEAL resources to strengthen 
these capacities, and a relevant MEL system to monitor change. This strategy should be fully operational 
early in Year 4 and it should be mainstreamed across IDEAL work. The performance measures should focus 
on measuring Kirkpatrick Level 3 and 4 change in specific terms linked to these priority capacities.
BHA Should Consider:
ANTICIPATING THE EXTENSION OF IDEAL AND THE SUCCESSOR BHA ACTIVITY. 
For the many reasons cited in the MTE, IDEAL has only recently begun to develop momentum towards its 
mission of improving food security program quality, impact, and sustainability. As a pause and reflect on 
where the activity is, IDEAL could be granted a funded or unfunded extension so that it can realize its 
mission. Given the many changes at USAID, BHA also should begin to contemplate the successor program 
to IDEAL. Sustainably improving food security is increasingly challenged by the global context of risks and 
the changing USAID institution.
INTERNAL REVIEW OF BHA MANAGEMENT
An internal assessment of the BHA-related bureaucratic challenges that IDEAL has encountered, many of 
which have caused unreasonable delays in IDEAL activities, impeding adaptive management. IDEAL’s ability 
to adaptively manage is an important determinant of its success.
CONTINUE AND INTENSIFY IDEAL ADVOCACY INTERNALLY AND EXTERNALLY
Continuing and intensifying efforts to identify and engage appropriate BHA/USAID advocates and key 
stakeholders outside the USAID community to participate in IDEAL workstreams.
Redoubling its efforts to promote IDEAL’s activities/work within USAID’s Bureaus, Regional Offices, and 
Missions. These efforts would be best targeted to opportunities in the field to leverage IDEAL’s impact. 
IDEAL Mid-Term Evaluation: Findings
27
Annex 1: Scope of Work for MTE
IDEAL Activity Mid-Term Evaluation 
CONSULTANT SCOPE OF WORK 
Hiring Organization: Save the Children US 
Location: Washington, DC 
Background & Context 
IDEAL is an activity funded by USAID’s Bureau for Humanitarian Assistance (BHA) that works to support the 
United States Government’s goal of improving food and nutrition security among the world’s most vulnerable 
households and communities. IDEAL addresses knowledge and capacity gaps expressed by the food and 
nutrition security implementing community to support them in the design and implementation of effective 
emergency and non-emergency food security activities. 
IDEAL’s three purposes (updated in conjunction with the IDEAL Theory of Change developed in Year 2 of 
programming and in midst of the COVID-19 pandemic) are: 
 Purpose 1: Improved capacity for integrated activity design and implementation of essentialelements 
of quality programming
 Purpose 2: Consistent application of adaptive management practices in food security programs
 Purpose 3: Improved coordination and collaboration (between IPs, BHA, and other 
stakeholders)around common goals
This Scope of Work (SoW) provides the framework for the mid-term evaluation consultant. The purpose of 
the mid-term evaluation is to assess specific aspects of the relevance, efficiency and effectiveness, 
sustainability, and performance of the IDEAL activity. The evaluation will inform the readiness of IDEAL to 
execute on its overall purposes during the end of Year 3 and all of Years 4 and 5 of the award. 
Objectives and Scope of Evaluation 
The purpose of this mid-term evaluation is to take stock of the IDEAL activities implemented to-date and 
compare it to its original intent, with an emphasis on recent adjustments (including the comprehensive 
Theory of Change developed in Year 2) as well as its pivot and propel work related to the COVID-19 
pandemic. The mid-term evaluation will: 
 Inform activity leadership on effective efforts to date to improve food security programming 
whichIDEAL can continue to catalyze; and
 Provide recommendations for the IDEAL team structure and programming to better meet the 
needsof the food security implementing community. Specifically, is the IDEAL activity currently 
structuredand coordinating across its functional teams and stakeholders (staffing, Program 
AdvisoryCommittee, and Technical Advisory Panels) to achieve its stated objectives?
The mid-term evaluation should be focused on strategic and participatory reflection on completed activities 
and the current Year 3 Work Plan to inform and improve the quality, reach, and breadth of IDEAL’s 
programming through the Life of Award (LOA). 
This mid-term evaluation has four broad objectives, listed below. Each objective has suggested lines of 
research, which the mid-term evaluation team should strive to address. 
RELEVANCE 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
28
Assess the degree to which the IDEAL activity’s functional and organizational design enables the activity to 
meet its goal and purposes. Specifically: 
 Is IDEAL’s strategic direction and programming around the seven core content areas relevant 
andappropriate to meet BHA and IP needs in Year 4 and Year 5?
 Has IDEAL sufficiently adapted to COVID-19 and other contextual changes in Year 1 and Year 2? 
Whatfurther adaptations are required to ensure the activity remains relevant to BHA and IP 
stakeholders?
EFFECTIVENESS AND EFFICIENCY 
Assess if the IDEAL activity is being implemented effectively and efficiently. Identify barriers the activity 
encountered and assess if the activity adapted appropriately. Specifically: 
 Is the IDEAL activity being implemented according to its Theory of Change? What is working and 
whatis not working? Specifically, are IDEAL’s internal processes and ways of working effective and 
efficientin achieving their desired outputs and outcomes?
 The merger of FFP and OFDA into BHA and the onset of the COVID-19 pandemic changed 
theoperating environment for IDEAL and its stakeholders in significant ways. How effective was 
IDEAL’spivot and adaption to these factors?
 What changes can be made to strengthen and enhance the IDEAL activity’s performance through 
theLOA?
 To what extent has the IDEAL activity been successful in achieving its stated objectives/outcomesand 
meeting its reporting targets?
 To what extent are the IDEAL activity’s Theory of Change and reporting indicators in the most 
recentM&E plan appropriate? Is the internal monitoring system sufficient to capture and report on 
progresstowards the Theory of Change and indicators?
Is IDEAL’s current size and structure appropriate to meet the continuing IP and BHA needs and to achieve its 
purposes and goal? Structure includes the functional teams within IDEAL as well as the working relationships 
with its Associate Awards, consortium members, Program Advisory Committee, and Technical Advisory 
Panels. 
EXIT STRATEGY PREPAREDNESS 
Assess the IDEAL activity’s current preparedness for exit. Specifically: 
 How has the sustainability plan been put into practice? What changes are needed to best prepare 
theactivity for successful exit?
 Have IDEAL IP stakeholders begun to establish their own capacity to do capacity strengthening?
Evaluation Design and Methods 
In addition to the external consultant(s), the mid-term evaluation team will include one (1) evaluation team 
member from each consortium member (Save the Children, The Kaizen Company, Mercy Corps, and TANGO 
International). BHA will have final approval on team composition. The IDEAL team will work with the 
evaluation lead (external consultant) to clearly define the scope and role of the evaluation team members 
during the evaluation’s inception phase. Internal consortium mid-term evaluation team members will not 
have had greater than 50 percent of their time allocated to IDEAL (life of award to-date). The consortium 
evaluation team members will be able to navigate and coordinate needed interviews and data within each 
consortium partner while maintaining a level of objectivity (not having worked full time on IDEAL). 
It is expected the consultant will first conduct a desk review and interview IDEAL leadership and consortium 
leads to inform the final methodology. The mid-term evaluation is expected to primarily utilize qualitative 
methods, although the use of secondary quantitative data may be helpful for some lines of inquiry. IDEAL will 
provide the evaluation team with full access to monitoring data and activity documentation. Detailed 
methodologies for the evaluation will be agreed between IDEAL, BHA, and the external consultant. 
IDEAL Mid-Term Evaluation: Findings
29
A detailed evaluation protocol will then be developed by the evaluators. The evaluators will work in 
conjunction with the IDEAL Senior Management Team (SMT) to identify appropriate consortium evaluation 
team members, and develop a scope of work for their individual contributions to the mid-term evaluation. 
Sources of existing activity information include: 
 Original activity proposal 
 Year 1, 2, and 3 Work Plans 
 Quarterly Reports to-date 
 Year 1 and 2 Annual Results Reports 
 IDEAL CRM data 
 Activity TOC (in Miro) 
 Activity/event/engagement evaluations 
 Other internal and external documents as requested 
Key information to inform budgeting for this SoW: 
 IDEAL is made up of four consortium members 
 IDEAL’s mandate includes providing support to BHA implementing partners 
 IDEAL has between 25-30 full time staff across its four consortium members 
 IDEAL has a close working partnership with 10-15 BHA staff 
 The IDEAL Program Advisory Committee (PAC) has 22 members 
 IDEAL held 37 events in the last reporting year (Year 2) 
 Labor costs for the internal evaluation members (one from each of the four consortium members) 
should not be included in costs 
Timeline, Activities, and Deliverables
April 1 –
May 1 
Inception phase, including determining team 
composition and member roles, develop 
detailed work plan for entire period 
Begin desk review of activity documents and 
preliminary key informant interviews (KIIs) to 
inform the research protocol 
Brief description of rationale and criteria used 
to establish team composition and roles 
Detailed mid-term evaluation research 
protocol 
May 1 –
July 1 
Comprehensive desk review and data collection Bi-weekly progress report (<2 pages) to IDEAL 
leadership, included as an annex in the final 
report 
Report on results of the desk review 
July 1 –
August 1 
Analysis, interpretation, and report drafting Presentation slide deck of preliminary results 
(to IDEAL 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
30
Annex 2: List of Key Informants Interviewed
Dalmar Ainasha
James Akai
Laura Alexander
Suzanne Amari
Luisa Angelsmith
Brian Bacon
Lloyd Banwart
Alex Bekunda
Cathy Bergman
Piers Bocock
Buck Bradshaw
Geraldine Brick
Tara Clerckin*
Hilary Cook
Kimberly Cook
Annette Fay
Bianca Flokstra
Karyn Fox
Tim Frankenberger
Eric Friesth
Mark Fritzler
Bradley Fusco
Karine Garnier
Gary Glass
Brian Hunter
Maggie James-Lucas
Kim Jurczyk
Andrew Kaiser
Mette Karlsen
Dasha Kosmynina
Lisa Kuennen-Asfaw
Justus Liku
Michael Manske
Salomon Matchoudo
Gitau Mbure
Jenny Morgan
Michael Mulford
Tim Ogborne
Melissa Opryszko
Darin Ottenhoff
Maja Persson
Olga Petryniak
Elizabeth Pearce
Andrea Procopio
Arif Rashid
Adam Reinhart
Karen Romano
Mara Russell
Barri Shorey*
Beatrice Scheuermann
Thomas Scialfa
Lawren Sinnema
Thomas Spangler
Anne Swindale
Laurie Starr
Camille Tacastacas
Kirsten Tanifum
Sara Titus
Kevin Weseni
Joan Whelan
Laura Zillmer
IDEAL Mid-Term Evaluation: Findings
31
Annex 3: Focus Group Discussions/Group Discussions and 
Observations:
Bi-weekly IDEAL-BHA check-in
IDEAL All-staff meeting
SMT meeting
MTE steering committee meeting (weekly)
SCSA TAP
Data TAP
FSST team
PAC meeting in May 2021
Sustainability and Integration TAP
SBC TAP
SLCD functional team
SLCD-CLA cross team meeting
CLA functional team
MTE Steering Committee
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
32
Annex 4: IDEAL Timeline
IDEAL Mid-Term Evaluation: Findings
33
Annex 5: IDEAL’s Theory of Change (Overview)
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
34
Annex 6: References
In addition to sources listed below, the evaluator reviewed IDEAL internal meeting notes, COVID-19 and 
other event recordings and materials on the FSN Network website.
Glenzer, K., Mubukanu, T., Struss, C. (2019). Technical and Operational Performance Support (TOPS) Program 
Summative Evaluation Final Report.
Goeldner Byrne, K. (2020). Literature Review: Adaptive Management in Humanitarian Settings. Washington, 
DC: Implementer-Led Design, Evidence, Analysis and Learning (IDEAL).
IDEAL. (2019a). IDEAL Monitoring and Evaluation Plan.
IDEAL. (2019b). IDEAL Year 1 Annual Results Report.
IDEAL. (2020a). IDEAL Year 2 Annual Results Report.
IDEAL. (2020b). Organizational Self-Assessment Transformation Tool.
IDEAL. (2021a). Activity Concept Note: Emergency Practitioners’ Forum.
IDEAL. (2021b). IDEAL Revised Monitoring and Evaluation Plan.
IDEAL. (2021c). Stakeholder Consultation: RFSA RFA Design and Process. Washington, DC: Implementer-Led 
Design, Evidence, Analysis and Learning (IDEAL).
IDEAL. (2021d). Terms of Reference for Steering Committee for Emergency Practitioners’ Forum.
Karlsen, M. (2020). Emergency Food Security Community of Practice and Knowledge Management Platform 
Analysis: Findings and Recommendations for IDEAL Engagement. Washington, DC: Implementer-Led 
Design, Evidence, Analysis and Learning (IDEAL).
Netzer, S. (2019). Assessment of Emergent Opportunities for IDEAL to Support Implementers of Food for 
funded Emergency Programs. Washington, DC: Implementer-Led Design, Evidence, Analysis and Learning 
(IDEAL).
Save the Children. (2018). Revised Technical Application, Implementer-led Design, Evidence, Analysis and 
Learning (IDEAL), USAID FFP72DFFP18RFA00001.
USAID. (2019). Office of Food for Peace Food Assistance and Food Security Programmatic Learning 
Agenda.
IDEAL Mid-Term Evaluation: Findings
35
Annex 7: Additional Tables and Figures
Table 6: Consortium roles and rationale (Revised Proposal, August 16, 2018)
Consortium 
Member
Rationale for Selection IDEAL Roles and Responsibilities
Save the 
Children (Prime 
Applicant)
● Successful management and 
implementation of the TOPS
Program
● Currently implementing 13 FFP￾funded emergency and 
development programs
● Proven experience managing 
subawards to international and 
national organizations
● 25 years of innovative 
programming in food security and 
nutrition
● Extensive experience managing 
large, complex USAID projects
● Lead IDEAL’s strategic, financial, and operational 
planning and delivery
● Apply adaptive management to ensure flexible, 
integrated program implementation
● Manage systems for subgrants, smalls grants, and 
finances, performance monitoring, and reporting
● Provide leadership and technical expertise in 
systems approaches, emergency, SBC, youth, and 
health/nutrition
The Kaizen 
Company
● 12 years of experience in 
implementing its demand-driven 
model for CoPs
● Its innovative and sustainable 
organizational development 
model that empowers 
organizations to build their own 
capacity
● Expertise in innovation and 
entrepreneurship to address food 
security/conflict resolution
● Provide leadership and technical expertise for 
knowledge management, strategic learning, and 
capacity and organizational development 
activities including peer-to-peer learning and 
stakeholder consultations
● Manage the FSN Network
Mercy Corps ● Implementing 130 food security 
projects in 29 countries, including 
7 FFP-funded emergency and 
development programs
● Expertise in adaptive 
management and resilience 
programming
● Provide leadership and technical expertise in 
integrated food security and resilience 
programming, adaptive management, gender, 
social cohesion/accountability
● Support emergency focus area
TANGO 
International
● Recognized leader in resilience 
measurement, M&E, TOC design 
and training, project design, and 
risk and vulnerability assessments
● Provide leadership and technical expertise in 
monitoring, evaluation, research, and resilience 
measurement
● Build M&E capacity by training partners on using 
effective models and tools
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
36
Table 7: Partner Roles
Principal Partner IDEAL Technical Leadership Scope
Save the Children (Prime) IDEAL Activity Management, Small Grants, Systems Approaches, Emergency, Social 
and Behavior Change (SBC), Youth, Health/Nutrition
The Kaizen Company Knowledge Management, Capacity Development, Communities of Practice, 
Sustainability and Exit Planning
Mercy Corps Integrated Food Security/Resilience Program Design and Implementation, Adaptive 
Management, Gender, Social Cohesion/Accountability
TANGO International Monitoring & Evaluation (M&E), Research, Resilience Measurement and Analysis
Table 8: Consortium Selection Rationale, and Roles and Responsibilities
Consortium 
Member
Rationale for Selection IDEAL Roles and Responsibilities
Save the Children 
(Prime Applicant)
● Successful management and 
implementation of the TOPS Program
● Currently implementing 13 FFP-funded 
emergency and development programs
● Proven experience managing subawards 
to international and national 
organizations
● 25 years of innovative programming in 
food security and nutrition
● Extensive experience managing large, 
complex USAID projects
● Lead IDEAL’s strategic, financial, and 
operational planning and delivery
● Apply adaptive management to ensure 
flexible, integrated program 
implementation
● Manage systems for subgrants, small 
grants, and finances, performance 
monitoring, and reporting
● Provide leadership and technical
expertise in systems approaches, 
emergency, SBC, youth, and 
health/nutrition
The Kaizen 
Company
● 12 years of experience in implementing 
its demand-driven model for CoPs
● Its innovative and sustainable 
organizational development model that 
empowers organizations to build their 
own capacity
● Expertise in innovation and 
entrepreneurship to address food 
security/conflict resolution
● Provide leadership and technical 
expertise for knowledge management, 
strategic learning, and capacity and 
organizational development activities 
including peer-to-pear learning and 
stakeholder consultations
● Manage the FSN Network
Mercy Corps ● Implementing 130 food security projects 
in 29 countries, including 7 FFP-funded 
emergency and development programs
● Expertise in adaptive management and 
resilience programming
● Provide leadership and technical 
expertise in integrated food security and 
resilience programming, adaptive 
management, gender, social 
cohesion/accountability
● Support emergency focus area
TANGO 
International
● Recognized leader in resilience 
measurement, M&E, TOC design and 
training, project design, and risk and 
vulnerability assessments
● Provide leadership and technical 
expertise in monitoring, evaluation, 
research, and resilience measurement
● Build M&E capacity by training partners 
on using effective models and tools
IDEAL Mid-Term Evaluation: Findings
37
Figure 2: Frequency of Participation in IDEAL Events (CRM database, events = 75, participants = 2,264
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
38
Table 9: MTE SWOT Summary
Strengths:
-team spirit and recent leadership
-COVID workstream
-Virtual P2P learning methods
-strong understanding of developmental projects
-continuity of organizations and staff from TOPS
-linkages with LEARN activity
-monitoring and evaluation workstreams
-comms/km capacity
-recent improvements in staffing
-recent improvements in technical leadership
Weaknesses:
-process orientation, lack of focused strategy for change
-fragmented and ill-defined roles among partners and between BHA and 
IDEAL (including sectoralizing food security in one partner’s area of 
responsibility)
-limited engagement of IDEAL with BHA (M and E is exception)
-lack of foundational definitions-implementer-led, etc.
-lack of consensus on IP target community
-ill-defined decision structures between and within BHA and IDEAL
-CFAs too numerous and don’t match with organization workstreams and 
organizational team composition
-CFAs may not match field priorities related to ToC, R and I
-emergency programs not emphasized, culture of rfsa
-ToC doesn’t amplify the importance of BHA connectivity with IDEAL and IPs
-IDEAL versus TOPS rather than building upon TOPS success
-tension: global versus local focus of IDEAL
-IDEAL doesn’t have clear identity and appeal to IPs
-insufficient senior food security expertise on staff
-Opportunities:
-COVID-19 Supplemental funding (2 billion)
-FAO partnership
-Interaction links
-Major funding through annual cycles and 
supplementals (8 billion)
-ELRHA (COVID stream)
-merging of FFP and OFDA into BHA provides 
opportunities to integrate
-IDEAL well placed to leverage the “Nexus”, has 
expertise in integrated multi sectoral 
programming
Threats:
-USAID reorganization: sectoralizing resilience, poor integration of food 
security within BHA; stove piping of sectors in legacy OFDA projects
-Constrained time frame of emergency programs
-competitiveness of PVOs
Figure 3: Logic Model Direction
IDEAL Mid-Term Evaluation: Findings
39
Table 10: IDEAL Revised Monitoring and Evaluation Plan, March 2021
Indica
tor #
Indicator Y1 
value
Y2 
value
Y3 
target
Y4 
target
Y5 
target
Target rationale
1 Percentage of 
IDEAL 
participants[A1] 
[A2] who cite 
improved 
capacity as a 
result of 
engagement with 
IDEAL
3 
activiti
es
11 
activi
ties
80% 80% 80% *Unit has changed
In Y2, around 80% of ex-post 
respondents[A3] [A4] indicated that the 
quality of their work had improved as a 
result of attending IDEAL events. As we 
broaden this the audience and increase the 
sample size of this respondents for this 
survey, we anticipate seeing a similarly high 
percentage of participants citing improved 
capacity.
2 Percentage of 
IDEAL 
participants who 
cite the 
application of 
adaptive 
management[A5] 
[A6] as a result of 
engagement with 
IDEAL
1 
activit
y
12 
activi
ties
60% 60% 60% *Unit has changed
In our Y2 annual survey, 57% of respondents 
who engaged with IDEAL within Y2 noted 
improved use of adaptive management as a 
result of IDEAL engagement. We will aim to 
slightly increase improved use of adaptive 
management by IPs in Y3-Y5, but set this 
target lower than those of indicator P1 since 
fewer IDEAL activities focus explicitly on 
adaptive management than general 
improved capacity.
3 Percentage of 
IDEAL 
participants who 
coordinated or 
collaborated with 
another food 
security 
stakeholders as a 
result of 
engagement with 
IDEAL
2 
activiti
es
6 
activi
ties
50% 50% 60% *Unit has changed
In our Y2 annual survey, around 29% of 
respondents who engaged with IDEAL within 
Y2 noted improved linkages as a result of 
IDEAL engagement. We believe reported 
linkages were limited in Y2 due to lack of in￾person interactions (and a low annual survey 
response rate that may not have captured 
the extent of improved 
coordination/collaboration amongst 
stakeholders). We will aim to improve 
coordination/collaboration between 
implementers in Y3-Y5, noting that this 
percentage will likely be lower in Y3 and Y4 
than Y5 as interactions continue remotely.
4 Number of IDEAL 
events[A7] [A8] 
[A9] [A10]
11 31 35 25 25 We have counted 35 events per our Y3 work 
plan: Learning from COVID-19 Adaptations 
(A1), P2P event on supporting staff training 
in emergency programs (A4), event on case 
studies in knowledge managements (A6), 
Field Voices Webinar Series (A7), Virtual 
Grants Fair (A9), workshop on capturing and 
disseminating lessons learned from final 
reports (A11), stakeholder consultation(s) 
on essential capacities for program design 
and implementation (A16), P2P exchange 
with R&I RFSAs (A19), lessons learned events 
with IMPEL (A19), P2P event around 
fostering a culture of learning (B1), virtual 
capacity strengthening workshop(s) on 
priority topic (B1), workshop on scenario 
planning (B2), stakeholder consultations on 
adaptive management (B3), DQA (Extended 
PDM) (B17), Data for Decision-Making 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
40
Indica
tor #
Indicator Y1 
value
Y2 
value
Y3 
target
Y4 
target
Y5 
target
Target rationale
regional showcases (x4) (B20), M&E Systems 
Workshop (B22), Gender Assessments (x3) 
(B25), M&E emergency workshop (B28), 
Qualitative Guidance Roll-Out (B30), virtual 
exhibit booth for support mechanisms (x2) 
(C1), App Lab Info Sessions (x2) (C2), App 
Lab Workshops (x3) (C2), App Lab Showcase 
(C2), HuMEL meetings (x3).
We will aim to have about 25 events in Y4 
and Y5, balancing quantity with quality and 
dedicating more of our efforts toward 
sustainable, non-event initiatives.
5 Number of 
organizations 
engaged in IDEAL 
events
85 475 150 150 150 We anticipate having more events in Y3 than 
Y2, but keeping the focus on food security 
organizations (Y2 had an upward spike in 
number of organizations engaged in IDEAL 
events due to a large number of non-food 
security organizations attending early IDEAL 
COVID-19 events). The projected number for 
Y3-Y5 is in line with Y1 actuals (and Y2 
actuals minus the early, less targeted 
COVID-19 events). We expect more 
organizations engaged in Y3 and Y4 than Y5 
when events may be shifting back to in￾person [A11] [A12] [A13] [A14].
6 Number of 
individuals to 
attend IDEAL
events
102 1,644 500 500 300 We anticipate having more targeted events 
in Y3-Y5 than in Y2 (when we had just begun 
working virtually and were not always 
limiting the scope or participation of our 
events) and aim to touch around 500 
individuals/year in Y3 and Y4. We expect 
more participants in Y3 and Y4 than Y5 when 
events may be shifting back to in-person.
7 Percentage of 
event 
participants who 
state the event 
objective(s) were 
fully met
n/a 79.2
%
80% 80% 80% We will aim for the vast majority of 
participants to indicate objectives were fully 
met, with the understanding that a minority 
of participants will occasionally indicate an 
objective was only 'partially' met.
8 Percentage of 
IDEAL event 
participants who 
cite application 
of event 
learnings[A15] 
[A16]
100% 69.1
%
70% 70% 70% We anticipate about the same percentage 
for Y3-Y5 as Y2. This target assumes that 
most people will apply what they've learned 
from events, but as events continue to be 
held virtually with fewer participant 
restrictions, we anticipate a number of these 
participants might participate to gain 
knowledge but don't have the opportunity 
to apply learnings to their program, similar 
to the last quarters of Y2.
9 Number of IDEAL 
products[A17] 
[A18] [A19] [A20]
7 17 28 23 23 Products include webinar recordings, IDEAL￾authored Resource Library products, e￾courses, and outputs from events (full 
definition of ‘products’ available is available 
in the PIRS in Annex 2). We anticipate about 
1/2 of the events will have a webinar 
recording, as well as an average of one 
product per learning stream (five learning 
streams total) per year and one product per 
TAP (5 TAPs total) per year.
IDEAL Mid-Term Evaluation: Findings
41
Indica
tor #
Indicator Y1 
value
Y2 
value
Y3 
target
Y4 
target
Y5 
target
Target rationale
10 Number of active 
communities of 
practice 
supported by 
IDEAL
2 2 5 5 5 We have counted five communities of 
practice from our Y3 work plan: HuMEL 
(B27), QualME, EP Forum (C3), TOC 
knowledge sharing group (B10), SLA 
knowledge sharing group (B7).
In Y4-Y5, we anticipate a few of the above 
might drop off (having served their purpose) 
and a handful will be added (potentially 
communities of practice for resilience and 
strategic management). In Y5, we will work 
on sustaining the communities of practice 
rather than adding new ones.[A21] [A22]
11 Number of 
resources 
uploaded to the 
FSN Network 
Resource Library
89 153 100 100 75 We saw a particularly large jump in 
resources uploaded in Y2 due to the mass 
upload of WASH resources by PRO-WASH 
and new resources uploaded as part of the 
review of resources during the FSN Network 
website refresh.
We believe 100 newly uploaded resources in 
Y3 and Y4 is an ambitious target that 
encourages widespread knowledge sharing 
while maintaining that resources uploaded 
to the FSN Network are of high quality. 
Additions will include TAP-identified 
technical resources and small grants￾generated products, amongst other IP￾submitted resources.
12 Number of times 
resources have 
been viewed via 
the FSN Network 
Resource Library
74,410 59,88
9
66,000 72,600 79,860 The number of resources was particularly 
high in Y1 due to the unprecedented 
popularity of the TOC curriculum and 
relatively lower in Y2 due to lack of new 
content and resource promotion in 
preparation for the FSN Network refresh.
We will aim to increase page views by about 
10% each year (slightly more in Y3, as we 
aim to see increased traffic from the FSN 
Network website refresh) by making more 
resources available on the FSN Network and 
expanding our outreach to new users.
13 Number of 
submissions to 
the FSN Network
18 
orgs
18 
subs.
26 
orgs
65 
subs.
65 65 50 *Unit has changed
The number of submissions request to the 
FSN Network increased significantly during 
Y2 with the FSN Network website refresh. 
IDEAL aims to maintain that level of interest. 
In the final year, we will aim to sustain 
interest in submissions while shifting some 
focus to other priorities related to 
sustainability and close-out.
14 Number of FSN 
Network 
newsletter 
subscribers
2,513 4,394 5,053 5,306 5,571 *This indicator is continuing/cumulative
We saw a significant jump in subscribers 
between Y1 and Y2 largely due to an initial 
COVID-19 event registration where we 
received 647 new subscribers. We expect 
IDEAL | Implementer-Led Design, Evidence, Analysis and Learning
42
Indica
tor #
Indicator Y1 
value
Y2 
value
Y3 
target
Y4 
target
Y5 
target
Target rationale
the increase to be less significant in Y3 (a 
10% increase) and a 5% increase in Y4 and 
Y5, as we suspect most potential subscribers 
will have already subscribed toward the end 
of the activity.
15 Number of small 
grants awarded
0 11 34 44 44 *This indicator is continuing/cumulative
For Y3, we anticipate the awarding of 7 
micro grants, 12 PIAs (4 for the 2019 RFA, 4 
for the qual RFA, 4 for the quant RFA), and 4 
research awards. For Y4, we estimate 
awarding around 10 new PIAs. We will not 
award any additional grants in the final year 
of the program.
16 Number of 
products 
generated via the 
Small Grants 
Program
n/a n/a 14 29 61 *This indicator is continuing/cumulative
We anticipate 1 product on average for each 
micro grant, 1.5 product on average for each 
PIA, and 2 products on average for research 
awards.
In Y3, we anticipate 8 micro grants and 4 
PIAs will close, making final products 
available. In Y4, we anticipate 6 micro grants 
and 6 PIAs will close. In Y5, we anticipate 16 
PIAs and 4 research awards will close.
17 Number of times 
small grants￾generated 
products have 
been viewed via 
the FSN Network 
website
n/a n/a 700 2,320 7,320 *This indicator is continuing/cumulative
The TOPS Small Grants evaluation 
(conducted in 2016, about three years after 
the beginning of the implementation period 
for small grants) showed 111 page 
views/product on average. IDEAL will aim for 
50 page views/product in Y3 (as many 
products will not be uploaded to the 
Resource Library until the end of Y3), 80 
downloads/product in Y4, and 120 
downloads/product in Y5 (aligning with the 
data we saw under TOPS) .
18 Number of 
organizations 
that participated 
in stakeholder 
consultations
n/a n/a 19 23 23 [A23] [A24] We have contacted 19 
organizations in preparation for Y3's 
stakeholder consultation. For all stakeholder 
consultations, IDEAL will target current BHA 
primes and subs as well as past FFP or OFDA 
awardees who don't currently have an 
award.
19 Percentage of 
participants 
based in 
countries of 
implementation 
engaged in 
stakeholder 
consultations
n/a n/a 50% 50% 50% We believe stakeholder consultations 
participation is equally important at the 
implementation country- and HQ-level and 
will equally target each. We will be doing 
more virtual consultations in Y3 (and likely 
beyond), allowing us to feasibly aim for 50% 
of implementation country-based 
participants. For Y3's stakeholder 
consultation, IDEAL has asked for two 
representatives per organization - one HQ￾based and one based in the country of 
implementation.
IDEAL Mid-Term Evaluation: Findings
43
Indica
tor #
Indicator Y1 
value
Y2 
value
Y3 
target
Y4 
target
Y5 
target
Target rationale
20 Percentage of 
stakeholder 
consultation 
participants who 
cite positive 
changes in 
process or 
practices as a 
result of 
stakeholder 
consultation
n/a n/a 80% 80% 80% In Y2, 69% of IDEAL event participants cited 
application of event learnings, a similar 
indicator that looks at change in behavior as 
a result of participation in an IDEAL event. 
With the audience being more targeted for 
stakeholder consultations, we hope to see 
close to 80% of stakeholder consultation 
participants citing positive changes in 
process or practices as a result of 
involvement in the stakeholder consultation.
21 Number of 
changes to BHA 
policy and/or 
guidance IDEAL 
contributed 
to[A25] [A26] 
[A27] [A28]
n/a n/a 2 2 2 Assuming BHA releases 4-5 policy/guidance 
documents a year, IDEAL would hope to 
affect half of the policy/guidance released 
(as a result of stakeholder consultations, 
feedback from IDEAL TAPs, and other 
engagement). In Y3, IDEAL has already 
contributed to the M&E Emergency 
Guidance document.