USAID LEARN MIDTERM EVALUATION
FEBRUARY 2018
MONICA MATTS LOOKS ON AS USAID/CAMBODIA PARTICIPATES IN THE COLLABORATING, LEARNING AND ADAPTING ACTION PLANNING PROCESS. CREDIT: USAID LEARN.
By: Norma Kriger, Ph.D.
This publication was produced for review by the United States Agency for International 
Development. It was prepared by the LEARN mechanism for the USAID Learning, Evaluation 
and Research (LER) Office in the Bureau for Policy, Planning and Learning (PPL). LEARN is 
managed by Dexis Consulting Group.
TABLE OF CONTENTS
ACRONYMS LIST .............................................................................................................................................. i
EXECUTIVE SUMMARY.................................................................................................................................iii
INTRODUCTION AND BACKGROUND..............................................................................................1
METHODS AND DATA.................................................................................................................................3
FINDINGS............................................................................................................................................................7
CAMBODIA FINDINGS.............................................................................................................................7
GUATEMALA FINDINGS ....................................................................................................................... 15
COMPARING AND CONTRASTING CAMBODIA AND GUATEMALA MISSIONS ...... 21
RECOMMENDATIONS ............................................................................................................................... 23
ANNEX 1: CAMBODIA MISSION INTERVIEW QUESTIONS....................................................... 26
ANNEX 2: USAID/GUATEMALA MISSION INTERVIEW QUESTIONS..................................... 28
ANNEX 3: FEEDBACK SURVEY — PPL/LEARN MIDTERM........................................................... 30
ANNEX 4: LITERATURE REVIEW ........................................................................................................... 31
ANNEX 5: GLOSSARY OF USAID LEARN TERMS ........................................................................... 36
ANNEX 6: LEARN’S CLARIFICATION ON EVALUATION FINDINGS .................................... 37
ENDNOTES ..................................................................................................................................................... 39
i
ACRONYMS LIST
A/COR Agreement Officer’s Representative/Contracting Officer’s
Representative
ADS Automated Directives System
AOR Agreement Officer’s Representative
BAA Broad Agency Announcement
CDCS Country Development Cooperation Strategy
CLA Collaborating, Learning and Adapting
CLA Team Collaborating, Learning and Adapting Team
CoP Chief of Party
COR Contracting Officer’s Representative
DGO Democracy and Governance Office
DRG Center Center for Excellence on Democracy, Human Rights and
Governance in Bureau for Democracy, Conflict and 
Humanitarian Assistance
DRC Democratic Republic of the Congo
EDO Economic Development Office
EFM Eligible Family Member
FEVS Federal Employee Viewpoint Survey
FMO Financial Management Office
FO Front Office
FSE Office of Food Security and Environment
FSN Foreign Service National
FSO Foreign Service Officer
IP Implementing Partner
KM Knowledge Management
LAC USAID's Bureau for Latin America and the Caribbean
LEARN Learning and Knowledge Management
LER Office of Learning, Evaluation and Research in USAID's Bureau 
for Policy, Planning and Learning
M&E Monitoring and Evaluation
MD Mission Director
MEL Monitoring, Evaluation and Learning
MERL Monitoring, Evaluation, Research and Learning
MERLIN Monitoring, Evaluation, Research and Learning Innovations
MTE Midterm Evaluation
OAA Office of Acquisitions and Assistance
OPHE Office of Public Health and Education
OU Operating Unit
PAD Project Appraisal Document
PMR Performance Monitoring Report
ii
PO Program Office
POC Point of Contact
PPL Bureau for Policy, Planning and Learning in USAID
PPSO Planning and Program Support Office
TDY Temporary Duty
ToC Theory of Change
USAID United States Agency for International Development
WHIP Western Highlands Integrated Project
iii
EXECUTIVE SUMMARY
USAID’s Learning and Knowledge Management mechanism (LEARN) is a five-year contract (2014-2019) 
to support USAID’s strategic learning and knowledge management (KM) approach — known as 
Collaborating, Learning and Adapting (CLA) – to improve the effectiveness of USAID programs in 
achieving significant development outcomes.1 LEARN is funded by USAID’s Bureau for Policy, Planning 
and Learning (PPL) through its Office of Learning, Evaluation and Research (LER). The PPL/LER and 
LEARN Midterm Evaluation (MTE) was conducted between March and December 2017.
EVALUATION PURPOSE
The objective of the PPL/LER and LEARN MTE is to test the postulate in LEARN’s theory of change 
(ToC): CLA champions drive CLA integration in USAID missions, operating units (OUs) and 
implementing partners (IPs). Please see Figure 1 below for LEARN’s Results Framework.
EVALUATION QUESTIONS
The evaluation aimed to answer the following questions:
• Q1: Has PPL/LER and LEARN’s engagement with USAID missions led to CLA integration, and if 
so, how?
• Q2: Have PPL/LER and LEARN’s efforts contributed to CLA champions’ behavior change, and if 
so, how? In turn, how have champions' new CLA behaviors contributed to changes in 
organizational behavior, processes and policies? What does this change process look like?
• Q3: Outside of LEARN’s ToC about CLA champions driving CLA integration, what are 
alternative explanations for increased CLA integration?
METHODS
PPL/LER identified six missions as promising cases because LEARN and PPL/LER had engaged in two 
Temporary Duty (TDY) travel assignments at each mission. From these missions, Cambodia and 
Guatemala were selected because they suggested interesting similarities and differences with the 
potential to illuminate how LEARN and PPL/LER might have contributed to CLA integration and to test 
LEARN’s ToC.
DATA
The primary data sources were semi-structured interviews with key informants: mission staff (by 
telephone/Skype), as well as LEARN and PPL/LER staff. Other sources included brief online surveys 
completed by mission staff, mission documents and LEARN documents.
LIMITATIONS OF THE MTE
Limitations included the absence of CLA baselines at the missions; the specialized CLA language that 
respondents, often not native English-speakers, need to know; less than optimal cooperation from the 
Guatemala mission; and respondents’ imperfect recollections of the TDYs.
CAMBODIA FINDINGS
• Q1: LEARN’s two TDYs (using the CLA Maturity Tool Self-Assessment and Action Planning 
process as well as the Country Development Cooperation Strategy (CDCS) midcourse 
stocktaking) contributed to an increase in CLA integration.2 The first TDY introduced the CLA 
iv
concept and framework and explained how to apply CLA, making CLA’s application more 
systematic, explicit and concrete than before. The TDYs also contributed, inter alia, to the 
mission incorporating CLA requirements in documents (notably contracts and agreements) and 
CLA integration in working groups, as well as changing a development objective in the CDCS. 
The Program Office (PO) played a crucial role in supporting CLA in the enabling conditions,
helping contribute to CLA integration.
• Q2: The TDYs contributed to numerous new CLA champions by raising awareness of CLA, and 
champions were important in driving CLA integration. In contrast to LEARN’s identification of 
one champion, mission staff identified as many as 17 champions. The PO and other mission 
leaders' support of CLA in the enabling conditions also contributed to new champions.
• Q3: LEARN’s ToC about CLA champions driving CLA integration receives strong support from 
the interview data. Most striking is the way in which CLA champions integrated CLA in the 
mission’s culture, processes and resources (enabling conditions). One example is how CLA 
champions in the PO contributed to communicating information about CLA resources in its 
weekly newsletter.3 Another example is how CLA champions contributed to incorporating CLA 
requirements in contracts and agreements as well as draft mission orders.4 These changes then 
likely foster further individual CLA behavior change at the mission. The case analysis thus 
suggests that CLA champions and the changes they made in the enabling conditions came to 
mutually reinforce CLA integration. This finding has implications for expanding or refining 
LEARN’s ToC to make explicit a role for enabling conditions as a potential driver of 
behavior change.
GUATEMALA FINDINGS
• Q1: LEARN’s TDYs (facilitating a Broad Agency Announcement, BAA, process)5 contributed to 
a deeper experience of implicit or tacit CLA — that is, using CLA practices without naming 
them as CLA — among the four-member BAA mission team. The mission’s and LEARN’s scopes
of work, the LEARN facilitator and the BAA team did not conceive of the BAA process as a 
CLA process, although the BAA process is a co-creation approach grounded in CLA principles 
and practices.
6 However, the data support the critical role of LEARN’s facilitation in the success 
of the BAA process.
• Q2: Of the three BAA team members interviewed, two spoke only of BAA champions rather 
than CLA champions.7 None of the BAA team members promoted the BAA process as CLA 
even after the TDYs. LEARN identified no CLA champions and LEARN’s facilitator equivocated 
on whether the BAA team should be identified as CLA champions, though acknowledging that 
they did implicitly model CLA behaviors. However, given that the BAA process is based on CLA 
principles and practices, it follows that BAA champions must also be CLA champions.
• Q3: LEARN’s ToC about CLA champions driving CLA integration receives implicit support from 
interview data: BAA champions were the driving force behind the mission’s acceptance of the 
BAA process, inherently a CLA process. Beyond the BAA process, the mission provided other 
kinds of support for CLA integration, notably a new Monitoring, Evaluation and Learning (MEL) 
support mechanism. Two of the three BAA team members believed that the mission had no 
CLA champions, but this clearly cannot represent mission-wide perceptions about the existence 
of CLA champions. Hence, with the exception of the BAA process, the interview data can 
v
neither support nor invalidate LEARN’s ToC about CLA champions driving CLA integration. 
However, instances of CLA integration, such as the MEL contract, arguably should advance CLA 
integration.8 As in the Cambodia case, the finding of mission support for CLA integration has 
implications for expanding or refining LEARN’s ToC to make explicit enabling conditions as a 
potential driver of behavior change.
RECOMMENDATIONS
Based on the evidence, LEARN and PPL/LER co-created the following recommendations:
• BE EXPLICIT AND INTENTIONAL ABOUT FAMILIARIZING MISSIONS WITH CLA: Our 
data suggest that language matters and integrating CLA language across our technical assistance 
work will likely strengthen the impact of our work. Regardless of the type of technical 
assistance, this includes helping missions identify where they are already applying CLA practices 
and approaches to their work as well as ensuring that a “CLA 101” becomes a standard part of 
TDY agendas. 
• KEEP THE FOCUS ON CLA CHAMPIONS: Our midterm confirmed the essential role 
champions play in increasing and sustaining CLA integration. To the best of our ability, we 
should strive to maintain regular contact with CLA champions from missions post-TDY. These 
conversations will hopefully offer opportunities for us to provide champions with tools, 
evidence and other guidance needed to apply CLA. 
• CONSIDER INTEGRATING A DEVELOPMENTAL EVALUATION INTO OUR 
STANDARD MONITORING, EVALUATION, RESEARCH AND LEARNING (MERL) 
PROCESSES: The data presented in the midterm raised many additional questions. 
Implementing a developmental evaluation would allow LEARN to follow up on a select number 
of missions for the remainder of the contract. Illustratively through this methodology, we could 
deepen our understanding about change processes in missions, the most effective type or 
combination of CLA interventions and longer-term impact of TDYs. The developmental 
evaluation should fold into existing MERL processes and will be designed with a low level of 
effort in mind. 
• UPDATE LEARN’S RESULTS FRAMEWORK: The midterm process showed us that we need 
to clarify the difference between enabling conditions and systems-level change. Currently, our 
results framework lacks clarity on LEARN’s ability to influence enabling conditions (culture, 
processes, resources); yet, our data tells us that our work contributes to changes in the enabling 
conditions. We do not have sufficient influence over broader changes at the policy and systems 
level, however, and the results framework should be updated accordingly
.
1
INTRODUCTION AND BACKGROUND
EVALUATION PURPOSE
The objective of USAID’s Bureau of Policy, Planning and Learning (PPL)/Office of Learning, Evaluation 
and Research (LER) and LEARN’s Midterm Evaluation (MTE) is to test and explore LEARN’s theory of 
change (ToC). This theory postulates that Collaborating, Learning and Adapting (CLA) champions drive 
CLA integration in organizations. Figure 1 depicts LEARN’s results framework:
Figure 1: LEARN's Results Framework
2
EVALUATION QUESTIONS
• Q1: Has LEARN and PPL/LER’s engagement with USAID missions led to CLA 
integration? If yes, how have the CLA approaches implemented by LEARN and PPL/LER 
led to CLA integration?
• Q2: Have LEARN and PPL/LER’s efforts contributed to behavior change in CLA 
champions, and if so, how? In turn, how have champions' new CLA behaviors 
contributed to changes in organizational behavior, processes and policies? What does 
this change process look like?
• Q3: Outside of LEARN’s ToC about CLA champions driving CLA integration, what are 
alternative explanations for increased CLA integration?
BACKGROUND
LEARN CONTRACT
USAID LEARN is a five-year contract (2014-2019) to support USAID’s strategic learning and KM 
approach — known as CLA — to improve the effectiveness of USAID programs in achieving significant 
development outcomes.9
LEARN is funded by PPL through 
LER. The contract has a funding 
ceiling of $57 million, with 
approximately $23.5 million of 
“core” funding anticipated from 
LER, and the balance available for 
buy-ins from USAID missions, 
bureaus and operating units 
(OUs).
10
LEARN was envisioned as PPL’s 
principle mechanism to support 
CLA throughout USAID’s 
program cycle. LEARN would 
primarily target USAID missions, 
but also support USAID bureaus 
and OUs and their implementing 
partners (IPs).11 PPL/LER 
anticipated a continually increasing 
demand for learning from 
missions and Washington units.12
The LEARN contract identifies 
five performance objectives, 
which LEARN was to approach 
in an integrated way to achieve 
maximum cumulative impact.13 Figure 2: Collaborating, Learning and Adapting Framework
3
RATIONALE FOR MIDTERM ASSESSMENT’S FOCUS ON LEARN’S CONTRIBUTIONS TO CLA 
INTEGRATION AT MISSIONS
LEARN initially proposed the MTE focus on its contractual performance and a potential follow-on 
contract.14 According to LEARN’s Chief of Party (CoP) and Deputy CoP, PPL/LER’s Contracting 
Officer’s Representative (COR) — a member of PPL/LER’s CLA Team — gave LEARN an A+ grade and 
considered an evaluation focused on LEARN’s performance to be unnecessary.15 LEARN and PPL/LER 
then shifted the MTE’s focus to understanding how LEARN was contributing to CLA integration at 
missions and to testing LEARN’s ToC. After the evaluator completed this report, she conducted a 
literature review to validate and/or identify gaps in LEARN’s ToC. The review can be found in Annex 4. 
The MTE’s focus on LEARN’s mission buy-ins excludes two important components of LEARN’s 
portfolio:
• LEARN’s core-funded work. LER has provided LEARN with $5 million each year, except in the 
first year when it provided $3.5 million. LEARN’s annual budget is approximately $9-$10 million.
• LEARN buy-ins at bureaus and operating missions. In September 2017, these buy-ins constituted 
approximately 70 percent of LEARN’s buy-in funds. The remaining 30 percent of buy-in funds 
were from missions.
LEARN’s COR justified the decision to focus on LEARN’s efforts to integrate CLA at missions on the 
grounds that when LEARN was first established, PPL/LER had made a conscious decision to focus its 
work, and therefore LEARN’s, in the field. The rationale was that it was at the missions where 
development was done, and the CLA Team still sees the missions as primary clients for CLA 
integration.16 LEARN’s CoP elaborated on the reason for the MTE’s focus on LEARN’s TDYs at 
missions: “Our goal is to strengthen CLA practice throughout USAID’s program cycle, which by 
USAID’s definition, happens at the missions.”17
LEARN previously aspired to long-term mission buy-ins, potentially with its staff resident at missions, as 
a way to “help ensure real CLA sustainability.” 18 LEARN also found it “harder to recruit for and manage
these one-offs than it would be for a longer-term engagement.”19 Because of the lower demand for 
mission buy-ins, LEARN now sees bureaus and OUs as its primary sources of buy-in funding.20
METHODS AND DATA
CASE SELECTION
PPL/LER identified six missions — Cambodia, Democratic Republic of the Congo (DRC), Guatemala, 
Jamaica, Southern Africa and Uganda — for possible inclusion as case studies in LEARN’s MTE. PPL/LER 
considered these missions promising cases because PPL/LER and LEARN had engaged with them in a 
more sustained way — defined as at least two TDYs at each mission. LEARN reported CLA integration 
at all six missions in its Performance Monitoring Report (PMR).21
Given time constraints and the desire for in-depth understanding of how learning occurs, the CLA 
Team, LEARN and the assessment lead agreed to select two missions as case studies. They chose the 
Cambodia and DRC missions, with the Guatemala mission as a substitute should either Cambodia or the 
DRC mission decline to participate. When PPL/LER eliminated the DRC mission as a possible case study, 
the Guatemala mission replaced the DRC mission.22
4
CAMBODIA AND GUATEMALA
The Cambodia and Guatemala cases were selected because their similarities and differences that 
emerged from preliminary research had the potential to illuminate how LEARN might have contributed 
to CLA integration and to test LEARN’s ToC.
The two missions offered interesting contrasts with respect to the CLA approaches that LEARN used at 
the missions, the length of the TDYs, the extent to which LEARN categorized CLA instances during the 
TDYs as mission- versus team-specific, the missions’ different reputations and experiences with 
Monitoring and Evaluation (M&E) or learning contracts, and the composition of the TDY teams.
CLA APPROACHES
● In Cambodia, the two LEARN TDYs used PPL/LER’s primary CLA approaches: the CLA 
Maturity Tool Self-Assessment and Action Planning Process and the CDCS midcourse 
stocktaking.
● In Guatemala, the two LEARN TDYs used CLA approaches either entirely or relatively new to 
PPL/LER and LEARN. In the first TDY, LEARN’s facilitation of the BAA process to identify 
Opportunities for Youth in Guatemala with the mission’s BAA team and the mission’s IPs was 
the first and only time that PPL/LER or LEARN has facilitated a BAA process. In the second 
TDY, LEARN conducted Options Research on pre-primary school education for the mission’s 
education department including through interviews in Guatemala, and then facilitated 
stakeholder discussions on the Options Research.23
LENGTH OF TDYS
● Of the six possible missions, LEARN spent the least time at the Cambodia mission (a total of 24 
days). The Cambodia case therefore suggested that, under certain conditions, progress in 
learning might be achieved in a very short time.
● Of the six possible missions, LEARN spent the most time in Guatemala (a total of 44 days).
LEARN’S CATEGORIZATION OF CLA INSTANCES AS MISSION- OR TEAM-SPECIFIC
• LEARN categorized most CLA instances at the Cambodia mission as mission-wide and most 
CLA instances at the Guatemala mission as team-specific.
MISSIONS’ CLA REPUTATIONS AND PRIOR M&E OR LEARNING SUPPORT CONTRACTS 
• The Cambodia mission had no known M&E or learning contracts –this was verified during the 
research phase – or reputation as a mission advanced in CLA integration. 
• The Guatemala mission had a reputation as “one of the more advanced missions in terms of 
CLA, with its Western Highlands Integrated Program (WHIP) serving as a model for 
collaboration across sectors within the mission and with implementing partners and local 
governments”24 and as having considerable organic CLA.25 The mission had also had an M&E 
support contract.26
5
COMPOSITION OF TDY TEAMS
• The Cambodia mission TDYs were conducted by a combination of PPL/LER/CLA and LEARN 
team members.
• The Guatemala mission TDYs were conducted by only LEARN staff.
The two missions shared important similarities. First, LEARN-identified CLA champions were notably 
lacking or entirely absent. Second, both missions had previously received technical assistance from 
PPL/LER. Third, LEARN’s TDYs at both missions had components of buy-in funding, albeit of significantly 
different amounts.
PRESENCE OF CLA CHAMPIONS
• Despite LEARN’s PMR finding that LEARN TDYs had contributed to increased CLA integration 
at both the Cambodia and Guatemala missions, LEARN had identified no champions at the 
Guatemala mission (comprising about 100 people),27 and only one at the Cambodia mission 
(comprising some 77 people). A priori, an increase in CLA integration at missions with no 
champions or only one champion calls into question LEARN’s ToC, according to which CLA 
champions drive CLA integration.
PRIOR PPL/LER TDYS
• PPL/LER had conducted a TDY at the Cambodia mission in 2015.28
• PPL/LER had conducted TDYs at the Guatemala mission in 2013 and 2015.29
BUY-IN COMPONENT IN TDYS 
• At the Cambodia mission, the first TDY (12 days) in March 2016 was core-funded; the second 
(12 days) in September 2016 was mostly a buy-in. At the Guatemala mission, the first TDY 
(March/April and August 2016) was almost entirely a buy-in; the second TDY (December 2016 
and May 2017) was a buy-in. However, the buy-in funding for the TDYs at the Guatemala 
mission was significantly greater than for the second TDY at the Cambodia mission.30
DATA
The MTE relied on multiple sources of data: LEARN and mission documents, online surveys for mission 
staff and semi-structured interviews with key informants. Interviews typically were an hour to an hour 
and a half in length. Key informant interviews were the primary data source. Interview protocols can be 
found in Annexes 1 and 2.
DOCUMENTS
• LEARN documents reviewed included trip reports, knowledge drops, the CLA integration 
checklist analysis, the PMR,31 the champions tracker,32 the TDY tracker,33 the analysis of Federal 
Employee Viewpoint Survey (FEVS) CLA-related data34 and CLA case competition entries.
• PPL/LER shared its CLA mission engagement Excel sheet that prioritized missions, based on 
LER’s evolving criteria, as candidates for core-funded TDYs to advance CLA integration.
• The Cambodia mission provided documents with CLA requirements in such forms as mission 
orders and project appraisal documents (PADs).
6
ONLINE SURVEYS
The online surveys for Guatemala and Cambodia mission staff were intended to obtain data on 
perceptions of CLA integration at the missions before and after the TDYs as well as other data to 
inform the semi-structured interview questions. Cambodia mission staff who participated in the two 
TDYs were invited to take part in online surveys. Fifteen people from the Cambodia mission responded 
(including some who had not participated in both or any of the TDYs). The Guatemala mission’s points 
of contact (POCs) agreed that all four BAA team members would respond to the online survey. Three 
did. Please see Annex 3.
SEMI-STRUCTURED INTERVIEWS WITH KEY INFORMANTS
Key informants included LEARN staff, PPL/LER/CLA Team staff, mission staff and mission IPs. Face-to￾face interviews were held with staff based in Washington DC and telephone or Skype interviews with 
mission-based staff and IPs not based in Washington DC.
At least eight LEARN staff were interviewed, sometimes more than once, between April and September 
2017. The interviewees, and primary focus of the questions, included:
• Two facilitators of the TDYs in Cambodia, one facilitator of the TDYs in Guatemala and a senior 
project associate who attended the first of two workshops (on how the CLA approaches may 
have contributed to CLA integration).
• The Senior Operations Manager (on budget issues).
• The M&E specialist (on measures of TDY contributions to CLA integration at missions).
• The CoP and Deputy CoP (on the LEARN contract’s key accomplishments and challenges).
Three of the six CLA Team members located in LER were interviewed, including LEARN’s COR (on 
two occasions) and the CLA Team lead. Interviews focused on gathering contextual information about 
the LEARN contract and its relationship to PPL/LER. A CLA Team member was interviewed twice 
before he left the team.
The assessment lead conducted interviews with 13 key informants from the Cambodia mission from July 
23, 2017 to August 15, 2017. LEARN’s POC at the mission identified most of these interviewees. The 
informants were from the PO, the Office of Public Health and Education (OPHE), the Office of Food
Security and Environment (FSE) and the Financial Management Office (FMO). Of the 13 key informants, 
nine were Foreign Service Nationals (FSNs), three were Foreign Service Officers (FSOs) and one was an 
Eligible Family Member (EFM). Interviewees were given the questions in advance as well as LEARN’s 
two-page document, Collaborating, Learning and Adapting Framework & Key Concepts.
The USAID/Guatemala interviews took place between August 24, 2017 and September 18, 2017. 
Interviews were held with three of the four members of USAID/Guatemala’s BAA team. One BAA team 
member, citing time constraints, responded to the questions by email. The BAA team leader was from 
the Health and Education Office. The other three were from the Planning and Program Support Office 
(PPSO), the Office of Acquisitions and Assistance (OAA) and the Economic Development Office (EDO). 
All four BAA team members were FSNs.
Twelve BAA participants/partners from 12 organizations took part in the BAA process. Nine 
participants were invited to participate in the MTE. (Three were not invited because the LEARN 
7
facilitator of the BAA process knew they would not be able to conduct interviews in English.) Three 
participants responded and followed through with interviews held from September 19, 2017 to 
September 22, 2017.
A Presidential Management Fellow from the Learning Division of USAID’s Center for Excellence on 
Democracy, Human Rights and Governance (DRG Center), who had trained USAID/Guatemala’s 
Democracy and Governance Office (DGO) in CLA, was interviewed for any insights about the status of 
CLA at the mission.
LIMITATIONS
A major limitation of the MTE is the absence of any CLA baseline at the missions under study.35
Interviews conducted by telephone/Skype are not as good as in-person interaction. In addition, most of 
the interviews with mission staff and their IPs were conducted in English, which is not the first language 
of the interviewees.
Compounding this language limitation, questions often required respondents to be familiar with the 
specialized terminology of CLA. LEARN’s concern that respondents might have difficulty understanding 
the questions that required knowledge of the CLA language led to the decision to share LEARN’s two￾page document, Collaborating, Learning and Adapting Framework & Key Concepts.
The MTE does not include LEARN’s TDY for the Guatemala mission where it conducted Options 
Research on three potential areas of intervention for an education project. For each possible 
intervention area, LEARN gathered information on existing activities in relation to: pros, cons, risks, 
assumptions, unmet needs, key stakeholders and best practices. LEARN then held focus groups on the 
options with stakeholders. The TDY was conducted for one branch (education) of a single technical 
office, the Health and Education Office. With only two potential interviewees — LEARN’s facilitator and 
the education office member — it seemed better to focus on LEARN’s facilitation of the BAA process at 
the Guatemala mission.
For people to recall details of events that occurred some time ago is challenging. Examples of the 
problems of remembering were illustrated in the Cambodia case. Several interviewees could not recall 
whether they had attended one or both of the TDYs (or that LEARN, as opposed to simply another 
Washington DC office, had provided services to the mission.) Also, some interviewees were uncertain 
about the relationship, if any, between the TDYs and the mission’s incorporation of CLA requirements 
in documents and CLA integration in Working Groups. In this latter case, the PO CLA Action Plan 
provided some clarification.36
The Guatemala mission agreed to have only its four BAA team members participate in the online survey 
and the interviews. Only three of the team members actually participated in the online survey and the 
interviews. The PO representative in the BAA team, who had agreed to send documents and contact 
information of others who had been involved in the BAA process before the facilitation, never followed 
through.
FINDINGS
CAMBODIA FINDINGS
Q1: Has LEARN and PPL/LER’s engagement with missions led to CLA integration? If yes, 
how have the CLA approaches implemented by LEARN and PPL led to CLA integration?
8
LEARN and PPL/LER’s two TDYs contributed to an increase in CLA integration. Quantitative and 
qualitative data from the interviews support this finding.
First, the average of interviewees’ ratings and the median of these averages comparing the extent of 
CLA integration at the mission since PPL/LER and LEARN’s TDYs with the current extent of CLA 
integration indicated an improvement in CLA integration in the program cycle processes, in the enabling 
conditions and in the mission as a whole. The scale for ratings was 1=poor, 2=fair, 3=OK, 4=good and 
5=excellent. Whether using averages or medians of the averages of interviewees’ ratings over the 
specified time period, the data show an improvement from “OK” to “good” in CLA integration in the 
whole mission, the program cycle processes and the enabling conditions. Most individual interviewees’ 
ratings also show an improvement in CLA integration in each of the two building blocks, CLA in the 
program cycle processes and enabling conditions, since the beginning of 2016.
Second, overall, interviewees conveyed that the TDYs led to CLA integration at the mission by 
introducing the concept of CLA and how to apply it. Although learning had been taking place before the 
TDYs, interviewees noted that the language of CLA made learning and its application more focused, 
more systematic, more explicit and more institutionalized than before the TDYs. Most respondents gave 
significant credit to the PO and the Mission Director (MD), and sometimes also the Technical Office
directors and the M&E team, for important leadership roles in then applying the CLA concept to the 
mission’s work. Examples included the incorporation of CLA requirements in contracts and agreements 
and other documents, as well as of CLA in Working Group activities.
RATINGS
The ratings — averages, medians and for a significant minority of individuals — point to a mission that 
provides “good” support to CLA in the enabling conditions (see Figure 3).
Figure 3: Ratings of CLA Integration in the program cycle and enabling conditions
CONTRIBUTIONS OF TDYS TO INCREASED CLA INTEGRATION AT THE MISSION
Generally, the positive ratings for CLA integration in the enabling conditions are consistent with 
qualitative data in which interviewees depicted the PO, the MD, the Technical Office Directors and the 
M&E specialists as championing CLA. These contextual factors aid in understanding not only the reasons 
9
but the ways the TDYs contributed to increased CLA at the mission. The “how” and the “why” — and 
therefore also the direct and indirect contributions of the TDYs to CLA integration — are often difficult 
to disentangle.
Interviewees cited the following examples, among others, of how the TDYs contributed to increased 
CLA integration.
• Introduced the CLA concept and the CLA framework and how to apply CLA, making CLA’s 
application more systematic, explicit and concrete than the more diffuse learning that the 
mission had been practicing before the first TDY.
• Integrated CLA requirements in new contracts and agreements, the new OPHE PAD and the 
existing M&E Plan templates by the PO.
• Inclusion of CLA in the M&E Working Group’s roles and responsibilities by the PO.
• Inclusion of CLA in the Gender Working Group’s Gender Development Dialogue by the PO.
• Inclusion of a section on CLA resources in the PO’s weekly news bulletin.
• Impetus of the CDCS midcourse stocktaking exercises for the mission to take the time to 
officially change the CDCS development objective and make other changes, previously only 
proposed at the office level.
• Explicit incorporation of CLA in the Portfolio Reviews by the PO.
• Holding monthly meetings to discuss CLA by all the TOs after the first TDY.
An interviewee in the PO acknowledged how, following the March 2016 TDY, the PO decided to 
include CLA requirements in agreements and contracts, and documents more generally. The 
interviewee also discussed how the TDYs led to making official changes to the CDCS 2014-2019. 
Running through the interviewee’s response was an effort to recognize not only the contributions of the 
TDYs but also the prior CLA that had been occurring, albeit in a less explicit way, and the role of the 
PO in promoting CLA in the mission.
TDYs were twice. You can’t expect that to have so much influence. TDYs won’t have so much 
effect on mission environment. PPL is not in our environment. It’s very far away. It’s not direct. 
Now I’m thinking aloud. Through the PO, PPL has changed the environment.
After first TDY, the PO decided to include CLA requirements in agreements or contracts. We 
didn’t have that before the TDY and new guidance [Automated Directives System or ADS]. 
We didn’t document it well enough and we didn’t talk about it that much.
Even within the mission, we are trying to document more. For example, evaluations — but this 
is not because of TDYs. We did it two years ago. Each evaluation must result in an action plan 
to address recommendations in the report. Some recommendations lead to changes in scope of 
work; some lead to changes in interventions. A lot of documents for evaluation actually. But 
TDYs did help us to document a lot more CLA.
In the midcourse CDCS 2014-2019 review, we made a lot of changes. . . . Changed a 
Development Objective. . . . Even before the midcourse review, changes had been made at the 
office level but they had not received official approval from mission. The midcourse review 
made the changes official and as a mission rather than individually by office. The TDY was still 
very useful — without it, we wouldn’t make time to sit down and make changes as a mission.
Changes had already happened at office level. The development hypothesis was revised to 
10
reflect changes made at office level. The TDY was not alone the reason for change. Change is 
happening every day.
A supervisory financial analyst/Agreement Officer’s Representative (AOR) in the FMO captured how the 
TDYs had made the mission’s approach to CLA more systematic, emphasizing that CLA was now 
scheduled, routine, more frequent and more explicit. Whereas previously people may have been 
engaged tacitly in CLA the TDYs had raised people’s awareness that they were doing collaborating, 
learning or adapting.
Now we are more frequently integrated; have a schedule for CLA, and also ad hoc changes. It’s 
more routine and frequent now. Sometimes (before) I couldn’t link meetings to learning. Now 
it’s spelled-out as are actions and next steps.
The M&E specialist in the FSE also stressed how the introduction of the CLA concept had led to the 
mission as a whole and every Technical Office having a new focus on CLA and conducting more regular 
CLA activities. He gave the example of how all the TOs, including his own office, began to meet every 
month immediately after the March 2016 TDY. Like many others at the mission, the M&E specialist 
referred to the TDYs as “CLA facilitations” rather than as LEARN facilitations.
Before the CLA facilitation, we do something related to CLA but we did not really focus on that. 
For example, after CLA in our mission we have more regular Portfolio Reviews, regular Chief of 
Party meetings, regular IPs and donor meetings. . . . Every office holds regular meetings to 
address things in their office to improve work in their office. We take actions after CLA was 
introduced to us in March 2016: training, planning, results. And we focus on the C, L and A. 
The first things after learning about CLA, to the best of our knowledge our mission had an 
Action Plan on CLA and made sure office provides open mind, ideas, discuss, learn and improve 
in the office. Mission leadership is very supportive of CLA activity. . . . For example, in my office 
(FSE) we hold regular monthly meetings. We . . . raise issues on [our] culture differences; then 
we address management issues (who should make decisions) to make day-to-day management 
more efficient to avoid problems inside the office, ensure that people can maximize their work 
more effectively. We identify obstacles and come up with solutions to take. Basically, we use the 
concept of CLA in our office. 
The Development Assistance Coordinator in the PO called attention to how the March 2016 TDY had 
introduced the concept of CLA as an “institutionalized package of tools,” reinforcing how the new 
language became official practice at the mission.
Don’t think I heard CLA as one of these hot topics. It was being done but not like a packaged 
set of tools we were supposed to be using called CLA. Some of the things were being done. But 
not as an institutionalized package of tools. It’s not like the mission was not doing evaluations, 
etc. 
Q2: Have LEARN and PPL/LER efforts contributed to behavior change in CLA champions, 
and if so, how? In turn, how have champions' new CLA behaviors contributed to changes in 
organizational behavior, processes and policies? What does this change process look like?
• A number of interviewees either self-identified as champions and/or were viewed as such by 
their colleagues. One may infer that PPL/LEARN’s TDYs have contributed either directly or 
indirectly to these CLA champions for two reasons: First, CLA language itself was only 
11
introduced after the TDYs and was the first encounter with the language for most of the 
interviewees. Second, it was possible to relate respondents’ replies to questions about their role 
as champions to their responses to questions about how the TDYs changed the way they did 
their work.
• The number of CLA champions (up to 17) identified by respondents is in stark contrast to 
LEARN’s identification of only one champion at the mission.
• The role of FSN champions located in the PO, both enjoying high social capital, had a significant 
impact on CLA integration.
• Interviewees associated champions with characteristics such as passion, knowledge, influence, 
people skills, encouragement of others to ask critical questions and the ability to motivate 
people.
• All interviewees considered CLA champions to be important for CLA integration to occur. 
Those asked how many CLA champions were needed to integrate CLA responded that it would 
require many and provided additional qualifiers.
• Proposed strategies to best increase the number of CLA champions included training 
Agreement Officer’s Representatives/Contracting Officer’s Representatives (A/CORs), a 
mission-wide behavior change strategy, creating a CLA Working Group and having PPL include 
CLA topics at high-level conferences for all missions. Several interviewees saw individual 
behavior change as requiring the MD to be a CLA champion; others stressed the need for CLA 
champions at all levels of the organization.
• Notably, no one felt there were any major obstacles at the mission to increasing CLA 
integration, although several respondents focused on the need to train A/CORs — a plea made 
by a couple of A/CORs, too.
PPL/LER AND LEARN’S CONTRIBUTION TO INDIVIDUAL BEHAVIOR CHANGE
To understand how the TDYs contributed to individual behavior change in CLA champions, the voices 
of those often mentioned by interviewees to be champions are instructive.
The gender specialist in the PO explained that he incorporated CLA in his job following the TDYs 
because he understood CLA’s benefits both for his professional development and for the mission’s 
program results and capacity building, and he did not see CLA as a time-consuming burden. To generate 
more CLA champions, he believes this understanding of CLA is crucial.
I see learning about CLA as an opportunity — how I can expand my own career. . . . I see 
benefits through CLA lens — helps us build capacity and get better results from programming. . 
. . Myself, I did it [incorporated CLA] because it was my job that I knew everything about 
gender integration. If I move to technical area, I must be well informed. Personal interest. . . . I 
didn’t think that way [that CLA was an additional time-consuming burden.] I saw it as 
opportunity to increase knowledge and perform better; personal upward mobility.
An interviewee in the PO highlighted how the PPL/LER and LEARN TDYs had amplified her CLA 
knowledge.
Before the March 2016 TDY, my knowledge of CLA was very limited. The TDY was an 
opportunity to explore it more. The TDY started with “what is CLA.” It’s what we’ve been doing 
12
everyday. It’s in our DNA. Now when people talk about their activities, I tell them it’s the C or 
the L or the A. . . . For me, now I know I do CLA. It used to be a fancy word. It’s not really 
fancy, I learn. It’s what I do. People keep talking about it more. Certain Project Managers share 
with me the example of something and say it’s CLA and I assume it must be through the TDY. . 
. . TDYs contributed a lot for me personally. . . . PPL/LEARN TDYs show me how to do this even 
better. TDYs raise consciousness about CLA in Portfolio Reviews, CLA in new project design, CLA 
in introducing new data to analyze, and CLA in gender. The TDY has made me to relate CLA to 
these different topics, and learn from each other. Talking with partners, I share a lot about their 
CLA activities. . . . Two design staff and the M&E specialist in the PO now meet once a week to 
learn from each other. 
NUMBERS OF CLA CHAMPIONS
Using the names of CLA champions provided by the few interviewees who were explicitly asked to 
name CLA champions, 17 individuals were named. While these interviewees named some of the same 
CLA champions, those named also diverged. One interviewee, perhaps helped by her PO vantage point 
of the entire mission, identified 11 CLA champions, specifying them by position and office. On the other 
extreme, another interviewee (an M&E specialist) named only one CLA champion. To the extent that 
these CLA champions are a product of the TDYs, the numbers reflect significant individual behavior 
change. 
Like several others, the director of OPHE, saw “the vast majority of people” at the mission to be CLA 
champions.
The vast majority of people in the mission are [CLA champions]. For the most part, the team 
here is truly passionate and dedicated to development. They must therefore be willing to review, 
learn lessons and learn what others are doing, adapt and change.
CHANGE PROCESSES: NUMBERS OF CHAMPIONS NEEDED, WHO AFFECTS CHANGE AND STRATEGIES 
OF CLA PROMOTION
Interviewees agreed that CLA champions were important to integrate CLA and felt many such 
champions were needed for CLA integration at the mission. They generally identified the most influential 
CLA champions to be in the PO, and the mission leadership, the M&E specialists and the Technical 
Office directors as necessary champions for CLA to get integrated at the mission.
On the numbers and location of CLA champions needed for CLA integration, a project management 
specialist in the FSE said:
Everyone in PO, FO [Front Office] and all Office Directors and maybe at least one champion 
from each Technical Office. When a leader asks critical questions, it’s really powerful. My office 
director in FSE supports CLA.
An interviewee in the PO felt the number of champions needed depended on the size of the mission and 
the champions’ influence. She emphasized learning as a bottom up and top down process that she 
thought was occurring with the right combination of energy at present.
Needs to be enough to gather a mass. It also depends on their influence and position (working 
level, middle, top level) and size of organization. Learning occurs both bottom up and top down. 
. . . Right now seems like a good combination.
13
The OPHE’s director stressed the need for CLA champions to come from the ranks of FSNs and senior 
FSOs but emphasized that without mission leadership (MD and PO director), CLA integration would 
not take place. 
Two levels of CLA promoters are needed. Senior FSO, Office Director at least. And FSNs who 
can translate CLA into the local language. CLA language is fairly nuanced — I don’t want to say 
sophisticated — of English language, understanding of Agency objectives and of theory. Need a 
strong FSN dedicated and passionate about M&E broadly (to refer to CLA matrix — forgive me 
if using old jargon) — how do you take M&E and learn and adapt. Being an Office Director in 
early 2016 (i.e., speaking personally). . . . If you have Office Directors who understand what PPL 
is trying to achieve — it’s the Agency lead in Learning and Adapting — able to understand CLA 
and buys-in and understand importance for all of us. Even if you have FSOs on board but who 
don’t have support from Mission leadership (Mission Director and Director of PO) . . . CLA 
promoters won’t be able to achieve integration, is what I’m saying.
An interviewee from the PO did not think it was necessary to have an MD champion CLA, albeit 
desirable, and praised the central role of the PO in championing CLA for the rest of the mission.
In my opinion, Program Office is most influential CLA promoters at mission because of their 
roles and responsibilities. We are responsible for crosscutting issues and gender. PO also needs 
to coordinate with all offices. Must advise Front Office in terms of guidance, rules and best 
practices. We are in a good position to make either good or bad influence on whole mission. 
Also, PO people are keen to learn and collaborate more and always want to do better job. 
Personality now in PO is now making us able to influence the whole mission. I don’t want to be 
saying it’s only the PO doing CLA. That is not to say that CLA role of other offices aren’t 
important.
Some people would say Mission Director is most influential in CLA. I disagree. Sometimes we 
don’t have Mission Director who’s like that. If Mission Director is not supportive of CLA, people 
take it as excuse not to apply CLA. Change starts from us and can influence even our 
leadership. Though it would be easier to have a supportive environment, we also influence our 
environment.
LEARN’s facilitator during the second TDY drew attention to several features of the FSN CLA 
champions at the Cambodia mission — their social capital, the location of two (referred to by name) in 
the PO and their strong working relationships — that amplified their influence.37 LEARN’s facilitator also 
remarked on the unusual empowerment of FSNs in the mission’s leadership.38
I think it’s awesome [that FSNs are champions]. They feel very comfortable speaking to FSOs 
and other senior people. Lida is happy to push back. She has a lot of her own skills. . . . Lida as 
FSN. Sopheap as FSN. . . . Both have high social capital. The ambassador calls on them. 
Sopheap and Lida are close. Lida has been there for a long time. Lida brings credibility to CLA. . 
. . That FSNs are empowered in the leadership makes a big difference. There are more 
Cambodians in senior type positions than in other missions.
In regards to the best ways to promote CLA champions, interviewees proposed a variety of strategies. 
Training A/CORs — something a couple of A/CORs reported that they desired and needed — was one 
approach that the mission was going to implement soon, both because of the importance of A/CORs 
with respect to IP execution of CLA practices and because of awareness that they lacked sufficient CLA 
understanding.
14
The M&E specialist in the OPHE identified the need to train A/CORs in the CLA framework and noted 
that he and the PO M&E specialist planned to hold trainings for them.
Lida and I will do training of A/CORs at mission (about 25 of them) in September 2017. They 
haven’t been receiving training on CLA. CLA is new jargon for them. When we observe what 
they do, it’s CLA. Their practices are good in terms of CLA. We’ll see if we can encourage 
A/COR as a role model for other AOR. OPHE has largest number of A/CORs: 12-13. The 
planned two sessions — one and a half hours each — in September will be their first training 
in CLA.
The OPHE director recommended that PPL take advantage of MD, Technical Office and PO conferences 
as well as MD consultations held in Washington DC to promote CLA information. 39
Got to get Mission Directors make sure they understand.
 Annual Mission Directors conference. PPL has some presentations. Have CLA be a topic 
[presented by PPL].
 Typically before Mission Directors go to missions they have consultations in DC. They 
usually go to PPL. Make CLA a topic. There’s a regional POs conference. Would be good to 
have those questions (like you’re asking about CLA) in these conferences. Structurally it is 
the PO that is responsible for leading this; obviously it’s all our responsibility for doing it. If 
we don’t have a PO that understands, they won’t look for CLA when we do designs, 
strategies. Almost any document goes through PO for approval. Given the PO is lead on 
CDCS and partner with any Technical Offices in their design work, it’s crucial POs review 
and discuss CLA as often as possible as POs around the world.
 Technical Offices have conferences on various topics: agricultural officers’ conferences, 
environmental officers’ conferences, etc. I’m not sure the extent to which PPL inserts 
themselves in Technical Offices’ conferences.
Q3. Outside of LEARN’s ToC about CLA champions driving CLA integration, what are 
alternative explanations for increased CLA integration?
LEARN’s theory that advancing CLA integration in the Cambodia mission is driven by individual behavior 
change appears to have strong support from interviewees. In less than two years since the TDYs 
introduced the mission to the CLA framework, interviewees identified numerous champions and 
multiple ways in which CLA has been integrated not only in the program cycle but also in the enabling 
conditions. Nobody identified structural obstacles or individuals in management who were obstacles to 
greater CLA integration.
Some evidence suggests that CLA support in the enabling conditions — mission culture, business 
processes and resources (“structures”) — is relatively institutionalized. First, the ratings for CLA 
integration in the enabling conditions were positive (sometimes more positive than for CLA integration 
in the program cycle processes). Second, qualitative data also supported these positive ratings for CLA 
integration in the enabling conditions. Third, the mission has a number of mechanisms and documents 
(with more pending) addressing CLA promotion and requirements as well as a newsletter that 
incorporates CLA resources — all instances of mission support for CLA integration in the enabling 
conditions.
15
The Cambodia case finding of substantial mission support for CLA in the enabling conditions, rather than 
pointing to an alternative to LEARN’s ToC, draws attention to a positive dynamic between CLA 
champions and greater support for CLA integration in the enabling conditions. As CLA champions 
contribute to improving support for CLA in the enabling conditions, CLA becomes more 
institutionalized, perhaps at some stage reducing the need to generate new CLA champions. LEARN’s 
ToC does not take explicit account of how such increased institutional support may reinforce the role 
of CLA champions in promoting CLA integration because it does not explicitly identify a role for 
structural or institutional support (or enabling conditions) for increasing CLA integration. 
GUATEMALA FINDINGS
Q1: Has LEARN and PPL/LER’s engagement with missions led to CLA integration? If yes, 
how have the CLA approaches implemented by LEARN and PPL/LER led to CLA 
integration?
The BAA process is co-created and grounded in CLA principles and practices. LEARN’s BAA facilitation 
contributed to a deeper experience of implicit or tacit CLA — that is, using CLA practices without 
naming them as CLA — among the BAA team members (and IPs).
Evaluating how, if at all, LEARN’s facilitation of the BAA process contributed to CLA integration at the 
mission is complicated. By definition, the mission’s decision to use a BAA process, a new approach to 
procurement at USAID, indicated CLA integration at the mission, and the LEARN facilitation of the BAA 
process involved the BAA team and its partners in more intense CLA practices than usual. While the 
BAA team and its partners credit the LEARN facilitation with making it possible to come up with a co￾created youth project design, neither the LEARN facilitator (at least initially) nor the BAA team 
conceived of the BAA process as a CLA exercise. Even after the BAA process, members of the BAA 
team resisted viewing the BAA process through a CLA lens and insisted they hired LEARN’s services to 
facilitate the BAA process and not CLA. Hence, the label implicit or tacit CLA is given both to the 
mission’s decision to use the BAA process and LEARN’s facilitation of that process. Two other 
examples are briefly noted to illustrate that the BAA process is not a unique instance of implicit CLA at 
the mission.
PRE-LEARN FACILITATION: IMPLICIT CLA IN MISSION’S DECISION TO ADOPT THE BAA APPROACH
The mission’s decision to use the BAA process required the use of implicit CLA. When the education 
office wanted to develop a youth project in the western highlands to address the crisis of 
unaccompanied youth migrating to the United States in 2014, the mission embraced the BAA process 
for two reasons. The mission was looking for outside expertise to provide non-traditional ideas to 
develop a youth project. Also, it understood that the BAA procurement process would be quicker than 
normal procurement.40 The FO provided support immediately. The then deputy MD asked each 
Technical Office director to designate a member to form a Working Group to explore the possibility of 
the mission using a BAA process. The Working Group had to justify the merits of the BAA approach to 
the mission. The Working Group also reached out to USAID’s Bureau for Latin America and the 
Caribbean (LAC) in Washington, DC, for information and expert assistance.41
The BAA team, led by the education office, replaced the Working Group after the mission formally 
approved the use of the BAA process in the fall of 2015.42 The BAA team continued to engage with the 
LAC office, in part because the education office had only one person from January 2016 to June 2017, 
when the mission made the BAA award. According to the partners who were interviewed, the BAA 
team’s selection of the eight expressions of interest demonstrated knowledge of the different types of 
16
expertise of organizations working on youth (e.g., employment creation, education) as well as on the 
Maya culture in the western highlands. It helped, the BAA partners interviewed said, that most of them 
already knew one another and some had worked together.43
As it transpired, the BAA process took much longer than a normal procurement process largely because 
of mission leadership changes that led to questions about the appropriateness of the mission investing in 
another youth project and because the mission anticipated a huge budget increase for work on Central 
America that did not materialize.44 These internal mission issues meant that LEARN had to facilitate an 
unanticipated second workshop.
THE BAA PROCESS AND LEARN’S FACILITATION: DEEPENING IMPLICIT CLA PRACTICES
The BAA team and its partners praised LEARN’s facilitation, often using superlatives. Several 
interviewees expressed their belief that without LEARN’s facilitation, the BAA process might not have 
culminated in the successful design of a youth project. LEARN’s facilitation team worked collaboratively 
with the BAA team and facilitated collaboration among its partners.45 However, all the participants who 
were interviewed stated categorically that the BAA process had not been an explicit CLA process. Even 
so, LEARN’s facilitation of the BAA process, by definition, contributed to more intense CLA than would 
occur in a normal procurement process.
All interviewees heaped accolades on LEARN’s facilitation. The PO representative said LEARN’s 
services were expensive but the mission got more than it had expected.
LEARN had experience on facilitation. And they did research on BAA and could propose 
activities we had not conceived of, like a conference call with IPs, webinar with IPs. That was a 
good contribution to the results we had.46
The partners deeply respected LEARN’s facilitation and its contribution to the final youth project design. 
One partner put his appreciation in this way:
Facilitators they hired were amazing. To build the process was amazing. Without facilitators, it 
was impossible to make it.47
Two of the partners with significant development experience referred to LEARN’s role in restraining 
them from monopolizing the process and creating the opportunity for less experienced development 
organizations to have a voice in the process. Though this slowed the project design process making for a 
painful and frustrating experience, in hindsight they said they had learned the importance for 
collaboration of proceeding at a pace that enabled all participants to have a voice.48
As successful as the facilitation was as a BAA process, all interviewees acknowledged that CLA was 
rarely, if ever, made, explicit. LEARN’s facilitator noted that the Guatemala buy-in was only LEARN’s 
third buy-in. Moreover, it was the first time LEARN was facilitating a BAA process, still a new USAID 
procurement process. Neither USAID’s nor LEARN’s scopes of work for the BAA process refer to 
CLA.49 Not until late in the process did the facilitator realize how the BAA process was an example of 
CLA. In the facilitator’s words:
It took a while to integrate CLA and BAA. That was part of my learning. . . . After working on it 
for a while, I realized it is CLA for the mission — external collaboration, learning from outside 
partners, enabling conditions of openness, building relationships/networks with partners to find 
out what was happening on the ground. . . . The BAA team was practicing CLA without knowing 
17
it. . . . Maybe because our CLA contribution in the BAA process was not made clear to 
participants, Angelina encouraged me to offer to help Megan [LAC office in Washington, DC] to 
write a CLA plan. Megan declined.
50
The BAA team’s PO representative questioned the appropriateness of the mission’s participation in the 
MTE when first contacted by LEARN’s COR — and again, when first contacted by LEARN — because 
the mission had contracted LEARN’s services to facilitate the BAA process and not CLA.
We saw our contribution/experience was not a CLA experience per se and hard for us to see 
how we could contribute to a CLA assessment.
51
Another BAA team member also emphasized that the facilitation was for the BAA process and not CLA. 
Moreover, CLA had only come up late in the facilitation process.
We used LEARN not because we were thinking about CLA. Not a term familiar at that time. 
We needed a third party to facilitate [the BAA process]. . . . First half of the facilitation, we 
never heard of CLA approach.
52
Nonetheless, the implicit CLA in the BAA process is made explicit in the CLA case competition entry as 
well as in interviews with the BAA partners, who were asked to reflect on the BAA process through a 
CLA lens. Only the observations of the BAA partners are drawn on here.
One partner lauded the mission’s adaptability in designing a contract with a prime and six 
subcontractors, despite missions typically avoiding such contracts because they are so difficult to 
manage. According to the BAA partner, the mission recognized that the contract design needed to 
better reflect the co-creation process.
Contracting Officers don’t usually like subcontractors managing sub-grants. They like the prime 
to manage all. The mission accepted subcontractors have their own grants: a prime with 6 
subcontractors. Each sub has 2 to 3 grantees below them. It rapidly becomes a nightmare. The 
mission agreed you can’t change that without changing the whole bottom-up process, so 
accepted it. . . . Not every member is now equal, so it is a different kind of collaboration.
53
Another partner found the relationship with the BAA team “very good” and acknowledged the 
surprising degree of the BAA team’s openness and its knowledge of youth issues. 
Megan and Michael (USAID LAC DC) and the Guatemala mission were surprisingly open, 
helpful and a lot more sophisticated than people in some of the organizations and within 
LEARN itself in terms of . . . knowledge of issues [relating to youth].54
Although the BAA process was an implicit CLA experience, it was also deeper than usual. Internally, the 
mission’s cross-sector BAA team had to collaborate with each other intensely during the process. 
Externally, the BAA team had to collaborate and draw on partners’ knowledge in entirely new ways to 
co-create the final concept note, requiring an unprecedented degree of collaboration and openness in 
the pre-award phase.
18
POST-LEARN FACILITATION AND CLA
The BAA team concurred that the BAA facilitation had no consequences for CLA integration at the 
mission. As a result of the positive feedback about LEARN’s facilitation, however, the mission decided to 
use LEARN’s services to review the CDCS (see Question 3).
Responding to the online survey question about the reason for changes, if any, in collaboration at the 
mission between early 2016 and August/September 2017, a BAA team member who recorded no 
improvements in collaboration — ratings were “fair” and “poor” for internal and external collaboration 
respectively — commented:
The facilitation services were specific to BAA process and they did not affect relations among 
teams.
The PO representative on the BAA team stated that the mission failed to use the CLA lessons from the 
BAA process, and some at the mission remained ignorant about even the BAA process, despite its wide 
discussion within the mission.
LEARN’s facilitation did not result in changes to how we use CLA but not because the 
information was not useful or we didn’t have information, but because BAA team and PO did 
nothing to use CLA lessons. I can think of excuses — workload — but we don’t systematically 
make sure lessons learned are used by others. . . . We don’t have a systematic process for CLA 
even though it’s included in the CDCS. . . . I think I’m going to criticize myself. No standard 
procedure to share this. . . . Despite many different internal discussions about the BAA process 
at the mission, some people don’t know what BAA is and how it works.
55
With hindsight, LEARN’s facilitator, encouraged by LEARN’s CoP, documented the BAA experience as 
an exemplary case of using CLA and entered it in LEARN’s case competition in 2017.56 The case won 
the award for the best example of CLA submitted by a mission. LEARN’s facilitator said:
I wrote the case for them. They may not realize BAA was CLA.
57
A BAA team member said the mission did not document the lessons of CLA arising from the BAA 
process because the team’s focus was on making a BAA award and it had no time to document the CLA 
lessons.
We were just trying to save the BAA, which was at risk of being cancelled. We were not really 
thinking about systematizing documentation. It was really Bari who told us this is about 
learning. Bari documented the process. Because she did it, we won the case competition. We 
were too busy. It was not our priority at the time.58
According to this BAA team member, the mission’s response to winning the CLA case competition was 
indifference, and because the BAA process took much longer than anticipated to result in an award, the 
mission would be unlikely to use the BAA process in the foreseeable future.
The Mission has new FSOs, new office directors. They are not familiar with the BAA process. For 
example, we won case competition award. The BAA Team was so excited. We were sharing the 
news with everyone. The chief of the PO asked: so why is that important? I don’t have much 
expectation that winning the case competition for CLA in the BAA process will affect CLA at the 
19
mission. . . . The perception is that BAA is longer as a procurement process. Everyone is just 
thinking about making awards rather than using BAA or introducing co-creation in the process.59
TWO OTHER EXAMPLES OF IMPLICIT CLA AT THE MISSION: THE EDUCATION OFFICE AND THE 
DEMOCRACY AND GOVERNANCE OFFICE
In early 2017, the DGO invited a presidential fellow from the DRG Center to help draft their Project 
Monitoring, Evaluation and Learning (MEL) Plan. On his arrival, the team asked for help incorporating a 
CLA Plan in the Project MEL Plan. The Fellow did not know why the DGO team decided to ask for CLA 
assistance. The team did not know what CLA meant at the start of his three and a half week TDY, but 
the Fellow noted that the team was already practicing CLA in many ways.
The DGO didn’t know what CLA meant at the start. The DGO were strong enthusiasts of CLA. 
They want to do it. Actually they are doing it. Financials for evaluations include CLA activities.
60
The DGO’s request for CLA training suggests an appetite for explicit CLA at the mission. Further 
evidence of such an appetite might be inferred, inter alia, from the mission’s recent sourcing of a five￾year supporting mechanism for CLA.
LEARN’s second TDY in which it conducted options research for the education office and then 
facilitated stakeholder discussions around the various options for a project design also demonstrated the 
education office’s practice of tacit CLA integration. The education office demonstrated openness in 
hiring LEARN’s services to survey existing education projects in regards to their strengths, weakness
and risks, and then to invite stakeholders to discuss the identified options. Without detracting from the 
education office’s practice of external collaboration and openness in seeking outside knowledge, the 
education office at the time had only one staff member and likely could not have designed the project 
itself. LEARN’s facilitator remarked:
She uses LEARN to do tasks she’d have her staff do if she had them. It’s one reason she likes 
LEARN.
61
Q2: Have LEARN and PPL/LER’s efforts contributed to behavior change in CLA 
champions, and if so, how? In turn, how have champions' new CLA behaviors contributed 
to changes in organizational behavior, processes and policies? What does this change 
process look like?
Consistent with the finding that the mission experienced LEARN’s facilitation as a BAA rather than CLA 
process, most BAA team members spoke only of BAA champions rather than CLA champions. 
However, given that the BAA process is inherently a CLA process, BAA champions must also be CLA 
champions.
LEARN has not identified CLA champions at the Guatemala mission. Asked whether the BAA team 
could be called CLA champions, the LEARN facilitator said that although LEARN had a definition of a 
CLA champion, in practice LEARN’s team members applied different thresholds when defining a 
champion. For some, taking a CLA training program was sufficient; for others (herself included), practice 
and modeling was essential. LEARN’s facilitator equivocated, though, on whether members of the BAA 
team might be labeled CLA champions, even though they met the higher threshold of modeling CLA 
practices.
20
The BAA team modeled openness but they don’t use CLA language. It’s a tough one.62
Would the BAA team members consider themselves CLA champions? One BAA team member thought 
not, and went on to say that many others at the mission worked to get the BAA process approved.
Many people at the mission helped overcome challenges to get approval to use the BAA process 
but they never saw it as a CLA process. . . . I wouldn’t refer to us as CLA champions but as BAA 
champions.
63
Asked if the mission had any CLA champions, the same BAA team member responded:
I wouldn’t go that far. I have to admit there are different efforts of incorporating CLA more 
consistently. . . . Maybe I have very high expectations. I don’t see somebody going above and 
beyond that this [CLA] happens. Then I’d call that person a champion.
64
Another BAA team representative was also willing to identify BAA champions but was dismissive of the
notion of CLA champions.
I am familiar with the term champion. BAA champion is the deputy mission director who 
formed the Working Group on BAA . . . and met with the Working Group weekly. Also the 
Director of Contracts. . . . CLA champions . . . not very important topic for the mission. CLA is 
used in some offices in design of activities and to coordinate. That’s as much as I know.
65
In contrast to the views of these BAA team members, a third BAA team member self-identified as a 
CLA champion and named four others: two of whom were on the BAA team, one in the DGO and the 
M&E officer in the PO (who LER/CLA identified as a champion).
I know the term champion. A lot of people are interested. Most, like me, are FSN. We must 
respond to our Office Directors changing every two to four years. Yes. I do consider myself a 
champion.
66
In summary, though none of the BAA team members, including the self-identified CLA champion and 
two of her team members whom she named as fellow champions, promoted the BAA process as CLA 
during or after the TDYs. Nonetheless, given that the BAA process is by definition a CLA process, BAA 
champions must also be CLA champions, albeit tacit ones. The question arises whether the BAA 
champions ceased to function as CLA champions once the BAA process was terminated or continued to 
operate as implicit CLA champions in other ways.
Q3: Outside of LEARN’s ToC about CLA champions driving CLA integration, what are 
alternative explanations for increased CLA integration? 
LEARN’s ToC about CLA champions driving CLA integration is validated by the role of BAA (or implicit 
CLA) champions advocating for the mission to use a BAA process (incorporating CLA practices and 
principles through co-creation), and implementing it. Beyond the BAA process, the mission provided 
other kinds of support for CLA integration. With only three mission interviews, the data is insufficient 
to understand the driving force behind mission support for these other examples of CLA integration, 
although, arguably, they should advance CLA integration. Like the Cambodia case, the finding of mission 
support for CLA integration has implications for expanding or refining LEARN’s ToC to explicitly 
include enabling conditions as a potential driver of behavior change.
21
SUPPORT FOR CLA IN THE ENABLING CONDITIONS
The following examples illustrate CLA support in the enabling conditions.
• Positive feedback from the BAA team about LEARN’s facilitation led the PO to decide to hire 
LEARN to facilitate a CDCS review. After PPL said missions could take another two years to do 
CDCS reviews because of uncertainty under the new administration, the mission opted not to 
do the exercise.67
• CLA is being incorporated as part of the new Metrics Project. Metrics Project (Metrics) is a five￾year contract that will provide monitoring, evaluation design and learning support services to 
the Guatemala mission. Metrics is the successor to the mission’s Monitoring and Evaluation 
Program (2013-2017).68
• In July 2016, the mission introduced a new mission order that requires design teams to include 
representatives from various TOs.69 A respondent to the online survey commented that the 
introduction of the mission order was independent of LEARN’s facilitation of the BAA process.70
The PO representative on the BAA team, who is cited earlier as not viewing anyone going above and 
beyond his/her job responsibilities, responded:
The Program Office. We promote the use of evaluation results and implementation of them —-
not in a police role. The Program Office has representatives in each technical team. . . . Making 
sure IPs share information/results is more of a Technical Office role. They make sure 
performance evaluations are shared and results are used. Complying with the Mission Order is 
the Program Office’s responsibility.
71
It is tempting to infer from the PO representative’s comments that the PO played the role of CLA 
champions in driving support for these cases of CLA integration. Such an inference, however, would 
remove the explanatory power of LEARN’s ToC as every instance of CLA integration could be 
attributed to a CLA champion.
The PO representative on the BAA team saw no particular obstacles to CLA integration at the mission 
insofar as all mission staff accepted the need to implement CLA and nobody resisted implementing CLA, 
even as she acknowledged how it is human nature to not always enact change.
Sometimes human nature. It is hard for us to change. We’re happy doing it this way. Takes 
time to change. I don’t see anyone or any team being more resistant and obstacle to overcome. 
We’re trying to implement the CLA process. Consistently, we need to be flexible, incorporate 
lessons, adapt to lessons. All staff are convinced that’s the way to go. We are slowly 
incorporating it — that’s the way learning is being incorporated in our processes.72
COMPARING AND CONTRASTING CAMBODIA AND GUATEMALA MISSIONS
The most important distinctions affecting CLA integration were the PPL/LER and LEARN approaches 
employed at the missions, the role of champions and the extent to which the PO prioritized and 
intentionally promoted CLA. Other factors mattered, though much less. To the extent that the MTE
hypothesized the significance of some of these factors, they played out in mostly unanticipated ways.
CLA APPROACHES
One major factor accounting for differences in the contribution of the TDYs to CLA integration was the 
specific PPL and LEARN approach during the TDY. At the Cambodia mission, PPL and LEARN’s 
22
facilitation of the CDCS midcourse stocktaking and the CLA Maturity Tool Self-Assessment and Action 
Planning process contributed to a conscious, intentional and systematic use of CLA at the mission and 
was an important impetus for an overall increase in the level of CLA integration. At the Guatemala 
mission, the novel BAA process inherently involved more CLA than usual for the BAA team and its 
partners, and LEARN’s facilitation of that BAA process thus contributed to a deepening of the BAA 
team’s implicit use of CLA.
The explicit use of CLA at the Cambodia mission and the implicit use of CLA during the BAA process at 
the Guatemala mission sharply raise the question of what difference implicit versus explicit CLA might 
make for CLA integration and development outcomes.
CLA CHAMPIONS
LEARN had identified only one champion at the Cambodia mission and none at the Guatemala mission. 
This led the assessment lead to postulate that an increase in CLA integration at missions with no 
champions or only one champion would call into question LEARN’s ToC, according to which CLA 
champions drive CLA integration.
However, the Cambodia mission identified numerous CLA champions located in different offices 
throughout the mission. The active, promotional role of the PO, which included at least two FSN 
champions with high social capital, was crucially important to advances in CLA integration, including the 
incorporation of CLA in several documents. The Guatemala mission could not identify CLA champions, 
although the PO had introduced CLA explicitly in at least one document and had recently acquired an 
MEL contract mechanism. The PO had also planned to hire LEARN to facilitate a CDCS midcourse 
stocktaking before PPL’s directive giving missions an extension on revising their CDCS documents. This 
year (2017) the DGO had requested the presidential fellow from the DRG Center, who had been 
invited to help draft their Project MEL Plan, to assist with incorporating a CLA Plan in the Project MEL 
Plan, again suggesting an appetite for explicit CLA at the mission.
MISSION-WIDE VERSUS TEAM-SPECIFIC TDYS
PPL/LEARN’s use of the CDCS midcourse stocktaking and the CLA Maturity Tool Self-Assessment and 
Action Planning process at the Cambodia mission, both mission-wide tools, resulted in a wider impact 
on the mission in terms of CLA integration than LEARN’s team-specific facilitation of the BAA process 
at the Guatemala mission for reasons beyond the different CLA approaches. At the Cambodia mission, 
the PO contained CLA champions who played a pivotal role in integrating CLA throughout the mission. 
In contrast, at the USAID/Guatemala mission, the PO made no effort to extract the CLA lessons of the 
BAA process and share them with the mission. The BAA team lead, the only person in the education 
office during the entire facilitation process until the BAA award, was not recognized at the mission as a 
CLA champion, and made no effort to spread the CLA process embedded in the BAA experience.
LENGTH OF TDYS
Prior to the interviews with mission staff, the hypothesis was that if the TDYs totaling only 24 days at 
the Cambodia mission had contributed to increased CLA integration, the conditions at the mission 
making progress in learning achievable in such a short time would warrant further exploration. As it 
transpired, the length of engagements at the two missions mostly reflected each mission’s overall 
management. 
The Cambodia mission always envisaged two TDYs, and both were well designed.73 The Guatemala 
mission had anticipated only one TDY, but because the BAA process got entangled in internal mission 
23
issues arising from mission leadership changes and unmet budget expectations, the BAA process 
required a second workshop in which the partners were given new ground rules by the mission (e.g., for 
the first time in the process, the BAA team informed partners their proposals had to meet a specified 
budget, asked them for one rather than two concept notes, and said one rather than two primes had to 
be identified). LEARN had anticipated only one workshop with the BAA team.74 Since LEARN had over￾budgeted for the TDY, additional buy-in funds were not a problem.75
MISSIONS’ CLA REPUTATIONS
Subsequent to the interviews, it came to the attention of the assessment lead that the Monitoring, 
Evaluation, Research and Learning Innovations (MERLIN) Program, an initiative led by the USAID Global 
Development Lab and designed by PPL, showcased the Cambodia mission’s strong support for CLA 
integration.76 Cambodia’s CLA reputation was therefore initially understated. In contrast, the 
expectations about the Guatemala mission might have been exaggerated. The mission’s Western 
Highlands Integrated Project (WHIP), part of the current CDCS (2012-2017), received positive publicity 
for its collaborative design. According to one mission member, however, WHIP was: 
“ . . . too ambitious. Our interventions when implementing were not being integrated, just co￾located. IPs got together to ensure they were not stepping on each other’s toes. . . . But because 
of how the mechanisms were written, there were limits on flexibility and how IPs could be 
integrated and work together.
77
COMPOSITION OF TDY TEAMS
It is difficult to assess if LEARN’s facilitation at the Guatemala mission might have been different if the 
LER/CLA Team had participated, as it has on all other LEARN TDYs. According to a LEARN team 
member, PPL brings to the TDYs a USAID insider perspective and social capital vis-à-vis mission 
leadership which LEARN does not have.78
RECOMMENDATIONS
Based on the findings provided by the evaluator, LEARN facilitated a participatory recommendation 
building process within the team and with PPL/LER/CLA team. There were two phases to the 
recommendation building process:
• CREATING RECOMMENDATIONS BASED ON FINDINGS: The evaluator presented 
her findings to key stakeholders from the LEARN and PPL/LER/CLA team. Participants then 
reacted to those findings by generating recommendations for further action by the LEARN 
team. This initial list of recommendations was provided to the entire LEARN team, and the 
LEARN Manager for Monitoring, Evaluation, Research and Learning (MERL) facilitated a session 
where the entire LEARN team expanded and subsequently prioritized a final list of 
recommendations. 
• ACTION PLANNING: Based on the final recommendations, the MERL Manager synthesized 
all recommendations into the four recommendation categories listed below and facilitated a 
session where team members action planned the top priorities under each of these categories. 
The idea was to plan actions that were relatively low level of effort and potentially high impact, 
given the already full workload the LEARN team is managing. 
24
Based on this process, LEARN and PPL/LER co-created the following recommendations: 
• BE EXPLICIT AND INTENTIONAL ABOUT FAMILIARIZING MISSIONS WITH CLA
Our data suggest that language matters and integrating CLA language across our technical 
assistance work will likely strengthen the impact of our work. Regardless of the type of technical 
assistance, this includes helping missions identify where they are already applying CLA practices 
and approaches to their work as well as ensuring that a “CLA 101” becomes a standard part of 
TDY agendas. To help missions further highlight their work as it relates to CLA and better 
development, case competition entry idea capture and follow-up support could also become 
part of the standard operating procedure for all TDYs.
• KEEP THE FOCUS ON CLA CHAMPIONS
Our midterm confirmed the essential role champions play in increasing and sustaining CLA 
integration. To the best of our ability, we should strive to maintain regular contact with CLA 
champions from missions post-TDY. These conversations will hopefully offer opportunities for 
us to provide champions with tools, evidence and other guidance needed to apply CLA. The 
data also suggested that we are unaware of the actual number of CLA champions at each 
mission. In our continued contact with known champions, we can ask for their help in identifying 
other individuals who exhibit the behaviors of a champion. Lastly, the USAID/Guatemala case 
shows us that people do not always self-identify as CLA champions. If needed, we can frame our 
conversations around ‘championing CLA’ as this subtle shift could encourage more uptake of 
CLA approaches and principles.
• CONSIDER INTEGRATING A DEVELOPMENTAL EVALUATION INTO LEARN 
STANDARD
The data presented in the midterm raised many additional questions. Implementing a 
developmental evaluation would allow LEARN to follow up on a select number of missions for 
the remainder of the contract. Illustratively through this methodology, we could deepen our 
understanding about: 
o change processes in missions.
o the connection between language and behavior change.
o the most effective type or combination of CLA interventions.
o the number or quality of touchpoints needed to obtain optimal CLA integration in 
missions.
o longer-term impact of TDYs.
The developmental evaluation should fold into existing MERL processes and will be designed 
with a low level of effort in mind.
• UPDATE LEARN’S RESULTS FRAMEWORK
The midterm process showed us that we need to clarify the difference between enabling 
conditions and systems-level change. Currently, our results framework lacks clarity on LEARN’s 
ability to influence enabling conditions (i.e., culture, processes, resources); yet, our data tells us 
that our work contributes to changes in the enabling conditions. We do not have sufficient 
influence over broader changes at the policy and systems level, however, and the results 
25
framework should be updated accordingly. Additionally, post-LEARN, a follow-on contract may 
warrant a more complex results framework depicting the dynamics between people and 
institutions and how changes in either magnify changes in the other. Right now, our framework 
does not account for these types of interactions and potential amplifying effects.
26
ANNEX 1: CAMBODIA MISSION INTERVIEW QUESTIONS
1. Which PPL/LEARN engagements did you participate in?
• Did you participate in the PPL/LEARN facilitation of the CLA Maturity Matrix self￾assessment and action planning in March 2016?
• Did you participate in the PPL/LEARN facilitation of the CDCS midcourse stocktaking 
exercise in March 2016?
• Did you participate in the PPL/LEARN facilitation of the CDCS midcourse stocktaking 
exercise in September 2016?
2. What kinds of changes, if any, did you experience personally or observe in the use of CLA in the 
program cycle processes and in the enabling conditions to support CLA after the PPL/LEARN 
engagements? How, if at all, did PPL/LEARN’s approaches to CLA contribute to any of these 
CLA changes?
3. Do you think any of the positive changes you have identified with respect to CLA in the 
program cycle and/or the enabling conditions at the mission as a result of PPL/LEARN 
engagements are sustainable? Please be specific about the changes to which you are referring.
• What would it take to institutionalize these changes?
4. How do you think these positive changes in CLA in the program cycle and/or changes in the 
enabling conditions (if any) have come to be integrated in the mission?
5. If you believe positive changes have occurred in CLA integration after the PPL/LEARN 
engagements, do you think individuals at the mission played an important role in the change 
processes? If no, why? If yes:
• How many people at the mission do you think are promoting CLA?
• Do individuals who promote CLA share any characteristics? If so, could you describe 
their key characteristics?
• Do individuals who promote CLA do particular types of work that require particular 
skills and knowledge at the mission? Discuss.
• Do individuals who promote CLA occupy particular ranks at the mission?
• Are they in particular offices?
• Are promoters of CLA at the mission more likely to be FSNs or FSOs? Why?
• Do promoters of CLA have particular attitudes? Discuss.
• Do CLA promoters have particular behaviors? Discuss.
• Do individuals who promote CLA at the mission enjoy particular respect or influence 
among mission staff? Discuss.
• Would having more CLA promoters at the mission foster increased CLA? If so, how 
many CLA promoters do you think are needed to successfully integrate CLA?
27
6. Are there individuals who hinder the promotion of CLA at the mission?
• Can you characterize them by rank/work status (FSN/FSO), office, attitudes, behavior, 
etc.?
• How do they impede CLA?
7. Are there any key obstacles (internal or external) to the mission achieving more CLA 
promoters?
• Can any of these obstacles be diminished or eliminated? How?
8. The Cambodia mission ranked among the top five adaptable missions in a LEARN analysis of 
CLA using FEVS data (2013-2016). Does this finding fit with your experience of the mission? 
Discuss.
9. Could you comment on the length of the two PPL/LEARN engagements — each was 12 days.
• Were the engagements too short, just right, too long?
• If the engagements were too short, what more would you have wanted PPL/LEARN to 
do at the mission?
• If the engagements were too long, what could PPL/LEARN have done differently and 
why?
10. Did PPL/LEARN’s engagements contribute to a change in “learning” at the mission? Using a scale 
of 1-5 [1 = poor, 2=fair, 3=okay, 4=good, 5=excellent], could you rate the level/quality of CLA 
in the program cycle processes (including the design and implementation of strategy, projects
and activities)?
• Before PPL/LEARN’s first engagement (in March 2016)? 
• After PPL/LEARN’s second engagement (in September 2016)?
11. Using a scale of 1-5 [1 = poor, 2=fair, 3=okay, 4=good, 5=excellent], could you rate the 
level/quality of enabling conditions to support CLA (e.g., does the mission create a comfortable 
environment for criticism and alternative perspectives?; does the mission leadership encourage 
staff to take time for learning beyond required training?; does the mission have good 
communication channels for sharing knowledge?; does the mission use exit interviews with staff 
to capture knowledge?; does the mission allocate resources for hiring staff with CLA skills?)
• Before PPL/LEARN’s first engagement (in March 2016)? 
• After PPL/LEARN’s second engagement (in September 2016)?
12. Besides your experience with the PPL/LEARN contract, has the Cambodia mission had 
experience with any other monitoring, evaluation and learning (MEL) contracts?
• If so, please name and describe their length and purpose.
13. Would you be willing to provide the name and email of a potential interviewee who might have 
a different perspective from you on CLA at the mission?
28
ANNEX 2: USAID/GUATEMALA MISSION INTERVIEW 
QUESTIONS
1. Do you prefer confidentiality?
2. Why did the mission use a BAA process rather than a traditional approach to project design? 
Could you discuss some of the obstacles to getting the mission to approve using the BAA 
process?
3. Why did the mission contract with LEARN to facilitate the BAA process? Why did the mission 
not view the BAA process as a CLA contract?
4. If you were to use LEARN to facilitate another BAA process, would you recommend it adopt 
any changes?
5. Would you seek any changes in the length of LEARN’s engagements in the BAA process? Why?
6. Did LEARN’s facilitation of the BAA Identifying Youth Opportunities contribute to any changes 
in (a) how you use CLA in your work (b) how your office uses CLA (c) how CLA is used in 
inter-office work and (d) how you work with partners?
7. Did your office create any documents relating to your experience with the BAA mechanism? 
Was your experience shared with the rest of the mission? Why?
8. Does your office/mission have any CLA-related documents? Share.
9. Apparently, a mission champion played an important role in overcoming resistance at the 
mission to the use of a BAA process. Who was the champion? Discuss the champion’s 
characteristics and what made the champion influential in overcoming opposition? Role of 
mission leadership is mentioned too. Discuss.
10. Are there other champions/CLA promoters at the mission? Who/How many/Offices/FSN￾FSO/levels in the organization? What role, if any, did PPL/LEARN play in creating these 
champions?
11. How important are CLA promoters/practitioners for increasing CLA integration? Why?
12. What’s the best way to increase the number of CLA promoters/practitioners?
13. Are there any obstacles to greater promotion of CLA at the mission — (a) individuals — who? 
If any, how and why? (b) institutional factors (c) other?
14. Other than using CLA promoters, what other strategies might/does the mission use to: (a) 
promote CLA in program cycle processes; (b)support CLA in the Enabling Conditions?
15. Guatemala mission has a reputation for innovative approaches to development and has often 
been viewed as a leader in collaboration with partners and others. Response?
16. Any prior MEL contracts at the mission/your office to your knowledge?
29
17. Suggest IPs involved in BAA process to talk to?
18. Anything you’d like to comment on about CLA integration at the mission?
FOR EDUCATION & HEALTH OFFICE (EHO):
1. Why did the EHO use the LEARN contract to do the Options Research on different potential 
education projects? Does the office contract out research routinely?
2. Did your office’s experience of LEARN’s Option Research and consultations with stakeholders 
contribute to CLA at your office? How?
3. Have you shared the Options Research findings with the mission leadership yet?
30
ANNEX 3: FEEDBACK SURVEY — LEARN MIDTERM
The Bureau of Policy, Planning and Learning and its mechanism, the Learning and Knowledge 
Management Contract, (PPL/LEARN) is conducting a Mid-Term Assessment of its efforts to facilitate 
Collaboration, Learning and Adaption (CLA) at USAID missions. The Cambodia mission is one of 
two missions that has been selected for in-depth study of PPL/LEARN’s contribution to CLA at 
missions. PPL/LEARN facilitated CLA at the Cambodia mission in March and September 2016. In 
March, PPL/LEARN facilitated the CLA Maturity Matrix self-assessment and action planning and a 
CDCS Midcourse Stocktaking retreat; in September, PPL/LEARN facilitated a second CDCS 
Midcourse Stocktaking retreat. As part of LEARN’s Mid-Term Assessment, PPL/LEARN is 
conducting an online survey for mission staff who participated in either or both of these PPL/LEARN
engagements.
The purpose of the anonymous online survey is to provide mission staff with an opportunity to 
share their perspectives of the impact, if any, that PPL/LEARN’s engagements have had on CLA at 
the mission. The survey should take at most 10 minutes to complete.
1. Did you take part in PPL/LEARN’s March 2016
engagement? Mark only one oval.
Yes
No
2. Did you take part in PPL/LEARN’s September 2016 
engagement? Mark only one oval.
Yes
No
3. How, if at all, did the PPL/LEARN engagement(s) contribute to any changes in the 
way you and the mission conduct your work? If you have not observed any changes, 
why do you think this has been the case?
4. What, if any, contribution do you believe that PPL/LEARN could make to CLA at 
the mission in the future? How best could PPL/LEARN make such a contribution? If 
you think PPL/LEARN can make no contribution to CLA at the mission, why do you 
think this to be the case?
31
ANNEX 4: LITERATURE REVIEW
This literature review examines the role of champions or change agents in organizational learning or 
organizational change. The review draws on a limited number of articles and books, selected partly 
through personal recommendation and partly through a library search using terms such as change agent
and organizational learning. 
The literature under review falls into two broad categories: 1) a focus on the important role of change 
agents (or change leaders) and the individual skills, appeals and relationships that make them successful;
2) an incorporation of the role of organizational structures (systems, processes, rules and routines) —
in LEARN’s language, enabling conditions (culture, processes, resources) and system-level factors —
either as providing the conditions for the emergence of change agents or as operating in a dynamic 
relationship with change agents to advance organizational learning and change.
LEARN’s theory of change that postulates champions as the primary drivers of organizational learning is 
validated insofar as the literature identifies change agents as critical actors. However, the review also 
suggests that the success of change agents depends, not only on individual-level factors, but on 
organizational structures. The CLA framework explicitly incorporates enabling conditions and LEARN is 
aware of the potential role of organizational structures as drivers of organizational learning.1 However, 
the results framework could be updated to better represent the potential dynamic between individuals 
and organizational structures. Additionally, LEARN’s theory of change could fruitfully develop the 
individual-level factors that make champions successful.
HOW CHANGE HAPPENS: CHANGE AGENTS 
The literature that concentrates on change agents, as presented below, contains some useful lines of 
inquiry for LEARN’s theory of change. These include: becoming a change agent and necessary change 
agent skills, appealing to emotions, and developing change agent’s networks and relationships with those 
ambivalent about change.
BECOMING A CHANGE AGENT AND NECESSARY CHANGE AGENT SKILLS: Cowsey & Deszca 
(2007) highlight the role of change agents in transforming organizations. Their model for change suggests 
that “change agents move systematically from awareness of the need for change, through initiation, 
planning and implementing the change, to measuring and confirming the change” (367-8). Change agents 
will continue to need basic skills in facilitation, influencing, negotiation and visioning. They will need to 
understand project management, and be able to implement projects. Some change agents will be 
generalists and some will be specialists with knowledge of a particular industry, sector, or change target 
(376).
APPEALING TO EMOTIONS: While Cowsey and Deszca focus on the process of becoming a change 
agent and the skills a change agent needs, Kottner & Cohen (2002) ask how “change leaders” (middle 
managers and executives) can achieve behavior change in others. Based on stories told by middle 
managers and senior executives, Kottner & Cohen identify changing people’s behavior as the single 
biggest challenge for organizational adaptation and change. According to the authors: “The key to this 
behavioral shift . . . is less about analysis and thinking and more about seeing and feeling” (179).
____________________________________
1 Sarah Schmidt, LEARN’s deputy chief of party, personal communication, December 21, 2017. Despite its 
awareness of the potential role of organizational structures as drivers of organizational learning, LEARN made a 
strategic decision to focus on promoting the development of champions where it could achieve greatest impact. 
This decision was based on LEARN’s understanding of its manageable interest and the mandate of PPL to support 
CLA through throughout USAID’s program cycle.
32
Whatever the method, successful “change leaders” supply valid ideas that have an emotional impact.2
The feelings change behavior, enabling people to move through the various stages of large-scale change 
(182). While the number of change leaders needed in organizations seeking transformation will be 
subjective, the authors claim is that most have less than half the number they need (184). Departing 
from the stories told by managers and executives, the authors envisage people who probably would not 
"look like 'leaders' in a traditional sense” helping to provide “very real change leadership” (185).
CHANGE AGENT'S NETWORKS AND RELATIONSHIPS: Battilana & Casciaro (2013) examine how 
the personal networks of change agents affect their success in implementing change initiatives. The 
authors tracked 68 such initiatives in the U.K.'s National Health Service for one year after their 
inception to understand what makes some people successful at leading organizational change. Beyond 
the finding that their personal networks were critical, change agents were found to be more successful 
when:
• they were central to the organization’s informal network, regardless of their position in the 
formal hierarchy;
• people who bridged disconnected groups and individuals were more effective at implementing 
dramatic reforms, while those with cohesive networks were better at instituting minor changes.3
• they were close to “fence-sitters,” or people ambivalent about the change. 
HOW CHANGE HAPPENS: DYNAMICS OF CHANGE AGENTS AND STRUCTURES
Similarly, models or theories of change based on a dynamic relationship between change agents and 
organizational structures suggest how LEARN’s theory of change may be modified and present 
interesting areas for investigation. Studies under review focus variously on ways: 
• internal organizational processes are needed either to hire or enable change agents to be 
effective.
• change agents at each organizational level require different competencies that must be 
integrated into systems.
• change agents at each organizational level are most likely to succeed in creating new ideas and 
institutionalizing them by employing specific political strategies tailored to defined processes of 
change.
• institutional entrepreneurs are enabled and constrained by culture, institutions and social relations 
and how these systemic factors shape institutional entrepreneurs.
• different systems may shape shared interest group learning.
ENABLING CHANGE AGENTS: Honig and Gulrajani (2017) focus on ways development aid 
organizations can improve their performance. They suggest that such organizations will only be able to 
accomplish organizational transformations if they are able to adapt to local realities. To do so requires 
____________________________________
2
Successful change leaders identify a problem or solution, show it to people in ways that are concrete, make their 
points in ways that are emotionally engaging, model needed behavior, ensure that the result is visible (e.g., on 
bulletin boards) and provide a means for people to continue to see the result every day through physical objects 
or through the (re)telling of vivid stories (182).
3 Cohesive network: “The people in your network are connected to one another. This builds trust and mutual 
support, facilitating communication and coordination” (65). Bridging network: “Your network contacts are not 
connected to one another. You are the bridge between disparate individuals and groups, giving you control over 
what, when and how you communicate with them” (65).
33
development agencies to change their internal processes (e.g., remove central command and control) to 
create the motivation, autonomy and trust that change agents need to be able to respond effectively to 
changing local contexts.
CHANGE AGENT COMPETENCIES THROUGHOUT THE ORGANIZATIONAL HIERARCHY: 
McFarland and Goldsworthy (2014) identify how organizations must recognize that change leaders 
require specific competencies and embed them as requirements in organizational systems. Further, 
different organizational levels require different change leader competencies. The authors interviewed or 
conversed with more than 60 executives in numerous organizations about individual and organizational 
change. Executives consistently mentioned leadership as the key factor in successful change efforts and 
the importance of building a cadre of change leaders. Other findings include:
• Great change leaders are proven business executives; balance the technical, political and 
developmental aspects of leading change; shape the organizational environment during change; 
create balance between control and flexibility, and can change themselves (171-2).
• To develop a cadre of change leaders, organizations need to formally identify a new, 
organizational-specific competency for leading change at every leadership level, integrate the 
new competency into the organization’s overall leadership model, and modify the performance 
management system to formally assess and promote leaders on their performance in leading 
change (173-4).
• Enabling learning in individuals, teams and the whole organization is the key role for a change 
leader, as learning is necessary for an organization to continuously improve at organizational 
change. Change leaders at each level enable learning in different ways, acting as: a coach and 
mentor to create an environment conducive to learning about change at the individual level; a 
facilitator at the team level; and an integrator, building the structures and processes needed to 
share and use learning at the organizational level (208).
CHANGE AGENTS AND POLITICAL STRATEGIES: Lawrence, Mauws, Dyck and Fleysen (2005) 
argue that change leaders perform different roles at different organizational levels, but highlight the 
politics of why only some valuable new ideas become institutionalized while others do not. The authors 
begin with Crossan, Lane and White ‘s model of organizational learning that posits learning begins with 
individuals, leads to group learning and then to organizational learning. The model describes four 
processes — intuiting (tacit knowledge), interpreting (explicit knowledge), integrating, institutionalizing 
— that connect individual to organizational learning.4 Lawrence et al. then build power and politics into 
the model to explain or predict which new ideas will be institutionalized and which organizational 
institutions (e.g., systems, routines) will provide the basis for further intuitions, the starting point for 
new ideas. They argue for a set of specific connections between political strategies and each of the four 
processes of organizational learning. More specifically, influence, according to the authors, is useful in 
overcoming the ambiguity and uncertainty associated with interpretation; force facilitates the 
accomplishment of collective action in the integration process; domination overcomes potential 
resistance to change and thus supports institutionalization; and, discipline supports the development of 
____________________________________
4
Intuiting and interpreting are individual-level processes; integrating occurs at the group level; and institutionalizing 
is at the organizational level. Together these four processes form a learning loop through the effect of new 
institutions (systems, structures, procedures and strategy) on organizational members’ experiences that feed into 
their individual intuitions.
34
the expertise that is necessary to foster intuition” (188-9). The model’s most important practical 
implication (188) is “that the championing of an idea may require very different skills and resources (and 
consequently different champions) as the idea passes through the different processes of organizational 
learning.”5
INSTITUTIONAL ENTREPRENEURS: Like Lawrence et al., Battilana, Leca & Boxenham (2009) 
develop a theory in which organizational actors and systems interact to enable and constrain change. 
Battilana et al. add to the sizeable literature on institutional entrepreneurship. Institutional 
entrepreneurs are change agents who must both initiate divergent change (major organizational change) 
and actively participate in implementation of these changes. Their approach tries to understand how 
culture, institutions and social relations influence actors’ cognition and actions, either constraining or 
enabling actors’ choices, which in turn shape institutions. In this way, the authors seek to overcome the 
weaknesses of agency models that do not take into account structures and build in a dynamic 
relationship between agency and structures, with each able to influence the other.
SYSTEMS AND GROUP LEARNING: Finally, Laurie Field (2017), like Lawrence et al., argues that 
political conflict in organizational learning is an inevitable and potentially positive force for learning. 
Drawing on the sociology of work, Field examines the different interests of employees and managers in 
a large Australian finance sector company that introduced a new pay and performance management 
system. The study raised questions about whether much of what the literature terms organizational 
learning may be merely shared interest group learning — in this case, by managers and employees.
BIBLIOGRAPHY
Battilana, Julie, Bernard Leca and Eva Boxenbaum. “How Actors Change Institutions: Towards a Theory 
of Institutional Entrepreneurship.” The Academy of Management Annals 3, no. 1 (2009): 65-107.
Battilana, Julie and Tiziana Casciaro. “The Network Secrets of Great Change Agents.” Harvard Business 
Review 91, nos 7/8. (July-August 2013): 62-68.
Belle, Stuart. “Organizational learning? Look Again.” The Learning Organization 23, no. 5 (2016): 332-341. 
Boucher, Johanne and Joanne M. Roch. “Absorptive Capacity: North-South Partnerships and 
Organizational Learning within Development Networks.” European Journal of Development 
Research 29, no. 2 (2016): 404-422.
Cowsey, Tupper and Gene Deszca. Toolkit for Organizational Change. Thousand Oaks, CA: SAGE
Publications, 2007.
____________________________________
5
“The chances of an idea passing through the interpreting process will be improved if a champion can use 
influence, access informal networks, frame the idea in a compelling manner and manage the ambiguity of the 
process. To integrate an idea into group activities, the champion ideally should have the authority to ensure that 
collective action is enforced and should have a direct link to the organization’s dominant coalition. To 
institutionalize an idea at the organizational level will then need a champion who is able to design and implement 
systems of domination. Such a person will likely come from within the organization’s dominant coalition and will 
understand the dynamics of pan-organizational systems" (188). “Finally, for the new idea to generate further 
intuitions that are consistent with and build on it, it will need members who are sufficiently steeped in the 
disciplinary systems of the organization to recognize or imagine novel patterns, and who can transform those 
patterns into compelling metaphors that might restart the whole set of processes” (188, 190).
35
Field, Laurie. “Interest Differences and Organizational Learning.” Administrative Sciences 7, no. 26 (2017). 
doi: 10.3390/admsci7030026
Honig, Dan and Nilima Gulrajani. “Making Good on Donors’ Desire to Do Development Differently.” 
Third World Quarterly (2017). doi: 10.1080/01436597.2017.1369030
Kotter, John P. and Dan S. Cohen. The Heart of Change: Real-Life Stories of How People Change Their 
Organizations. Boston: Harvard Business School Press, 2002.
Lawrence, Thomas B., Michael K. Mauws, Bruno Dyck and Robert F. Fleysen. “The Politics of 
Organizational Learning: Integrating Power into the 4I Framework.” Academy of Management 
Review 30, no. 1 (2005): 180-191.
McFarland, Walter and Susan Goldsworthy. Choosing Change. How Leaders and Organizations Drive Results 
One Person at a Time. McGraw-Hill Education, 2014.
Schwandt, David R. and Michael J Marquardt. Organizational Learning: From World-Class Theories to Global 
Best Practices. Boca Raton, FL: St Lucie Press, 2000.
36
ANNEX 5: GLOSSARY OF USAID LEARN TERMS
CHAMPIONS TRACKER is an online Google sheet where LEARN keeps track of CLA 
champions.
CLA CASE COMPETITION is hosted annually by USAID to capture real-life case studies of USAID 
staff and implementing partners using a Collaborating, Learning and Adapting (CLA) approach for 
organizational learning and better development outcomes.
CLA CHAMPIONS are individuals in organizations who promote, model and/or integrate CLA in their 
workplace.
CLA FRAMEWORK identifies key components and subcomponents that help define what CLA is in 
practice. The framework gives CLA structure, clarity and coherence across two dimensions: CLA in the 
program cycle and enabling conditions.
• CLA in the program cycle: how CLA is incorporated throughout program cycle processes, 
including strategy, project and activity design and implementation.
• Enabling conditions: how an organization’s culture, business processes and resource allocation 
support CLA integration.
CLA INTEGRATION refers to the intentional, systematic and resourced application of the CLA 
approach (as outlined in the CLA framework) by development practitioners.
CLA INTEGRATION CHECKLIST DATA refers to LEARN’s capture of instances of CLA integration 
by sub-components of the CLA framework.
CLA MATURITY TOOL includes one key concept and five stage cards for each subcomponent in the 
CLA Framework. The stage cards describe a spectrum of practice from Not Yet Present to 
Institutionalized.
COLLABORATING, LEARNING AND ADAPTING (CLA), USAID’s approach to improving 
organizational learning, is intended to help USAID and its implementing partners address common 
challenges that confront international development assistance and improve development results. In 2012, 
USAID’s Bureau for Policy, Planning and Learning introduced the concept of CLA as a way to 
operationalize adaptive management throughout USAID’s program cycle.
COUNTRY DEVELOPMENT COOPERATION STRATEGY (CDCS) MIDCOURSE 
STOCKTAKING is a CDCS midterm review that provides a structured opportunity for a mission to 
determine if its strategy is still appropriate to country context and Agency priorities and adapt strategy 
based on learning to date. USAID LEARN works with missions to design and facilitate this exercise to 
capture emerging knowledge and lessons learned. For more information on CDCS midcourse 
stocktaking, please see [Automated Directives System] ADS 201 and ADS Reference 201MAG.
KNOWLEDGE DROP is an internal LEARN blog that allows staff to share in real-time learning and 
reflections.
TDY TRACKER is an online Google sheet where LEARN tracks its TDYs.
37
ANNEX 6: LEARN’S CLARIFICATION ON EVALUATION FINDINGS
The LEARN contract is adding this annex to clarify some points made in the mid-term evaluation, 
particularly in response to the evaluation’s question 3 (Outside of LEARN’s theory of change about CLA 
champions driving CLA integration, what are the alternative explanations for increased CLA 
integration?).
LEARN’s theory hinges on the idea that CLA champions drive CLA integration. While the evaluator 
does not point to an alternative to LEARN’s TOC,6 she does indicate that LEARN’s ToC does not take 
into account enabling conditions. The evaluator writes, “LEARN’s TOC does not take explicit account 
of how such increased institutional support may reinforce the role of CLA champions in promoting CLA 
integration because it does not explicitly identify a role for structural or institutional support (or 
enabling conditions) for increasing CLA integration” (pg. 15).
LEARN wanted to clarify that we believe our theory of change does take enabling conditions into 
account based on how we define CLA integration, which includes consideration for enabling conditions 
as defined in the CLA framework. CLA integration is both CLA in the Program Cycle (Collaborating, 
Learning and Adapting components) and Enabling Conditions (Culture, Processes and Resources). We 
intentionally included a focus on enabling conditions within the CLA framework because we recognize 
the dynamic interplay between individual champions and their enabling conditions. 
While some champions can integrate CLA regardless of the enabling conditions, it is much easier for 
champions to promote, model and integrate CLA practices when enabling conditions are ripe (meaning, 
they are operating in a context where the culture supports CLA, CLA-related processes are in place, 
and there are sufficient resources to systematically and intentionally integrate CLA). 
In addition, positive enabling conditions come into existence because of what individuals (i.e., champions) 
decide to do or not do. Those enabling conditions can improve or worsen based on the actions of 
individual champions within missions and implementing partners, and champions can help create better 
enabling conditions for CLA to thrive. 
In recognition of this confusion that arose during the mid-term evaluation, LEARN saw the need to 
update its results framework to clarify that CLA integration is related to both CLA integration in the 
Program Cycle and organizational enabling conditions. The revised results framework (below) includes 
specific mention of CLA integration in the Program Cycle and organizational enabling conditions at the 
goal level (as opposed to the previous version that only mentioned integration in the Program Cycle). In 
addition, we have adjusted the assumption box to the right from “USAID and partner enabling 
conditions for CLA are improved” to “USAID’s operational environment remains conducive to CLA 
integration.” This assumption box was always intended to refer to systems-level issues within USAID 
(performance incentives, flexibility in contracting, supportive policy, etc.) but the language lacked 
precision and was confusing given its reference to “enabling conditions,” which also appears in the CLA 
framework. 
As described to the evaluator, LEARN sees the CLA framework as dynamic and interconnected (no 
component exists in isolation), demonstrating the importance of enabling conditions to support 
individual and team actions when implementing the Program Cycle. 
____________________________________
6 “The Cambodia case finding of substantial mission support for CLA in enabling conditions, rather than 
pointing to an alternative to LEARN’s ToC, draws attention to a positive dynamic between CLA 
champions and greater support for CLA integration in the enabling conditions.” (pg. 15)
38
Figure 4: LEARN’s Updated Results Framework
39
ENDNOTES
1
For more information on CLA, please see the Glossary on p. 35 of this document.
2
For more information on CDCS, please see the Glossary on p. 35 of this document.
3
LEARN CLA Integration Checklist. Examples of CLA Integration within Culture, Processes and Resources. 
Processes, Knowledge Management: “Having good communication channels and approaches for sharing knowledge 
(i.e., newsletters, etc.).” 
4
LEARN CLA Integration Checklist. Examples of CLA Integration within Culture, Processes and Resources. 
Resources, Implementing Mechanisms: “Solicitation requirements around CLA (including CLA in evaluation criteria, 
language encouraging adaptive management, collaboration, or learning activities, etc.).”
5
For more information on BAA, please see the Glossary on p. 35 of this document.
6
LEARN CLA Integration Checklist. Examples of CLA Integration within Culture, Processes and Resources. 
Resources, Mission: “Using co-creation approaches (i.e., BAAs, etc).”
7 While four mission staff participated in the BAA, only three were available for interviews.
8
LEARN CLA Integration Checklist. Examples of CLA Integration within Culture, Processes and Resources. 
Resources, Mission: “CLA or MEL support mechanisms.”
9
LEARN’s LEARN Contract Non-Confidential.
10 LEARN Contract Non-Confidential.
11 LEARN Monitoring, Evaluation, Research and Learning Plan, Updated FY 2016, 2.
12 LEARN Contract Non-Confidential
13 LEARN Contract Non-Confidential
14 Interview with LEARN’s CoP and Deputy CoP, September 14, 2017.
15 Interview with LEARN’s CoP and Deputy CoP, September 14, 2017.
16 Interview with LEARN’s COR, September 22, 2017. 
17 LEARN’s CoP, comment on draft MTE report, December 6, 2017.
18 Piers Bocock, Knowledge Drop, June 24, 2016; Interview with LEARN’s COR, September 22, 2017. 
19Interview with LEARN’s COR, September 22, 2017; Piers Bocock, Knowledge Drop, June 24, 2016. 
20 Interview with LEARN’s CoP and Deputy CoP, September 14, 2017.
21 USAID, LEARN, Performance Monitoring Report, September 29, 2016-March 31, 2017. Prepared by Dexis 
Consulting Group, Submitted May 12, 2017.
22 PPL/LER had learned that the DRC mission had scrapped its performance management plan and was rethinking 
its strategy and indicators. PPL/LER therefore judged it would be a difficult time to engage the mission as a 
participant in the MTE.
23 Options Research refers to LEARN’s research on three potential areas of intervention for an education project. 
For each possible intervention area, LEARN gathered information on existing activities in relation to: pros, cons, 
risks, assumptions, unmet needs, key stakeholders and best practices.
24 About the Mission, Knowledge Drop.
25 Out-brief Memo to USAID Guatemala PPSO from PPL/LER, October 4, 2013.
26 Out-brief Memo to USAID Guatemala PPSO from PPL/LER, October 4, 2013.
27 In contrast to LEARN, LER/CLA lists four champions at the Guatemala mission and Angel Lopez, an M&E 
specialist and FSN, is identified as a primary champion (About the Mission, Knowledge Drop).
28 LEARN’s Deputy CoP, meeting notes, May 4, 2017.
29 Out-brief Memo to USAID Guatemala PPSO from PPL/LER, Zan Larsen, Monica Matts, October 4, 2013, 
describes the main purpose of the TDY assisting the Guatemala mission in finalizing a draft performance 
management plan and to provide recommendations for systematizing the mission’s CLA plan. USAID/PPL 
conducted a CLA stocktaking exercise at the Guatemala mission in early 2015 (About the Mission, Knowledge 
Drop).
30 Interview with Eric Friesth, LEARN’s Senior Operations Manager, July 18, 2017. According to LEARN’s Senior 
Operations Manager, LEARN’s obligations for the Cambodia TDY were $23,176 and for the Guatemala TDYs 
were $246,404.
40
31 USAID, LEARN, Performance Monitoring Report, September 29, 2016-March 31, 2017. Prepared by Dexis 
Consulting Group, Submitted May 12, 2017.
32 For more information on champions tracker, please see the Glossary on p. 35 of this document.
33 For more information on TDY tracker, please see the Glossary on p. 35 of this document.
34 USAID LEARN, Ilana Shapiro, CLA Analysis of the Federal Employee Viewpoint Survey by USAID Mission, 2017.
35 LEARN says that LER/CLA rejected its suggestions to establish CLA baselines at all the missions at the start of 
the LEARN contract. Interview with Bari Rabin, LEARN’s Senior Capacity Building Specialist, April 13, 2017; 
Interview with LEARN’s CoP and Deputy CoP, September 14, 2017.
36 USAID/Cambodia, CLA Self-Assessment and Action Planning Session Report, Program Office, March 17, 2016.
37 Jenny Barker, LEARN Technical Assistance Manager, September 13, 2017. The LEARN facilitator’s identification 
of two CLA champions is at odds with LEARN’s internal tracker of CLA champions that identified only one CLA 
champion.
38 Jenny Barker, LEARN Technical Assistance Manager, September 13, 2017.
39 According to CLA’s team lead, the CLA team and LEARN go to TOs’ conferences (but must be invited by the 
regional bureau or pillar), PO conferences and evaluation conferences. LER/CLA also consults with newly 
appointed MDs when they are in Washington, DC, before they go to their missions. PPL is fortunate if it gets one 
session at MDs’ conferences, and then it has many topics to cover. That said, the CLA team lead said PPL itself has 
been slow to buy into the value of CLA as a concept, although the CLA team’s advocacy has begun to have a 
positive impact within PPL. “Maybe what we are doing at these venues is not visible to people at missions but we are 
doing these things except for the participation at MD conferences where we must get invited” (Interview with CLA team 
lead, September 20, 2017).
40 USAID Broad Agency Announcements identifies as one benefit among others for USAID: “Much more time 
efficient in terms of design to award lead time.” https://www.usaid.gov/partnership-opportunities/respond￾solicitation/broad-agency-announcements
41 Interview with Ligia Alfaro, PPSO, August 24, 2017.
42 BAA team interviews; 2017 CLA Case Competition, “Building bridges to collaboration: How co-creation led to 
the Puentes Project,” Laura Villegas, USAID/Guatemala.
43 Interview with Fernando Rubio, Juarez & Associates, September 22, 2017; Interview with Felix Alvarado, 
Vitruvian Consulting LLC, September 19, 2017.
44 Interview with Ligia Alfaro, PPSO, August 24, 2017.
45 Interview with Leia Cozier, LEARN Senior Project Associate, September 13, 2017.
46 Interview with Ligia Alfaro, PPSO, August 24, 2017.
47 Interview with David Martinez, FundaSistemas, September 19, 2017.
48 Interview with Fernando Rubio, Juarez & Associates, September 22, 2017; Interview with Felix Alvarado, 
Vitruvian Consulting LLC, September 19, 2017.
49 USAID/Guatemala, Broad Agency Announcement (BAA), Guatemala 2015, Addendum No. 1, Identifying 
Opportunities for Youth, Draft Statement of Work: BAA Facilitation Partner; LEARN Contract, Statement of 
Work, Guatemala Buy-in Activity Manager, Expected Period of Performance: March 1 to June 30, 2016.
50 Interview with Bari Rabin, September 19, 2017. Megan Meinen, USAID LAC, Washington, DC, did not respond 
to an email request (October 23, 2017) to participate in an interview.
51 Interview with Ligia Alfaro, PPSO, August 24, 2017.
52 Interview with BAA team member A, September 18, 2017 (confidential).
53 Interview with Fernando Rubio, Juarez & Associates, September 22, 2017.
54 Interview with Felix Alvardao, Vitruvian Consulting LLC, September 19, 2017.
55 Interview with Ligia Alfaro, PPSO, August 24, 2017.
56 Interview with Bari Rabin, September 19, 2017.
57 Interview with Bari Rabin, September 19, 2017.
58 Interview with BAA team member A, September 18, 2017 (confidential).
59 Interview with BAA team member A, September 18, 2017 (confidential).
60 Interview with Andrew Greer, September 6, 2017. See also Andrew Greer’s blog, June 30, 2017, that refers to 
other CLA activities of the DGO: They are collaborating with stakeholders and the whole mission had plans for 
regular meetings of cross-sectoral Working Groups, and with respect to learning, the team was already engaged in 
multiple research activities such as impact evaluations. https://usaidlearninglab.org/lab-notes/usaidguatemalas￾journey-incorporate-cla-their-project-mel-plan
41
61 Interview with Bari Rabin, April 13, 2017.
62 Interview with Bari Rabin, September 19, 2017.
63 Interview with Ligia Alfaro, PPSO, August 24, 2017.
64 Interview with Ligia Alfaro, PPSO, August 24, 2017.
65 Interview with BAA team member B, September 1, 2017.
66 Interview with BAA team member A, September 18, 2017 (confidential).
67 Interview with Bari Rabin, April 13, 2017; Interview with Ligia Maria Alfaro, PPSO, August 24 2017.
68 https://www.fbo.gov/spg/AID/OM/GUA/SOL-520-17-000009/listing.html. The link was provided by Fabiola Loy, 
OAA, September 13, 2017. See also 
https://www.fbo.gov/index?s=opportunity&mode=form&id=fa8b0fe2025ac65c246437ca949f5d2b&tab=core&_cvie
w=1; The M&E contract was with DevTech; Interview BAA team member B, September 1, 2017.
69 Interview with Ligia Maria Alfaro, PPSO, August 24, 2017.
70 Mission survey results. If you would like to request this information, please email Kristin Lindell at 
klindell@learning4dev.org
71 Interview with Ligia Maria Alfaro, PPSO, August 24, 2017.
72 Interview with Ligia Maria Alfaro, PPSO, August 24, 2017.
73 Interview with Jessica Ziegler, September 19, 2017; Interview with Jenny Barker, September 13, 2017.
74 MERL interview with Bari Rabin, emailed by Kristin Lindell on August 11, 2017.
75 Interview with Eric Friesth, LEARN Senior Operations Manager, July 18, 2017.
76 Comment on draft MTE, Monalisa Salib, LEARN Manager of Organizational Learning and Research, November 7, 
2017; On MERLIN, see Cambodia’s Child Protection https://www.usaid.gov/GlobalDevLab/about/monitoring￾evaluation-research-and-learning-innovations-program
77 Interview with Ligia Alfaro, August 24, 2017.
78 Interview with Jessica Ziegler, LEARN’s Senior Learning Specialist, September 19, 2017.