EVIDENCE AND LEARNING (E&L) 
UTILIZATION MEASUREMENT ANALYSIS 
(UMA)
March 2022
DISCLAIMER: The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International 
Development (USAID) or the United States Government.
USAID/TANZANIA
The E&L UMA was conducted by The Cloudburst Group, namely Miriam Counterman (Team Lead); Simon 
Conte, Aleta Starosta, and Kate Marple-Cantrell (Senior Evaluation Specialists); and Makayla Barker and 
Ryan Hatano (Data Analysts).
This document was produced for review by the United States Agency for International Development, 
Democracy, Human Rights and Governance Center under the Learning, Evaluation and Research 
Activity II (LER II) contract: GS10F0218U/7200AA18M00017.
Prepared by: 
The Cloudburst Group
8400 Corporate Drive, Suite 550
Landover, MD 20785-2238
Tel: 301-918-4400
CONTENTS
ACRONYMS II
EXECUTIVE SUMMARY 1
1.0 BACKGROUND AND PURPOSE 6
1.2 RQS 7
2.0 EVALUATION METHODS AND LIMITATIONS 7
2.1. METHODOLOGY 7
2.2. CHALLENGES AND LIMITATIONS 8
3.0 RQ 1: HOW WERE THE FINDINGS AND/OR REPORT SHARED AND DISSEMINATED? TO WHOM 
AND FOR WHAT PURPOSE? WERE THERE DERIVATIVE COMMUNICATION OR DISSEMINATION 
PRODUCTS PRODUCED? IF SO, HOW WERE THEY USED? 9
3.1 FINDINGS 9
3.2 CONCLUSIONS 13
3.3 RECOMMENDATIONS 14
4.0 RQ 2: WERE THERE FINDINGS FROM THE LEARNING PRODUCT/REPORT THAT WERE 
PARTICULARLY IMPORTANT? FOR EXAMPLE, ARE THERE FINDINGS THAT PEOPLE STILL REFER TO 
MONTHS AFTER THE EVALUATION? 15
4.1 FINDINGS 15
4.2 CONCLUSIONS 17
4.3 RECOMMENDATIONS 18
5.0 RQ3: WAS THERE ANY FORMAL PROCESS OF DECIDING WHAT TO DO WITH THE FINDINGS OR 
RESULTS (E.G., A WORKSHOP OR MEETING)? WAS A PEAP PRODUCED? WAS IT UPDATED? 19
5.1 FINDINGS 19
5.2 CONCLUSIONS 20
5.3 RECOMMENDATIONS 20
6.0 RQ 4: WERE THERE ANY RECOMMENDATIONS FROM THE REPORT THAT WERE ACTED UPON? 
WHAT RECOMMENDATIONS AND WHAT WAS DONE? HOW DID THE EVALUATION INFLUENCE 
USAID STRATEGY, PROGRAM DESIGN, ACTIVITY IMPLEMENTATION, AND UNDERSTANDING? 21
6.1 FINDINGS 21
6.2 CONCLUSIONS 24
6.3 RECOMMENDATIONS 25
7.0 RQ 5: WHAT ARE SOME OF THE BARRIERS TO USING EVALUATION FINDINGS OR ACTING ON 
EVALUATION RECOMMENDATIONS? DID THE STRUCTURE OF THE REPORT SUPPORT ITS USE? 25
7.1 FINDINGS 25
7.2 CONCLUSIONS 27
7.3 RECOMMENDATIONS 28
8.0 RQ 6: HOW COULD EVALUATIONS BE MADE MORE USEFUL? HOW COULD USAID AND IPS USE 
EVALUATION FINDINGS MORE? 28
8.1 FINDINGS 28
8.2 CONCLUSIONS 30
8.3 RECOMMENDATIONS 31
9.0 CONCLUSION 31
ANNEX 1. INTERVIEW GUIDE 32
ANNEX 2. SUMMARY OF KIIS 40
ANNEX 3. LIST OF DOCUMENTS REVIEWED 41
ii | USAID REPORT TITLE HERE USAID.GOV
ACRONYMS
CDCS Country Development Cooperation Strategy
DEC Development Experience Clearinghouse
DRG Democracy, Human Rights, and Governance
E&L Evidence and Learning
ET Evaluation team
IP Implementing partner
KII Key informant interview
LER II Learning, Evaluation, and Research Activity II
PEAP Post-evaluation action plan
POC Point of contact
RQ Research question 
SOW Statement of work
UMA Utilization Measurement Analysis
USAID United States Agency for International Development
1 | E&L UMA FINDINGS REPORT USAID.GOV
EXECUTIVE SUMMARY
The Utilization Measurement Analysis (UMA) is intended to provide the Evidence and Learning (E&L) team 
at the United States Agency for International Development (USAID) Center for Excellence in Democracy, 
Human Rights, and Governance (DRG) with a systematic review of uptake and use of its learning products. 
The UMA explores how learning products from 22 taskings under the Learning, Evaluation, and Research 
Activity II (LER II) task order have been used and disseminated and provides recommendations for 
improving the presentation of findings, encouraging utilization, and expanding dissemination efforts. 
The findings are based on 20 interviews, reflecting the views of 25 individuals, on 21 taskings. Taskings 
included performance evaluations, assessments, frameworks, and other guides. Interviews were semi￾structured and addressed the utilization and dissemination of research findings. The findings are organized 
by research questions (RQs) and summarized here, the findings and conclusions are discussed separately
in more detail in the full report.
Although the research acknowledges the challenges in conducting effective learning activities, the value of 
these products is clear. The purpose of the research is to identify factors that facilitate or hinder the 
utilization of research in order to improve future products. The recommendations provide actionable 
suggestions capitalizing on facilitating factors and overcoming hindrances.
RQ 1: HOW WERE THE FINDINGS AND/OR REPORT SHARED AND DISSEMINATED? TO 
WHOM AND FOR WHAT PURPOSE? WERE THERE DERIVATIVE COMMUNICATION OR 
DISSEMINATION PRODUCTS PRODUCED? IF SO, HOW WERE THEY USED?
Dissemination activities across the 22 taskings evaluated varied significantly: there were 
outbriefs, findings workshops, dissemination presentations, and launch events. Thirteen held 
an outbrief event to present preliminary findings to key stakeholders (out of 15 evaluations and 
assessments that would be expected to have an outbrief). The audience for these events most often 
included Mission and project staff, although, in select instances, the relevant Mission chose to present 
findings to a wider audience (such as government counterparts or other donors). Five additional taskings 
held a findings workshop or presentation specifically designed to provide key stakeholders with an 
opportunity to critique and discuss the prioritization of recommendations. Additionally, learning products 
from three interconnected taskings informed a thematic launch event that was externally facing. Audience 
members of this event included staff from the DRG Center and Mission staff and stakeholders working 
outside of USAID. 
There were mixed views on the efficacy of these dissemination products. Four interviewees
noted the outbriefs’ effectiveness at efficiently conveying findings to a targeted audience and three posed 
the likelihood that audience members would find them redundant or forgettable. Four interviewees 
advocated generally for the use of presentations to disseminate findings rather than written materials 
because of the opportunity to include interactive elements such as Q&A segments, pause-and-reflect 
sessions, and guided discussions. Nine interviewees also discussed written materials supplementary to a 
report that could provide practitioners and key stakeholders an overview of the findings/relevant 
recommendations. Ideas included two-pagers, executive summaries, infographics, and comics. The 
advantage of these types of summative materials is that their brevity and non-technical nature make 
findings more accessible to some field staff and other non-DRG personnel who may find a targeted learning 
product useful but would otherwise not engage with the full findings. 
2 | E&L UMA FINDINGS REPORT USAID.GOV
RECOMMENDATIONS
● Require dissemination plans at the concept note stage, with clear expectations in the 
statement of work (SOW). This better ensures a holistic dissemination plan and checks that 
the planned deliverables meet the dissemination goals for the research. Plans should outline events 
and include a defined purpose, audience, and tool(s) for each activity. 
● Scopes and designs should consider the audience and purpose of dissemination activities to tailor 
dissemination products to the intended audience. While some audiences read and benefit 
from a full findings report, many do not. Depending on the audience, summaries, abstracts, 
presentations, infographics, and other visuals may be a more effective means of communication. 
● In considering the purpose and intended audience for the dissemination plan, leverage digital 
tools to encourage participation in presentations, workshops, and other events. More 
interactive activities are more likely to engage audiences and effectively deliver information. 
Particularly in an age of “Zoom fatigue,” it is important to deliver presentations engagingly.
RQ 2: WERE THERE FINDINGS FROM THE LEARNING PRODUCT/REPORT THAT WERE 
PARTICULARLY IMPORTANT? FOR EXAMPLE, ARE THERE FINDINGS THAT PEOPLE STILL 
REFER TO MONTHS AFTER THE EVALUATION? 
Few interviewees reported specific findings that were important, although many referenced 
using findings collectively. For a majority of interviewees, too much time had elapsed to remember 
specific findings and, as discussed in RQ 3, only one had a written record of what was done with the 
findings and recommendations for reference. However, interviewees reported minor to moderate 
influence on program design, activity implementation, USAID strategy, and general learning or 
understanding (discussed further in RQ4). Interviewees reported using findings from learning products 
for: activity design, arguing for increased funding, continuing a certain type of programming, starting a 
program in a new sub-sector, observation by a third party outside the implementation or USAID team, 
and relationship-building and getting stakeholders on the same page.
In discussing why findings were utilized or not utilized, interviewees praised both broad, 
high-level findings and specific, highly targeted findings and recommendations. This largely 
results from different products used for different purposes; however, in at least one case, an interviewee 
praised both broad findings and specific, targeted findings in a single product. Broad findings and 
recommendations inform thinking, engage a maximum number of people, and inform strategy. However, 
others felt that specific, actionable findings and recommendations were more useful to implementation. 
Interviewees expressed a need for easily digestible information. Useful findings and recommendations may 
be buried in long reports and complex language. Many interviewees also raised infographics, abstracts, and 
two-page summaries as useful tools; however, these must be tailored to the needs of the audience. 
RECOMMENDATIONS
● Determine the intervals for utilization follow-up between the E&L team and the 
commissioner of the research and incorporate it into the post-evaluation action plan 
(PEAP). The correct timing for follow-up is likely to vary from one tasking to another; one will 
not use findings for the first year, while another will have forgotten findings after a year. Regular 
follow-up is resource-intensive and risks being annoying to the user, but best ensures the 
collection of accurate information and encourages utilization. 
● Consider the use of findings outside of activity design or implementation. When looking 
at the utility of a report, there are many uses outside of activity design and implementation that 
are still worthwhile. These may include relationship-building and providing an objective or outsider 
3 | E&L UMA FINDINGS REPORT USAID.GOV
opinion. The way that findings are presented can be tailored to better suit those uses—for 
example, targeted two-pagers or summaries can be used for meeting with counterparts and 
communicating findings. 
● Incorporate visually appealing and brief presentations of the findings in tasking 
scopes, where appropriate. Interviewees indicated that the audience for a findings report is 
limited; infographics, brief summaries, and visual presentations are more useful for a broad 
audience. The tasking timeline and budget need to account for these products.
RQ 3: WAS THERE ANY FORMAL PROCESS OF DECIDING WHAT TO DO WITH THE 
FINDINGS OR RESULTS (E.G., A WORKSHOP OR MEETING)? WAS A PEAP PRODUCED? WAS 
IT UPDATED?
In general, few taskings had updated PEAPs or other formal processes to decide what to do 
with the findings or results. Among 17 Cloudburst LER II taskings that had recommendations (some 
learning products were guides and toolkits that would not have had actionable recommendations 
comparable to an evaluation or assessment), five taskings had action plan templates required in the scope; 
Cloudburst provided the template filled in with recommendations and suggested timelines after 
completion of the findings report. Of those five, one tasking had taken the template provided and updated 
it with actions undertaken. An additional six taskings had validation or recommendation workshops to 
formally discuss recommendations. Other examples of processes to decide what to do with findings are 
the creation of a guide to the next internal steps, a learning and synthesis event, and other workshops. 
RECOMMENDATIONS
● Consider ways to encourage the use of the PEAP. PEAPs are now consistently being 
produced for applicable research products. These can be a tool to help Missions discuss and decide 
how to use recommendations; however, they do not appear to be well used, even within taskings 
with recommendations workshops. Following up to ask about the PEAP may help to encourage 
its use in some cases and additional education for Missions or others using evaluations would help 
users understand the purpose of the PEAP, what to do with it, and how it can benefit them. 
● Support users with other processes for determining recommendation 
implementation. In some cases,Missions or other users may have alternative processes that 
they use to decide how to implement recommendations. Interviewees mentioned written plans 
for implementation in other formats, or “roadmaps,” and various forms of meetings and 
discussions to determine implementation. In these instances, users could be guided on adapting 
the PEAP to their process to make it most effective for tracking utilization. 
RQ 4: WERE THERE ANY RECOMMENDATIONS FROM THE REPORT THAT WERE ACTED 
UPON? WHAT RECOMMENDATIONS AND WHAT WAS DONE? HOW DID THE EVALUATION 
INFLUENCE USAID STRATEGY, PROGRAM DESIGN, ACTIVITY IMPLEMENTATION, AND 
UNDERSTANDING?
A large majority of interviewees were able to discuss ways in which the recommendations from their 
learning products were used—only one felt that the recommendations were not useful at all. Interviewees 
rated most learning products as having a moderate to major effect on activity implementation (n=8, 53 
percent), activity design (n=12, 75 percent), strategy (n=8, 50 percent), and general learning (n=9, 60 
percent). Activity design had the most interviewees who considered their learning product to be 
moderately or majorly influential. This likely reflects the purpose of most of the evaluations and 
assessments, which was to inform program design. In 15 interviews, respondents struggled to name 
specific examples of recommendations that were particularly important, either because they used the 
4 | E&L UMA FINDINGS REPORT USAID.GOV
report more holistically for background information, because not enough time had elapsed for 
implementation, or because they simply could not recall individual recommendations. Six interviewees 
were able to name specific recommendations, the lessons from which are discussed below. 
A common theme among interviewees was that evaluations and their recommendations need to 
be carefully designed to serve the needs of the Mission or other users. Five interviewees felt that 
this had significantly contributed to the learning product’s success while three felt doing so would have 
resulted in a better product. A related common perception was that close collaboration between the 
research team and Mission or other users was beneficial.
RECOMMENDATIONS
● Clearly communicate the needs of the user to the research team so that 
recommendations can be properly tailored. As in the recommendations for findings, with 
a wide variety of learning products, recommendations must fit the intended use. 
Recommendations must also take into account the donor landscape and the intended scope of 
the use. It may be better to recommend continuing an activity as-is rather than recommending an 
impractical or impossible activity, given the context. 
● Consider USAID’s role in the evaluation. This, too, depends on the specific learning product; 
however, Mission or other USAID staff have an important role in the design, implementation, and 
dissemination of research. This may include E&L methodological expertise in writing the scope 
and design, user collaboration in the research methodology, participation in fieldwork, or 
championing dissemination and utilization. View additional guidance on USAID’s role in external 
evaluations. 
● Select the timing of the research carefully. The E&L team should help Missions or other 
commissioners to understand the timeframe of a research project and assess whether the
proposed approach can meet their needs in the allotted time. This will also help to ensure that 
the research is done at an appropriate time, given utilization goals. 
RQ 5: WHAT ARE SOME OF THE BARRIERS TO USING EVALUATION FINDINGS OR ACTING 
ON EVALUATION RECOMMENDATIONS? DID THE STRUCTURE OF THE REPORT SUPPORT 
ITS USE?
Respondents identified a variety of factors that present barriers to using or acting upon 
evaluation recommendations. Most barriers can generally be categorized as either external factors,
such as the broader political context in a country, or internal factors, such as a lack of bandwidth within 
USAID to act upon or document the use of recommendations. Some barriers, however, straddle this 
divide and are applicable both externally to and internally within USAID. The agency’s foreign assistance 
activities do not occur in a vacuum, and an array of circumstances need to be aligned for an evaluation to 
lead to impactful recommendations that can be acted upon. Among other things, the political and 
economic context within the country must be conducive to reform efforts, there must be sufficient buy￾in both from stakeholders and within USAID itself, the report must provide realistic and actionable 
recommendations, and there must be sufficient bandwidth and funding within USAID to support follow￾up activities.
RECOMMENDATIONS
● Encourage closer collaboration between the technical team commissioning the 
evaluation and the evaluation team (ET), such as collaboration on the design to ensure that 
the research provides analysis that achieves USAID’s goals.
5 | E&L UMA FINDINGS REPORT USAID.GOV
● Build buy-in from implementing partners (IPs) by adequately preparing them for 
evaluations, such as by crafting evaluation guidelines that emphasize the benefits of a 
collaborative process and that frame the evaluation as a learning opportunity and a constructive 
process. In addition, commissioner of the research should provide clear communication to the IP 
on the purpose of the research or evaluation, with support from the ET, where necessary. 
● Provide specific parameters to the ET that aid them in crafting actionable 
recommendations; e.g., rapid-response vs. long-term, directed toward specific partners, the 
general level of funding anticipated for follow-on activities, relevant political context, known 
obstacles to implementation, etc. View additional guidance on developing an effective evaluation 
SOW. 
● Realistically consider the viability of evaluation use before an evaluation is conducted.
Do the timing, resources, and political context provide space and opportunity for an evaluation 
to be impactful?
RQ 6: HOW COULD EVALUATIONS BE MADE MORE USEFUL? HOW COULD USAID AND IPS 
USE EVALUATION FINDINGS MORE? 
Many of the suggestions for improving evaluations focused on dissemination—pushing 
research out to more stakeholders and in more easily accessible formats. As has been discussed 
throughout the preceding RQs, ensuring that the research scope and design fit the needs of the Mission 
or commissioning body is the most important way to ensure that the findings and recommendations are 
useful. One interviewee pointed out that evaluators often feel pressure to provide recommendations 
when sometimes the best recommendation is to continue successful activities. 
Further requests are to ensure the creation of a research team with the correct combination of knowledge 
and experience and also to consider the role of USAID staff in research scoping and design, fieldwork, 
and dissemination. In addition, the presentation of the findings and recommendations is key. Interviewees 
expressed a desire for more visual representations, summaries and briefs, and interactive workshops and 
presentations. Finally, research can be difficult to access and is often not synthesized in a way that would 
make it useful to a broader audience. 
RECOMMENDATIONS
● Determine the correct level of engagement of USAID with the research team. While 
highly dependent on the context of the learning product, there is an important role for USAID 
staff in the learning product. This may include contributing research expertise to the scope, 
collaborating on the research design and workplan, participating in fieldwork where appropriate, 
and championing the research.
● Make research more accessible. The Development Experience Clearinghouse (DEC) does 
not currently contribute to disseminating research more widely. A more useful repository of 
research would make reports and summaries easier to find. Organizing reports by subsector and 
providing a synthesis of reports to suggest useful best practices would enable greater utilization. 
● Engage in research synthesis activities. These activities serve to build on existing research 
to improve and grow a larger body of knowledge of lessons learned and best practices, but they 
also serve to stimulate engagement with and interest in research.
6 | E&L UMA FINDINGS REPORT USAID.GOV
1.0 BACKGROUND AND PURPOSE 
The UMA is intended to provide the E&L team at the USAID Center for Excellence in DRG with a 
systematic review of uptake and use of its learning products. The UMA explores how learning products 
from 22 taskings under LER II have been used and disseminated and provides recommendations for 
improving the presentation of findings, encouraging utilization, and expanding dissemination efforts. 
Since the release of USAID’s Evaluation Policy in 2011, reviews and meta-analyses1 have shown an increase 
in both the number of evaluations and the quality of evaluations commissioned by USAID; however, more 
and better evaluations do not ensure evaluation use. As Fowle et al. (2020) demonstrated, utilization is a 
key mechanism to understanding the link between research products and improvements in development 
programming and policy.2 According to one 2016 study of USAID evaluations, they are used in project 
design and implementation and strategy and policy formation. The most common effects attributed to 
evaluations are refocusing ongoing activities such as revised workplans, extended timelines, or expanded 
geographic reach.3 However, without a regular, systematic process to examine utilization, there remains 
unmeasured potential for improving the utilization of both evaluations and other learning products.
Previous analyses of evaluation utilization, including USAID’s 2016 report4 and a World Bank review,5
found key themes in highly utilized learning products. Learning activities should be conducted when there 
is a clear question and application for the research, which requires careful timing, design, and presentation. 
To achieve this, previous analyses recommend engaging stakeholders, including beneficiaries and other key 
informants, throughout the research process to improve buy-in and uptake of recommendations. This 
type of engagement can improve receptivity, which has also been identified as key to utilization. In addition, 
findings and guidance on best practices should be shared widely in a format that is easily understood and 
clearly applicable to policy and practice. Producing accessible products, such as infographics or summaries, 
can also help to disseminate the findings. Tools, like trackers, that link evaluation recommendations to 
specific actions make the uptake of recommendations easier for the intended audience. Finally, the findings 
of individual evaluations can be capitalized upon by synthesizing findings across multiple evaluations.
Using a structured set of follow-up interviews with stakeholders of 20 E&L-commissioned taskings 
completed under the LER II task order, the UMA provides insight to the team on what aspects of these 
learning products are and are not utilized by end-users. Building on the Program and Policy Change 
Framework, the findings explore to what extent the findings of learning products are implemented and 
what barriers exist to disseminating and utilizing findings. 
1 USAID. (2013). “Meta-Evaluation of Quality and Coverage of USAID Evaluations.”
https://www.usaid.gov/sites/default/files/documents/1870/Meta￾Evaluation%20of%20Quality%20and%20Coverage%20of%20USAID%20Evaluations%202009-2012.pdf
2 Fowle, K., B. Wells, M. Day, A. Kumar, C. Bess, B. Bingham, and A. Wayman. 2020. “The Program and Policy Change 
Framework: A new tool to measure research use in low- and middle-income countries.” Research Evaluation, 1–14, doi: 
10.1093/reseval/rvaa017.
3 USAID. (2011). “Evaluation: Learning from Experience.” USAID Evaluation Policy.
https://www.usaid.gov/sites/default/files/documents/2151/USAIDEvaluationPolicy.pdf
4 USAID. (2016). “Evaluation Utilization at USAID.” https://data.usaid.gov/api/views/gp7w-vu6q/files/7a656d92-bc16-406f-b468-
c67c9fa20d9f. 
5World Bank. (2004). “Influential Evaluations: Evaluations that Improved Performance and Impacts of Development Programs.” 
https://documents1.worldbank.org/curated/en/609091468329361924/pdf/328790ENGLISH01tial1evaluations1ecd.pdf
7 | E&L UMA FINDINGS REPORT USAID.GOV
1.2 RQS
1. How were the findings and/or report shared and disseminated? To whom and for what purpose? 
Were there derivative communication or dissemination products produced? If so, how were they 
used?
2. Were there findings from the learning product/report that were particularly important? For 
example, are there findings that people still refer to months after the evaluation? 
3. Was there any formal process of deciding what to do with the findings or results (e.g., a workshop 
or meeting)? Was a PEAP produced? Was it updated? 
4. Were there any recommendations from the report that were acted upon? What 
recommendations and what was done? How did the evaluation influence USAID strategy, program 
design, activity implementation, and understanding?
5. What are some of the barriers to using evaluation findings or acting on evaluation 
recommendations? Did the structure of the report support its use? 
6. How could evaluations be made more useful? How could USAID and IPs use the evaluation findings 
more? 
2.0 EVALUATION METHODS AND LIMITATIONS 
2.1. METHODOLOGY
The study used a mixed-methods approach to collect data on the RQs, including conducting key informant 
interviews (KIIs), performing a desk review, and compiling data on the selected taskings to complete a 
tracker on selected indicators.
KEY INFORMANT INTERVIEWS
The main source of data for the UMA came from interviews with the main points of contact (POCs) for 
21 taskings conducted by Cloudburst between 2018 and 2021. A total of 22 taskings were selected for 
inclusion in the UMA. Table 1 below shows the types of learning products produced under the 22 selected 
taskings. These taskings were all those conducted between 2018 and 2021, excluding those products that 
were so unique in nature as to not be comparable (such as conferences). The taskings ranged from closing 
two months prior to the interview to closing 34 months prior, measured by the date on the final 
deliverable. The mean period elapsed was 14 months and the median was 10 months.
TABLE 1 
TYPE FREQUENCY
Evaluation 9 (41%)
Assessment 7 (32%)
Evidence review 3 (14%)
Framework/guide 2 (9%)
Other 1(4%)
At least one POC was interviewed for each tasking and multiple POCs were interviewed for some of the 
taskings. A full list of KIIs is included in Annex 1. The interview guide (included in Annex 1) was 
coordinated with the NORC/SI team, which conducted a similar UMA to generate comparable data. 
8 | E&L UMA FINDINGS REPORT USAID.GOV
The research team contacted a total of 39 individuals regarding 22 taskings. Twenty interviews were 
successfully conducted with 25 individuals. All but one tasking is represented with at least one interview 
in this findings report. However, 14 individuals could not be interviewed, most of whom were contacted 
at least three times. 
All interviews were conducted remotely and included at least one interviewer and one note-taker. The 
research team conducted interviews in which the interviewer had been involved in the research itself and 
where the interviewer was completely new to the tasking. Although the research team was concerned 
that interviewees would be hesitant to be critical of products when the interviewer had helped to produce 
the research or that the interviewer’s own views would bias the interview, that did not appear to be the 
case—interviewees and interviewers alike were aware of and able to discuss challenges during the 
research process. There was value in an interviewer who was familiar with the research done, 
remembered the process, and had a relationship with the POC; this was true even in cases where the 
interviewee was not entirely happy with the result. However, there was also a benefit in having an 
interviewer whose perspective was uncolored by any previous experience. It is helpful, where possible, 
to have an interview team that includes one person already familiar with the tasking and one fresh to the 
research. 
Using notes and interview transcripts, the research team used inductive analysis to answer the RQs. All 
interviews were coded for keywords and emerging themes, which were noted and compiled for writing 
the findings report. Where possible, these themes were triangulated with documentation and the research 
team’s own knowledge of the tasking. 
DESK REVIEW AND SECONDARY DATA ANALYSIS 
The ET reviewed project documents from Cloudburst files to provide the indicators included in the 
tracker. The desk review informed the development of the research design and the interview tool. In 
addition, the ET used project documents to prompt interviewee recall and encourage further discussion. 
Finally, the ET used documents to validate findings where indicated by interviewees; for example, 
referencing a Country Development Cooperation Strategy (CDCS) where findings were cited. 
The tracker completed in conjunction with this report includes basic indicators such as the number of 
outbriefs held and PEAPs created. These indicators were also used to inform findings throughout the UMA 
and are cited in this report. Due to the diverse nature of the learning products included in this study, not 
all indicators are appropriate for all taskings. For example, a guide or framework may not have an outbrief 
or PEAP but may have other dissemination events. 
The original research methodology included two to three case studies, which were intended to include a 
deeper examination of selected case studies, based on documentation and additional interviews. Taskings 
were selected for potential case studies to reflect diversity in utility (both positive and negative). Seven 
individuals were contacted at least three times each, but only one agreed to an interview. In most cases, 
additional documentation was either not available or not sharable. As a result, rather than case studies, 
the report includes topics for additional focus, where additional information or analysis was possible. 
2.2. CHALLENGES AND LIMITATIONS
Section 2.1 references many of the challenges and limitations of this report. The timing proved to be 
particularly problematic, both in terms of respondents’ memories of the relevant activities fading, and also 
9 | E&L UMA FINDINGS REPORT USAID.GOV
concerning a lack of institutional knowledge as relevant staff moved to new positions over time. These 
factors may have impacted the quality of the KII data underpinning the research team’s analysis.
Some caution must be taken with the interviewees’ responses. Interviewees who were unsure of an 
answer or who had a positive bias due to their own role in the evaluation may have elected not to answer 
a question or overestimated the utility of the learning product, resulting in inflated findings. Overall, 
interviewees’ willingness to criticize suggests this is not a huge concern; nevertheless, it must be taken 
into account. 
As noted in Section 2.1, the findings, conclusions, and recommendations of this report are based on a 
fairly small sample size of only 20 interviews with 25 individuals. While this does represent 22 of the 23 
identified taskings, the research team was unable to conduct interviews with 15 individuals, who may have 
added different perspectives and information to the findings, conclusions, and recommendations.
Additionally, in the 22 taskings included in the sample, there are at least four different types of research 
products covered. Findings are, therefore, drawn from a relatively small number of interviews. For 
example, a finding could represent the views of all respondents who commissioned an evidence review, 
which would be three taskings. The ET has focused findings on evaluations and assessments (which are 
the bulk of the taskings covered), identified themes that carry across learning product types, and noted 
the number of interviews on which findings are based. 
Finally, as the original implementer of these taskings, there is the risk that Cloudburst’s analysis of their 
impact is not completely objective due to a potential positive bias. There is also a risk that interviewees, 
having been heavily involved in the original research product, are also positively biased. Where possible, 
the ET reviewed additional documentation to verify results; however, that was possible in only a few cases. 
To mitigate this risk, the ET probed interviewees for specific examples and recommendations for 
improvement.
3.0 RQ 1: HOW WERE THE FINDINGS AND/OR REPORT SHARED 
AND DISSEMINATED? TO WHOM AND FOR WHAT PURPOSE? 
WERE THERE DERIVATIVE COMMUNICATION OR 
DISSEMINATION PRODUCTS PRODUCED? IF SO, HOW WERE 
THEY USED?
3.1 FINDINGS
Dissemination activities across the 22 taskings evaluated varied significantly. Thirteen held an outbrief 
event to present preliminary findings to key stakeholders (out of 15 evaluations and assessments that 
would be expected to have an outbrief). The audience for these events most often included Mission and 
project staff, although, in select instances, the relevant Mission chose to present findings to a wider 
audience (such as government counterparts or other donors). The number of people in attendance at 
these outbrief presentations generally varied between 5 and 25 people. Five additional taskings held a 
findings workshop or presentation specifically designed to provide key stakeholders with an opportunity 
to critique and discuss the prioritization of recommendations. These were more recent taskings that 
incorporated discussions of recommendations as part of USAID’s push to encourage research utilization. 
Additionally, learning products from three interconnected taskings informed a thematic launch event that 
10 | E&L UMA FINDINGS REPORT USAID.GOV
was externally facing. Audience members of this event included staff from the DRG Center and Mission 
staff and stakeholders working outside of USAID. 
Only one tasking did not hold a primary interactive dissemination activity; however, this tasking is ongoing 
and designed around the iterative dissemination of DRG learning digests, videos, and infographics. Aside 
from this tasking, three other taskings used infographics as a dissemination tool, and four distributed two￾pagers. Only one tasking used both infographics and two-pagers to share findings. In general, stakeholders 
agreed that there is no one-size-fits-all approach to the successful dissemination of learning products. As 
one respondent summarized, “In general, successful dissemination activities are those that are specific and 
relevant to respondents.”
TABLE 2: COUNT OF TASKINGS WITH DISSEMINATION ACTIVITIES
DISSEMINATION 
ACTIVITY
PURPOSE NUMBER OF 
TASKINGS 
Findings report Provide a detailed account of a study with findings, conclusions, and 
recommendations
216
Two-pager Summarize findings and recommendations 6 
Infographic Summarize findings and recommendations 4 
Outbrief Present preliminary findings 13 
Workshop Provide key stakeholders with an opportunity to discuss findings and 
recommendations
5 
Launch event Initiate the utilization of a new learning product 2 
Final presentation Present final findings and recommendations 8 
As discussed above, interviewees reported utilizing a variety of dissemination tools, including outbriefs, 
workshops, reports, two-pagers, infographics, and other events. Across LER II taskings, outbrief 
presentations were used most frequently. Interviewees were split on their opinions of outbriefs, with four 
noting the outbriefs’ effectiveness at efficiently conveying findings to a targeted audience, three finding the 
outbriefs beneficial for other purposes, and two expressing the likelihood that audience members would 
find them redundant or forgettable. When presented with the question “What made this outbrief 
successful or not successful?” one respondent tellingly replied, “To be frank, I don’t really remember this 
one, but because nothing jumps out [in my mind], it was probably fine.” That said, interviewees generally 
indicated that the more interactive an outbrief presentation, the better. The same interviewee who could 
6 LER II tasking CB002 “DRG Learning Dissemination Products” was designed to produce products for dissemination to USAID and 
non-USAID audiences that summarize and communicate research and evaluation findings, DRG learning priorities, and other 
messages. Therefore, this tasking has produced two-pagers, infographics, videos, and other dissemination products, but does not 
have a findings report.
11 | E&L UMA FINDINGS REPORT USAID.GOV
not recall the details of the outbrief also said, “We usually provide guidance that the presentation shouldn’t 
be too long and that there should be time to ask questions.” Interviewees from two other taskings 
specifically credited the utility of oubriefs to the opportunity to ask questions, while six interviewees in 
total listed “interactive” or some variation thereof as an attribute of a successful outbrief presentation. 
While these interviewees spoke specifically to interactive components of specific dissemination events, 
three other interviewees advocated generally for the use of presentations to disseminate findings rather 
than written materials because of the opportunity to include interactive elements such as Q&A segments, 
pause-and-reflect sessions, and guided discussions. One of these individuals summarized, “As long as the 
engagement is not static, anywhere there is two-way communication, it will have a different effect.” 
In keeping with the interviewees’ general preference for interactive presentations, several interviewees 
spoke to the utility of workshops. Indeed, all interviewees who participated in workshops agreed that the 
exercise was useful. One interviewee shared that they used Jamboards, interactive digital whiteboards 
developed by Google, to facilitate participation during a recommendations workshop on improving the 
use of impact evaluations across USAID activities. The tool allowed workshop participants to provide 
written feedback on recommendations and vote on the prioritization of recommendations. The technique 
not only encouraged participation among attendees but also ended up being useful after the workshop 
because the team could point to a record of widespread consensus on recommendations. Another 
interviewee discussed using various performance and evidence reviews on Global Labor for a co-creation 
workshop to design the new Global Labor New Frontiers project. This workshop not only effectively 
disseminated the learning products to a wider audience but also ensured that the findings were directly 
utilized in the new activity. 
Despite the positive feedback regarding workshops and other presentations, when asked how the 
dissemination of learning products could have been improved, several interviewees unsurprisingly 
discussed written materials supplementary to a report that could provide practitioners and key 
stakeholders with an overview of the findings and/or relevant recommendations. Ideas included two￾pagers, executive summaries, infographics, and comics. The advantage of these types of summative 
materials is that their brevity and non-technical nature make findings more accessible to some field staff 
and other non-DRG personnel who may find the learning product useful but would otherwise not engage 
with it. One interviewee, who worked on programming in the Dominican Republic, specifically affirmed 
that, in hindsight, they would have done more dissemination in a non-technical, approachable format.
Among the interviewees who spoke to these types of supplementary materials, six stated that, though the 
discussed learning product did not have a two-pager, they think it would have aided dissemination; five 
interviewees said the same for infographics. A majority of these individuals felt that most practitioners do 
not have the time to read long reports, but would take the time to review findings in this distilled format. 
One interviewee explained, “If we were to share anything with the Mission or more broadly, I think that 
a two-pager would have been more useful. Especially for stakeholders that support us financially—we 
could have used it to advocate for budgets.” 
Interestingly, among taskings that did have either infographics or two-pagers, none of the interviewees 
explicitly mentioned the utility of these summary products, suggesting that individuals who have not used 
two-pagers or infographics as a dissemination tool may be overconfident in their utility. One interviewee 
felt that two-pagers were not helpful because they believed that anyone interested in the topic would 
want to read the whole report to understand the findings and recommendations. Though rare, this 
12 | E&L UMA FINDINGS REPORT USAID.GOV
sentiment suggests that two-pagers are not inherently useful and should be written for and disseminated 
to an appropriate audience. 
Other interviewees corroborated this finding by expressing that one-off dissemination methods are often 
not as effective as they could be if they were deployed within a holistic approach like a dissemination 
tracker. An interviewee from one tasking designed to communicate general research and evaluation 
findings to USAID and nonUSAID staff explained that it is unlikely that a single dissemination activity such 
as a training, presentation, newsletter, or two-pager would reach everyone who might find the learning 
product useful, but that multiple dissemination activities at different times would reach a wider audience. 
Other interviewees concurred with this sentiment, relating that timing plays an important role in the 
overall success of dissemination activities. When speaking about the positive reception of a presentation, 
one interviewee said, “it wasn’t like the content was for the future—it was something they could use 
tomorrow.” Another interviewee who worked on a Sustainable Landscapes guide for DRG staff agreed 
that timing aided their dissemination efforts because “ultimately, [they] launched at a time when the 
interest and reception was a lot greater and more positive [than previously].” 
DISSEMINATION AUDIENCE
In addition to speaking to the importance of disseminating learning products at times that are advantageous 
to promoting utilization, several interviewees indicated the type of audience they believed was most 
appropriate for the dissemination tool they deployed. For disseminating general findings, interviewees 
generally agreed that it is important to target a broad and diverse audience. One interviewee felt that 
research teams should make a habit of doing more presentations with diverse groups of stakeholders, 
while another indicated that, in hindsight, they would have done a presentation for beneficiaries as well as 
project and Mission staff. A third interviewee who had experience with presentations done for 
USAID/Zimbabwe said that simply inviting more people at the Mission to presentations improves the 
utilization of learning products while a fourth felt that it is important to invite any USAID or project staff 
who may find the topic relevant, even if they work outside the DRG sector. One interviewee also noted 
the value of engaging the local development community in the dissemination in addition to project 
beneficiaries.
One of the benefits of inviting a variety of stakeholders to participate in dissemination events is the ability 
to amplify diverse voices. One interviewee professed that incorporating voices from countries wherein 
the activity took place had a significant positive impact: “The launch event was a really great event with 
voices from the Philippines, Indonesia, and Cambodia. Staff that were cited in the guide gave presentations 
on what worked and what was challenging and I think that had a way of bringing the subject to life—if we 
just had Washington staff, it wouldn’t have worked as well.” This insight also validates the above finding 
that more interactive and engaging presentations are more successful at disseminating learning. 
In contrast to inviting a wide variety of people to dissemination events aimed at presenting findings, as 
noted above, interviewees found that it was more appropriate to adopt a highly targeted dissemination 
approach for sharing recommendations. This finding confirms the conventional wisdom shared by one 
interviewee that only people who need recommendations will use them. For example, two different 
interviewees explained how their team had reached out to individual Missions that they thought might be 
in a good position to use the learning product, which in their opinion had a higher success rate of 
promoting the implementation of recommendations than larger presentations to DRG USAID staff. These 
same interviewees also incorporated an elections-related learning product into core elections trainings 
13 | E&L UMA FINDINGS REPORT USAID.GOV
for USAID staff and the Automated Directives System respectively, therein ensuring that 
recommendations had a direct impact. Two additional interviewees advocated incorporating 
recommendations into a targeted training curriculum. Used strategically, this finding could help streamline 
the transfer of knowledge from research teams developing the learning products to practitioners who can 
implement recommendations. 
3.2 CONCLUSIONS
Overall, interviewees generally acknowledged that they could have done more dissemination for learning 
products. However, it is also clear that simply increasing dissemination is unlikely to increase 
utilization without first developing a holistic plan for effective dissemination that addresses the three key 
issues of timing, audience, and communication style. For further discussion of an example of a 
dissemination strategy, see the Focus On: Global Labor Review. 
Interviewees found that timing can either facilitate or be a barrier to successful dissemination. One 
interviewee noted that findings and recommendations do not always have an immediate use case, making 
dissemination and utilization challenging. Therefore, a holistic dissemination approach can help 
practitioners take advantage of auspicious timing for sharing recommendations and findings. 
Moreover, planning for iterative dissemination activities increases the likelihood that more people will be 
aware of and capable of utilizing findings and recommendations. 
A synthesis of interviewee comments made evident that a holistic dissemination strategy not only 
requires multiple types of dissemination but also necessitates matching each dissemination 
tool with the appropriate audience; doing so ensures that dissemination activities are a part 
of a continuous rather than compartmentalized approach. Because the dissemination plan is 
highly specific to the learning product, the intended audience and purpose for dissemination should be 
elucidated in the SOW. For example, nearly one-third of interviewees thought that using two-pagers or 
infographics as dissemination tools would have been useful to the activity, while interviewees who did use 
two-pagers did not feel compelled to comment on their utility. The finding suggests that it is possible that 
supplementary summary materials such as two-pagers may not have been targeted to the right audiences, 
or may have been overshadowed by other products. Often, two-pagers are brief distillations of executive 
summaries (already 3–5 pages, generally) and may not add significant value. It may be more effective to 
target the two-pager to the intended audience and focus on the findings or recommendations most useful 
to that audience. Additionally, it is plausible that two-pagers and infographics might have little effect at 
promoting utilization on their own, but that there could be synergies to disseminating two-pagers and 
infographics in conjunction with other events and presentations.
The table below identifies audience types that might be the best fit for each type of dissemination tool. 
The audiences listed below are general and should be adapted to each specific learning product. Most 
content is produced in English; therefore, the audience for a findings report in many cases is likely to be 
an international IP. It may be more practical to engage non-English speaking IPs in presentations and events 
and translate executive summaries.
14 | E&L UMA FINDINGS REPORT USAID.GOV
TABLE 3: POTENTIAL AUDIENCES FOR DISSEMINATION ACTIVITIES
DISSEMINATION ACTIVITY PURPOSE POTENTIAL AUDIENCE
Findings report Provide a detailed account of a study 
with findings, conclusions, and 
recommendations
● USAID technical staff
● International IPs (and local where 
appropriate)
Two-pager Summarize findings and 
recommendations 
● Broad USAID audience
● Local government counterparts
● Beneficiaries
Infographic Summarize findings and 
recommendations
● Broad USAID audience
● Local government counterparts
● Beneficiaries
● Broad public release
Outbrief Present preliminary findings ● USAID Mission staff/research 
commissioners
● IPs
Workshop Provide key stakeholders with an 
opportunity to discuss findings and 
recommendations
● USAID Mission staff/research 
commissioners 
Launch event Initiate the utilization of a new 
learning product
● Broad USAID audience
● IPs
● International and local civil society
● U.S. foreign policy decision-makers 
(e.g., congressional committees)
● International donor community
● Academia
Final presentation Present final findings and 
recommendations
● Interested USAID staff
● IPs
● International and local civil society
● Local government counterparts
● International donor community
Finally, communication style is unsurprisingly an important factor in the relative success of dissemination 
activities. Most practitioners do not have the spare time to review large amounts of 
information, making it important for written materials to be concise and easily digestible 
and presentations to be relevant and engaging. The general consensus of interviewees regarding 
the importance of audience engagement during presentations indicates that workshops where there is a 
brief presentation and significant time for discussion may be generally more useful than outbrief 
presentations with little time for Q&A. More targeted dissemination will likely increase engagement with 
learning products and the uptake of utilization and recommendations. 
3.3 RECOMMENDATIONS
● Require dissemination plans at the concept note stage, with clear expectations in the 
SOW. This better ensures a holistic dissemination plan and checks that the planned deliverables 
15 | E&L UMA FINDINGS REPORT USAID.GOV
meet the dissemination goals for the research. Plans should outline events and include a defined 
purpose, audience, and tool(s) for each activity. 
● Scopes and designs should consider the audience and purpose of dissemination activities to tailor 
dissemination products to the intended audience. While some audiences read and benefit 
from a full findings report, many do not. Depending on the audience, summaries, abstracts, 
presentations, infographics, and other visuals may be a more effective means of communication. 
● In considering the purpose and intended audience for the dissemination plan, leverage digital 
tools to encourage participation in presentations, workshops, and other events. More 
interactive activities are more likely to engage audiences and effectively deliver information. 
Particularly in an age of “Zoom fatigue,” it is important to deliver presentations engagingly.
4.0 RQ 2: WERE THERE FINDINGS FROM THE LEARNING 
PRODUCT/REPORT THAT WERE PARTICULARLY IMPORTANT? 
FOR EXAMPLE, ARE THERE FINDINGS THAT PEOPLE STILL REFER 
TO MONTHS AFTER THE EVALUATION?
4.1 FINDINGS
Few interviewees reported specific findings that were important, although many referenced using findings 
collectively.7 For a majority of interviewees, one issue may have been that too much time had elapsed to 
remember specific findings and, as discussed in RQ3, two mentioned a written record of what was done 
with the findings for reference. However, interviewees reported minor to moderate influence on program 
design, activity implementation, USAID strategy, and general learning or understanding (discussed further 
in RQ4). 
While interviewees rarely recalled a specific finding that was very influential, they often spoke to a specific 
application of the findings. For example, the findings were used heavily in the design of a follow-on 
program, they were used to justify a new program in a sector in which Mission had not been working, or 
they were used in writing the CDCS. 
CHALLENGES WITH COLLECTING DATA ON FINDINGS UTILIZATION
Overall, interviewees struggled to recall specific findings and how and why they were used or not used. 
The recall period for interviewees ranged from two months to over two years. Moreover, up to two 
years after a learning product was produced, the most knowledgeable person on that tasking may have 
moved to a new position or left USAID. In at least three cases, the interviewee expressed that they were 
not the best person to address the interview questions. There were also issues with shorter recall periods. 
An interviewee stated that even a one-year follow-up may not be sufficient for the Global Elections 
Framework because globally, only a few elections happen per year. 
All interviewees were asked how they would prefer to engage with follow-up on utilization and 
dissemination. Four preferred a call, three preferred a follow-up in writing, and three did not have a 
preference between the two. In total, ten interviewees did express support for some sort of follow-up 
7 RQ2 and RQ4 address findings and recommendations, respectively. The research team found that interviewees did not regard 
findings and recommendations separately and spoke about the two in an interrelated way. As a result, while the research team 
has sought to respond to RQ2 and RQ4 thoroughly, there is overlap between the two and the distinction between findings and 
recommendations is somewhat imposed. 
16 | E&L UMA FINDINGS REPORT USAID.GOV
contact. While the remaining interviewees did not have a direct response to this question, no interviewee 
rejected the utility of a follow-up. 
HOW FINDINGS ARE USED
Interviewees reported using findings from learning products in several ways. As noted above, learning 
products are frequently used for activity design—two interviewees specifically cited the learning product’s 
use for design. For instance, in the case of a midterm review of gender-based violdence programming, 
findings were used to tweak the design to take a more comprehensive view of sexual and gender-based 
violence and engage more stakeholders, despite the fact that a follow-on activity had already been 
designed. An additional interviewee also used findings to improve a project’s functioning for the remaining 
months of an existing agreement. 
Findings were also used to argue for an increase in funding, continuing a certain type of programming, or 
starting a program in a new sector. One interviewee used findings to support the design and launch of a 
rapid response mechanism. The same interviewee also used findings to start a rule of law program where 
the Mission had not had any rule of law programming for years. Another interviewee discussed using 
findings as a general argument for continuing a funding mechanism that the findings showed was working 
well. 
The importance of learning products providing objective, external expert assessments was noted by four 
interviewees. These interviewees felt that having an outside view of a program can help to confirm the 
existing view of what is happening and provide backup for making an argument for maintaining activities 
under threat or undertaking controversial changes. One example of this was a program where the 
technical officer wanted to eliminate a small grants program that was not working. With support from the 
findings of the evaluation, all parties agreed to eliminate the program and received no push-back. In another 
example, an interviewee referenced the benefit of objective findings in terms of delivering controversial 
decisions to counterparts or IPs. Another interviewee stated, “We wouldn’t be good at our jobs if we 
didn’t have hunches about what is going well or wrong in our projects after managing them for a few 
years. But having an objective assessment that gets at these same areas, that observes issues in a stark 
way, not based on perspectives—that can be really helpful.” Another interviewee agreed, noting that it 
was reassuring to continue activities when a midterm performance evaluation provided evidence that the 
activity was successful to date. 
Finally, there were also less concrete, but nonetheless important, uses. One interviewee stated that the 
learning product had “helped everyone to see the same thing” when they were trying to share information 
across diverse groups at USAID, and another found findings useful in getting stakeholders on the same 
page. Similarly, a third interviewee found findings beneficial for relationship-building with counterparts. Yet 
another interviewee used broad findings to suggest a fresh approach with a government counterpart. The 
Electoral Management Bodies Assessment was useful for helping USAID to strengthen its relationship with 
the local government’s electoral body. Used strategically, these interviewees felt that findings can provide 
a common basis for discussion, objective view, and useful reference. 
PRESENTATION OF FINDINGS
A majority of interviewees highlighted the need for easily digestible information. Interviews repeatedly 
raised the fact that few people read a full findings report. They are often too long and may use academic 
language that is difficult to understand and uninteresting. Useful findings may be buried in long reports and 
17 | E&L UMA FINDINGS REPORT USAID.GOV
complex language, which becomes less useful and applicable. One interviewee suggested that an approach 
to break up long, technical products (such as guides or toolkits) and make them more usable is to use 
modules. Users can then select modules that are pertinent to their work in a shorter format. The Election 
Assessment Framework took this approach, as did the Media Assessment Tool. 
Other suggestions for addressing this issue, which are discussed more in RQ1 on dissemination, were 
varied. Many interviewees raised infographics and two-page summaries as useful tools; however, two-page 
summaries may even be too long in some cases. One interviewee suggested including abstracts in reports 
rather than two-pagers or longer executive summaries, which has been required for some taskings in the 
past. 
Interviewees also suggested focusing on infographics and visuals—even illustrations—that are more eye￾catching and informative. One interviewee gave an example of presenting findings in a comic format, noting 
that while non-traditional, it was highly effective in communicating key information in contexts where 
language barriers made traditional reports and summaries unapproachable. There are certainly tradeoffs 
with taking a more creative approach to presenting findings in terms of time and budget, but doing so has 
the potential to create a much greater impact. 
4.2 CONCLUSIONS
A majority of interviewees struggled to remember specifics of the research or how findings were used, 
including those whose learning product was produced six months previously. In addition, up to two years 
after a learning product was produced, the most knowledgeable person on that tasking may have moved 
to a new position or left USAID. However, shorter recall periods also have drawbacks in that interviewees 
may not have sufficient time to implement findings or be able to take a higher-level assessment of process 
and utilization. One option is to adjust the follow-up period to the specific tasking, which threatens 
to become cumbersome and overly complicated. A second option is to conduct multiple follow-ups at 
regular intervals over time, as one unit at the World Bank does. Project managers are contacted monthly 
for very brief feedback on how research is being used. However, this approach has a downside in the time 
and resource investment in conducting follow-ups, as well as the risk of annoying respondents. In addition, 
while the information gained from the follow-up itself may be more or less useful, the contact itself can 
spur revisiting a product and encouraging use, as is discussed further in Section 5. Whatever the means 
of follow-up, communicating clear expectations between the E&L team and the commissioner 
of the research during the scoping phase may help to ensure responsiveness. In addition, the 
timing for the follow-up can be determined and incorporated into the PEAP. 
Regarding the means of follow-up, there was no clear preference among interviewees between written 
and verbal discussion. The means of follow-up may not be as important as keeping the interaction brief. It 
is also possible that if POCs are aware in advance that follow-up will occur, they will be more likely to 
respond to these requests.
Additionally, users are more focused on how they applied findings, rather than the specific 
findings that they applied. As a result, it is difficult to draw clear conclusions about what makes a 
finding particularly impactful. 
It is useful to note the less concrete applications of findings that interviewees found important. While the 
effect on activity design or implementation is more common, findings were also considered valuable 
18 | E&L UMA FINDINGS REPORT USAID.GOV
in relationship- and consensus-building or as a perceived objective view on a controversial 
topic. Program and technical officers can take the results of evaluations and assessments to counterparts 
and stakeholders for relationship-building, strengthening a shared understanding of an issue, and providing 
support for controversial decisions. Options for additional use categories could include relationship- and 
consensus-building or institutional knowledge, for example, documenting how changes in the political or 
policy context shaped project outcomes. 
Useful findings are tailored to the needs of the evaluation. For example, four interviewees found 
broad and high-level findings useful and informative, while that was a specific complaint for three, who 
found specific and targeted findings to be most useful. This is a concern that can and should be discussed 
in planning phases as the Mission or other user explains how the learning product will be used. 
This contradiction points to the need to very carefully consider the end-user of the evaluation. In some 
cases (for instance, where the learning product is informing a follow-on design), the users may need very 
specific information that they are lacking. Those that are using the learning product for design and 
implementation need practical findings with an eye toward implementation, taking into account, for 
example, financial and material limitations. However, other uses will be better served with more general 
findings, which may be useful for shaping thinking, strategy consideration, reference, and relationship￾building. 
Finally, the presentation of findings is key to utility. Interviewees largely agree that a long 
report, written in academic language, is unlikely to be used widely. While two interviewees 
wanted this type of product for reference, they also stated that it would need to be combined with shorter 
and more visual products for a wider audience. One example given was to use abstracts. Abstracts deliver 
key information in as little as a half-page. This could help to both convey the findings and enhance 
comparability across evaluations and assessments. However, significant information is lost in distilling 
findings to such a short format and the highlights from the report may be different depending on the 
audience. Where targeted dissemination is important, the abstract could be tailored to the interests of 
the intended audience and may be useful in more taskings than currently used. Other examples included 
comics or infographics. To implement this, USAID would need to adjust the timelines and budgets for 
taskings that do want additional dissemination products. 
4.3 RECOMMENDATIONS
● Determine the intervals for utilization follow-up between the E&L team and the 
commissioner of the research and incorporate it into the PEAP. The correct timing for 
follow-up is likely to vary from one tasking to another; one will not use findings for the first year, 
while another will have forgotten findings after a year. Regular follow-up is resource-intensive and 
risks being annoying to the user, but best ensures the collection of accurate information and 
encourages utilization. 
● Consider the use of findings outside of activity design or implementation. When looking 
at the utility of a report, there are many uses outside of activity design and implementation that 
are still worthwhile. These may include relationship-building and providing an objective or outsider 
opinion. The way that findings are presented can be tailored to better suit those uses—for 
example, targeted two-pagers or summaries can be used for meeting with counterparts and 
communicating findings. 
● Incorporate visually appealing and brief presentations of the findings in tasking 
scopes, where appropriate. Interviewees indicated that the audience for a findings report is 
19 | E&L UMA FINDINGS REPORT USAID.GOV
limited; infographics, brief summaries, and visual presentations are more useful for a broad 
audience. The tasking timeline and budget need to account for these products.
5.0 RQ3: WAS THERE ANY FORMAL PROCESS OF DECIDING 
WHAT TO DO WITH THE FINDINGS OR RESULTS (E.G., A 
WORKSHOP OR MEETING)? WAS A PEAP PRODUCED? WAS IT 
UPDATED?
5.1 FINDINGS
In general, two taskings mentioned updated PEAPs or other formal processes to decide what to do with 
the findings or results.nt, effective October 2020, requires that Missions develop a PEAP to guide 
discussion of evaluation findings and recommendations and determine which will be adopted, by whom, 
and when.8 Among 17 Cloudburst LER II taskings that had recommendations (some learning products 
were guides and toolkits that would not have had actionable recommendations comparable to an 
evaluation or assessment), PEAPs began to be included in evaluation scopes beginning with tasking 15; not 
all taskings after 15 were evaluations, where a PEAP would have been appropriate. Five taskings had action 
plans required in the scope and received them after completion of the findings report. Of those five, one 
tasking had updated a PEAP template to add what recommendations had been implemented. An additional 
six taskings had validation or recommendation workshops to formally discuss recommendations. 
TABLE 4: NUMBER AND PERCENTAGE OF TASKINGS BY RECOMMENDATION PROCESS TYPE
FORMAL PROCESS FOR 
CONSIDERING 
RECOMMENDATIONS
PEAP UPDATED PEAP
6 (35 percent) 5 (29 percent) 1 (6 percent)
Other interviewees that did not have recommendation workshops did mention returning to the 
recommendations. For example, in Ukraine, the program office provided the technical officer with a 
written plan to implement recommendations. The technical officer was then able to return to the written 
plan over time and found it very helpful. Another activity manager stated they had no formal plan to 
implement recommendations, but the team discussed a plan informally. This interviewee did not find the 
recommendations themselves useful but did use the informal process of discussing those 
recommendations to make sure the team was on the same page and to discuss best practices. In yet 
another example, an interviewee explained that while they did not have a formal process for discussing 
recommendations, they were used: “Specific recommendations were shared with individual activities and 
IPs for their awareness and capacity-building. They were discussed with them individually by respective 
project managers.”
Another approach mentioned was to have an event to share findings and discuss recommendations. One 
example given was an evidence and synthesis week, during which the Mission had presentations on the 
evidence collected and time for discussion of the recommendations. While this type of event could be 
8 USAID. (2021). “Post-Evaluation Action Plans: Guidance and Templates.” Available at: 
https://usaidlearninglab.org/sites/default/files/resource/files/et_post_evaluation_action_plans_final2021.pdf
20 | E&L UMA FINDINGS REPORT USAID.GOV
more focused on the dissemination of findings, it could also be structured to allow for the pertinent 
stakeholders to understand the findings and then discuss implementation. 
One interviewee noted that following up on the use of the PEAP was itself a good way to encourage 
updating it. In general, requesting a conversation about the use of the PEAP may remind the pertinent 
stakeholders to return to the document and add updates. Without some encouragement, it is unlikely 
that learning product users will update a PEAP. 
5.2 CONCLUSIONS
A formal process for discussing and deciding what to do with findings and recommendations was very rare 
among interviewees, both before and after PEAPs were required. The subset of taskings produced 
after PEAPs were required is small, but the addition of that requirement does not seem to 
have significantly changed formal processes for considering recommendations. While these 
findings are not surprising, they do call for more creativity in how to support learning product users in 
implementing findings and recommendations.
5.3 RECOMMENDATIONS
● Consider ways to encourage the use of the PEAP. PEAPs are now consistently being 
produced for applicable research products. These can be a tool to help Missions discuss and decide 
how to use recommendations; however, they do not appear to be well used, even within taskings 
with recommendations workshops. Following up to ask about the PEAP may help to encourage 
its use in some cases and additional education for Missions or others using evaluations would help 
users understand the purpose of the PEAP, what to do with it, and how it can benefit them. 
● Support users with other processes for determining recommendation implementation. In some 
cases, the PEAP may not be the most effective way for a Missions or other users may have 
alternative processes that they use to decide how to implement recommendations. Interviewees 
mentioned other written plans for implementation in other formats, or “roadmaps,” and various 
forms of meetings and discussions to determine implementation. In these instances, users could 
be guided on adapting the PEAP to their process to make it most effective for tracking utilization. 
As with other recommendations, this process should be tailored to the user’s needs and specific 
learning product and may include learning events or other workshops. 
21 | E&L UMA FINDINGS REPORT USAID.GOV
6.0 RQ 4: WERE THERE ANY RECOMMENDATIONS FROM THE 
REPORT THAT WERE ACTED UPON? WHAT 
RECOMMENDATIONS AND WHAT WAS DONE? HOW DID THE 
EVALUATION INFLUENCE USAID STRATEGY, PROGRAM 
DESIGN, ACTIVITY IMPLEMENTATION, AND UNDERSTANDING?
6.1 FINDINGS
As shown in figure 1 below, interviewees rated most learning products as having a moderate to major 
effect on activity implementation (n=8, 53 percent), activity design (n=12, 75 percent), strategy (n=8, 50 
percent), and general learning (n=9, 60 percent). Activity design had the most interviewees who 
considered their learning product to be moderately or majorly influential. This likely reflects the purpose 
of most of the evaluations and assessments, which was to inform program design. Five respondents 
struggled to name very specific examples of recommendations that were particularly important, either 
because they used the report more holistically for background information, because not enough time had 
elapsed for implementation, or because they simply could not recall individual recommendations. Six 
interviewees were able to name specific recommendations, the lessons from which are discussed below. 
Figure 1: Level of effect on implementation, design, strategy, and general learning rated by interviewees
EFFECT ON IMPLEMENTATION, DESIGN, STRATEGY, AND GENERAL LEARNING 
The ET asked interviewees to rate the efficacy of learning product recommendations on a scale from one 
(no effect) to four (major effect) separately for activity implementation, activity design, strategy, and 
general learning. Interviewees rated 15 learning products (some were multiple products within a single 
tasking) on usefulness to implementation with an average score of 2.67—or a minor to moderate effect. 
The average score among DC-based staff (2.17) was slightly lower than among Mission-based staff (3). 
22 | E&L UMA FINDINGS REPORT USAID.GOV
Those who did not respond to this question either were not involved in the design, did not remember 
how the recommendations were used, or were involved with a learning product that did not have 
recommendations (such as a guide or handbook). 
Few of the interviewees who rated the effect on implementation as moderate to major further elaborated 
why they did so. In part, this was because it was difficult for interviewees to differentiate design from 
implementation. One interviewee considered the effect on implementation to be major because the 
Mission had used the assessment recommendations in designing a new project, which was currently being 
implemented. Multiple respondents felt that having the recommendations confirm what they already 
thought was helpful. In addition, having an objective evaluator present recommendations, even if those 
recommendations were merely confirmatory, helped to support the direction that implementation may 
have already been taking. 
The same question was asked regarding program design. Interviewees rated 16 learning products and 
gave an average score of 3, or a moderate effect. This was slightly higher among Mission-based staff (3.28) 
than among DC-based staff (2.75). While this suggests greater utility to design than to implementation, it 
is probably more accurately a reflection of the intent of the learning products. Missions may have been 
more interested in using assessments and evaluations for the design of follow-on or new projects than for 
ongoing projects. 
Two interviewees noted highly targeted and specific recommendations that helped them design a follow￾on program. One example from a media assessment in Zimbabwe is a recommendation to include 
capacity-building for women in the media and investigative journalism to support diversity and pluralism 
of media voices. This was cited as a highly targeted recommendation that the Mission was then able to 
use. 
When asked to rate the effect of learning products on strategy, interviewees also rated 16 learning 
products. The average score was 2.69, or minor to moderate effect (2.29 among DC-based staff, 3 among 
Mission-based staff). Two interviewees understood this question to refer specifically to the CDCS and 
found the recommendations to be useful for background information and general understanding. Other 
interviewees understood the question to refer to a sectoral strategy, such as climate change, or for the 
Mission’s approach to a specific topic, such as gender-based violence. 
Regarding strategy, interviewees who found recommendations useful largely cited it as background 
information that informed their understanding of an issue. For example, the governance and civil society 
and media evidence reviews produced for the Armenia Mission under LER II CB003 were both cited in 
the Armenia CDCS. In this and other examples, the learning product was useful for strategy development 
in combination with other research and sources. 
Finally, interviewees rated 15 learning products on their utility for general learning. The average score 
was 2.8, or minor to moderate. As above, DC-based staff gave a slightly lower score of 2.42, while Mission￾based staff gave a slightly higher score of 3.06. A moderate or major effect on general learning often came 
from higher-level recommendations that informed the background understanding of an issue. One 
interviewee found comparative recommendations to other countries to be particularly useful in gaining a 
nuanced understanding of the topics studied; a second interviewee found the lack of comparison to other 
donors to be a shortcoming of an evidence review. One interviewee stated that they were able to take 
23 | E&L UMA FINDINGS REPORT USAID.GOV
the recommendations from one global evaluation of a funding mechanism and apply it to other, similar 
funding mechanisms. 
Overall, there may be some inflation of these scores for two reasons. First, interviewees who were unsure 
or doubtful of the utility of recommendations may have elected not to answer the question or provide a 
rating. For implementation, design, and strategy, no interviewee stated that the learning product had a 
negligible effect. However, for general learning, two interviewees did give a score of negligible effect, which 
suggests that interviewees may not have been hesitant to answer honestly. Second, interviewees were 
heavily involved in managing the evaluations. While they frequently voiced criticism of the products, they 
may still have been more likely to see their utility than someone with more distance from the research. 
NATURE OF THE RECOMMENDATION
In discussing why recommendations were utilized or not utilized, interviewees praised both broad, high￾level recommendations and specific, highly targeted recommendations. Two interviewees (one in a Mission 
and one in DC) stated that having broad recommendations that informed thinking was a strength of their 
learning product. One interviewee felt that having those broad recommendations was useful for engaging 
a maximum number of people, but that having specific recommendations would be more useful for 
implementation. Another interviewee found high-level recommendations useful to help inform background 
for a CDCS but acknowledged that those higher-level recommendations were off-putting to some 
technical officers who wanted more applicable recommendations. In addition, one interviewee felt that 
high-level recommendations were useful in that they provided a reference product to which the 
interviewee could return. 
Four interviewees expressed that recommendations that were highly specific and targeted for follow-on 
design and ongoing implementation were most useful. For example, one interviewee felt that actionable, 
technical recommendations were most effective. Another interviewee concurred that highly targeted 
recommendations were useful and helped the team to strengthen their program design. In another case 
that involved a program review of work on global elections and political transition, the interviewee felt 
that more input from the field would strengthen recommendations and better ensure that they are 
actionable in that context. In addition, interviewees felt that these recommendations should take into 
account the goals of the users and the limitations in what they could implement. Even highly specific 
recommendations are not useful if they are impossible to implement. As one interviewee working on a 
civil society and media activity in sub-Saharan Africa explained, “There were some recommendations 
around social movements that didn’t quite make it in the [new] activity in the way that I would have liked. 
The language [of the recommendation] was around in-kind support since they don’t have the structure to 
support large cash awards. But it was hard to translate that into actual programming.” 
EVALUATION DESIGN
A common theme among interviewees was that evaluations and their recommendations need to be 
carefully designed to serve the needs of the Mission or other users. Five interviewees felt that this had 
significantly contributed to the learning product’s success, while three felt doing so would have resulted 
in a better product. These three interviewees raised this idea—they felt that spending time at the beginning 
of the tasking to ensure that the design fit their needs and communicating this to the research team 
resulted in an improved result. One interviewee felt that they had to spend significant time communicating 
to the ET what they wanted to know, particularly that they were interested in what the data showed, 
24 | E&L UMA FINDINGS REPORT USAID.GOV
even if that recommendation was controversial. However, that interviewee was then very pleased with 
the product, suggesting that in that case, the additional time was worthwhile. 
A related common perception was that close collaboration between the research team and Mission or 
other user was beneficial. One interviewee discussed the intense collaboration process between the 
Mission and the ET in revising the evaluation questions and writing the workplan. There were numerous 
rounds of iteration between the two to ensure that the Mission got the information that they needed to 
achieve their objectives. The interviewee stated that this “looked like a struggle at first,” but resulted in a 
more useful report. Four additional interviewees raised collaboration between the research team and the 
Mission or other end users as being key to ensuring that the learning product met their needs. However, 
this kind of collaboration can also be frustrating to participants and have timeline implications. 
Finally, interviewees discussed timing as very important to the utility of the final product. One interviewee 
stated that it was important to perform the evaluation with sufficient lead time to implement 
recommendations. Similarly, another stated that the utility to implementation was limited because the 
program had only a few months remaining. Other interviewees mentioned that the process of performing 
an evaluation took longer than they had anticipated and they received recommendations after they were 
useful. There is a critical need for DRG Center staff to educate Missions on realistic timelines for research 
to ensure that the scope achieves the users’ goals. Given the recommendation included here that there 
be additional time for design development, that could be a longer timeline than currently assumed. 
One interviewee suggested in the future doing a preliminary recommendations presentation that would 
allow them to begin using recommendations before the full report was available, which has been done in 
some other Cloudburst taskings. A more robust outbrief presentation may require slightly more 
preparation time after the completion of fieldwork, although it is a matter of days rather than weeks. It 
requires an understanding between USAID and the research team that, while unlikely, recommendations 
could change between the outbrief and the findings report. 
As discussed in RQ1, an outbrief that includes initial findings can help to bridge the gap between the 
completion of fieldwork and a finalized report. The conclusions that can be drawn in an outbrief, however, 
are limited and should not be overly emphasized, as they could change upon further analysis. 
6.2 CONCLUSIONS
Overall, interviewees had a positive view of the learning products and their utility to USAID 
program design, implementation, strategy, and general learning. Interviewees on average rated 
learning products as having a minor to moderate influence on the various phases of program planning and 
implementation. As one interviewee stated, “All the recommendations were useful and informative; 
whether we used them or not was a different matter. Sometimes resources limit how well we can act on 
a recommendation.” However, four interviewees were able to give specific examples of recommendations 
that were useful to them, suggesting that the positive perception may have been somewhat inflated or 
that the time between the report and follow-up was too long. 
The examples of effective recommendations that were given highlighted the importance of careful 
evaluation design to ensure the results actually serve the needs of the Mission or other commissioning 
party. Interviewees suggested taking additional time at the outset of an evaluation or assessment
to ensure that the design matched the needs of the Mission or other commissioner of the research. While 
25 | E&L UMA FINDINGS REPORT USAID.GOV
this process can be frustrating and time-consuming and may not be possible or necessary in all cases, 
interviewees felt it better ensured they got the information they needed. This may go hand-in-hand with 
the involvement of USAID with the research team; the nature of that involvement will likely vary from 
one tasking to another but can help to ensure buy-in within USAID and guide the research in the desired 
direction. Finally, the timing of research is key. For some taskings, taking extended time to collaborate 
on research design paid off; for others, USAID did not commission the research with enough time to use 
the results. 
6.3 RECOMMENDATIONS
● Clearly communicate the needs of the user to the research team so that 
recommendations can be properly tailored. As in the recommendations for findings, with 
a wide variety of learning products, recommendations must fit the intended use. 
Recommendations must also take into account the donor landscape and the intended scope of 
the use. It may be better to recommend continuing an activity as-is rather than recommending an 
impractical or impossible activity, given the context. 
● Consider USAID’s role in the evaluation. This, too, depends on the specific learning product; 
however, Mission or other USAID staff have an important role in the design, implementation, and 
dissemination of research. This may include E&L methodological expertise in writing the scope 
and design, user collaboration in the research methodology, participation in fieldwork, or 
championing dissemination and utilization. View additional guidance on USAID’s role in external 
evaluations. 
● Select the timing of the research carefully. The E&L team should help Missions or other 
commissioners to understand the timeframe of a research project and assess whether the 
proposed approach can meet their needs in the allotted time. This will also help to ensure that 
the research is done at an appropriate time, given utilization goals. 
7.0 RQ 5: WHAT ARE SOME OF THE BARRIERS TO USING 
EVALUATION FINDINGS OR ACTING ON EVALUATION 
RECOMMENDATIONS? DID THE STRUCTURE OF THE REPORT 
SUPPORT ITS USE?
7.1 FINDINGS
Respondents identified a variety of factors that present barriers to using or acting upon evaluation 
recommendations. Most barriers can generally be categorized as either external factors, such as the 
broader political context in a country, or internal factors, such as a lack of bandwidth within USAID to 
act upon recommendations. Some barriers, however, straddle this divide and are applicable both 
externally to and internally within USAID.
EXTERNAL FACTORS
External factors generally consist of those outside USAID’s control. Perhaps the most obvious of these is 
the overall reform environment within a country (i.e., the political, economic, and social conditions that 
either facilitate or, in many instances, present, obstacles to reform). Several respondents referenced the 
difficult political landscape within a country—such as a new and inexperienced government or poor 
relations between the U.S. and host-country governments—as obstacles to acting upon evaluation 
recommendations. In some cases, given the constantly shifting sands of the international environment, 
these types of changing conditions can even occur during an evaluation or research activity. Naturally, 
26 | E&L UMA FINDINGS REPORT USAID.GOV
economic conditions within a country can affect its reform priorities and willingness to engage with USAID 
on evaluation recommendations. Notably, several respondents referenced the challenging political and 
economic landscape caused by the COVID-19 pandemic, which understandably focused countries' 
attention on more immediate health and economic concerns. Additionally, factors such as closing space 
for reform or a rapidly changing reform environment can make it challenging to follow up on evaluation 
recommendations in a timely and opportune manner.
In addition to the general political context, a more specific barrier identified by respondents is a lack of 
buy-in by relevant stakeholders. In some instances, evaluation findings may threaten the interests of 
particular stakeholders who are content with the status quo. In others, this lack of buy-in can, in part, 
stem from a lack of collaboration during the evaluation process. Stakeholders that are more involved in 
the evaluation process are, naturally, more likely to buy into evaluation recommendations due to both a 
sense of ownership and their concerns, perspectives, and ideas being reflected in the report. One 
respondent observed that “...ownership among the technical team would have helped—we tried to do 
that but it backfired, they did not feel ownership. It’s all well and good to talk about being open to 
recommendations, but when it regards a specific sector that so many jobs are reliant on and the 
recommendations are threatening to the work, it just won’t go over well.”
Finally, an additional external factor that impedes the implementation of evaluation findings is time. Not 
just the inability to act upon recommendations promptly or while the window for reform is still open, but 
simply in terms of whether individuals see the report as being worth their time and effort. One respondent 
noted that:
The chief constraint in all of these things is time; people don’t have enough of it, so for every incremental 
improvement in the quality, it takes time…every incremental improvement takes someone’s time and they 
get annoyed. So we can’t engage someone without taking their time, so we have to make it useful for 
people. Because, for a lot of people, it doesn’t always feel that the incremental improvement that takes 
50 percent more time produces 50 percent utility.
While this barrier is equally true within USAID, for external stakeholders there is the additional challenge 
of engaging them with a report when there is a multitude of sources and information vying for their 
attention.
INTERNAL FACTORS
Internal factors that represent barriers to the implementation of evaluation recommendations are those 
either specific to USAID or within its control. For example, while a lack of buy-in was previously raised 
as an external factor, lack of buy-in within USAID can also be a significant obstacle to the implementation 
of recommendations. In some instances, this may arise from technical teams that have preconceived ideas 
regarding the evaluation process, and recommendations that differ from those preexisting perspectives 
can find difficulty finding acceptance. As with a lack of buy-in from external stakeholders, USAID teams 
that are more involved with the evaluation process are more likely to accept and act upon evaluation 
recommendations. Interestingly, one respondent observed that not only is participation by the USAID 
team important for building ownership of the evaluation and its results; having the right team involved can 
be just as important. In one instance, when asked if anything about the tasking should have been changed, 
one respondent replied, “I would have put the DRG team in the lead; I think they felt that because the 
program office was leading, it diminished the buy-in of the work.” Finally, two respondents noted that, 
27 | E&L UMA FINDINGS REPORT USAID.GOV
separate from the lack of buy-in from specific teams, a lack of buy-in from USAID leadership in evaluation 
results also makes acting upon recommendations challenging.
Another internal factor that is specific to USAID is the lack of opportunity to act upon evaluation 
recommendations. This can stem from many causes, such as a lack of bandwidth or a lack of institutional 
knowledge from turnover at the USAID Mission. Political and funding priorities may change and, as a 
result, the will and funding to follow up on evaluation recommendations may fail to materialize. Similarly, 
one respondent noted that internal politics also plays a role in whether evaluation recommendations are 
implemented. USAID is an understandably large, complicated, and bureaucratic organization with various 
interests competing for a finite amount of resources in terms of political will, funding, and staffing. USAID 
is simply unable to follow up on every evaluation, let alone every recommendation. As strategic priorities 
and interests are weighed, some evaluations and recommendations will simply be deemed less important 
than others.
A discrete set of factors that can raise barriers to the implementation of evaluation recommendations is 
the structure and format of the evaluation report and the recommendations. As noted in RQ2, 
respondents observed that the evaluation reports can be overly academic and fail to communicate key 
points in a succinct and easily digestible format. For example, one respondent observed, “I do feel that a 
lot of these efforts are academic or have an academic bent, which for certain audiences is great, but if 
you’re working in a context where that isn’t approachable to your partners, that’s not useful.” Academia 
provides extremely useful perspectives, expertise, and research that can help inform and guide 
international development. The challenge is that academics frequently do not speak the same language as 
the international development community or approach research and analysis the same way. When 
engaging academic perspectives, the ET needs to frame and even translate their inputs in a way that speaks 
to the intended USAID audience. 
Additionally, several respondents simply found the reports to be too lengthy, which relates to the prior 
barrier regarding a lack of time and bandwidth to fully engage with the report. This can stem from a variety 
of factors, including a broad scope of work that requires a large number of KIIs, numerous and complicated 
issues and evaluation questions for analysis, or the need to extensively support and document findings and
recommendations before they will be accepted. The audience for the report, whether external 
stakeholders or internal USAID staff, must continually prioritize their information sources. Regrettably, 
reports that simply require too much time to digest are less likely to be acted upon. 
Several respondents noted that, on occasion, some evaluation findings and recommendations may not be 
crafted to be as realistic or as actionable as they could be. In some cases, recommendations are too vague 
or academic in nature; in other instances, they can be too specific and difficult to implement. 
Recommendations may not consider the realities of implementation, such as the specific political context 
within a country, buy-in by relevant stakeholders, logistics, funding, or similar work being done by other 
implementers or donors. While these considerations can be difficult for an ET to weigh or even be fully 
informed of, such as internal USAID priorities and funding restrictions, the lack of synergy between the 
recommendations and their audience can nevertheless be a barrier.
7.2 CONCLUSIONS
Overall, respondents identified a range of internal and external factors that provide barriers 
to the implementation of evaluation recommendations, most of which are outside the direct 
28 | E&L UMA FINDINGS REPORT USAID.GOV
control of the technical team leading the evaluation. USAID’s foreign assistance activities do not 
occur in a vacuum, and a somewhat remarkable array of circumstances need to be nearly perfectly aligned 
for an evaluation to lead to impactful recommendations that can be acted upon. The political and economic 
context within the country must be conducive to reform efforts; there must be sufficient buy-in both from 
stakeholders and within USAID itself; the report must provide realistic, actionable recommendations; and 
there must be sufficient bandwidth and funding within USAID to support follow-up activities.
7.3 RECOMMENDATIONS
● Encourage closer collaboration between the technical team commissioning the 
evaluation and the evaluation team (ET), such as collaboration on the design to ensure that 
the research provides analysis that achieves USAID’s goals.
● Build buy-in from implementing partners (IPs) by adequately preparing them for 
evaluations, such as by crafting evaluation guidelines that emphasize the benefits of a 
collaborative process and that frame the evaluation as a learning opportunity and a constructive 
process. In addition, commissioner of the research should provide clear communication to the IP 
on the purpose of the research or evaluation, with support from the ET, where necessary. 
● Provide specific parameters to the ET that aid them in crafting actionable 
recommendations; e.g., rapid-response vs. long-term, directed toward specific partners, the 
general level of funding anticipated for follow-on activities, relevant political context, known 
obstacles to implementation, etc. View additional guidance on developing an effective evaluation 
SOW. 
● Realistically consider the viability of evaluation use before an evaluation is conducted.
Do the timing, resources, and political context provide space and opportunity for an evaluation 
to be impactful?
8.0 RQ 6: HOW COULD EVALUATIONS BE MADE MORE USEFUL? 
HOW COULD USAID AND IPS USE EVALUATION FINDINGS 
MORE?
8.1 FINDINGS
Many of the suggestions for improving evaluations focused on dissemination—pushing research out to 
more stakeholders and in more easily accessible formats—rather than utilization. One interviewee, 
confirming that they would not have changed the evaluation, stated that if the outcome had not been 
useful, they would not have been doing their job at the outset. 
As has been discussed repeatedly throughout the preceding RQs, ensuring that the scope and design fit 
the needs of the Mission or commissioning body is the most important way to ensure that the findings 
and recommendations are useful. The interviewee who earlier noted that some recommendations are not 
sensitive to local logistical parameters pointed out that evaluators often feel pressure to provide novel 
recommendations when sometimes the best recommendation is to continue successful activities. 
Specifically, this interviewee was critical of a recommendation to dedicate resources to activities that were 
outside the scope of their project. Learning products are diverse in their use and purpose and each may 
require a slightly different approach. However, other recommendations arose across a variety of taskings.
29 | E&L UMA FINDINGS REPORT USAID.GOV
STAFFING
Having the correct staffing on both the research team and within the USAID team is crucial to evaluation 
success. A research team with the correct credentials and experience is key. This includes strong technical 
expertise in the topics of interest, which requires the scope to make clear what those topics are for 
recruiting purposes, geographic experience, and knowledge of USAID and the specific development 
context. Without the latter, the recommendations may be impractical or directed toward the incorrect 
actor. One interviewee found that the mismatch between the research team and the needs of the Mission, 
in both sectoral expertise and country experience, led to a report that did not meet the objectives of the 
Mission or give them the information they needed. This led the Mission to conduct rapid assessments 
internally, which they found more useful. 
Interviewees also suggested that the E&L team, as experts in research methodology, could help to ensure 
that the scope of work was clear and accurately communicated the needs of the Mission or commissioning 
body. One interviewee expressed discomfort in approving the design of the research without having the 
background to know if the proposed methodology would achieve their goals. However, this is not desired 
in all cases; another interviewee expressed skepticism that deeper E&L team involvement would be useful, 
feeling instead that additional reviewers would delay the process. 
In other cases, interviewees felt that the Mission’s involvement in the research itself would help ensure 
success. Two interviewees wanted Mission technical staff to participate in interviews pertinent to their 
technical area or even in all interviews. In one tasking, the activity manager participated in all interviews, 
which contributed to Mission buy-in and uptake of the results. USAID staff played a role in the research 
team on at least four Cloudburst taskings—a strategy that was considered successful. However, it is not 
feasible or appropriate in all cases and can raise questions of evaluation independence. 
PRESENTATION
Four interviewees raised the need for brief summaries, visual representations, and innovative illustrations. 
As discussed in RQ2, multiple interviewees called for brief summaries that convey major findings and 
recommendations written with approachable language, as many stakeholders would be unlikely to read 
full reports. These two-pagers or abstracts could be tailored to the intended audiences, providing them 
with the information that would be most useful. This includes preparing materials that Mission 
representatives can have to distribute to stakeholders, such as in the interest of relationship-building. 
Eleven interviewees mentioned presentations being more effective than written material. Of those, five 
also stated that interactive presentations, as opposed to passive, were the best option. Interviewees felt 
that interaction helped stakeholders to better understand and engage with the material and discussion 
could lead to more implementation. One interviewee suggested engaging government counterparts and 
other important stakeholders with presentations of the findings to improve buy-in and another urged 
presentations as a way to disseminate findings to beneficiaries. 
SYNTHESIS AND ACCESS TO RESEARCH
Five interviewees noted that as a clearinghouse for research products, the DEC is very difficult to use. 
Interviewees struggle to find specific reports, topics, or research methodologies, which would allow for 
comparison across contexts. Interviewees acknowledged that research often exists, but believe that it is 
30 | E&L UMA FINDINGS REPORT USAID.GOV
difficult to access. It would be beneficial to have an easily accessible repository of research for 
disseminating findings within a given context. 
In addition, two interviewees took a broader view, expressing a wish for the synthesis of research across 
contexts by subsector. For example, a technical officer designing a program on anti-corruption may want 
to access research from other contexts doing similar work. Similarly, if that technical officer was writing 
a scope for a research project, they could benefit from seeing research using different methodologies on 
a similar topic. What is more, this type of synthesis could be made even more useful with summaries 
highlighting key similarities, differences, and lessons learned. This would help to identify activities that 
work in multiple contexts and shape best practices. 
8.2 CONCLUSIONS
Interviewees had limited specific suggestions in terms of improving evaluation reports; most were focused 
on disseminating findings. That being said, ensuring the correct expertise on the research team 
and engaging USAID staff are important to a successful evaluation. Members of the research 
team must possess country experience combined with USAID knowledge and technical expertise. As
noted throughout the report, USAID engagement may look different on different taskings, but it plays a 
key role in the evaluation’s success. For further discussion on USAID’s role, see the Focus On: Role of 
USAID staff in Research. 
As discussed in RQ1, producing an accessible and engaging research product to complement a findings 
report helps to encourage uptake of the results. The most recommended approach was 
presentations, but infographics, visuals, and two-pagers were also frequently referenced. The 
strong preference for presentations raises the question of why presentations are not more widely used; 
from the learning partner point of view, they are frequently offered. For example, in one tasking, a slide 
deck was requested and created, but never presented. This may reflect “Zoom fatigue” and weariness of 
planning and participating in more meetings. It may also be because many research products lack a 
champion who is interested in pushing research out to a wide audience and encouraging its use. A Mission 
technical officer may use the learning product for a design or implementation and, due to time and budget 
constraints, in addition to project fatigue, leave it at that. This is, of course, successful utilization, but may 
not fully capitalize on the information gathered.
Finally, both access to and synthesis of research play a role in making findings more useful. A 
depository of research that allows for easy searching and access would help to spread research more 
widely. It is widely recognized that the DEC is flawed and difficult to use. The DEC, or another learning 
repository, could be made more useful with better search functions and filtering tools. In addition, other 
knowledge repositories, such as the Learning Lab, could play a role in making research products more 
accessible. One example is organizing links to research in a way that supports its use, such as a list of links 
and synopses by topic. 
Synthesizing research across contexts is a way to build best practices. There is a great deal of 
information being generated through these learning products that is not synthesized. Synthesis activities 
should include performance evaluations and other less rigorous research that still has valuable information 
to contribute to a body of knowledge. Examples of existing work include 3ie’s evidence gap maps and 
their organization of evidence impact summaries. 
31 | E&L UMA FINDINGS REPORT USAID.GOV
8.3 RECOMMENDATIONS
● Determine the correct level of engagement of USAID with the research team. While 
highly dependent on the context of the learning product, there is an important role for USAID 
staff in the learning product. This may include contributing research expertise to the scope, 
collaborating on the research design and workplan, participating in fieldwork where appropriate, 
and championing the research.
● Make research more accessible. The Development Experience Clearinghouse (DEC) does 
not currently contribute to disseminating research more widely. A more useful repository of 
research would make reports and summaries easier to find. Organizing reports by subsector and 
providing a synthesis of reports to suggest useful best practices would enable greater utilization. 
● Engage in research synthesis activities. These activities serve to build on existing research 
to improve and grow a larger body of knowledge of lessons learned and best practices, but they 
also serve to stimulate engagement with and interest in research.
9.0 CONCLUSION
The results of the UMA are open to contradictory interpretations. On one hand, it would be easy to focus 
on the limitations of the various learning products. Even those respondents mostly closely associated with 
the taskings have difficulty remembering specifics of the learning products or the details of how they were 
disseminated. Reports and evaluations are often regarded as too academic and lengthy, with 
recommendations that can be unrealistic, divorced from the relevant political context, and either too 
vague or too specific. Follow-on activities seem haphazard at best due to a variety of factors largely outside 
the control of the technical team leading the evaluation or assessment. These range from unfavorable 
political or economic conditions, lack of buy-in from stakeholders and/or USAID itself, lack of funding, and 
even the fluid nature of USAID staffing. In many instances, the utilization and impact of the taskings appear 
limited.
On the other hand, acknowledging the inherent difficulty of conducting effective learning activities in no 
way undermines their potential utility. While many respondents may have difficulty recalling the specifics 
of their reports, many also provided examples of how the findings and recommendations helped provide 
a useful, external perspective that facilitated strategic thinking, discussions, and planning for current and 
future work, both with stakeholders and within USAID. 
The challenge and objective of this exercise is to help identify those factors that can either facilitate or 
hinder an impactful learning activity. International development and foreign assistance are innately 
complicated and uncertain fields, and the varied experience and impact of the learning activities reflect 
this. As represented in the report recommendations, it is possible to draw conclusions to enhance the 
effectiveness of learning activities and increase their chances of success. This begins with USAID 
understanding and communicating its objectives for the learning activity in light of the relevant political 
context, time restrictions, funding limitations, and other factors that should guide not only the learning 
activity from the outset but also effective dissemination strategies. Recognize that only a small portion of 
the intended audience will consume a full report, so tailoring a learning activity to its audience with 
complementary dissemination products and activities is essential. Interviewees often identified closer 
collaboration between the research and technical teams as a key factor in both building buy-in with the 
technical team and facilitating the crafting of effective recommendations. These, and other actionable steps, 
can enhance the efficacy, efficiency, and impact of USAID learning activities.
32 | E&L UMA FINDINGS REPORT USAID.GOV
ANNEX 1. INTERVIEW GUIDE 
Assent Statement: Thank you for your participation in this brief questionnaire about the use and 
dissemination of [the evaluation/learning product]. The questionnaire will be used to inform an assessment 
of the utilization of learning products produced under the LER II task order. The interviewer you speak 
with at a later date will have access to the information you provide here and completing the questionnaire 
will make your later interview more efficient. All data and identifying information will be anonymized in 
the findings report; it will be impossible to reconstruct your answers. No one will be identified by name 
and it will be impossible to attribute any findings or comments to you. This interview will take 
approximately one hour.
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
Question Indicator
Tasking no.
Region
Countries
Title
Publication year
DEC link
Was there a two-pager or other infographic created? 
[link]
Percent of taskings with a two-pager or other infographic
Were there any other learning products created? [link]
Was there an outbrief presentation? [link] Percent of taskings with an outbrief presentation
Number of attendees Number of attendees at the outbrief
33 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
Attendees:
1. IP staff
2. Foreign government partners
3. USAID staff
4. Academics
5. Media
6. Public
7. Study participants
8. Other
Attendee groups represented at the outbrief
Was there a dissemination presentation to share the 
findings from this study with a larger group of 
stakeholders? [link]
Percent of taskings with a broader dissemination 
presentation 
Number of attendees Number of attendees at the broader dissemination 
presentation
Attendees:
1. IP staff
2. Foreign government partners
3. USAID staff
4. Academics
5. Media
6. Public
7. Study participants
8. Other
Attendee groups represented at the outbrief
Was there a post-study workshop or meeting with core 
project stakeholders to discuss which recommendations 
to accept, which to amend, and which to reject?
Percent of taskings with a post-study workshop or 
meeting to discuss recommendations
Was a PEAP created? [link] Percent of taskings with a PEAP
Utility of study findings and recommendations
Question Answer options Indicator
34 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
In your view, to what extent did the evaluation/learning 
activity influence USAID’s ACTIVITY 
IMPLEMENTATION for the topic(s) covered in the 
study?
1=Negligible or no 
influence
2=Minor influence
3=Moderate influence
4=Major influence
97=Do not know
Percent of projects with a 
"minor," "moderate," or 
"major" influence on USAID 
activity design
You mentioned this project had a 
[negligible/minor/moderate/major] influence on activity 
implementation. 
[If negligible/minor] Why do you think the 
evaluation/learning product did not have a greater 
influence on implementation?
[If moderate/major] Why do you think the 
evaluation/learning product had an influence on 
implementation? Can you provide any specific examples 
of how the evaluation/learning activity informed USAID’s 
activity implementation?
Open-ended 
In your view, to what extent did the evaluation/learning 
product influence USAID’s PROJECT/ACTIVITY DESIGN for 
the topic(s) covered in the study?
1=Negligible or no 
influence
2=Minor influence
3=Moderate influence
4=Major influence
97=Do not know
Percent of projects with a 
"minor," "moderate," or 
"major" influence on USAID 
project/activity design
You mentioned this project had a 
[negligible/minor/moderate/major] influence on 
project/activity design. Why do you say this?
[If negligible/minor] Why do you think the 
evaluation/learning product did not have a greater 
influence on project/activity design? 
[if moderate/major] Why do you think the 
evaluation/learning product had an influence on 
Open-ended
35 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
project/activity design? Can you provide any specific 
examples of how the evaluation/learning activity 
informed USAID’s project/activity design?
In your view, to what extent did the evaluation/learning 
product influence USAID’s STRATEGY (e.g., the CDCS or 
mid-course stocktaking on the topic[s] covered in the 
study)?
1=Negligible or no 
influence
2=Minor influence
3=Moderate influence
4=Major influence
97=Do not know
Percent of projects with a 
"minor," "moderate," or 
"major" influence on USAID 
strategy
You mentioned this project had a 
[negligible/minor/moderate/major] influence on USAID 
strategy. Why do you say this?
[If negligible/minor] Why do you think the 
evaluation/learning product did not have a greater 
influence on strategy?
[if moderate/major] Why do you think the 
evaluation/learning product had an influence on 
strategy? Can you provide any specific examples of how
the evaluation/learning activity informed USAID’s 
activity strategy?
Open-ended
In your view, to what extent did the evaluation/learning 
product contribute to GENERAL LEARNING within USAID 
on the topic(s) covered in the study?
1=Negligible or no 
contribution
2=Minor contribution
3=Moderate 
contribution
4=Major contribution
97=Do not know
Percent of projects with 
"minor," "moderate," or 
"major" contribution to USAID 
general learning
36 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
You mentioned this project had a 
[negligible/minor/moderate/major] influence on general 
learning at USAID. Why do you say this?
[If negligible/minor] Why do you think the 
evaluation/learning product did not have a greater 
influence on general learning?
[if moderate/major] Why do you think the 
evaluation/learning product had an influence on general 
learning? Can you provide any specific examples of how 
the evaluation/learning activity informed USAID’s 
activity general learning?
Open-ended 
If you were to do this evaluation/learning activity again, 
is there anything you would change about how the 
findings and recommendations were presented? How 
could the presentation of the findings and 
recommendations be changed to make them more 
useful? (Probes: Were they clear, specific, and 
actionable? Were they practical and feasible? Did they 
clearly define who should be responsible for 
implementation?)
Open-ended 
Implementation of study recommendations
Question Answer options Indicator
[If a PEAP was created] Has the action plan been 
updated? 
1=Yes
2=No
97=Do not know/do 
not recall
99=Not applicable
Percent of projects with an 
action plan
[If the PEAP has been updated] Could you provide us 
with a soft copy of the updated PEAP (if procurement 
sensitivities could be sent to the DRG Center directly)?
Open-ended
37 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
Other than the post-study workshop or meeting and the 
action plan, were there any other post-study action 
planning activities? What were they?
Open-ended
Can you remember any recommendations from the 
report that were influential? What were they? How were 
influential recommendations used?
Open-ended
Can you remember any recommendations from the 
report that were not useful? What were they? Why 
weren’t they useful?
Open-ended
Were there recommendations that were rejected? Why 
were they rejected?
Open-ended
Were there recommendations that were not acted 
upon? Why were they not acted upon (timing, clarity, 
applicability, external factors)?
Open-ended
Were there recommendations that you wanted to act 
upon, but were not able to? Why were you not able to 
(impractical, timing, responsible party, external factors)?
Open-ended
Dissemination products and activities
Question Answer options Indicator
[For projects with an outbrief] Was the outbrief useful? 
In your view, what did the outbrief achieve? What made 
it successful? What would have made it more 
successful?
Open-ended
[If no outbrief]: Why was there not an outbrief for this 
evaluation/learning activity?
Open-ended
[If no second outbrief]: Why was there not a 
dissemination presentation to share the findings with a 
broader audience? In retrospect, do you feel a second, 
broader presentation would have been useful? 
Why/why not?
Open-ended
38 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
Overall, to what extent were these activities successful 
in clearly, concisely, and accessibly communicating the 
key study findings to the intended audience?
1=Very unsuccessful
2=Unsuccessful
3=Somewhat 
successful
4=Successful
5=Very successful
Percent of projects where 
USAID POC viewed 
dissemination activities as 
“successful” or “very 
successful” in clearly, concisely, 
and accessibly communicating 
the key study findings to the 
intended audience
Do you feel that any of the dissemination activities we 
have discussed were particularly successful? Which 
one(s)? What about them made them successful?
Open-ended
In hindsight, what (if anything) do you wish you had 
done differently to disseminate these learnings? Do you 
see any missed opportunities for broadening/deepening 
engagement?
Open-ended
Wrap-up
Question Answer options Indicator
What are some of the barriers to using evaluation 
findings or acting on evaluation recommendations?
[Probe for potential explanations: quality of the report, 
actionability of the recommendations, reception by the 
Mission, reception by the IP, leadership push, learning 
culture at the Mission, timeliness of the report, cost, 
activity cycle.]
Open-ended
How could USAID incentivize and encourage the use of 
evaluation findings?
What changes could be made to evaluations or other 
learning activities to make them more useful?
Open-ended
What is the best way for the E&L team to follow up with 
you to ask about utilization on future taskings (e.g., 
survey, interview, written follow-up)?
Open-ended
39 | E&L UMA FINDINGS REPORT USAID.GOV
INFORMATION TO BE PRE-FILLED USING PROJECT DOCUMENTATION[1]
Do you have any other comments on utilization and 
implementation that you want to share with us?
Open-ended
[1] Any missing information will be filled in or verified in interviews where necessary. 
40 | E&L UMA FINDINGS REPORT USAID.GOV
ANNEX 2. SUMMARY OF KIIS
GROUP NUMBER OF INTERVIEWEES PERCENT OF INTERVIEWS
USAID/DRG Washington 12 46%
USAID/Mission 12 46%
IP/Other donor 2 8%
Male 9 35%
Female 17 65%
FSN 9 35%
FSO/Other 17 65%
41 | E&L UMA FINDINGS REPORT USAID.GOV
ANNEX 3. LIST OF DOCUMENTS REVIEWED 
Tasking documents, including scopes of work, workplans, findings reports, infographics, two-pagers, 
outbrief slide decks, and dissemination presentation slide decks from the following taskings:
● CB002 Dissemination Activities
● CB003 Armenia Rule of Law, Governance, and Civil Society and Media Assessments
● CB004 Zimbabwean Political Parties and the Promotion of Democracy Assessment
● CB005 Global Labor Collective Action Evidence Review
● CB006 Election Assessment Framework
● CB007 DRG Sustainable Landscapes Guide
● CB008 Global Labor Future of Work Evidence Review
● CB009 Global Elections and Political Transitions Performance Review
● CB010 Ukraine Parliamentary Sector Assessment
● CB012 Global Labor Program Performance Review and Related Evidence Review on Selected 
Topics
● CB013 Sudan DRG Assessment
● CB014 Ukraine New Justice Activity Midterm Performance Evaluation
● CB015 Dominican Republic Family and Gender-Based Violence Literature Review and Human 
Rights Grants Program
● CB016 Belarus Civil Society Midterm Performance Evaluation
● CB017 Nigeria Electoral Management Bodies Assessment
● CB018 Zimbabwe Media Assessment
● CB019 Zimbabwe Civil Society Assessment 
● CB020 Malawi Local Governance Assistance Program Performance Evaluation
● CB021 Ukraine DG East Midterm Performance Evaluation 
● CB023 Impact Evaluation Retrospective 
● CB024 Human Rights Support Mechanism Rapid Response Mechanism Midterm Performance 
Evaluation
● CB025 Georgia Good Governance Initiative Evaluation
Fowle, K., B. Wells, M. Day, A. Kumar, C. Bess, B. Bingham, and A. Wayman. (2020). “The Program and Policy Change Framework: 
A new tool to measure research use in low- and middle-income countries.” Research Evaluation, 1–14, doi: 
10.1093/reseval/rvaa017.
USAID. (2016). Evaluation Utilization at USAID: Findings from an Independent Study, February 2016. Available at: 
https://www.usaid.gov/sites/default/files/documents/1870/Evaluation%20Utilization%20Two￾Page%20Brief%20Final%20%281%29.pdf
USAID. (2013). Meta-Evaluation of Quality and Coverage of USAID Evaluations: 2009–2012. Available at: 
https://www.usaid.gov/sites/default/files/documents/1870/Meta￾Evaluation%20of%20Quality%20and%20Coverage%20of%20USAID%20Evaluations%202009-2012.pdf 
USAID. (2011). Evaluation: Learning from experience. USAID Evaluation Policy. Available at: 
https://www.usaid.gov/sites/default/files/documents/2151/USAIDEvaluationPolicy.pdf 
World Bank. (2004). Influential Evaluations: Evaluations that improved performance and impacts of development programs. 
Washington, DC. Available at: 
https://documents1.worldbank.org/curated/en/609091468329361924/pdf/328790ENGLISH01tial1evaluations1ecd.pdf