November 2015
This publication was produced at the request of the United States Agency for International Development. It 
was prepared independently by Kourtney Pompi, Panhavuth Long, Samdy Lonh, and Amanda Stek of Social 
Impact.
MID-TERM PERFORMANCE EVALUATION OF 
USAID/CAMBODIA’S DEVELOPMENT 
INNOVATIONS (DI) PROJECT
MID-TERM 
PERFORMANCE 
EVALUATION OF 
USAID/CAMBODIA’S 
DEVELOPMENT 
INNOVATIONS (DI) 
PROJECT 
SUPPORTING A MORE COLLABORATIVE ENVIRONMENT FOR 
CIVIL SOCIETY AND TECHNOLOGY SERVICE PROVIDERS IN 
CAMBODIA 
November 2015 
AID-486-I-14-00001 and AID-442-TO-15-00002 
DISCLAIMER 
The author’s views expressed in this publication do not necessarily reflect the views of the United States 
Agency for International Development or the United States Government. 
COVER PHOTO 
An ICT advisor from NGO World Education, one of Development Innovations Project grantees, 
demonstrates how to use TEST app to a young student in Kampong Cham. The TEST app helps teachers 
quickly and easily test the reading skills of Cambodian children. Because the assessment process is 
digitized, teachers can get results immediately and can lead student to other relevant literacy tools based 
on the result. 
ACKNOWLEDGEMENTS 
This evaluation could not have been possible without the contributions of dedicated USAID, implementing 
partner, and beneficiary organization staff whose inputs were instrumental to the evaluation team’s efforts. 
The team expresses its deepest gratitude to the Contracting Officer’s Representative, Ms. Peoulida Ros, 
for her tremendous support and guidance from the evaluation’s inception to its close-out. The team would 
also like to thank Ms. Elizabeth Davnie-Easton, the Alternate Contracting Officer’s Representative, for 
taking the time to provide critical information to the team. 
The team also wishes to acknowledge the support it received from DI project staff. Specifically, the team 
expresses its appreciation for the generosity of DI Chief of Party, Ms. Kate Heuisler, in her willingness to 
welcome the team, provide critical input to numerous inquiries, supply project data, and facilitate 
introductions to partner organizations. 
The team would like to convey its bottomless appreciation for the staff of each organization and partner 
sampled for this evaluation, as well as the beneficiaries who made themselves available to participate in 
focus group discussions and interviews. These staff and beneficiaries consistently greeted the team with 
warmth and enthusiasm, and willingly provided the information upon which this evaluation and its findings, 
conclusions, and recommendations is based. 
CONTENTS 
Executive Summary ............................................................................................................................................................ i
Project Background ....................................................................................................................................................... i
Evaluation Purpose, Questions, and Use .................................................................................................................. i
Evaluation Design and Limitations ............................................................................................................................ iii
Findings, Conclusions, and Recommendations...................................................................................................... iii
Evaluation Purpose and Questions ................................................................................................................................ 1
Evaluation Purpose ........................................................................................................................................................ 1
Evaluation Audience and Use ..................................................................................................................................... 1
Evaluation Questions .................................................................................................................................................... 1
Project Context ............................................................................................................................................................. 3
Project History .............................................................................................................................................................. 3
Results Framework/Theory of Change ................................................................................................................... 4
Evaluation Methods and Limitations .............................................................................................................................. 5
Evaluation Framework ................................................................................................................................................. 5
Phase I: Desk Review ................................................................................................................................................... 5
Phase II: Qualitative Data Collection........................................................................................................................ 6
Data Analysis .................................................................................................................................................................. 7
Limitations....................................................................................................................................................................... 8
Findings, Conclusions, and Recommendations ......................................................................................................... 11
Evaluation Question 1 ................................................................................................................................................ 12
Evaluation Question 2 ................................................................................................................................................ 17
Evaluation Question 3 ................................................................................................................................................ 20
Evaluation Question 4 ................................................................................................................................................ 22
Evaluation Question 5 ................................................................................................................................................ 24
Annexes ............................................................................................................................................................................. 27
Annex I: Evaluation Statement of Work ................................................................................................................ 27
Annex II: Evaluation Matrix ....................................................................................................................................... 39
Annex III: Data Collection Instruments ................................................................................................................. 44
Annex IV: Sources of Information ........................................................................................................................... 70
Annex V: Data Collection Schedule ....................................................................................................................... 72
Annex VI: Disclosure of any Conflicts of Interest............................................................................................... 73
TABLE OF FIGURES 
Figure 1: DAI’s Program Approach ............................................................................................................................... 4
Figure 2: DI Approach According to the Evaluation Team’s Analysis................................................................. 11
Figure 3: Overlap of Tech Solution Development ................................................................................................... 19
Figure 4: Tech Solution Leads to Desired Change .................................................................................................. 21
ACRONYMS 
API Advocacy and Policy Institute
CBCLO Capacity Building for Cambodian Local Organizations
CCC Cooperative Committee for Cambodia
CCHR Cambodian Center for Human Rights
CDCS Country Development Cooperation Strategy
COP Chief of Party
CSO Civil Society Organization
DAI Development Alternatives, Inc.
DCHA Democracy, Conflict, and Humanitarian Assistance
DEC Development Experience Clearinghouse
DG Democracy and Governance
DI Development Innovations
FGD Focus Group Discussion
FY Fiscal Year
ICT Information and Communication Technology
ICT4D ICT for Development
INGO International Non-Governmental Organization
ISC Information Safety and Capacity
IVR Interactive Voice Response
KII Key Informant Interview
RFA Request for Applications
SILK Social Innovation Lab Kampuchea
SMS Short Message Service
SOW Statement of Work 
SPICE Structuring Partnerships for an Innovative Communications Environment
TEST Technology for Education Systems Transformation
TRANSMIT Tracking and Sharing Multi-Sector Issues with Technology
TSP Technology Service Provider
USAID U.S. Agency for International Development
VSO Voluntary Services Overseas
i 
EXECUTIVE SUMMARY 
Technology advances within Cambodia have evolved rapidly in recent years, but gaps remain in the use of 
technology by citizens. Often the actual technical limitations are a result of traditional market forces. 
Therefore, many civil society organizations (CSOs) across the country have not been exposed to the 
various uses of technology for advancing their programs and goals. As a result, many CSOs are not able 
to recognize the possibilities that technology could provide them as they look to advance their 
development goals. 
The USAID-funded Development Innovations (DI) project seeks to build capacities of Cambodian citizens 
and CSOs to use technology for development. To achieve this objective, the project used a strategy of 
events, trainings, the 5D Lab, and mentorship to help CSOs make the best use of new and emerging 
technology for their activities. 
PROJECT BACKGROUND 
Development Alternatives, Inc. (DAI) is currently implementing the three-year DI project (previously 
named Social Innovation Lab Kampuchea (SILK)), a $7,499,744 project. The period of performance is from 
August 7, 2013 to August 6, 2016. The primary goal of the project, as stated in the project’s Monitoring 
and Evaluation Plan, is “to enable a vibrant and sustainable community of Cambodia’s top talent to create 
and utilize technology to facilitate the development of Cambodia by expanding the outreach and improving 
the impact of development programs and by increasing the efficiency and effectiveness of the CSOs 
engaged in USG-assisted activities.”
The DI project specifically aims to build collaborative relationships between Cambodian civil society 
organizations and technology service providers in order to increase the use of technology solutions as a 
means of helping CSOs achieve their development goals. DI’s stated project objectives are the following: 
1. CSOs have an expanded understanding of what’s possible with ICT and have the capacity, tools, 
and incentives to implement. 
2. Tech and service providers have an increased understanding of and capacity to address ICT for 
development and Cambodian CSO needs. 
3. Innovation facilitated through collaboration between CSOs, techies and private sector.
EVALUATION PURPOSE, QUESTIONS, AND USE
This mid-term performance evaluation assessed the workability of the development hypothesis of the DI 
project, measured the implementing partner’s performance in achieving the results expected, and 
documented any obstacles encountered in implementation. The evaluation captured lessons learned from 
DI in order to inform decisions regarding the remainder of the project as well as potential future 
USAID/Cambodia interventions in the Information and Communications Technology (ICT) for 
Development sector. The evaluation also examined how gender was incorporated into program design 
and implementation. In addition, the evaluation looked at program design and implementation for USAID 
more broadly, as this project is the first of its kind for USAID. 
Evaluation Questions 
The evaluation examined DI’s first two years of implementation (August 2013 - August 2015) and took 
place from August to October 2015. The evaluation team reviewed, analyzed, and evaluated the DI project 
ii
along the following criteria, and, where applicable, identified opportunities and recommendations for 
improvement. In answering these questions, the team assessed the performance of both USAID and its 
implementing partner(s). 
1. To what extent has the project succeeded in achieving all of its objectives? Was there a significant 
departure between the Request for Applications (RFA), the project’s Program Description, and 
the work plans? If there were objectives that were not, or are unlikely to be achieved, what were 
the key factors for such findings (please consider the role of project design and project 
implementation)? What recommendations does the evaluation team have for a) the remainder of 
the project, and b) future design and implementation of similar ICT for Development projects? 
Were there any unintended results (positive or negative) of the project? If so, what were they? 
2. To what extent has the project successfully engaged with civil society organizations, especially in 
order to assist CSOs to utilize technological tools/solutions to amplify their programmatic impact 
and support civic outcomes? Have CSOs been able to use the project’s assistance to develop 
solutions to assist them in achieving their development goals? If no or if only partially, why? How 
could the project improve it? 
3. To what extent has the project as a whole (including all of its various activities) contributed to 
achieving the objectives of the USAID/Cambodia Office of Democracy and Governance? 
4. In what ways has the project succeeded or failed in terms of financial sustainability? In what ways 
has the project succeeded or failed in terms of participatory sustainability (i.e. stimulating enduring 
participation in the Lab which could be sustained after the end of the project)? What evidence is 
there that the project will be sustainable at its completion? 
5. How has the Cambodian country context (state of civil society and human capacity in the IT 
sector) affected the implementation of the project? 
Social Impact’s organizational practice is to ensure that, in addition to the above evaluation questions, the 
evaluation team collects data and presents it in a way that is useable by USAID. Specific utilization 
questions that USAID/Cambodia requested this evaluation answer include the following: 
 What are the overall results of the project to date? 
 What is the developmental impact of DI on the CSO and ICT for Development (ICT4D) 
community in Cambodia? 
 This project was funded out of the DG office within the Mission. Did the project have a 
developmental impact within the DG sector? Should it be managed out of a different office? 
 Once the DI project ends, what are next steps? Is this program self-sustainable or could it be 
supported in other ways (different donors, create a stand-alone entity, etc.)? 
 Is the DI model viable, across a sector or across the entire mission? Can/should it be rolled into 
other capacity building programs? Should it be rolled into other support mechanisms within the 
mission? 
Audience and Use 
The primary audiences for the evaluation are the USAID/Cambodia Office of Democracy and Governance, 
the Office of Acquisition and Assistance, Mission leadership, the Mission as a whole, the USAID Global 
Development Lab, USAID Asia Bureau, the Bureau for Democracy, Conflict and Humanitarian Assistance 
(DCHA), other overseas Missions, and DAI as the implementer of the DI project. A wider audience will 
also benefit from the midterm findings once the final evaluation report is submitted to the Development 
Experience Clearinghouse (DEC). 
It is expected that these audiences will use the midterm evaluation results to support and enhance their 
internal decision-making regarding new or expanded activities to promote the use of technology to 
develop efficient and effective civil society groups and organizations. Specifically, the evaluation is intended 
iii 
to inform USAID and DAI decisions to be made for the remainder of the DI project. Findings are also 
intended to help USAID/Cambodia decide on a future course of action with regards to technology 
programming. Additionally, the evaluation may have findings with implications for USAID as a whole. 
EVALUATION DESIGN AND LIMITATIONS 
The evaluation’s data collection was conducted in two phases. Phase 1 included a comprehensive desk 
review of all project documents in order to build evaluation team members’ understanding of the initial 
project concepts and desired outcomes based on the RFA and the cooperative agreement, and to better 
understand DAI’s program design and implementation plan. Phase II consisted of qualitative data collection 
using key informant interviews (KIIs) with key stakeholders, focus group discussions (FGDs) with project 
partners and participants, and observation of scheduled project activities/labs. In total, the evaluation team 
reviewed more than 50 project documents, conducted 38 KIIs, held five FGDs, and conducted one 
observation of a DI activity. 
The Evaluation Matrix in Annex II details the data sources and key respondent categories that informed 
the evaluation team’s answers to each of the evaluation questions. The Matrix references Program 
Indicators, which include a compilation of indicators listed in the latest quarterly reports provided to the 
evaluation team. 
Limitations 
The most significant limitation during fieldwork was the availability of respondent groups. First, it was 
challenging for the evaluation team to schedule FGDs with the desired 6-10 participants per FGD. For 
example, the evaluation team contacted 60+ participants for the 5D Lab-member FGDs and was only able 
to speak with four men and two women. The evaluation team tried to mitigate this challenge by 
rescheduling FGDs, holding FGDs at the DI 5D Lab and offering one-on-one interview times as 
alternatives. Second, the evaluation team was unable to speak with the former Chief of Party (COP) of 
the DI project. Despite multiple communications, this critical respondent was not available for an 
interview. 
While the study findings cannot be generalized beyond the DI project, the evaluation team believes that, 
should the evaluation be conducted again, similar findings to those included in this report would be 
uncovered. Feedback was consistent across all respondent groups; there was no feedback or responses 
that were outliers from the other information collected. 
FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 
Based on the evidence collected, the evaluation team concluded that the DI project has achieved its stated 
objectives to-date. The evaluation team noted, however, that DAI’s slow project start-up, as well as both 
USAID and DAI not providing a clear vision of what success should look like and a lack of an agreed upon 
definition of what ‘innovation’ means, had a significant effect on DI’s overall achievements to-date. The 
programmatic and management changes that took place in early 2015 helped to rectify these deficits. The 
evaluation team also notes that there was a clear distinction amongst respondents’ attitudes toward the 
DI project that aligned with the course correction that took place in spring 2015. Those who encountered 
the DI project in the first 18 months of the project felt that it was too broad and unfocused, and found 
the grant program difficult to navigate. Respondents who have taken part in DI activities since the program 
and management shift have a more positive view of the program. Nearly all those interviewed, including 
sub-grantees, indicated positive feedback on DI’s trainings and advisory support services that went beyond 
the standard confirmation that a specific training was ‘useful.’ 
iv
Specific findings and conclusions related to each evaluation question are included below, followed by 
targeted recommendations for DAI as the implementing partner and USAID/Cambodia. 
Key Findings and Conclusions 
While the DI project overall has delivered some measureable successes to note, it has suffered from a 
variety of factors that affected the substantive results the project has been able to achieve. These factors 
include: revisions to the overall programmatic approach; management and personnel adjustments; lack of 
clear vision of what DI’s programmatic success should look like; different definitions of ‘innovation’; and 
complimentary, and often competing, technology-focused programs funded by USAID and others. 
All CSOs interviewed for this evaluation report stated that the DI trainings they attended were useful, 
and all named the ‘low-fuss video’, ‘digital security’, ‘social media communication strategies’ and ‘design 
thinking’ trainings as most useful. Over the course of the grant, DI provided ‘advisory services’ in a variety 
of packages to CSOs1, although they were not always packaged as such. These advisory services consisted 
of trainings, one-on-one consultations and project design sessions. The evaluation team asked every KII 
and FGD participant (aside from USAID and DI project staff) about their interaction with the DI project 
to assess which activities were viewed as most useful by participants. All respondents confirmed that these 
services would be useful, though only sub-grantee partners were aware of the advisory services that the 
DI team offers as a package of support. The evaluation team concludes that the targeted support DI 
provides through specific training sessions, grant funding, and advisory services to sub-grantees have 
yielded the most positive results. 
The evaluation team found that participating CSOs have benefited from the DI project to varying degrees 
depending on their own internal capacities and interests. Furthermore, the use of technology solutions by 
CSOs depends on individual CSO capacity and interest. The evaluation team also found that it is too early 
to tell what the developmental effect is from the technology solutions that CSOs are using. This is due to 
the slow start of grants and the recent re-focus towards the DG sector. While DI engaged with several 
women CSO leaders or women’s issue CSOs, facilitated women’s focused events and projects (such as 
the Technovations competition), and encouraged women and girls to participate in DI activities, these 
efforts did not yield the desired results of increasing women’s engagement with ICT2, even from the 
project staff’s perspective. 
It is too early to tell if the DI project will contribute to the goal of strengthening democracy and civil 
society, though the project has achieved results in USAID/Cambodia’s Sub IR 1.2.2: Expanded 
technological platforms increasing civic engagement. Several sub-grant projects have the potential, 
however, for positively affecting the DG sector including TosFund, Khmer Smart Keyboard, the Advocacy 
and Policy Institute’s (API’s) Interactive Voice Response (IVR) project, the Cambodian Center for Human 
Rights’ (CCHR’s) Land Right, Pact’s TRANSMIT (Tracking and Sharing Multi-Sector Issues with 
Technology) project, and the Cooperative Committee for Cambodia’s (CCC’s) digital M&E tool. 
The DI project, as currently implemented, is not sustainable beyond the life of the current grant. 
Specifically, the evaluation team found that the 5D lab drew attention away from other collaborative work 
environments that already existed and were more organic in their origin. The evaluation team found the 
 
1 The “Advisory Services” offered by DI have gone through a number of iterations and names (including rapid 
consultations and the Innovation Resource Center). Based on project records, 60 CSOs and 24 TSPs have engaged 
in some form of advisory service over the life of the project (some returning for multiple visits). 
2
 As described in program documentation including the cooperative agreement and annual work plans. 
v 
DAI’s July 2015 sustainability assessment to be both realistic and feasible, with many of the evaluation 
team’s findings being echoed in the assessment findings. This sustainability assessment replaced the original 
business plan for the project. If fully implemented, DI’s revised sustainability plan will likely yield more 
meaningful results and prepare USAID for any follow-on or next generation project. If implemented, this 
should increase participatory sustainability. The evaluation team does not see, however, how the DI 
project will reach financial sustainability by the end of the grant program. The 2015 sustainability 
assessment does not include a financial sustainability component3. 
The evaluation team found several contextual factors that inhibited the achievement of DI’s outcomes, 
including the Cambodian legal framework and political context, a general lack of women in technology 
fields4, and different operational and business cultures between the private sector and CSOs. 
Key Recommendations
Based on this evaluation’s key findings and conclusions, the evaluation team recommends the following for 
the DI project and USAID/Cambodia: 
 In order to increase local ownership of tech solutions, encourage local knowledge and ultimately, 
local ownership, USAID and DI should increase support of known local solutions and foster 
locally-driven tech solutions. This should hold true for other ICT4D projects USAID considers in 
the future. 
 USAID and DI should incorporate high tech local CSOs into future ICT project(s) as ‘tech 
influencers’. These local tech influencers can play a mentoring role to lower-capacity organizations 
and help make future projects locally sustainable. 
 USAID and DI should develop a flexible, streamlined grant program that is multi-staged/phased 
and is not risk-averse. The grant program should support prototype development, allow for beta 
testing, and subsequently support the scaling up of projects. Any grant funds should support 
monitoring the tech solutions long term impact and support maintenance and tech updates. 
 DI should continue to provide advisory services for CSOs. USAID should support DI’s advisory 
services for CSOs and make it into a sustainable model by identifying an appropriate local entity 
that can be groomed to take over the neutral broker role in the future. CSOs are eager for 
more/expanded advisory service, which the current DI provides. DI should identify and mentor a 
local partner, such as university or technical school, who can take over DI’s neutral broker role 
eventually. 
 
3
 The Evaluation team was provided a 23-page Sustainability Assessment Executive Summary during the data 
collection phase. If the full assessment report contains additional information, the evaluation team was not privy to 
it. 
4 This is further explained under evaluation question 5 in the ‘Findings, Conclusions, and Recommendations’ section.
1 
EVALUATION PURPOSE AND 
QUESTIONS 
EVALUATION PURPOSE 
Development Alternatives, Inc. (DAI) is currently implementing the three year Development Innovations 
(DI) project (previously named Social Innovation Lab Kampuchea (SILK)), a $7,499,744 project. The period 
of performance is from August 7, 2013 to August 6, 2016. The primary goal of the project, as stated in 
the project’s Monitoring and Evaluation Plan, is “to enable a vibrant and sustainable community of 
Cambodia’s top talent to create and utilize technology to facilitate the development of Cambodia by 
expanding the outreach and improving the impact of development programs and by increasing the 
efficiency and effectiveness of the Civil Society Organizations (CSOs) engaged in USG-assisted activities.”
This mid-term performance evaluation assessed the workability of the development hypothesis of the DI 
project, measured the implementing partner’s performance in achieving the results expected, and 
documented any obstacles encountered in implementation. The evaluation captured lessons learned from 
DI in order to inform decisions regarding the remainder of the project as well as potential future 
USAID/Cambodia interventions in the Information and Communications Technology (ICT) for 
Development sector. The evaluation also examined how gender was incorporated into program design 
and implementation, as well as the effect that technology had on the promotion of gender equality and 
female empowerment. In addition, the evaluation looked at program design and implementation for USAID 
more broadly, as this project is the first of its kind for USAID. 
EVALUATION AUDIENCE AND USE 
The primary audiences for the evaluation are the USAID/Cambodia Office of Democracy and Governance, 
The Office of Acquisition and Assistance, Mission leadership, the Mission as a whole, the USAID Global 
Development Lab, USAID Asia Bureau, the Bureau for Democracy, Conflict and Humanitarian Assistance 
(DCHA), other overseas Missions and USAID as a whole. A wider audience will also benefit from the 
midterm findings once the final evaluation report is submitted to the Development Experience 
Clearinghouse (DEC). 
It is expected that these audiences will use the midterm evaluation results to support and enhance their 
internal decision-making regarding new or expanded activities to promote the use of technology to 
develop efficient and effective civil society groups and organizations. Specifically, the evaluation is intended 
to inform decisions to be made for the remainder of the DI project. Findings are also intended to help 
USAID/Cambodia decide on a future course of action with regards to technology programming. 
Additionally, the evaluation may have findings with implications for USAID as a whole. 
EVALUATION QUESTIONS 
The midterm evaluation examined DI’s first two years of implementation (August 2013 - August 2015) 
and took place from August 2015 to October 2015. The evaluation reviewed, analyzed, and evaluated the 
DI project along the following criteria, and, where applicable, identified opportunities and 
recommendations for improvement. In answering these questions, the evaluation team assessed the 
performance of both USAID and its implementing partner(s). 
2 
1. To what extent has the project succeeded in achieving all of its objectives? Was there a significant 
departure between the RFA, the project’s Program Description, and the work plans? If there were 
objectives that were not, or are unlikely to be achieved, what were the key factors for such 
findings (please consider the role of project design and project implementation)? What 
recommendations does the evaluation team have for a) the remainder of the project, and b) future 
design and implementation of similar ICT for Development projects? Were there any unintended 
results (positive or negative) of the project? If so, what were they? 
2. To what extent has the project successfully engaged with civil society organizations, especially in 
order to assist CSOs to utilize technological tools/solutions to amplify their programmatic impact 
and support civic outcomes? Have CSOs been able to use the project’s assistance to develop 
solutions to assist them in achieving their development goals? If no or if only partially, why? How 
could the project improve it? 
3. To what extent has the project as a whole (including all of its various activities) contributed to 
achieving the objectives of the USAID/Cambodia Office of Democracy and Governance?5
4. In what ways has the project succeeded or failed in terms of financial sustainability? In what ways 
has the project succeeded or failed in terms of participatory sustainability (i.e. stimulating enduring 
participation in the Lab which could be sustained after the end of the project)? What evidence is 
there that the project will be sustainable at its completion? 
5. How has the Cambodian country context (state of civil society and human capacity in the IT 
sector) affected the implementation of the project? 
Social Impact’s organizational practice is to ensure that, in addition to above evaluation questions, the 
evaluation team collects data and presents it in a way that is useable by USAID. Specific utilization 
questions that USAID/Cambodia has requested this evaluation answer include the following: 
 What are the overall results of the project to date? 
 What is the developmental impact of the DI program on the CSO and ICT4D community in 
Cambodia? 
 This project was funded out of the Democracy and Governance (DG) office within the Mission. 
Did the project have a developmental impact within the DG sector? Should it be managed out of 
a different office? 
 Once the DI project ends, what are next steps? Is this program self-sustainable or could it be 
supported in other ways (different donors, create a stand-alone entity, etc.)? 
 Is the DI model viable, across a sector or across the entire mission? Can/should it be rolled into 
other capacity building programs? Should it be rolled into other support mechanisms within the 
mission? 
The SI evaluation team incorporated these utilization questions into the Findings, Conclusions, and 
Recommendation section of this report. 
 
5 During the USAID kick-off call with the evaluation team, USAID clarified that the team should focus on the goal 
of strengthening civil society as detailed in Development Objective 1: Strengthened democracy and government 
accountability, and enhanced respect for human rights, with a particular look at Intermediate Result 1.2: Increased 
capability of civil society to engage in political processes.
3 
PROJECT BACKGROUND 
PROJECT CONTEXT 
According to the World Bank6, nine out of every 100 people in Cambodia have access to the internet. 
Coupled with an increase in the affordability of mobile technology, including the use of Khmer-enabled 
smartphones, internet penetration is set to skyrocket in the next five years, with the Minister of Post and 
Telecommunications predicting a 65 percent penetration rate by 20207. This presents a growing 
opportunity for citizens and CSOs in Cambodia to utilize ICTs, mobile technology, and the internet to 
increase citizen access to information, identify innovative tech solutions that mobilize citizens around 
issues they care about, and to modernize how CSOs do their work. 
CSOs have taken some measures to utilize ICT. Interactive Voice Response (IVR) was greatly 
popularized by the USAID-funded Structuring Partnerships for an Innovative Communications 
Environment (SPICE) project. The SPICE project, implemented from October 2012 to March 2015, 
facilitated CSOs to use IVR in almost 30 distinct activities, including projects in the education, health 
and DG sectors. 
The general perception, however, is that citizens outside of the capital are still limited in their use of
ICT, primarily using computers and the internet only for email, websites, Facebook, and other desktop￾based software. Internet and data security remains an issue, especially for CSOs working on sensitive 
issues. The USAID Washington-funded Information Safety and Capacity (ISC) project has filled this gap 
to some extent by providing training and mentoring on information security to select CSOs. 
Cambodia has an active ICT business sector, although many of these businesses remain too profit￾oriented, operate under a different business culture than CSOs, or do not have the proper linkages 
or ability to effectively cater to the CSO market. While there are a number of small start-ups, larger 
ICT firms complain about the poor quality of Cambodian graduates in the sector. Some interlocutors 
point out that local ICT firms are still oriented towards foreign markets, rather than domestic ones. 
Despite positive trends, there is still more progress to be made in the utilization of ICT by civil society 
groups, particularly in the DG sector. Civil society struggles with how to use technology to address 
access to information, corruption, human rights, civic education and policy advocacy. 
PROJECT HISTORY 
In January 2012, USAID/Cambodia's Office of Democracy and Governance commissioned an 
assessment of social media and ICT usage to inform future project designs. On October 1, 2012, Open 
Institute was awarded the SPICE project to accomplish several specific tasks related to ICT in 
Cambodia. In particular, Open Institute was to facilitate the use of Khmer text in Android-based smart 
phones, facilitate a public-private partnerships to enable greater use of IVR, promote CSOs use of 
Short Message Service (SMS) to disseminate information, localize applications, and create an 
anonymized crowdsourcing platform. 
 
6
 http://data.worldbank.org/indicator/IT.NET.USER.P2 
7
 http://news.xinhuanet.com/english/2015-04/27/c_134188747.htm
4 
SPICE set the stage for the next CSO-ICT focused project. Following a full and open competition, DAI 
won the three-year award for the DI project. In January 2014, DI moved into their offices and opened 
the 5D Lab. DI was launched by the USAID Mission Director in March 2014. The project was not fully 
funded in the second year due to a shortfall in funds of the USAID/Cambodia DG Office. Additional 
funds were leveraged from other technical offices, including biodiversity and human rights focused 
projects, for the remaining project years. Based on program documents, primarily the monitoring and 
evaluation plan and annual work plans, the DI project, in addition to working with the CSO and 
Technology Service Provider (TSP) community based in Phnom Penh, included four cross-cutting 
themes: youth, women, sustainability, and populations within the provinces. 
RESULTS FRAMEWORK/THEORY OF CHANGE 
The theory of change stated in the RFA was as follows:
If USAID invests in the processes of social innovation, it will provide unique opportunities for civil society 
organizations and socially-minded entrepreneurs to advance their ideas and create new technological 
goods and services that benefit Cambodian society through increased communication and access to 
information.
Although the project description submitted by DAI does not explicitly state a development hypothesis, 
the following quote from the Executive Summary gives some indication of how DAI formulated the theory 
of change: 
This program will enable Cambodian-led 
technology products to meet CSO demands 
and trigger transformative changes in the 
ways CSO conduct programs. At the same 
time, investing in the domestic information 
and communications technology (ICT) 
sector will enable it to become an engine of 
growth in the economy. This emerging 
ecosystem of technology and civil society will 
result in CSOs' increased ability to access 
technology tools and increase their 
proficiencies in order to apply knowledge of 
mobile, Internet, and social media tools to 
amplify their programmatic impact and 
support civic outcomes.
DI’s stated program objectives are as follows: 
Objective 1: CSOs have an expanded understanding of what’s possible with ICT and have the capacity, 
tools, and incentives to implement.
Objective 2: Tech and service providers have an increased understanding of and capacity to address ICT 
for development and Cambodian CSO needs.
Objective 3: Innovation facilitated through collaboration between CSOs, techies and private sector.
Figure 1: DAI’s Program Approach 
(from July 2014 Monitoring and Evaluation Plan)
5 
EVALUATION METHODS AND 
LIMITATIONS 
EVALUATION FRAMEWORK 
This evaluation was conducted using a three-person evaluation team. Following a comprehensive desk 
review, as well as an in-brief with the current USAID Mission Director and staff, the evaluation team 
refined the evaluation work plan and developed data collection tools for this evaluation. The evaluation 
team used a qualitative approach that included document review, key informant interviews (KIIs), focus 
group discussions (FGDs) with project participants, and observation of scheduled project activities. 
The evaluation’s data collection was conducted in two phases. Phase 1 included a comprehensive desk 
review of all project documents in order to build evaluation team members’ understanding of the initial 
project concepts and desired outcomes based on the RFA, and to better understand DAI’s project design 
and implementation plan. Phase II consisted of qualitative data collection using KIIs with key stakeholders, 
FGDs with project partners and participants, and observation of scheduled project activities. In total, the 
evaluation team reviewed more than 50 project documents, conducted 38 KIIs, held five FGDs, and 
conducted one observation of a DI activity. 
The Evaluation Matrix in Annex II details the data sources and key respondent categories that informed 
the evaluation team’s answers to each of the evaluation’s questions. The Matrix references Program 
Indicators, which includes a compilation of indicators listed in the latest quarterly reports provided to the 
evaluation team. Annex V details the evaluation data collection schedule. 
PHASE I: DESK REVIEW 
The evaluation team conducted an extensive desk review of all available project documents supplied by 
both USAID and DAI. The evaluation used project data to corroborate findings from interviews and other 
data sources. The review included the following documents (see Annex IV for an exhaustive list of 
documents reviewed by the evaluation team): 
 Project Request for Application (RFA) 
 USAID Social Media Assessment 2012 
 Project cooperative agreement and modifications (SILK/DI) 
 Project reports (quarterly) 
 Financial reports (quarterly) 
 Work plans (annual) 
 Project monitoring and evaluation plan 
 Project grants documentation (proposals, reports, and tracking pipeline) 
 USAID Cambodia Country Development Cooperation Strategy (CDCS) 
 DI Business Plan
In addition, during Phase II of the evaluation, DAI provided the evaluation team with additional project 
literature that described individual DI-supported projects. The DAI team also provided a copy of the 
sustainability assessment conducted in the summer of 2015 that revised the original DI Business Plan. 
6 
PHASE II: QUALITATIVE DATA COLLECTION 
Phase II began when the evaluation team leader arrived in country. Upon her arrival, the evaluation team 
held an in-brief with USAID/Cambodia in order to clarify expectations and discuss future utilization of the 
evaluation to ensure that the evaluation was responsive to the Mission’s needs. Following the in-brief with 
USAID/Cambodia, the evaluation team finalized the work plan and all data collection instruments. 
Key Informant Interviews (KIIs) 
The team conducted KIIs to triangulate the data collected in the desk review and gain further insights into 
perceptions of the project’s effectiveness at its midpoint. The evaluation team conducted 38 KIIs with the 
following individuals/organizations/technology service providers (see Annex IV for an exhaustive list of 
respondents): 
 Current and former USAID staff 
 Current and former implementing partner (DAI) staff 
 Key CSOs and TSPs that have participated in DI 
 CSOs that have not participated in DI 
 DG Partners 
 Sub-grantees 
 Rejected Sub-grant applicants 
 University or school representatives 
Each KII protocol differed depending on the key informant’s role and “causal distance” from DI activities, 
as well as the extent of the key informant’s involvement in DI activities and the time available for 
interviewing. The questions addressed not just knowledge and general perceptions, but more importantly, 
probed for specific examples of attitude and behavior change. In total, the evaluation team held 38 
interviews during the three-week data collection phase. Final interview questions and protocols are 
included in Annex III. 
Focus Group Discussions (FGDs) 
The evaluation team originally planned to conduct separate FGDs with the following types of individuals 
in Phnom Penh: 
 5D Lab Members (male and female) 
 Training beneficiaries 
 Bloggers 
 Developer/Programmer community 
 5D Lab Advisory Board Members 
As part of the evaluation design, the evaluation team intended to hold two FGDs with 5D Lab members 
– one with at least eight male members and one with at least eight female members. The evaluation team 
intended to conduct at least one FGD with a random sample of training beneficiaries. To maximize the 
use of time, the evaluation team also sought to hold FGDs with groups of individuals with similar 
characteristics including bloggers, members of the local developer/programmer community, and the DI 
Advisory Board. 
The evaluation team held five FGDs in total, but with slightly different respondent groups than those 
planned and noted above. While the evaluation team did meet with 5D Lab advisors in their capacity as 
sub-grantees or key CSOs, the evaluation team was not able to meet with them in their capacity as 5D 
Lab Advisory Board members in a FGD as originally envisioned from the evaluation work plan. The 
evaluation team had considerable difficulty organizing and holding FGDs as originally envisioned. More 
7 
details on the difficulties and resulting data limitations are outlined below. In total, the evaluation team 
held five FGDs: one with male 5D Lab members, one with female 5D Lab members, one with bloggers, 
one with training beneficiaries, and one with DI staff not included in the KIIs. 
Each FGD protocol differed depending on the groups’ role and “causal distance” from activities, as well 
as the extent of the groups’ involvement in DI activities and the time available for interviewing. The 
questions did not address just knowledge and general perceptions, but more importantly, probed for 
specific examples of attitude and behavior change. Final FGD questions are included in Annex III. 
Observation 
During phase two, there was one project event scheduled for the evaluation team to observe. On 
September 22, the team attended and observed the Innovations in Action Session entitled "Putting an End 
to Violence against Women through Innovations in Mobile Technology" by international non￾governmental organization (INGO) The Asia Foundation and technology partner Golden Gekko. 
DATA ANALYSIS 
The evaluation team conducted daily de-briefs in order to discuss a) evidence collected, patterns, and 
discrepancies in answering the evaluation questions and b) any adjustments that were needed in the 
evaluation schedule. 
This evaluation relied on two main sources of data: existing project documents and other relevant reports 
as a secondary source, and a number of key informant interviews, focus group discussions, and 
observations as a primary source. Parallel analysis was used to analyze the evidence from both sources. In 
this analytical approach, each type of data for the evaluation question was first analyzed according to its 
respective source, and then across both data sources. For example, the team developed preliminary 
findings by first analyzing interviews with key informants; then, developed complementary preliminary 
findings from the key documents and other secondary materials. 
Given that the overall approach of this evaluation relies on in-depth interviews with a broad range of 
stakeholders as a primary data source, the evaluation team collected primarily qualitative data to be 
analyzed using appropriate qualitative analysis methods, specifically: 
 The evaluation team began data analysis by performing a round of initial open coding in order to 
identify and conceptualize common trends and themes that emerged from the collected data. 
 Once all the qualitative data were consolidated according to the evaluation questions, the 
evaluation team identified the most prevalent themes and concepts mentioned by respondents in 
order to yield in-depth analysis pertinent to the evaluation questions. 
This two-step coding approach allowed for generating both “horizontal” analysis (across main themes to 
create the big picture) and “vertical” analysis (in-depth understanding of the most important issues). 
Gender Integration 
Consistent with USAID evaluation policy and recognizing that effects of integration and the success of the 
project might vary across gender, the team applied a gender perspective to its data collection and data 
analysis. All data collection methodologies considered the privacy and confidentiality of respondents as 
well as included gender-appropriate questions. Both women and men were included in the sampling of 
stakeholder groups. Lastly, the team ensured that interviews and focus groups were conducted at times 
and places accessible to both men and women equally. The evaluation team included gender-specific 
questions in interview guides with all relevant stakeholder groups in order to evaluate the effect of gender 
equality on project results and vice versa. 
8 
LIMITATIONS 
There are few, but notable, limitations to the methods used and data collected during this evaluation. 
Below details limitations related to bias, availability of respondents, gender inclusion, and provincial 
populations and the evaluation team’s mitigation strategies to address each limitation.
Bias 
As is expected in any social research project, there are biases and other limitations that must be addressed 
through methodological or analytical methods. The evaluation team took special care in identifying and 
mitigating (as much as possible) the following issues. 
 Recall bias, such as key informants responding to team questions with answers related to a 
different assistance program (particularly the SPICE program). A similar problem is that 
participants in multiple training activities may blend their experiences into a composite memory 
or response, e.g., individuals have received training on several topics before and during the 
evaluation period, and subsequently do not distinguish between them as separate activities in their 
responses. Additionally, respondents may have encountered other tech labs/social entrepreneur 
centers such as the Impact Hub and may confuse assistance received through other similar 
projects. 
 Response bias, such as respondents giving the interviewer positive remarks about an activity like 
ICT training because they would like to participate in additional training in the future. The team 
fully expected that key points of contact, trainees, and partner organizations may understand that 
this assessment will shape future project opportunities and funding. 
 Selection bias in the form of contacts provided by DAI and/or USAID can mean that the team 
only hears from people with positive experiences. The list of respondent groups was provided by 
USAID to the evaluation team. Due to a limited data collection phase, the evaluation team did not 
have additional time to reach out to others beyond the provided list. The exception to this was 
to interview additional university/learning institutions, as the evaluation team felt this was an 
important group to meet.
The most effective approach to combating these threats was to use multiple sources of data to triangulate 
data for each evaluation question. By combining information found in documents and interviews from 
multiple sources, any one piece of biased data did not skew the analysis. Another approach used by the 
team that pertains specifically to interviews is the inclusion of key informants from organizations that did 
not directly participate in DI. The following groups were interviewed: 
 Voluntary Services Overseas (VSO) 
 Cooperative Committee for Cambodia (CCC) 
 SILAKA 
 Returnee Integration Support Center 
Availability of Respondents 
The most significant limitation to the data collected was the availability of respondent groups. Below details 
the challenges the evaluation team faced when attempting to contact respondents. 
 Declined to be Interviewed/Cancelled Scheduled Interview: Several respondents canceled 
scheduled interviews or declined to participate as a respondent in the evaluation. For those 
meetings that were cancelled, the evaluation team tried to reschedule for a time and location 
convenient to the respondent. Some of these meetings were rescheduled; some were not. Several 
9 
meeting requests were declined, as the respondents did not feel it was worth their time or they 
did not feel they had anything to contribute. This is true of both CSOs and TSPs. 
 No Response/No Shows: The evaluation team experienced the most frustration in trying to 
schedule FGDs with the desired 6-10 participants per FGD. Despite all the evaluation team’s 
efforts, all FGDs included less than the desired number of participants. For example, the evaluation 
team contacted 60+ participants for the 5D Lab member FGDs and was only able to convene four 
men and two women. The team had many more respondent confirmations, but when it came time 
for the scheduled FGD, there were several no shows for each respondent group. The evaluation 
team tried to mitigate this by rescheduling some FGDs, holding FGDs at the DI 5D Lab, and 
offering one-on-one interview times as alternatives. 
Of note, the evaluation team tried several times to speak with the former Chief of Party (COP) 
of the DI project and could not get a time scheduled for an interview. In fact, the last emails sent 
by the evaluation team went unanswered. 
 Schedule Conflicts: The evaluation team anticipated that interviews with government officials and 
advisory board members may be difficult to schedule because of existing demands on their time 
or the need to accommodate last-minute scheduling changes. In addition, the annual Pchum Ben 
holiday was scheduled for the week after the team’s departure, making it difficult to schedule 
interviews later in the fieldwork schedule. There were a number of requested interviews and 
focus groups that were not able to be scheduled due to respondent scheduling conflicts and/or 
travel plans. 
While the study findings cannot be generalized beyond the DI project, the evaluation team believes that, 
should the evaluation be conducted again, similar findings to those included in this report would be 
uncovered. Feedback was consistent across all respondent groups; there was no feedback or responses 
that were outliers from the other information collected.
Gender Inclusion and Provincial Populations 
Because the DI project was not as successful as originally intended in increasing gender inclusion across 
activities, the evaluation team had difficulty gathering feedback from female project participants. Despite 
reaching out to several women-led CSOs, and reaching out specifically to female 5D Lab members, the 
evaluation team was only able to capture feedback from five females (not including USAID or DI staff). 
This had an effect on the gender analysis component of this evaluation. The feedback the evaluation team 
received from female respondents, however, does not differ in any meaningful way from male feedback. 
Therefore, the limited female sample does not negatively impact the reliability or coverage of findings 
presented here. The evaluation team concluded that the limited number of female respondents did not 
necessarily weaken overall analysis. The evaluation team was able to gather some gender-specific data 
based on the inclusion of gender-specific questions in each KII and FGD protocol, however. 
Additionally, as described in program planning documents (cooperative agreement and work plans), the 
DI program had several cross-cutting themes including activities that support and benefit provincial 
populations. The DI program focused most program efforts at CSOs and TSPs that were based in Phnom 
Penh. As described by DAI, this was due to time constraints and the refinement of their efforts with CSOs 
that had some degree of technical capacity and awareness of how to use technology in their work. DI did 
support organizations whose project activities, including those funded through DI’s grant program, 
benefited citizens in the provinces. As a result, the DI project itself did not have a strong emphasis on the 
direct inclusion of provincial based CSOs or provincial training events other than the barcamps. Due to 
limited time, the evaluation team was not able to speak with attendees of the barcamps held outside 
10
Phnom Penh, for example, who were the only provincial populations the DI project directly worked with. 
In analyzing the results of the program, the evaluation team included how the project reached out to, 
included and benefited provincial populations. Analysis around sustainability is captured under evaluation 
question 4. All other cross cutting theme analysis is incorporated, where appropriate, under each of the 
additional evaluation questions. 
11
FINDINGS, CONCLUSIONS, AND 
RECOMMENDATIONS 
Based on the evidence collected, the evaluation team concluded that the DI project had positive outcomes 
and met its stated objectives. The evaluation team noted, however, that DAI’s slow project start-up, the 
project’s lack of a clear vision of what success should look like and an agreed upon definition of what 
‘innovation’ means had an effect on DI’s overall achievements to-date. The programmatic and management 
changes that took place in early 2015 helped to rectify these deficits. The evaluation team also noted that 
there was a clear distinction amongst respondents’ attitudes toward the DI project that aligned with the 
course correction that took place in spring 2015. Those who encountered the DI project in the first 18 
months of the project felt that it was too broad and unfocused, and found the grant program difficult to 
navigate. Respondents who have taken part in DI activities since the program and management shift have 
a more positive view of the project. Nearly all sub-grantees indicated positive feedback on DI’s trainings 
and advisory support services that went beyond the standard confirmation that a specific training was 
‘useful.’ 
While there was no significant departure from DI’s project design, the evaluation team’s analysis and 
understanding of the project differed from the project approach as stated by DAI in project documentation 
(see Figure 2). The evaluation team understood DI’s project focus as lying primarily at the intersection of 
objectives 1(CSOs) and 2 (Tech Providers) to bolster objective 3 (Partnerships). The evaluation team’s 
understanding from project documents was that the DI program would work with CSOs, regardless of 
baseline ability, to increase their skills for including ICT solutions in their work. The evaluation team 
recognized that choices must be made about which organizations to support due to available financial 
resources and time. The evaluation team concluded, that the DI project as implemented had a slightly 
different focus than the one outlined in 
program documents. While the DI 
project certainly worked with CSOs and 
TSPs to build collaborative relationships, 
the evaluation team’s analysis showed 
that the DI project focused mostly 
(though not exclusively) on the bottom 
half of Figure 2, with emphasis and 
support centering on CSOs that had an 
existing awareness of what tech 
solutions would be useful to their work 
and with larger TSPs that had the 
infrastructure and internal practices that 
could support CSO work.8 The 
evaluation team called these groups ‘high 
tech aware’. This was an important 
distinction, as 100% of respondents 
 
8 While most project public-private-partnerships have been with big TSPs that have significant funds to contribute, 
the project has begun working to introduce smaller TSPs to potential grantees in need of their services, recruit 
freelancers as mentors for project activities, including TSPs of all sizes in Tech Expos and Barcamp, and provide 
space for TSP-led IT training and mentorship. This information was clarified by the implementer post-evaluation 
fieldwork.
Figure 2: DI Approach According to the Evaluation 
Team’s Analysis
12
explained that one of the biggest barriers to greater usage of ICTs by Cambodians broadly, and CSOs 
specifically, is a lack of human and technical capacity. Working primarily at the bottom of Figure 2, with 
groups that were at a higher capacity and awareness for tech solutions, the project missed opportunities 
to help fill this gap with other lower capacity CSOs (represented at the top of Figure 2) which could have 
helped to level out the large deficit in capacity that currently exists. The evaluation team recognized, 
however, that the three-year grant length was a limiting factor in this regard. 
It is through this revised understanding of the DI programmatic approach that the evaluation team 
centered the mid-term findings, conclusions, and recommendations. For this report, analysis is organized 
by evaluation question. Each question begins with a breakdown of key findings and includes a summary of 
conclusions. Each question is then concluded with specific recommendations for USAID and/or DAI. 
Evaluation Question 1 
To what extent has the project succeeded in achieving all of its objectives? Was there a significant departure 
between the RFA, the project’s Program Description, and the work plans? If there were objectives that were not, or 
are unlikely to be achieved, what were the key factors for such findings (please consider the role of project design 
and project implementation)? What recommendations does the evaluation team have for a) the remainder of the 
project, and b) future design and implementation of similar ICT for Development projects? Were there any 
unintended results (positive or negative) of the project? If so, what were they?
Findings 
Overall, the DI project has met its stated objectives (one, two, and three) and has some measureable 
successes to note. The DI project suffered, however, from a variety of factors that affected the results 
the project has seen thus far. These factors include: revisions to the overall programmatic approach; 
management and personnel adjustments; lack of clear vision, by both USAID and DAI, of what DI’s 
programmatic success should or would look like; different definitions and understandings of what was 
meant by ‘innovation’; and complimentary, and often competing, tech focused programs funded by USAID 
and others. 
The evaluation team found that there was no significant departure from project work plans, project RFA, 
and the project’s Program Description to date. DI activities were refined throughout the grant period, to 
reflect program realities9, but the program objectives did not change. 
The evaluation team noted, however, that there may have been a flaw in the original program design 
process; the original RFA and the cooperative agreement were quite broad in terms of expected program 
achievements and did not mention USAID’s intention that the proposed program should have a democracy 
and governance focus. Though there was no major departure from project plans, the DI project refined 
its approach after consultation with USAID to focus more on organizations operating within the 
democracy and governance sector in early to mid-2015. In the original project RFA, there was no 
democracy and governance goal or development objective referenced, nor was it referenced in 
subsequent program documents. The original AOR for the DI project notes that there was no explicit 
reference to the DG sector in early project documents or in verbal guidance provided to DAI. The DAI 
proposal, therefore, did not mention the democracy and governance and plans to achieve impacts in the 
DG sector. Under guidance from the USAID DG Office Director, the 2015 project shift included the 
inclusion of DG-specific goals.10 Since this re-focus, the DI project is clearer in its activities and desired 
outcomes. 
 
9 This includes DAI’s learning curve within the CSO and TSP community in Cambodia through the startup and launch 
phases, the receptivity of CSOs and TSPs to proposed activities and changing priorities from the proposal to 
implementation phase. 
10 Additional findings related to this subject are included under evaluation question 3.
13
Findings Related to Objective 1 
The evaluation team found that the targeted support DI provided through specific training sessions, grant 
funding, and advisory services to sub-grantees seemed to have yielded the most positive results from the 
CSO perspective. For CSO respondents, including sub-grantees 
that the evaluation team spoke to who had attended a DI training, 
100% could articulate how they used the information from the 
training in their work. For objective one, CSOs that took part in 
DI trainings mentioned specifically their ability to utilize knowledge 
gained from the sessions. One organization mentioned their 
application of the design thinking training in their everyday work. 
Other groups discussed how the digital security training was beneficial, especially in relation to the 
potential for government surveillance of their work. 
Over the course of the grant, DI provided ‘advisory services’ in a variety of packages to CSOs11, although 
they were not always packaged as such. These advisory services consisted of trainings, one-on-one 
consultations, and project design sessions. The evaluation team asked every KII and FGD participant (aside 
from USAID and DI project staff) about their interaction with the DI project to assess which activities 
were viewed as most useful by participants. All respondents confirmed that these services would be useful, 
though only sub-grantee partners were aware of the advisory services that the DI team offered as a 
package of support. The evaluation team concluded that the targeted support DI provides through specific 
training sessions, grant funding, and advisory services to sub-grantees yielded the most positive results. 
Findings Related to Objective 2 
The evaluation team also asked all TSP respondents about their interactions with the DI project, including 
any events they may have attended. Two TSP respondents specifically stated that they found the events 
helpful as networking opportunities. Other TSPs, including those that use the 5D Lab, did not mention 
the events or trainings when asked about their interactions with the DI project. 
Findings Related to Objective 3 
With the exception of a few events, where one or two CSO-TSP collaborative relationships originated, 
most respondents did not find DI’s events (discussions, presentations, networking-style gatherings) useful. 
One notable exception was a speed dating style event where a future sub-grantee met a TSP with whom 
they developed a project to submit to the DI grant fund. This is in line with the DI revised approach that 
dropped tech focused events and refocused efforts on targeted trainings, advisory services and grant 
support. While TSPs did not find the DI events or trainings particularly useful, they did find utility from 
the role that DI played in linking TSPs and CSOs. Three TSP partners specifically mentioned they found 
this ‘interlocutor’ role helped them understand the CSO perspective and CSO needs. 
The evaluation team noted several projects supported through the grant fund that help groups meet their 
development objectives. These included the World Education student aptitude and learning app, 
Cambodian Center for Independent Media’s citizen journalism app, and Advocacy and Policy Initiative’s 
public information app. This speaks directly to meeting objective three – developing collaborative 
partnerships. The evaluation team did note that several DI supported projects, including the People in 
Need mHealth app and ALiEN DEV’s Khmer smart keyboard app either a) had support from other USAID 
projects like SPICE or b) originated outside of the DI project parameters. This does not diminish DI’s 
 
11 The “Advisory Services” offered by DI have gone through a number of iterations and names (including rapid 
consultations and the Innovation Resource Center). Based on project records, 60 CSOs and 24 TSPs have engaged 
in some form of advisory service over the life of the project (some returning for multiple visits). 
“The digital security training 
was good. It fit our needs and 
prompted us to develop an 
internal security policy.”
- Key CSO Respondent
14
influence in these projects’ successes, but does illustrate the overlap of tech-focused projects and the 
reality that idea generation does not always occur in confined programmatic spaces. 
Despite these grant program high points and the achievement of DI objectives, many current, former, and 
potential sub-grant applicants noted that the DI grant program was extremely inflexible and burdensome 
in requirements for the application process, and in the program and financial reporting process. In fact, 
one current sub-grantee said that they will likely not reapply for additional DI grant support as they found 
the grant procedures and processes too 
burdensome. Respondents explained that the 
grant process extremely slow and they were 
often asked to prototype ideas with their own 
resources. Those sub-grantees that received 
sub-grants since the mid-2015 cycle noted, 
however, that the process was smoother but still 
somewhat onerous. 
Several respondents commented that DI could have more impact if they allocated funds to content and 
not just to tech solutions. This was due to the understanding that the content pushed out or collected 
through the tech solution (such as an app or website) is equally as important as the tech solution itself. 
The People In Need mHealth project was an example of this, where the tech solution was funded via 
SPICE and DI funded the development of additional content for dissemination through the app. 
The evaluation team noted some unintended results of the project - most substantial was the 5D Lab. 
Originally envisioned as a standalone collaborative lab that would exist past the DI project, no one that 
the evaluation team spoke with felt that the DI 5D Lab was successful in that regard. In fact, the DI 
management team, self-identifying this unintended result, undertook a sustainability assessment mid-2015 
in order to revise their plans for how the 5D Lab could be sustainable without DI support. Though the 
Lab was not successful in facilitating the incubation of ideas, there were positive outputs or outcomes 
from the 5D Lab as a co-working space for civic-focused groups or other social entrepreneurs. The 5D 
Lab did provide a physical space for people to work, such as sub-grantee ALiEN DEV, who created the 
Khmer Smart Keyboard, which could assist in linking 5D Lab members with TSPs. 
The evaluation team also noted that gender inclusion in the DI project could have been more robust. 
DAI described to the evaluation team that they had difficulty getting women to attend DI events. DAI 
initially used a women development approach (hosting trainings and events specific to women or that 
highlighted gender specific topics) to encourage women to participate in the program. Some efforts were 
more successful than others. A highlight was the Technovations project that brought high school girls 
together to work on a technology competition. To increase female attendance at more DI events and 
trainings, DAI shifted to a gender mainstreaming approach, an approach that makes concerns and 
experiences of both women and men an integral dimension of the project design and implementation. This 
shift took place at the same time as the revisions to the project approach and management (early-to-mid 
2015). From the evidence gathered, the evaluation team could not conclude that this shift in approach 
resulted in an increase in attendance by women in the DI project. 
Finally, the evaluation team notes that the DI project did not have a 
local Cambodian counterpart as a part of this project. While the 
program certainly can consider sub-grantees and TSPs as partners, 
there was no local partner to act as a counterpart (or even to be 
mentored to eventually be a counterpart) for the project. Having a 
local partner to be more of a ‘co-implementer’ could help with both 
“At the beginning, they said it 
was collaborative [the DI 
project], but then it was just 
DI working alone.”
-Subgrant Partner
“We didn’t have the money to do a prototype, but 
that’s what DI wanted us to do [in order to apply 
for a grant]. The application process took more 
than a year. They said our proposal was still unclear 
and they wanted us to do another prototype.”
-Female Sub-grant partner
15
sustainability issues and combatting the perception that the DI project is driven by internationals rather 
than locally driven. As a corollary to this, several CSOs and tech community members that the evaluation 
team met with mentioned that the branding, by both DI and USAID, of project activities was viewed 
negatively by local civic activists and tech community members. Several respondents remarked that the 
events, which they originally felt were their own, had been “coopted” by the international community and 
that these events were no longer their own. 
Conclusions 
The evaluation team concluded that at the project’s inception, USAID did not provide sufficient guidance 
to DAI in order for them to meet USAID’s expected outcome of developmental results within the 
democracy and governance sector. Additionally, USAID and DAI did not have a clear vision of what DI’s 
success would look like, making it difficult for the implementing partner to implement a project aimed 
towards those unknown goals. There was also little guidance on who beneficiaries of the DI project should 
be. The evaluation team heard from DAI, key CSO partners and sub-grantees alike that they were not 
sure who the project was supposed to work with – was the project focused on issue-specific CSOs (DG, 
human rights, education, health) or anyone with an innovative idea? Should the project be working with 
groups with lower capacity in order to bring them up to speed, or is the program focused on working 
with CSOs that already have a certain level of existing capacity? Was the project working with learning 
institutions (such as Liger) or CSOs? The result of this was that the DI project cast a very wide net early 
on that hampered their ability to invest significant time and energy with a few select organizations that 
had ideas for how tech could be a solution for their developmental challenges. 
The evaluation team also concludes that several factors related to DAI’s initial staffing choices, as well as 
in-country experience, affected the DI project. DAI had difficulty in establishing a management team that 
had the skills and experience necessary for the project. A contributing factor was that the RFA requested 
that COP candidates have a strong media background. 
Second, the initial DAI management team did not have a 
strong civil society background. Third, DAI’s lack of 
experience in the sector within Cambodia slowed project 
start-up. At the same time, USAID did not identify 
management as one of DAI’s challenges to implementation 
until more than a year into the project. DAI undertook 
staff changes, which seemed to have improved implementation enough to lead to the achievement of more 
measureable successes. The evaluation team concluded that, had these challenges been identified and 
mitigated earlier, the DI project may have had more results by its midpoint. 
USAID and DAI operated with different definitions and interpretations of what was meant by ‘innovation’. 
This affected the types of projects DI was targeting for support and affected USAID’s expectations of what 
the project should be achieving. Specifically, the evaluation team was told by one respondent that he/she 
thought the project requirement was that innovative ideas “had to improve a CSOs challenge by 100 
percent and use new technology” only, instead of using innovative ideas and new or existing technologies 
to incrementally address CSO needs and/or goals. Another respondent explained that they thought 
USAID’s expectation of ‘innovation’ was “technology that was ‘disruptive’”– or technology that shakes up 
the industry (like the use of email as a communication tool or the smartphone replacing the ubiquitous 
Nokia phone). Other respondents, conversely, told the evaluation team that any tech solution, regardless 
of how flashy it was, should be considered innovative if it is new and innovative for that specific 
organization and its challenge. 
Because there are several USAID-funded tech-focused and/or capacity-building programs that were/are 
being implemented at the same time as DI in Cambodia, it is difficult to attribute sector successes solely 
“I wondered why the SILK/DI project 
‘wasn’t awarded to a group that had 
been in the space longer -- both 
Cambodia or in the ICT field.”
- Sub-grant Partner CSO
16
and specifically to the DI project, though findings detailed above do explain specific DI activities and 
projects that respondents identified as resulting in positive change. Several project ideas that were 
supported by DI were also, in part, supported by the USAID-funded SPICE project, for example. This 
includes the People in Need mHealth project, as well as other IVR-based projects. At the same time, there 
was a specific capacity-building project, the Capacity Building for Cambodian Local Organizations 
(CBCLO), that one non-partner organization said they would go to if they needed tech or ICT support –
they did not know DI offered similar tech solution support and advisory services. While CBCLO has a 
very different focus than DI, the perception of this local CSO (and likely others) was that DI was not a 
resource. There are also a number of other bilaterally-funded tech programs – including Daikonia and 
European Union support projects - that are competing within the CSO-ICT space in a similar way to DI’s 
objective three. 
Finally, the evaluation team concluded that DI missed an opportunity by not having any strong local 
partners as part of the implementation team. Local partners could have supported the DI management 
team in identifying local tech solutions that were at the right level for an organization’s capacity and in￾house technical capacity. A local organization, such as a university or vocational school, could have also 
helped support the now-defunct young professionals’ internship program or could have housed the 
innovation 5D Lab more effectively and sustainably. 
Recommendations 
Based on the above findings and conclusions, the evaluation team makes the following recommendations 
for evaluation question 1: 
Recommendations for USAID and DAI – Current Grant Period 
• For the remainder of the grant period, USAID should be clear regarding any additional changes 
to outcomes it expects the DI project to achieve by the end of the grant period. At the same 
time, DAI should continue to be very explicit in seeking guidance from USAID and ensuring that 
their project meets USAID expectations. 
• Streamline the current grant process and be less risk-averse in the ideas that are funded. 
• Facilitate donor coordination so that ICT and tech-focused programming is complimentary and 
does not foster competition among local organizations. Reconsider branding requirements to 
encourage local ownership of the project components – such as trainings, BarCamp style events 
or hackathons. 
Recommendations for USAID and DAI – Future Projects 
• From the program design phase, develop a clear vision of success that includes tangible outcomes 
with clearly defined beneficiary groups. Project beneficiaries should be clearly outlined, for 
example. A program focused on university students is different than one focused on youth more 
broadly. 
• USAID should also select implementing partners that have 1) a strong, successful track record on 
the technical areas required of the program and 2) substantial experience in the program country. 
• Use a flexible, streamlined grant process that is multi-staged/phased and is not completely risk￾averse. The grant program should support the development of prototypes, allow for beta-testing 
and piloting of ideas, and subsequently support the scaling-up of projects. Any grant fund should 
support monitoring the tech solutions’ long-term impact, maintenance, and tech updates. 
Recommendations for DAI – Current Grant Period
• Continue to provide targeted and tailored support to CSOs so that they may a) become better 
equipped to identify technology as a solution to their development challenges and b) improve 
internal capacity to manage tech projects. Additionally, incorporate high tech CSOs into the next 
17
project as ‘tech influencers’. These local tech influencers can play a mentoring role to lower 
capacity organizations and can help make future projects locally sustainable. 
• Increase support of known local solutions, such as Open Institute’s IVR services, and foster locally￾driven tech solutions, such as those promoted by Open Development Cambodia and by the tech 
community writ large. 
• Increase collaboration and partnerships with learning institutions in the project. This includes 
university and tech schools, as well as schools such as the Liger Learning Center that are focused 
on helping high school youth become more engaged in science and technology. Increasing 
engagement with learning institutions can provide a number of benefits including: learning 
exchanges that help showcase innovations in other countries and can spur new ideas for 
Cambodians, provide local partnerships that can help fill the human capacity need in the tech 
sector, increase women’s participation in technology-focused programs and employment 
opportunities, and encourage sustainable and locally-driven, collaborative solutions. Develop a 
communication and outreach strategy to potential partners and beneficiaries that better utilize 
partner communication styles. Communication should be multi-faceted and reach beyond just 
email and social media. Language used in communication should be simple and concise and convey 
how the topic/subject is pertinent to the end user. 
• Increase efforts to include women in programming. For example, continue to support the 
Technovations activities to increase girls’ involvement in tech-oriented learning opportunities. 
The evaluation team recognized that there is a deficit of women in the tech sector to begin with, 
which makes gender inclusion a challenge but still a programmatic need. This could be achieved 
by having a gender-specific outreach plan to increase women’s involvement in tech-related 
educational programs, by having a gender focused mentorship or internship program, or through 
continuing support of the Technovations team. 
EVALUATION QUESTION 2 
To what extent has the project successfully engaged with civil society organizations, especially in order to 
assist CSOs to utilize technological tools/solutions to amplify their programmatic impact and support civic 
outcomes? Have CSOs been able to use the project’s assistance to develop solutions to assist them in 
achieving their development goals? If no or if only partially, why? How could the project improve it?
Findings 
All CSOs interviewed for this evaluation report that the DI trainings they attended were useful, with most 
naming the low-fuss video, digital security, social media communication strategies and design thinking 
trainings as the most useful in helping them learn 
about the use of technology in their organization 
toward their ultimate development goal. All sub￾grantees that the evaluation team met with also 
stated that the suite of services DI offers, including 
one-on-one consultations, design planning sessions, 
TSP/CSO matching service and targeted trainings 
were also useful. The evaluation team used the term, 
‘advisory services’ (as also used by DI) to describe 
these CSO support activities. CSOs are beginning to 
understand the importance of ICT in their work and 
are willing to engage on how to use it more effectively. One training participant that attended an FGD 
described that he used the social media communication training in his daily work as the communications 
manager to promote his organization’s work in a more targeted and strategic way. 
“DI helped us figure out the right platform for 
our needs. We were working off a basic app, 
developed under SPICE. DI helped us work 
with a local TSP to customize their tech 
solutions for our needs. DI went with us to a 
meeting with the local TSP to make sure we 
were getting the right info to help us inform 
our proposal to DI.”
-Female CSO/Sub-grantee partner
18
The evaluation team found, however, that participating CSOs have benefited from the DI project to 
varying degrees depending on their own internal capacities and interests. Several respondents explained 
that if an organization’s leadership is excited about the use of technology, then the organization is more 
likely to incorporate tech solutions into their work. Unfortunately, the evaluation team found that most 
of the attendees at DI events and trainings are not organizational leaders, but instead program or IT staff. 
While these attendees were excited by what they learned at the DI event, several stated that they had 
difficulty putting into practice what they learned when they returned to their organization, as they often 
did not have the authority to promote these new ideas across their organizations or they did not have 
the buy-in from the organization’s leadership to incorporate these new tech solutions into their work. 
The evaluation team reviewed DI’s sustainability assessment, which included a survey of CSOs on training 
and tech needs, which will be useful as the DI team considers its training offerings for the remainder of 
the grant period. 
The evaluation team also found that it is too early to tell what the developmental effect is from the tech 
solutions that CSOs are using. This is due to the slow start of grants and the recent re-focus towards the 
democracy and governance sector. Additionally, identifying specific effects resulting from DI assistance is 
elusive because of the crowded tech sector donor landscape (as explained under evaluation question 1). 
For example, several DI-supported projects/initiatives overlapped with SPICE, and some tech solutions 
(such as the Khmer SmartKeyboard and TosFund) were identified separately from the DI project (although 
they did utilize DI resources to scale up or connect to key TSP partners). 
Conclusions
As described in Figure 2, the DI project worked primarily with organizations that had a relatively high 
awareness and capacity for incorporating technology into their work. While the DI team describes having 
to work with many CSOs to help them to clearly identify how a tech solution could address development 
challenges, these organizations were already at the higher end of the knowledge and skills spectrum. The 
evaluation team concluded that the DI project may have produced more results as detailed in this 
evaluation question, and to objective 1 of the DI project, if there had been more support provided to 
organizations that had lower awareness of and capacity in technology use. 
It is too early to identify, however, the result of DI’s activities on CSO development goal achievement. 
Many tech solutions identified by CSOs as impactful in their work are not directly attributable to DI: 
TosFund, Khmer Smart Keyboard, Baby Village Care mHealth, and Technology for Education Systems 
Transformation (TEST) app. Khmer smart keyboard was self-started by ALiEN DEV, for example. DI’s 
support to People In Need was strictly content development for Baby Village Care mHealth Program, as 
their app had already been developed through the SPICE program. 
Figure 3 below details the overlap in tech solution development, complicating the evaluation team’s ability 
to determine whether DI’s assistance has helped CSOs achieve their development goals. The programs 
included here are not exhaustive and serve to show how three projects overlapped between DI and the 
SPICE project. 
19
Figure 3: Overlap of Tech Solution Development
Recommendations 
Based on the above findings and conclusions, the evaluation team makes the following recommendations 
for evaluation question 2: 
Recommendations for USAID and DAI – Current Grant Period and Future Grants
• Target CSOs that have a development challenge where ICT solutions make sense and are 
necessary, regardless of where they lie on the spectrum of capacity and awareness for technology. 
The technology does not need to be flashy or the most recent solution. Often CSOs, especially 
those with low tech awareness or tech capacity, may need simple tech solutions that are 
innovative for them and their work. DI and future tech projects supported by USAID should be 
flexible in how ‘innovative’ is defined in order to be responsive to local organizations’ technology 
needs. For example, an appropriate and necessary tech solution for an organization may be to 
digitize records and reports so that they can more readily access their programmatic lessons 
learned. 
• Support CSOs that may currently have lower capacity for technology (in terms of skills or human 
capacity), as well as those that may have lower awareness on how technology can enhance their 
work. A prevalent comment that the evaluation team heard from respondents across most 
stakeholder groups was that Cambodia suffers from a lack of human capacity in terms of tech 
skills and knowledge. USAID and DAI should support CSOs with lower capacity of technology in 
order to help them increase their ability to use technology in their work. 
Recommendations for DAI – Current Grant Period
• Provide targeted support to CSOs that focuses on specific capacity needs. While the DI trainings 
have been useful, many groups asked for additional training support that is targeted to their 
organization’s specific needs as they seek to identify how and when tech solutions may be an 
appropriate tool for their organization. DI captured much of this in the recent sustainability 
assessment and should use those findings to revise their training offerings for the remainder of 
the grant period. 
20
• Continue to offer ‘packaged’ support to CSOs. This packaged support should include both training 
as well as follow-on support to see if training participants are facing challenges with implementing 
skills or information learned from the trainings. Follow-on support can also address trainees’
questions after they return to their daily routine and try to apply lessons learned to their work. 
• Revamp the DI training project, focusing on a revision of the training delivery method. For 
example, the training programs can be cyclical or academy style (having a package of training 
modules that are replicated multiple times per year to increase the number of people trained on 
IT basics). 
EVALUATION QUESTION 3 
To what extent has the project as a whole (including all of its various activities) contributed to achieving 
the objectives of the USAID/Cambodia office of democracy and governance? 
Findings 
To address this question, the evaluation team focused on collecting information on the extent that civil 
society was strengthened through DI, as detailed in Development Objective 1: Strengthened democracy 
and government accountability, and enhanced respect for human rights, with a particular look at 
Intermediate Result 1.2: Increased capability of civil society to engage in political processes from 
USAID/Cambodia’s CDCS.12
The evaluation team finds that the DI project contributed to Sub IR 1.2.2: Expanded technological 
platforms increasing civic engagement. All but two subgrant projects examined by the evaluation team 
used technology solutions to increase citizens’ access to information and freedom of expression in some 
way. The evaluation team also finds the DI project has improved the capacity of partners/CSOs as detailed 
in evaluation question 1 and 2 (contributing to DO 1). These CSOs are starting to use technology to 
better equip citizens with tools and information to engage on issues that affect their daily lives. The 
trainings, advisory services and grant support that CSOs received through their engagement with DI 
yielded the most positive results. 
The evaluation team finds, however, the DI project has not specifically promoted the use of 
technology/tech solutions to improve citizens engagement “with the political process”, as targeted in IR 
1.2 in the 2014-2018 CDCS. The one exception is Pact’s TRANSMIT project. While still in the early 
stages of their grant, the TRANSMIT project aimed to use technology to link citizens and local government 
authorities together around local community issues. This project has the potential to directly impact 
political processes by promoting greater government accountability. 
Upon reviewing the original RFA, the winning proposal submission, DI’s work plans, and DI’s monitoring 
evaluation plans, the evaluation team could find no evidence that this project had any intended focus within 
the democracy and governance sector (as explained in evaluation question 1). It was not until the revised 
program approach and management shift in early-to-mid 2015 that a focus on democracy and governance 
became a project-level priority. When USAID clarified the required DG focus, DI quickly responded and 
included appropriate revisions to the project focus in their 2015 adjustment. Prior to this re-focus, the DI 
project worked across multiple sectors, including health, education, the environment, and human rights. 
Specific funding streams were provided to the DI project from the USAID human rights office and 
biodiversity team. The DI project did work with several sub-grantees on the issue of land rights (CCHR) 
and human rights (DC-CAM). Since the re-focus, the DI project has increased outreach and support to 
DG-focused CSOs and project ideas, with the goal of strengthening civil society. 
 
12 USAID/Cambodia Country Development Cooperation Strategy 2014 - 2018
21
Though the program as a whole is limited in the effect it has had (and will have) in the DG sector, 
considering the findings above, the evaluation team identified some activities and projects that have the 
potential to increase the capability of civil society by 2016. The evaluation team found the following sub￾grant projects to have the potential for DG sector impact: TosFund, Khmer Smart Keyboard, API IVR 
project, CCHR’s Land Right, PACT’s TRANSMIT project, and CCC’s digital M&E tool. These sub-grant 
projects all include a developmental focus that center around core DG values. If successfully implemented 
as envisioned, these projects can have a meaningful influence on Cambodian citizens’ ability to access 
information and communicate. 
Conclusions 
The DI project has contributed to USAID/Cambodia’s Development Objective 1, though more through 
Sub IR 1.2.2 than IR 1.2 at project midpoint. The current sub-grant projects including TosFund, Khmer 
Smart Keyboard, API IVR project, CCHR’s Land Right, PACT’s TRANSMIT project, and CCC’s digital 
M&E tool have the potential to increase the capability of civil society by project close. It is too early to 
see the effect of these tech solutions at the DI project midpoint, however, as these projects are still in 
the early stages of development and have not been fully implemented. Overall, there is likely to be limited 
results in the DG sector from the DI project overall considering the focus on the DG sector was not 
clarified until 2015. 
Recommendations 
Based on the above findings and conclusions, the evaluation team makes the following recommendations 
for evaluation question 3: 
Recommendations for USAID – Current Grant Period and Future Grants
 From the program design phase, USAID should clarify any sector-specific objectives or desired 
outcomes, particularly those related to USAID Development Objectives. USAID should be clear 
and realistic regarding any additional DG-related objectives it expects the DI project to achieve 
by 2016. 
Recommendations for DAI – Current Grant Period
 Look for opportunities to continue coaching and mentorships with CSOs. This is especially useful 
within the DG sector, as there are a number of trust and confidence issues that may hinder DG￾focused organizations from using tech solutions in their work (as described under evaluation 
question 2). 
Recommendations for USAID and DAI – Current Grant Period
 Nurture CSOs that have identified how tech solutions could be transformational regardless of 
sector. The evaluation team developed Figure 4 (tech solution leads to desired change13, below) 
based, in part, from feedback received during the data collection phase. Specifically, USAID and 
DI should consider the component as well as the whole when it comes to ICT for development. 
The content that the tech solution generates, pushes out to the public or gathers is just as 
important as the trigger (or the motivation for the tech solution or tools use). These two 
components, content and trigger, must produce a specific result that will contribute to the desired 
change the CSO is looking to achieve. USAID and DI should look to support projects, regardless 
of sector, that marry these components together to solve development challenges for local civil 
society. 
 
13 The graphic is intended to provide a visual that these are component parts that require the other for the entire 
system to function properly. The gears are not meant to be ranked by order of importance based on size. 
22
TECH SOLUTION: 
 = DESIRED CHANGE 
Figure 4: Tech Solution Leads to Desired Change 
EVALUATION QUESTION 4 
In what ways has the project succeeded or failed in terms of financial sustainability? In what ways has 
the project succeeded or failed in terms of participatory sustainability (i.e. stimulating enduring 
participation in the Lab which could be sustained after the end of the project)? What evidence is there 
that the project will be sustainable at its completion?
Findings 
The evaluation team finds that the DI project, as designed and implemented currently, is not financially 
sustainable. This was stated by nearly every respondent the evaluation team met with. Specifically, the 
evaluation team found that the 5D Lab drew attention away from other collaborative work environments 
that already existed and were more organic in their origin, such as Small World, Hackerspace, Impact 
Hub, and Co-space. The evaluation team also found that some 5D Lab members decided to move to a 
paid space such as Impact Hub, as they provided expanded opportunities for engagement rather than 
limiting their work to only the CSO community. 
DAI recognized these sustainability issues and, after the program revisions and management adjustments 
made in early-to-mid 2015, DAI conducted a sustainability assessment14. The evaluation team notes that 
the sustainability assessment plan echoes many of the evaluation teams own findings. If fully implemented, 
DI’s revised sustainability plan should prepare USAID for any follow-on or next generation project. If 
implemented, the 2015 sustainability plan should increase participatory sustainability by “investing in 
Cambodian organizations to establish sustainable services that meet CSO demand”
15. The evaluation team, 
however, did not see how the DI project will reach financial sustainability by the end of the grant program. 
The 2015 sustainability assessment did not include a financial sustainability component16. The evaluation 
team could not assess current progress made on the implementation of this plan considering it was 
formalized in July 2015. 
 
14 This sustainability assessment replaces the original business plan for the project. 
15 As described in the Sustainability Plan Executive Summary that was provided to the evaluation team. 
16 The evaluation team was provided with the 23-page Sustainability Assessment Executive Summary during the data 
collection phase. If the full assessment report contains additional information, the evaluation team was not privy to 
it. 
23
The evaluation team finds that, through DI’s advisory services, the project was able to fill the role of a 
neutral broker between CSOs and TSPs. By providing unbiased recommendations for TSPs, a role that 
many respondents stated no other group in Phnom Penh is currently able to fill, the evaluation team found 
that DI is providing a valuable, trusted service to local organizations. This role is necessary but not yet 
sustainable beyond the life of the DI grant. It is not yet sustainable due to the lack of funds identified to 
support it outside of DI, and there is no other neutral party yet identified to fill the role after DI’s
completion. The evaluation team finds that these advisory services have the potential to contribute to 
participatory sustainability. 
Without any local implementing partners, the DI project is viewed as a top-down initiative with no local 
ownership. This was echoed in evaluation question one’s findings in relation to branding. 
Conclusions 
As currently implemented, the evaluation team found that the DI project is not sustainable (in terms of 
financial and participatory sustainability). In terms of financial sustainability, the 5D Lab was created as a 
free lab when other pay-for access labs were already available. This shifted users away from other 
collaborative work environments. As DI staff recognize, this ‘cannibalized’ an organically growing 
collaborative work environment. For the 5D Lab to be sustainable without DAI or USAID support, it 
would likely need to ask for membership fees of current 5D Lab members (to make it sustainable) when 
a) they cannot afford membership fees and b) there are other, more convenient co-working spaces 
available. This is something DAI, USAID, and the evaluation team recognize is not a viable option. 
Additionally, the 5D Lab space was created with CSOs in mind. Both 5D Lab members and other CSOs 
who have been invited to use the space agree that having a space for use by CSO staff is not within their 
organizational culture. Most CSOs have their own office space, with leadership who expects their staff to 
work on-site at their own offices. One respondent described their desire to work at a space like the 5D 
Lab, but that they needed to be in their own office in order to a) appease their boss and b) be able to be 
near colleagues working on similar projects and tasks. 
Recommendations 
Based on the above findings and conclusions, the evaluation team 
makes the following recommendations for evaluation question 4: 
Recommendations for USAID and DAI – Current Grant 
Period and Future Grants
 Foster and support locally generated tech solutions that 
have the potential to be self-supporting (beyond international donor support) and replicable, such 
as TosFund or Open Institute’s IVR technology17, through the grant fund and partnerships (such 
as mentorship opportunities and training partners). 
 Continue to provide advisory services for CSOs, but adjust the service model to be more 
sustainable by identifying local organization(s) that can eventually take over this role from the DI 
project. The DAI sustainability assessment and plan has good recommendations for this that the 
evaluation team supports. 
 CSOs are eager to continue using the current advisory services that DI provides. DAI should 
therefore identify and mentor a local partner (or partners), such as university or technical school, 
who can take over this role in 2016. 
 
17 The potential for these to start creating revenue is high. 
“I attended the opening of the 
lab, but I do not see how it was 
relevant to what we were 
doing.”
-Female CSO leader 
(non-partner)
24
Recommendations for USAID – Current Grant Period and Future Grants 
 Stop support for stand-alone lab/hub space and instead focus on funding improved advisory 
services both in the DI project and in other USG￾supported initiatives. 
 Technology is a cross-cutting issue and should be a part 
of all USAID-supported programs. By incorporating 
technology training and support across all USAID 
programs, the local human capacity for technology could 
increase significantly. 
EVALUATION QUESTION 5
How has the Cambodian country context (state of civil society and human capacity in its sector) affected 
the implementation of the project?
Findings 
The evaluation team found several contextual factors that inhibited the achievement of DI’s outcomes. 
Several respondent groups, especially those in the DG sector, expressed concern that the legal framework 
and political context will affect their ability to use tech solutions and ICTs in their work. Many respondents 
explicitly stated that they were concerned about the draft cybercrimes law and how it may affect their 
ability to use internet-based tech solutions to express themselves or organize their constituents, for 
example. CSO respondents expressed concern that their technology solutions may be accessed by 
unauthorized parties (such as the government or TSP employees). One large TSP stated explicitly that it 
does not want to be involved in any CSO project that is political or perceived as politically engaged 
[paraphrased]. 
The evaluation team found that there were few women in the technology sector, or who hold tech￾focused positions with CSOs. While the DI project needs to have a more robust gender inclusion strategy, 
the evaluation team recognized this is difficult when there are few women in the field to begin with. 
It was apparent to the evaluation team, as well as respondents 
involved in the DI project, that CSOs and TSPs have very 
different business cultures, motivations, and incentives. This 
directly affects their ability to naturally seek each other out as 
collaborative partners working together to identify 
development solutions. Based on the feedback from 
respondents, the nature of the CSO-TSP is more transactional 
versus transformative; TSPs seem to be singularly focused on a 
type of technology (internet service providers, website or app developers, mobile telephony and 
technology) as well as potential revenues. CSOs tend to work through a slower, more iterative process 
in identifying the right solutions (including tech solutions) for their development challenges. CSOs also 
tend to be less financially flushed and usually look for low-cost or discounted rates. These differing 
outlooks and financial motivations make for difficult partnerships and a complex context in which to 
implement a project like DI. 
Trust is also an issue, as evidenced in the analysis on the political context. Several CSOs that the evaluation 
team spoke with explicitly described that they do not trust that TSP employees would ‘keep their data 
safe’ or would not ‘copy’ the information for their own needs. There is also a perception that the 
government uses TSPs to conduct surveillance of CSOs work. This compounds the lack of trust by CSOs 
of TSPs. 
“We used the 5D lab for our own 
events (the lab has an event space). 
We did not use the lab to work.”
-Key CSO partner
“It’s hard to get more women 
involved. There should be more 
awareness raising on Tech with 
women. Start from school level 
first.”
-Key CSO Partner 
25
The evaluation team identified other contextual factors for DI to consider moving forward. Online 
penetration has increased as (1) electricity has become more available in rural areas, (2) internet coverage 
has spread and is cheaper, (3) smart phones are affordable, and (4) mobile phones use Khmer Unicode 
more widely. It will be critical for DI to support tech solutions that are quickly adaptable, work with TSPs 
that are flexible and can operate in this fluid environment, and refine the grant program to better respond 
to these changes. 
Conclusions 
There is a firmly held opinion among CSOs in DG sector that their ability to use ICT solutions is affected 
by the political environment and legal developments. The adoption of the Law on Associations and Non￾Governmental Organizations (LANGO), the drafting and development of cybercrime law, the amendment 
of the election law, the lack of rule of law, and self-censorship have a direct impact on both CSOs and 
users. 
There is a clear difference in organizational culture, difference in motivation and difference in time frames 
between CSOs and the private sector that made the partnering objective difficult. 
Given the scarcity of women in tech, DI was struggling to include them in activities in a meaningful way. 
For CSOs, women were intended users of the tech solutions, but did not seem to hold roles within the 
CSOs that were tech oriented (e.g. communication officers were male, IT staff were male). 
ICT solutions are constantly evolving, and the use of Internet and smartphones is increasing (although 80
percent of country is without both). Coupled with relatively low internet/data fees, the potential for 
expansion in this space is very high. Communications and/or online penetration has become increasingly 
necessary for CSOs - both high tech and low tech. There remains, however, a significant divide between 
knowledge and the technology. 
Recommendations 
Based on the above findings and conclusions, the evaluation team makes the following recommendations 
for evaluation question 5: 
Recommendations for USAID –Current Grant Period and Future Grants 
 Continue to support programs that encourage women in the IT field. Specific efforts to increase 
women’s access to IT training and to help women become more competitive for IT jobs is important 
to increase their participation in projects such as DI. USAID should consider increasing the number 
of scholarships, training, and internship opportunities for women in the tech and ICT fields. 
Recommendations for DAI – Current Grant Period
 Continue the unbiased brokerage role that DI plays between CSOs and TSPs. Similar to a customer 
relations manager in the private sector, this role has been valuable to both CSOs and TSP partners. 
This should continue as each group learns and adapts their operating culture to meet the needs and 
expectations of the other. 
 DI should identify local partner(s) who could eventually take over the beneficial ‘broker’ role between 
CSOs and TSPs. At present, there is no local organization or entity that CSOs or TSPs can agree upon 
to play this brokerage role besides DI. In order to increase sustainability and to promote local 
ownership, DI should identify and groom a local organization (such as a CSO or coalition of 
organizations) or an entity (such as a learning institution) to eventually take over this neutral brokerage 
role. 
26
 Refine the grants project to allow for a more nimble, flexible process that encourages partners to 
identify new technologies that are responsive the rapidly changing tech environment, especially for 
tech solutions that can engage or affect non-Phnom Penh or Siem Reap based populations. 
27
ANNEXES 
ANNEX I: EVALUATION STATEMENT OF WORK 
SECTION C - DESCRIPTION I SPECIFICATIONS/STATEMENT OF WORK 
Cl. TITLE OF PROJECT/ACTIVITY 
Development Innovations Mid-Term Performance Evaluation. 
C2. PURPOSE 
The Contractor will conduct a mid-term performance evaluation of the Development 
Innovations three-year project. The evaluation will assess the viability of the development 
hypothesis, the implementing partner's performance in achieving the results expected, as 
well as provide a clearer understanding of the obstacles and opportunities encountered in 
implementation. The evaluation will also capture lessons learned for the Development 
Innovations project, to inform decisions to be made for the remainder of the project as well 
as potential future USAID/Cambodia interventions. The evaluation will help 
USAID/Cambodia make a more informed determination on a future course of action with 
regards to approaches to utilizing technology to achieve development objectives and 
outcomes. 
C3. BACKGROUND
1) Identifying Information 
Title of the 
Evaluated Project/
Activity
Development Innovations
Award Number AID-442-A-13-00003
Life of Project/Activity From August 7, 2013 to August 6, 2016
Funding $7,499,744
Implementing Organization(s) Development Alternatives, Inc.
Agreement Officer's 
Representative 
(AOR)
Jean-Marc Gorelick, jgorelick@usaid.gov
2) Development Context 
Empowering civil society organization18 with information and communications technology 
(ICT) for civic education, networking, and policy advocacy is a key part of USAID/Cambodia's 
 
18 For the purpose of this evaluation, the evaluation team should use the definition of civil society organization contained in 
the Monitoring and Evaluation Plan approved on July 8, 2014. 
28
Democracy and Governance strategy. USAID/Cambodia also recognizes the potential of ICT 
to contribute to CSO activities in ways beyond Democracy and Governance objectives. Some 
limitations remain, however, as Cambodia's population is primarily rural, with low incomes 
and limited access to electricity. In addition, although the number of phones which are capable 
of using Khmer script has increased to 51%, many users with Khmer-enabled phones do not 
actually use Khmer frequently, and basic phones are difficult to use to enter Khmer text. In 
addition, many Khmer people are illiterate or marginally literate. Despite these challenges,
according to a Singapore-based consultancy firm, internet penetration grew to 25% in March 2015, 
an exponential growth over the last several years. Most users access the internet using mobile 
phones, and a survey conducted by Open Institute in 2014 found that 28% of phones in usage in 
Cambodia are smartphones. Cambodians are still not using the internet for a wide variety of 
purposes, however, with Facebook incredibly popular at 1.4 million accounts as of 2014. Other 
popular services include YouTube and instant messaging applications. An unknown number use 
the internet to access news items on websites, such as Radio Free Asia Khmer, Voice of 
Democracy, or other Khmer language newspapers. Despite the limited range of social or political 
content, it is widely perceived that content shared on Facebook played a key role in influencing 
youth, who were exceptionally active in advance of the July 2013 elections, which saw the ruling 
Cambodian People's Party lose a significant portion of its seats in the Assembly. 
The government has also considered a number of measures to limit the influence of the internet. 
A draft Cybercrimes Law, which would have punished a wide range of vaguely defined speech 
crimes, was dropped by the government in December 2014, although lingering concerns remain 
that it will be revived at a later date. Moreover, the government has instituted orders mandating 
Internet Service Providers (ISPs) and Mobile Network Operators (MNOs) to install surveillance 
equipment allowing the government to conduct surveillance on phone calls, SMS, and internet 
activity. The government has also announced a "cyber war room" to monitor public usage of online 
forums or discussion on Facebook. Government transparency is still quite low, despite the 
potential to use the internet to share information with the public and despite a draft Access to 
Information Law in the works. 
Civil Society Organizations (CSOs) have taken some measures to utilize ICT. Interactive Voice 
Response (IVR) was greatly popularized by the USAID funded Structuring Partnerships for an 
Innovative Communications Environment (SPICE) project. The project, which ended in March 
2015, facilitated CSOs to use IVR in almost 30 distinct activities. CSOs involved in education 
have utilized ICT to encourage reading in Khmer, while CSOs in the health sector have utilized 
SMS, IVR and multi-channel platforms for a variety of purposes. In the Democracy and 
Governance sector, CSOs have utilized IVR, a corruption reporting application, and an open data 
website to disseminate information. The general perception, however, is that CSOs based outside 
of the capital are still limited in their use of ICT, primarily using computers and the internet only 
for email, websites, Facebook, and other desktop-based software. Internet and data security 
remains an issue, especially for CSOs working on sensitive issues. The USAID Washington 
funded Information Safety and Capacity (ISC) project has filled this gap to some extent by 
providing training and mentoring on information security to select CSOs. 
29
Despite having an active ICT business sector, many of these businesses are still too profit- oriented 
to effectively cater to the CSO market. While there are a number of small start-ups, larger ICT 
firms complain about the poor quality of Cambodian graduates in the sector. Some interlocutors 
point out that local ICT firms are still oriented towards foreign markets, rather than domestic ones. 
There are several ICT Labs or start-up incubators in Phnom Penh such as Co-lab and Small World. 
Despite positive trends, there is still more progress to be made in the utilization of ICT by civil 
society groups, particularly in the Democracy and Governance sector. Civil society struggles with 
how to use technology to address access to information, corruption, human rights, civic education 
and policy advocacy. 
3) Project/Activity Information 
a) History
In January 2012, USAID/Cambodia's Office of Democracy and Governance commissioned an 
assessment of social media and ICT usage to inform discussion about future project designs. On 
October 1, 2012, Open Institute was awarded the SPICE project to accomplish several specific 
tasks related to ICT in Cambodia. In particular, Open Institute was to facilitate the use of Khmer 
text in Android based smart phones, facilitate a public-private partnership to enable greater use 
of IVR, promote CSOs use of SMS to disseminate information, localize applications and create 
an anonymized crowdsourcing platform. 
Following a full and open competition, Development Alternatives, Inc. (DAI) won the three year 
award for the Development Innovations project (previously named Social Innovation Lab 
Kampuchea(SILK)), with a period of performance from August 7, 2013 to August 6, 2016. In 
January 2014, Development Innovations moved into their offices and opened the Lab. The project 
was launched by the Mission Director, with special guest Minister of Education Hang Chuon 
Naron, on March 18, 2014. Development Innovations hosted Innovations in Development Tech 
Expo with Acting Assistant Administrator Denise Rollins on September 8, 2014. The project was 
not fully funded in the second year due to a shortfall in funds of the Democracy and Governance 
Office. Additional funds were leveraged from other technical offices to focus on protecting 
biodiversity in Cambodia using ICT tools. 
b) Results Framework/Theory of Change 
The theory of change stated in the RFA was as follows:
If USAID invests in the processes of social innovation, it will provide unique opportunities for civil 
society organizations and socially-minded entrepreneurs to advance their ideas and create new 
technological goods and services that benefit Cambodian society through increased communication 
and access to information.
Although the program description submitted by DAI does not explicitly state a development 
hypothesis, the following quote from the Executive Summary gives some indication of how DAI 
formulated the theory of change: 
30
This program will enable Cambodian-led technology products to meet CSO demands and trigger 
transformative changes in the ways CSO conduct programs. At the same time, investing in the domestic 
information and communications technology (JCT) sector will enable it to become an engine of growth 
in the economy. This emerging ecosystem of technology and civil society will result in CSOs' increased 
ability to access technology tools and increase their proficiencies in order to apply knowledge of 
mobile, Internet, and social media tools to amplify their programmatic impact and support civic 
outcomes. 
c) Approach and Implementation 
The Project Description of the Cooperative Agreement contains the following objectives: 
Component 1: Partner.silk 
Component I will identify, design, and formalize partnerships that match CSOs with companies to 
develop and introduce technologies that support democracy and governance development 
objectives into the Cambodian market. 
Component 2: Tech.silk 
DAI, through SILK19, will foster an active technology community focused on Cambodian 
challenges, cultivate and incubate social enterprises, and support the development of Khmer 
language content that supports development goals and can be accessed by average citizens. This 
program will connect and activate CSOs; programmers; small IT enterprises; and multinational 
companies, universities, bloggers/khloggers, informal technology user groups, and donors to 
develop a profitable and responsive Cambodian ICT market that serves CSOs and citizen 
information needs. 
Component 3: Learn.silk 
The Learn.silk component creates opportunities to increase technology literacy among citizens and 
CSOs; develop a body of knowledge and collection of best practices on technology and security 
among Cambodian CSOs and citizens; field test new technologies in training focused on practical 
application; and build awareness of information security concerns. Learn.silk will also build the 
community around CSO needs while connecting it with state-of-the-art mobile, Internet, and ICT 
tools. SILK presents a timely opportunity to provide balanced, wide-ranging training programs 
grounded in the international knowledge of DAI and its partners, the relationships across the 
network of SILK partners, and access to best practices and lessons learned from USAID and other 
programs. Implementing ICT innovations brings with it a responsibility to train users on 
appropriate uses and effective security practices. 
In its first workplan, Development Innovations adjusted these objectives as follows: 
 
19 SILK was the original name of the project, an acronym for Social Innovation Lab Kampuchea
31
Objective 1: PARTNER, Innovation facilitated through collaboration between CSOs, techies and 
private sector. 
Objective 2: TECH, Technology and service providers have an increased understanding of and 
capacity to address social issues and Cambodian CSO needs. 
Objective 3: LEARN, CSOs have an expanded understanding of what's possible with ICT and 
have the capacity, tools, and incentives to implement. 
d) Geographical Coverage and Targeted Beneficiaries 
Although the project nominally covers the entire country, many of its activities are based in 
Phnom Penh. Most of the activities outside of Phnom Penh are carried out under sub-grants 
(although not all sub-grants include work in the field). For example, the sub-grant to World 
Education for the TEST application utilized the TEST application in a small number of rural 
or provincial schools. In addition, BarCamp activities which were partly funded by the project 
also take place in selected provincial towns. 
The targeted beneficiaries of the project include primarily civil society organizations, staff of 
civil society organizations, technologists and technology service companies, youth (especially 
youth interested in technology), and entrepreneurs. Members of the general public may also 
participate in project activities, especially those who have access to the project facility in Phnom 
Penh. In addition, the general public is also target beneficiaries to the extent that ICT solutions 
implemented by CSOs lead to improved development results for their Cambodian beneficiaries. 
The sub-populations to be reached by these CSOs depend on which CSOs develop ICT solutions 
with assistance from the project, and which CSOs are successful in getting funding from the 
project. 
e) Expected Results 
The following results are contained in the project Cooperative Agreement: 
Objective 1: Partner
• Identification and facilitation of 10 partnerships per year; partnerships toolkit tailored to SILK; 
investor pitch days held biannually, product focus groups established and meeting quarterly. 
• At least 30 CSOs using secure mobile email; at least 30 CSOs completed information security 
training. 
• Five serious games available in Khmer; serious games used in SILK training programs. 
• OCR software available; Khmer Unicode usage increased among universities, government and 
CSOs; awareness of software availability increased. 
• (Illustrative) New solutions identified and funded for up to two intractable challenges annually. 
• (Illustrative) Test and roll out at least three mature technologies annually within the technology 
and CSO communities. 
Objective 2: Tech
32
• Annual apprenticeship program established; CSOs linked to technology firms for their ICT 
needs; SILK is a space for technologists to develop new products for marketing to CSOs and 
consumers. 
• At least five technology-oriented social enterprises developing tools for Cambodian development 
community with sustainable business plans. 
• An actual and perceived increase in Khmer content available as part of technology-based 
solutions. 
Objective 3: Learn 
• Approximately 25 percent of CSOs in Cambodia receiving mentoring; use of technology by 
mentored CSOs increased; numbers of stakeholders reached increased via technology. 
• Social media training curriculum established; CSOs trained; train the trainer program 
implemented. 
• Technology training curriculum established; bi-monthly training held; train-the-trainer program 
established. 
• Mass media and viral campaigns launched; knowledge of information security practices 
increased. 
• Quarterly youth outreach events conducted by mentee CSOs; annual video blogging challenge 
conducted in partnership with a CSO; SILK participants are engaging with youth. 
Other relevant content within the Cooperative Agreement, which are not listed as results, but 
which actually constitute results under the award are: 
• Establishing SILK, the iLab20
 - DAI will collaborate with our Cambodian partners￾including CSOs, private IT firms, mobile network operators, and international investors-to 
establish SILK (the iLab) within three months of award.
• Formulating and executing a sound, market-driven business strategy for SILK means that it must 
not only consistently deliver results but also be self-sustaining before the end of USAID funding. 
We propose that SILK will be sustainable21 by Year 3 of implementation. 
• Promote mobile technology as a platform for businesses to expand their markets: SILK will 
actively promote mobile technology as a platform to expand the market base for businesses. 
• Provide market insights, understanding, and access to seed capital: SILK will identify innovators 
within the business community and provide them with the skills necessary to expand their 
businesses. 
• Provide infrastructure to empower business ideas: SILK will support the development of 
business-accessible ICT infrastructure that will encourage testing and piloting tools for CSO use 
with minimal up-front investment. 
• Utilize innovative funding mechanisms to spur product development: DAI will design prize 
competitions that are attractive to the business community and provide incentives for businesses 
to design solutions for CSOs. 
 
20 The Lab is a publically accessible physical space implemented by the project where interested members of 
the public can access equipment and develop ICT ideas with other innovators. It is intended to remain self￾sustaining after the end of the project.
21 Additional analysis of the Cooperative Agreement demonstrates that "sustainable" means financially 
sustainable.
33
Key Indicators: 
1. The number of CSO beneficiaries using ICT-based solutions as a result of the CSOs' 
engagement with USG assistance 
2. Percent of CSOs that have engaged with USG supported ICT project that have implemented 
!ICT-based solutions. 
3. Number of ICT-based solutions designed and implemented as a result of CSO and TSP 
engagement, through a USG supported ICT project. 
4. Number of CSOs working in Cambodia (whether a local CSO or an international CSO working 
through its local affiliate) that have engaged with a USG-supported ICT project to address an
identified development problem through an ICT-based solution. 
C4. WORK REQUIREMENTS 
1) Evaluation Rationale
As the Development Innovations project has reached the half-way point, the Office of Democracy 
and Governance has some key questions about the project. 
2) Evaluation Purpose, Audience and Intended Use
a) Purpose
The evaluation will assess the workability of the development hypothesis, the implementing 
partner's performance in achieving the results expected, as well as a clearer understanding of the 
obstacles encountered in implementation. The evaluation will also capture lessons learned from 
Development Innovations, to inform decisions to be made for the remainder of the project as well 
as potential future USAID/Cambodia interventions in the ICT for Development sector. In addition, 
the evaluation may have findings with implications for USAID as a whole, as this project was a 
first of its kind for USAID. The evaluation will help USAID/Cambodia decide on a future course 
of action with regards to technology programming. 
The evaluation isto cover the period from the beginning of the project on August 7, 2013 until 
the time the evaluation is conducted.
b) Audience and Intended Use
The audience is the USAID/Cambodia Office of Democracy and Governance, The Office of 
Acquisition and Assistance, Mission leadership, the Mission as a whole, the USAID Global 
Development Lab, USAID Asia Bureau, the Bureau for Democracy, Conflict and Humanitarian 
Assistance (DCHA), other overseas Mission, USAID as a whole, and interested members of the 
international development community who access the final evaluation report through the 
34
Development Experience Clearinghouse (DEC). The Contractor will submit the final evaluation 
to USAID/Cambodia, who will ensure that the report is uploaded to the DEC. 
For the Evaluation's intended use, please see the "Purpose" section, above.
3) Evaluation Questions 
1. To what extent has the project succeeded in achieving all of its objectives? Was there a significant 
departure between the RFA, the project's Program Description, and the work plans? If there were 
objectives that were not, or are unlikely to be achieved, what were the key factors for such 
findings (please consider the role of project design and project implementation)? What 
recommendations does the evaluation team have for a) the remainder of the project, and b) future 
design and implementation of similar ICT for Development projects? Were there any unintended 
results (positive or negative) of the project? If so, what were they? 
2. To what extent has the project successfully engaged with civil society organizations, especially 
in order to assist CSOs to utilize technological tools/solutions to amplify their programmatic 
impact and support civic outcomes? Have CSOs been able to use the project's assistance to 
develop solutions to assist them in achieving their development goals? If no or if only partially, 
why? How could the project improve it? 
3. To what extent has the project as a whole (including all of its various activities) contributed to 
achieving the objectives of the USAID/Cambodia Office of Democracy and Governance? 
4. In what ways has the project succeeded or failed in terms of financial sustainability? In what 
ways has the project succeeded or failed in terms of participatory sustainability (i.e. stimulating 
enduring participation in the Lab which could be sustained after the end of the project)? What 
evidence is there that the project will be sustainable at its completion? 
5. How has the Cambodian country context (state of civil society and human capacity in the IT 
sector) affected the implementation of the project? 
4) Evaluation Design and Suggested Methodologies
a. Study Design
The Mid-term Performance Evaluation of the Development Innovations project will be a 
qualitative design in nature through examining project documents, indicator and additional 
tracking data results, interviews, observations, and (if necessary) focus group discussions. For 
further detail, please see "Data Collection and Analysis Methods," below. 
b. Data Collection and Analysis Methods
It is recommended that the evaluation team employ a combination of desk research, observations, 
focused group discussion(s), and interviews. The proposed evaluation methodology is as follows, 
but not limited to: 
1. Desktop Review of Key Documents and Initial Analysis
The Evaluation Team shall review relevant USAID documents, as well as key documents from 
USAID's implementing partners and outside sources, and as necessary, internet based research on 
35
ICT for Development in Cambodia. The Evaluation Team will use this literature to develop 
required evaluation tools, and to set forth a hypothesis that can be tested through limited field 
research and interviews. The list of documents is as below, but not limited to: 
 Cooperative Agreement and modifications Project Monitoring and Evaluation Plan Annual 
Workplans
 Quarterly reports Quarterly financial reports DQA V1orksheets 
 Indicator results as reported in quarterly reports,
 Additional tracking indicators contained in quarterly reports, 
 Sub-recipient reports (as available from Development Innovations) Sub-recipient 
proposals 
 Development Innovations Business Plan Development Innovations grant tracking pipeline 
USAID Social Media Assessment 
 USAID/Cambodia Country Development Cooperation Strategy (CDCS) Request for 
Applications (RFA) for the SILK/Development Innovations project 
2. Conduct Interviews
The Evaluation Team will conduct interviews with key informants chosen from the following 
categories: USAID personnel (including former AOR of the project); implementing partner staff 
and volunteers; key CSOs and Technology Service Providers (TSPs) that have participated in the 
project; CSOs and TSPs that have not participated in the project; Lab members; 
Developer/Programmer community using the Lab and attending the project events; Advisory 
Board members; Sub-grantees and sub-grant applicants; university or school representatives; 
relevant government officials (if any); and CSOs and others that have consulted with the 
Innovation Resource Center. The Team should create a sampling frame to conduct interviews of 
stakeholders and field visits with the few sub-grantees. The Evaluation Team's work plan should 
include an interview list and proposed field visits. The Mission's Office of Democracy and 
Governance will provide the evaluation team lists of individuals/organizations of each category 
and a suggested informant list for the interviews, after the award. The evaluation team can expand 
upon this list if they receive any useful leads. As the project is primarily based in Phnom Penh, it 
is not anticipated that extensive field trips will be needed. 
3. Conduct Observations
The team shall conduct observations of events and activities taking place at the project during the 
duration of the evaluation, including organized events as V1ell as daily operations of the project's 
Lab. These organized events may include, but not limit to, Hackathon; Tech and ICT Solutions 
Expo; and BarCamp Phnom Penh. A complete list of the confirmed events shall be provided to the 
evaluation team after the award.
4. Conduct Focus Group Discussions 
The evaluation team shall conduct Focus Group Discussions. 
36
5. Critically Analyze Reported Achievements 
The evaluation team shall probe more deeply into achievements reported by the implementing 
partner team to ascertain their accuracy. 
The Contractor will build on the proposed methodology and provide more specific details on the 
evaluation methodology in their Proposal. Once awarded, the evaluation team will be provided 
with necessary documents to review and it is permitted to make modification to the evaluation 
methodology with close consultation with USAID/Cambodia. The modification shall be reflected 
in the Evaluation Workplan. The methodology narrative should discuss the merits and limitations 
of the final evaluation methodology. The Evaluation Team will design appropriate tools for 
collecting data from various units of analysis. Any data collection tools developed will be 
submitted to USAID/Cambodia prior to fielding. 
The information collected will be analyzed by the Evaluation Team to establish credible answers 
to the questions and provide major trends and issues. USAID requires that evaluations explore 
issues of gender; thus, the evaluation will also examine gender issues within the context of the 
evaluation of Development Innovations. Sex-disaggregated data shall be presented where 
applicable. 
c) Limitations or Constraints of the Design and Methodologies 
In order to avoid a "participants' bias" by talking to those involved in the project and its 
beneficiaries, the evaluation team should make a significant effort to talk to stakeholders who 
have not been involved in the project, or those who were involved at some point, but later 
discontinued participation. In addition, participants or beneficiaries may generally give positive 
reports about the project without a broader awareness of the project's relationship to Mission 
objectives under the CDCS, thus, it is a responsibility of the evaluation team to ensure that the data 
collection tools and the analysis capture findings that can be linked to the relevant objectives of 
the CDCS. Further, the team needs to avoid being drawn into divergent objectives- objectives 
which have value, but are not the objectives of the project and the evaluation themselves. 
Another constraint is that some effects or impacts of the project, especially those related to events 
or trainings, may be difficult to assess. 
The evaluation team should be careful not to confuse the accomplishments of the SPICE project 
and the Development Innovations project. The Evaluation Team should also ensure that they 
clearly explain the difference between the SPICE project and the Development Innovations to 
informants prior to and during interviews as well. 
5) Team Composition and Roles and Responsibilities of Team Members 
a. Team Composition and Qualifications
The team should contain the following elements: ICT for development experience, Experience 
conducting monitoring and evaluation, especially mid-term or final project evaluations, 
37
experience in the Cambodian ICT for Development context, Civil society/DG experience, and a 
broader development context lens. These elements could be distributed among members of the 
team as follows: 
Team Leader, International (Evaluation Specialist): A senior-level development practitioner 
with at least a Master's degree in a relevant discipline and at least twelve (12) years of work 
experience in development. The Team Leader must have significant experience in the ICT for 
Development field, and significant experience conducting monitoring and evaluation, particularly 
high quality evaluations. The Team Leader should also be an excellent writer. Ideally, the Team 
Leader would have several years of Cambodia experience. Familiarity with USAID policy 
guidance and program design is preferred. The Team Leader will be responsible for development 
of the evaluation and overall team coordination, including ensuring that team members adequately 
understand their roles and responsibilities, and for assigning individual data/information 
collection, and reporting responsibilities. 
Team Member, Cambodian (Democracy, Rights and Governance Specialist): This team 
member must hold a Bachelor’s degree in development studies, human rights, social science, law, 
political science, or similar degree, with eight years of work experience in development, or a 
Master's degree with six years of work experience. Significant experience conducting evaluations 
is preferred. This team member will provide expertise to the team related to civil society and 
democracy and governance issues. The team member will understand the challenges of civil 
society in the Cambodian context, and be well versed in the Cambodian governance and political 
context. This team member should have strong English language skills, the ability to conduct 
interviews in English, and strong English language writing skills are preferred. The team member 
should not have political or other affiliations that would unduly influence (or could reasonably be 
perceived as influencing) their recommendations. 
Team Member, Cambodian (Other Cross-cutting Technical Specialist): This team member 
must hold a Bachelor's degree in development studies, human rights, social science, law, political 
science, or similar subject, or Management, or an ICT related field, with eight years of work 
experience in development, or a Master’s degree with six years of work experience. 
Significant experience conducting evaluations is preferred. This team member will provide 
expertise to the team related to ICT for development in the Cambodian context. The team member 
will be well versed in the state of ICT for development in Cambodia, and preferably the state of 
private sector technology firms as well. This team member should have strong English language 
skills, the ability to conduct interviews in English, and strong English language writing skills are 
preferred. The team member should not have political or other affiliations that would unduly 
influence (or could reasonably be perceived as influencing) their recommendations. 
Overall the team will need expertise in USAID or other major donor practices and expectations in 
program evaluation; program design and analysis; quantitative and qualitative data collection and 
analysis; survey design and analysis. 
38
b. Roles and Responsibilities of Team Members
The evaluation team can establish roles and division of labor among the team during preparation 
or at the Evaluation Team Planning Meeting. However, the Contractor is responsible for ensuring 
that all deliverables are received by USAID/Cambodia. Additional members of the team may be 
asked to contribute portions of the writing as per agreement among the team members. 
END OF SECTION C
39
ANNEX II: EVALUATION MATRIX 
Evaluation Question Key Related Indicator(s) Key Related KII/FGD 
Question22
Proposed Data 
Collection 
Methodology
Respondent 
Group
To what extent has the project 
succeeded in achieving all of 
its objectives? 
Was there a significant departure 
between the RFA, the project’s 
Program Description, and the 
work plans?
If there were objectives that 
were not, or are unlikely to be 
achieved, what were the key 
factors for such findings 
(please consider the role of 
project design and project
implementation)?
What recommendations does 
the evaluation team have for 
a) the remainder of the 
project, and b) future design 
and implementation of similar 
ICT for Development projects? 
Were there any unintended 
results (positive or negative) 
of the project? If so, what were 
they? 
Outcome Indicator 1: The number of CSO 
beneficiaries using ICT-based solutions as a 
result of the CSOs engagement with USG 
assistance
Outcome Indicator 2: Percentage of CSO, TSP, 
individual participants that show improved 
skills after participating in Training Events.
Outcome 1.1 including:
 Outcome Indicator 1.1: Percent of CSOs 
that have engaged with USG supported 
ICT project that have 
implemented ICT-based solutions
 Outcome Indicator 1.2: Number of 
CSOs receiving USG assistance engaged 
in advocacy interventions
 Output Indicator 1.1: Number of CSOs 
working in Cambodia (whether a local 
CSO or an international CSO working 
through its local affiliate) that have 
engaged with a USG-supported ICT 
project to address an identified 
development problem through an ICT￾based solution.
 Output Indicator 1.2: Percent of CSOs, 
TSPs, and individual participants that 
show improved skills after 
participating in Training Events.
Outcome 2 including:
Based on your interaction with the 
DI program, please describe your 
assessment of the program to 
date.
What project outcomes have you 
observed to date? Have you 
observed any 
unintended/unexpected results 
(positive or negative)?
Has the project encountered any 
challenges? If so, please describe 
(program design, implementation, 
or other challenges). If challenges 
did occur, how did they affect the 
program? How did the project 
address and overcome the 
challenges?
Have your activities included both 
male and female participants? 
How did you ensure activities 
were made available to both men 
and women equally?
What recommendations do you 
have for the project?
What do you think the project 
should do more of? What do you 
1.Key informant 
interviews
2. Focus Group 
Discussions
3. Observation
4. Review of 
program documents
1. USAID
2. DI Program 
Staff/Volunteers
3. Key Participating 
CSOs and TSPs
4. Sub-grantees
5. Training 
beneficiaries
6. 5D Lab Members
 
22 Questions included here are not exhaustive. See Annex III for complete data collection tools and protocols. 
40
Evaluation Question Key Related Indicator(s) Key Related KII/FGD 
Question22
Proposed Data 
Collection 
Methodology
Respondent 
Group
 Output Indicator 2.1: Number of 
Technology and Service Providers 
(TSPs) that have actively 
participated in USG assisted activities
 Output Indicator 2.2: Percentage of 
CSO, TSP, and individual participants 
that show improved skills after 
participating in Training Events
Outcome 3 including:
 Outcome Indicator 3.1: Number of ICT￾based solutions designed and 
implemented as a result of CSO and 
TSP engagement, through a USG 
supported ICT project.
 Output Indicator 3.1: Percentage of 
youths that participate in USG 
assistance’s youth-oriented activities 
that acquire new or improved 
knowledge from participating in 
program activities.
think the project should do less 
of?
To what extent has the project 
successfully engaged with civil 
society organizations,
especially in order to assist 
CSOs to utilize technological 
tools/solutions to amplify 
their programmatic impact 
and support civic outcomes?
Have CSOs been able to use 
the project’s assistance to 
develop solutions to assist 
them in achieving their goals?
Outcome Indicator 1: The number of CSO 
beneficiaries using ICT-based solutions as a 
result of the CSOs engagement with USG 
assistance.
Output Indicator 1.1: Number of CSOs working 
in Cambodia (whether a local CSO or an 
international CSO working through its local 
affiliate) that have engaged with a USG￾supported ICT project to address an identified 
development problem through an ICT-based 
solution.
Output Indicator 2.2: Percentage of CSO, TSP, 
and individual participants that show improved 
skills after participating in Training Events
Can you describe the 
collaborations that you observed 
between CSOs and TSPs that took 
part in the DI program? How 
could those collaborations be 
further strengthened or 
expanded?
What types of requests for 
support/training have you 
received from participating 
CSO/TSPs or 5D lab members?
What development problem is 
your CSO seeking to better 
address by utilizing technology? 
1.Key informant 
interviews
2. Focus Group 
Discussions
3. Review of 
program documents
1. Key Participating 
CSOs and TSPs
2. Sub-grantees
3. Training 
beneficiaries
4. 5D Lab Members
5. Blogger 
Community
6. Developer/Progr
ammer 
Community
7. DG Partners
41
Evaluation Question Key Related Indicator(s) Key Related KII/FGD 
Question22
Proposed Data 
Collection 
Methodology
Respondent 
Group
Outcome Indicator 3.1: Number of ICT-based 
solutions designed and implemented as a result 
of CSO and TSP engagement, through a USG 
supported ICT project.
What type of ICT-based solution 
are you using?
Please describe the type of 
assistance provided to you 
through the DI program. Did you 
find the information/relationships 
gained through your participation 
in the program useful for your 
work as a CSO? If yes, please 
describe in more detail.
On a scale of 1-5, how frequently 
do you use the information you 
gained from your participation in 
DI trainings/partnership? (1 being 
frequently, 5 being infrequently)
Please describe any relationships 
that you have developed as a 
result of your participation in the 
DI program. (Probe for 
collaboration, etc.)
To what extent has the project 
as a whole (including all of its 
various activities) contributed 
to achieving the objectives of 
the USAID/Cambodia Office of 
Democracy and Governance?
Outcome Indicator 1: The number of CSO 
beneficiaries using ICT-based solutions as a 
result of the CSOs engagement with USG 
assistance
Outcome 1.1 including:
 Outcome Indicator 1.1: Percent of CSOs 
that have engaged with USG supported 
ICT project that have 
implemented ICT-based solutions
 Outcome Indicator 1.2: Number of 
CSOs receiving USG assistance engaged 
in advocacy interventions
 Output Indicator 1.1: Number of CSOs 
working in Cambodia (whether a local 
What project outcomes have you 
observed to date (specifically 
related to outcome 1)?
Can you describe the 
collaborations that you observed 
between CSOs and TSPs that took 
part in the DI program?
Has civil society and TSPs been 
receptive to DI initiatives?
What development problem is 
your CSO seeking to better 
address by utilizing technology? 
1.Key informant 
interviews
2. Review of 
program documents
1. USAID Staff
2. DI Program 
Staff/Volunteers
3. Key Participating 
CSOs
4. Sub-grantees
5. DG Partners
42
Evaluation Question Key Related Indicator(s) Key Related KII/FGD 
Question22
Proposed Data 
Collection 
Methodology
Respondent 
Group
CSO or an international CSO working 
through its local affiliate) that have 
engaged with a USG-supported ICT 
project to address an identified 
development problem through an ICT￾based solution.
 Output Indicator 1.2: Percent of CSOs, 
TSPs, and individual participants that 
show improved skills after 
participating in Training Events.
What type of ICT-based solution 
are you using?
Please describe the programs 
gender inclusion efforts. How do 
project activities ensure 
participation by both men and 
women?
In what ways has the project 
succeeded or failed in terms of 
financial sustainability? 
In what ways has the project 
succeeded or failed in terms of 
participatory sustainability 
(i.e. stimulating enduring 
participation in the Lab which 
could be sustained after the 
end of the project)? 
What evidence is there that 
the project will be sustainable 
at its completion?
Outcome 2 including:
 Output Indicator 2.1: Number of 
Technology and Service Providers 
(TSPs) that have actively 
participated in USG assisted activities
 Output Indicator 2.2: Percentage of 
CSO, TSP, and individual participants 
that show improved skills after 
participating in Training Events
Outcome Indicator 3.1: Number of ICT-based 
solutions designed and implemented as a result 
of CSO and TSP engagement, through a USG 
supported ICT project.
Explain the sustainability steps 
the DI project has taken to ensure 
lasting outcomes (discuss 
financial sustainability of 
partners/sub-grantees).
If the DI program were to end 
tomorrow, would you continue 
utilizing technology to advance 
your CSOs mission? If yes, how? If 
no, why?
If the DI program were to end 
tomorrow, would you maintain 
the relationships with CSOs you 
built through the program on your 
own? If yes, how? If no, why?
Explain how you plan to continue 
the work you started with the DI 
program. If you are not planning 
to continue this work, can you 
explain why? 
1.Key informant 
interviews
2. Focus Group 
Discussions
3. Review of 
program documents
1. Lab Members 
and staff
2. Key Participating 
CSOs and TSPs
3. Sub-grantees
4. Training 
beneficiaries
5. 5D Lab Members
6. Blogger 
Community
7. Developer/Progr
ammer 
Community
How has the Cambodian 
country context (state of civil 
society and human capacity in 
NA Are there any barriers, in your 
estimation, standing in the way of 
the program achieving its stated 
objectives?
1.Key informant 
interviews
2. Review of 
program documents
1. USAID Staff
2. DAI Program Staff
3. CSO and TSP 
Partners
43
Evaluation Question Key Related Indicator(s) Key Related KII/FGD 
Question22
Proposed Data 
Collection 
Methodology
Respondent 
Group
the IT sector) affected the 
implementation of the project?
Have there been any contextual 
factors (Cambodia’s political 
environment, gender disparities, 
etc.) that has affected the DI 
project?
44
ANNEX III: DATA COLLECTION INSTRUMENTS 
USAID/Cambodia DI Evaluation 
Coversheet for All Key Informant Interview Protocols 
Date of Interview: Interviewee Name, Title: 
Team: Time Start: Time End:
Interviewer(s): Location:
Introduction: Good morning/afternoon and thank you for taking the time to speak with us today. As mentioned 
during our interview request, we are working with USAID/Cambodia to conduct a midterm evaluation of the 
Development Innovations Project (DI). The evaluation is intended to provide an informed assessment of progress 
to date and recommend potential modifications, if needed. 
Our team has had the opportunity to review some background documents to get a better sense of the design and 
implementation of the project. However, these documents can only tell us so much. We would like to speak with 
you today to hear about your experience, in your own words, in order to help us better understand how these 
projects look and function “on the ground.” 
Confidentiality Protocol 
 We will collect information on individuals’ names, organizations and positions. A list of key informants will 
be made available as an annex to the final evaluation report, but those names and positions will not be 
associated to any particular findings or statements in the report. 
 We may include quotes from respondents in the evaluation report, but will not link individual names, 
organizations or personally identifiable information to those quotes, unless express written consent is 
granted by the respondent. Should the team desire to use a particular quote, photograph or identifiable 
information in the report, the evaluators will contact the respondent(s) for permission to do so. 
 All data gathered will be used for the sole purposes of this evaluation, and will not be shared with other 
audiences or used for any other purpose. 
 Your participation in this interview is voluntary and if you do not feel comfortable answering a particular 
question please let us know and we will simply go on to the next question. 
Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get 
started? 
Inform interviewee we may follow-up with brief email survey at the end of fieldwork. 
45
USAID/Cambodia DI KIIs 
(Est. Time ~45 min)
USAID Staff
Introduction and Project Scope 
 What has been your experience with the grant/project? 
 Can you describe the project to date, as you understand it? 
Progress and Effectiveness 
 Have there been any changes to the scope or program activities since you have overseen the DI 
project? If yes, please describe any and all changes, including contextual factors, intervening 
circumstances or other reasons that may have affected the program. 
 Based on your interaction with the DI program, please describe your assessment of the program 
to date. 
 What project outcomes have you observed to date? 
o Have you observed any unintended/unexpected results (positive or negative)? 
 What do you think are the projects high points/successes? 
 Are there any barriers, in your estimation, standing in the way of the program achieving its 
stated objectives? 
 Has the project encountered any challenges? If so, please describe (program design challenges, 
implementation challenges, or other). 
o If challenges did occur, how did they affect the program? 
o How did the project address and overcome the challenges? 
 What do you think the project should do more of? 
 What do you think the project should do less of? 
CSO/TSP Collaboration 
 Can you describe the collaborations that you observed between CSOs and TSPs that took part 
in the DI program? 
o How could those collaborations be further strengthened or expanded? 
 What types of requests for support/training have you received from CSO/TSPs or 5D lab 
members? 
Cross-Cutting Issues 
 Can you describe any observations about women and youth participation in the DI program? 
 Explain the sustainability steps the DI project has taken to ensure lasting outcomes (discuss 
financial sustainability of partners/sub-grantees). 
Contextual Issues 
 Are there any contextual factors, such as Cambodia’s political environment, that have affected 
the DI project? 
 What are the major impediments to the use of technology by CSOs in Cambodia? 
Recommendations 
 What recommendations do you have for the project? 
 Is there any specific information that you would like the evaluation team to collect that will be 
helpful for you as you develop further funding or program plans? 
46
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
DI Program Staff
Introduction and Project Scope 
 Describe your role in the DI project. How long have you been with the project? 
 Can you describe the project to date? 
 Have there been any changes to the scope or program activities since you have been with the 
DI project? If yes, please describe. 
Progress and Effectiveness 
 What do you think are the projects high points/successes? 
 What have been the most effective program activities to support DI’s overall goal?
 Has the project encountered any challenges? If so, please describe (program design challenges, 
implementation challenges, or other). 
o If challenges did occur, how did they affect the program? 
o How did the project address and overcome the challenges? 
CSO/TSP Collaboration 
 Has civil society and TSPs been receptive to DI initiatives? 
 What do you think the project should do more of? 
 What do you think the project should do less of? 
 Can you describe the collaborations that you observed between CSOs and TSPs that took part 
in the DI program? 
o How could those collaborations be further strengthened or expanded? 
o Are there any hindrances to CSO and TSP collaboration? 
 What types of requests for support/training have you received from participating CSO/TSPs or 
5D lab members? 
Cross-Cutting Issues 
 Please describe the programs gender inclusion efforts. How do project activities ensure 
participation by both men and women? 
 Explain the sustainability steps the DI project has taken to ensure lasting outcomes (discuss 
financial sustainability of partners/sub-grantees). 
Contextual Issues 
 Are there any contextual factors, such as Cambodia’s political environment, that have affected 
the DI project? 
Recommendations 
 What recommendations do you have for the project? 
47
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Key CSOs
Introduction and Project Scope 
 What development problem is your CSO seeking to better address by utilizing technology? 
 What type of ICT-based solution are you using? 
Progress and Effectiveness 
 Please describe the type of assistance provided to you through the DI program. 
 How many ICT events / trainings have you participated in DI? List as many as you can.
 Did you find the information/relationships gained through your participation in the program 
useful for your work as a CSO? If yes, please describe in more detail. 
 How frequently did you visit the 5D Lab? 
 How have you used the knowledge you gained from your participation in DI 
trainings/partnership in your organization’s work?
 Since participation with DI, what do you see as the most significant improvement within your 
organization?
 Have you experienced any challenges with the DI program? 
CSO/TSP Collaboration 
 Have you ever developed an ICT solution for your organization? If yes, please list all.
 Please describe any relationships that you have developed as a result of your participation in the DI 
program. (Probe for collaboration, etc.) 
o With DI 
o With other CSOs 
o With TSPs 
o With the Developer/Programmer Community 
o With the Blogger community
 If the DI program were to end tomorrow, would you maintain the relationships you built 
through the program on your own? If yes, how? If no, why? 
 If the DI program were to end tomorrow, would you continue utilizing technology to advance 
your mission? If yes, how? If no, why? 
Cross-Cutting Issues 
 Please describe the programs youth and gender inclusion efforts. How do project activities 
ensure participation by both men and women? Such as: 
o Tracking the progress of women and girl’s access to ICT or women participating in 
policy making
o Developing capacity building for gender equality through ICT 
o Promoting good practices on the ways women use ICT 
o Increasing women’s networks through electronic connectivity
 Is the assistance you have received through the DI project sustainable? 
Contextual Issues 
 Will you use more ICT solutions for your communication and to improve your activities? 
48
 The ICT sector is growing in Cambodia. Do you see your organization using ICT solutions for 
communications in your activities? 
 Do you have any concerns about security in relation to using ICTs in your work? 
Recommendations 
 If additional training or support could be provided, what type of training/support would you find 
useful (try to dig past the obvious financial support request; try to urge answers that fit into 
scope of DI program)?
 Do you have any recommendations for the program going forward? 
 What types of technology would be useful for you in your CSO work? 
49
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Key TSPs
Introduction and Project Scope 
 Please describe the type of assistance provided to you through the DI program. 
• What is your main reason that you would use the 5D Lab? (possible answers: working space, Internet 
Access, Meeting Space, Access to Technology Materials, Getting Advice, Networking with other 
TSP/CSO/Technologist)
• What ARE your main business activities?
Progress and Effectiveness 
 What was the main benefit from participating with DI? (ICT Capacity Development, ICT 
Solution, Relationship with TSP & Technologist, Grant etc.)
 How did you use the knowledge or connections gained through your participation in the 
program? 
 Please describe any collaborations that you have developed as a result of your participation in 
the DI program. (Probe for collaboration, etc) 
o With other TSPs 
o With CSOs 
o With DI 
o With the Developer/Programmer Community 
o With the Blogger community
 Thinking about your interaction with the program, what do you wish would have been done 
differently? 
CSO/TSP Collaboration 
 How many CSO have you made contact with as a result of the DI program? Please provide 
details.
 Have you provided mentoring / training to CSOs through DI?
 How can the tech community provide additional support to CSOs? 
 Can you think of ways of collaborating with CSOs that could push your company’s mission 
forward? 
Cross-Cutting Issues 
 Please describe the programs youth and gender inclusion efforts. How do project activities 
ensure participation by both men and women? Such as: 
o Tracking the progress of women and girl’s access of ICT or women participating in 
policy making
o Developing the apacity building for gender equality through ICT 
o Promoting good practices on the ways women using ICT 
o Increasing women network through elctronic connectivity
 Is the assistance you have received through the DI project sustainable? If yes, how so? If no, 
why? 
Contextual Issues 
 Are there any contextual factors (such as Cambodia’s political environment) that affected your 
collaboration with CSOs? 
50
Recommendations 
• Do you have any recommendation to improve the 5D Lab?
• Do you want the 5D Lab continue to operate after the DI program ends?
51
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Non-Partner CSOs
Introduction and Project Scope 
 Have you heard of the 5D Lab and DI? 
 What do you know about the DI program? 
 What types of technology do you currently use in your work to support your mission? 
 What types of technology would be useful for you in your work? 
CSO/TSP Collaboration 
 Do you currently collaborate with TSPs to produce ICT solutions for your organization’s work? 
o If yes, please describe. 
 If no, how can collaborating with TSPs be possible?
Contextual Issues 
 How do you see the future of the technology landscape within Cambodia and within your 
organization? 
52
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Non-Partner TSPs
Introduction and Project Scope 
 Have you heard of the 5D Lab and DI? 
 What do you know about the DI program? 
CSO/TSP Collaboration 
 Do you collaborate with CSOs currently? If yes, for what purpose? If no, could collaborating 
with CSOs be useful for your work? 
 What could be done to strengthen CSOs ability to work with TSPs such as yourself? 
Contextual Issues 
 How do you see the future of the technology landscape within Cambodia and within your 
organization? 
53
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
DG Partners 
*Some of the partners in this category may also fall under another respondent category (Key CSO, sub￾grantee, etc). The Evaluation Team will supplement these DG specific questions with the appropriate 
protocol for the other questions. 
CSO/TSP Collaboration 
 Do you think CSOs and TSPs could collaborate on ICT solutions to further D&G project goals? 
If yes, please describe 
 If no, what are some of the barriers? 
Contextual Issues 
 How do you see the future of the technology landscape within Cambodia and within your 
organization? 
54
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Sub-grantees
*Some of the partners in this category may also fall under another respondent category (Key CSP, sub￾grantee, etc). The Evaluation Team will supplement these DG specific questions with the appropriate 
protocol for the other questions. 
Introduction and Project Scope 
 Please briefly describe your project that is being funded through the DI sub-grant. 
 Please describe the assistance DI provided through the sub-grant you received. 
 Please describe your interaction with DI in terms of sub-grant program implementation/grant 
management. 
Cross-Cutting Issues 
 Have your activities included both male and female participants? How did you ensure activities 
were made available to both men and women equally? 
Contextual Issues 
 How do you see the future of the technology landscape within Cambodia and within your 
organization? 
55
USAID/Cambodia DI KIIs 
(Est. Time ~60 min)
Rejected Sub-grantee Applicants
Introduction and Project Scope 
 What was your proposal to DI? 
 Why do you think your proposal idea was rejected? 
 Did DI give you a reason for why your proposal was rejected? 
 Was the call for proposal clear to you? 
 Would you consider reapplying for a DI sub-grant? 
 Explain your current work and how future collaboration or support by TSPs could enhance your 
program impact. 
CSO/TSP Collaboration 
 Do you currently collaborate with CSOs/TSPs to produce ICT solutions for your organization’s 
work? 
o If yes, please describe. 
 If no, how can collaborating with CSOs/TSPs be possible?
Contextual Issues 
 How do you see the future of the technology landscape within Cambodia and within your 
organization? 
56
USAID/Cambodia DI KIIs 
(Est. Time ~30 min)
Government Officials/University officials 
Introduction and Project Scope 
 Are you aware of the DI program? 
 How have you interacted with the DI program? What is your impression of the program? 
CSO/TSP Collaboration 
 How does the program most benefit the CSO and TSP community in Cambodia? 
 How could the DI program better benefit both CSOs and TSPs in Cambodia? 
 In your view, how could CSOs better utilize technology in their work? 
 In your view, how could TSPs provide better services or support to CSOs?
Cross-cutting Issues 
 How could women be better integrated into CSO-ICT collaboration programs such as DI? 
 How could youth be better integrated into ICT programs such as DI? 
Recommendations 
 Do you have any recommendations for the program going forward?
57
USAID/Cambodia DI Evaluation 
Coversheet for All Focus Group Discussion Protocols 
Date of Interview: Interviewee Name, Title: 
Interviewer(s): Time Start: 
Location: Time End:
Introduction: Good morning/afternoon and thank you for taking the time to speak with us today. As mentioned 
during our interview request, we are working with USAID/Cambodia to conduct a midterm evaluation of the 
Development Innovations Project (DI). The evaluation is intended to provide an informed assessment of progress 
to date and recommend potential modifications, if needed. 
Our team has had the opportunity to review some background documents to get a better sense of the design and 
implementation of the project. However, these documents can only tell us so much. We would like to speak with 
you today to hear about your experience, in your own words, in order to help us better understand how these 
projects look and function “on the ground.” 
Confidentiality Protocol 
 We will collect information on individuals’ names, organizations, and positions. A list of key informants 
will be made available as an annex to the final evaluation report, but those names and positions will not be 
associated to any particular findings or statements in the report. 
 We may include quotes from respondents in the evaluation report, but will not link individual names, 
organizations, or personally identifiable information to those quotes, unless express written consent is 
granted by the respondent. Should the team desire to use a particular quote, photograph, or identifiable 
information in the report, the evaluators will contact the respondent(s) for permission to do so. 
 All data gathered will be used for the sole purposes of this evaluation, and will not be shared with other 
audiences or used for any other purpose. 
 Your participation in this interview is voluntary and if you do not feel comfortable answering a particular 
question please let us know and we will simply go on to the next question. 
Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get 
started? 
Inform interviewee we may follow-up with brief email survey at the end of fieldwork. 
58
Focus Group Discussion Guide 
5D Lab Members (female cohort, male cohort) 
Time: 90 minutes 
Be sure to:
 Set up the room to facilitate a participatory discussion 
 Introduce yourselves
 Introduce SI and explain why you are conducting the focus group session
 Ask the participants to do the same
 Give a verbal agenda and length of the meeting
 Set goals
 Clarify your role as a facilitator
Evaluation’s objective
To better understand what the DI program has accomplished, document any challenges and identify opportunities 
for program improvement. 
Purpose of discussion
To hear from 5D Lab members about their experience in the Development Innovation Program. 
Focus Group facilitator’s guidelines for effective discussions:
 Everyone is clear on the topic
 Everyone participates; no one dominates the discussion - No speeches!
 One person talks at a time
 Comments and discussion stay on the topic
 Comments should be to the whole group - no side discussions
 Respect time limits
 Write down unanswered questions
 No divisive or confrontational language or tone
 Take notes of discussion, comments and observations so that you can write report
1. Introduction (10 min) 
Reaffirm points of the meeting 
Welcome participants 
Set 90 minute timeline 
Introduce the evaluation, give verbal agenda, objectives (as stated above) 
2. Setting rules (5 min) 
Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define 
how a group will function and how the participants will interact. 
Sample rules 
Each group member has the right to participate. 
The opinion of each group member is important and should be respected. 
Group members should be tolerant of different ideas. 
Each group member is important. 
3. Tell us about your involvement with the DI project (40 min) 
Ask participants: 
 Describe your role at the lab. How long have you been coming to the lab? 
 Can you describe the type of services or programs the lab offers?
59
o How have they been received by CSOs and TSPs? 
 Has the lab encountered any challenges? If so, please describe. 
o If challenges did occur, how did they affect the lab’s work?
o How did DI address and overcome the challenges? 
 What types of support/training have you requested from the lab? 
 What do you think the lab should do more of? 
 What do you think the lab should do less of? 
4. CSO/TSP Collaborations (20 min) 
Ask participants: 
 Can you describe the collaborations that you observed between CSOs and TSPs that 
come to the lab? 
o How do you think those collaborations could be further strengthened or 
expanded? 
5. Cross-cutting issues (10 min) 
Ask participants: 
 What efforts does the lab take to promote women and youth’s involvement in lab 
activities? 
 What could the lab do differently to encourage more women and youth to become 
involved in technology and civil society? 
6. Contextual Factors (15 min) 
Ask participants: 
 Will you use more ICT solutions for your communication and to improve your 
activities?
 The ICT sector is growing in Cambodia. Do you see your organization using ICT 
solutions for communications in your activities? 
 Do you have any concerns about security in relation to using ICTs in your work? 
7. Conclusion / Close (5 min) 
 Summarize the session 
 Ask if everyone agrees to what was discussed, offer any final comments and thank all those 
who participated. 
60
Focus Group Discussion Guide 
Training Beneficiaries 
Time: 90 minutes 
Be sure to:
 Set up the room to facilitate a participatory discussion 
 Introduce yourselves
 Introduce SI and explain why you are conducting the focus group session
 Ask the participants to do the same
 Give a verbal agenda and length of the meeting
 Set goals
 Clarify your role as a facilitator
Evaluation’s objective
To better understand what the DI program has accomplished, document any challenges and identify opportunities 
for program improvement. 
Purpose of discussion
To hear from training beneficiaries about their experience in the Development Innovation Program. 
Focus Group facilitator’s guidelines for effective discussions:
 Everyone is clear on the topic
 Everyone participates; no one dominates the discussion - No speeches!
 One person talks at a time
 Comments and discussion stay on the topic
 Comments should be to the whole group - no side discussions
 Respect time limits
 Write down unanswered questions
 No divisive or confrontational language or tone
 Take notes of discussion, comments and observations so that you can write report
1. Introduction (10 min) 
Reaffirm points of the meeting 
Welcome participants 
Set 90 minute timeline 
Introduce the evaluation, give verbal agenda, objectives (as stated above) 
2. Setting rules (5 min) 
Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define 
how a group will function and how the participants will interact. 
Sample rules 
Each group member has the right to participate. 
The opinion of each group member is important and should be respected. 
Group members should be tolerant of different ideas. 
Each group member is important. 
3. Tell us about your involvement with the DI project (35 min) 
Ask participants: 
61
 Please describe the training you received (name of training). 
 Did you appreciate the training organization and method? If yes, what specifically made 
the training a positive experience for you? If no, what could have been done better? 
 How have you used this information in your daily work? 
 Are there topics or sessions that you would like DI to host? 
 Have you visited the 5D lab? 
 Can you describe your visits to the 5D lab? 
4. CSO/TSP Collaborations (20 min) 
Ask participants: 
 Have you made any new connections or relationships as a result of your participation in 
the training?
 Have you collaborated with anyone from your training session as a result of the training 
session? 
5. Cross-cutting issues (10 min) 
Ask participants: 
 What efforts did the training facilitators take to promote women’s involvement? 
 What could the training facilitators do differently to encourage more women to become 
more involved in technology and civil society trainings and work? 
 Will you use more ICT solutions for your communication and to improve your activities 
in the future? If yes, how so? If no, why? 
6. Contextual Factors (15 min) 
Ask participants: 
 The ICT sector is growing in Cambodia, do you see your organization using ICT 
solutions for communications in your activities? 
 Do you have any concerns about security in relation to using ICTs in your work? 
7. Recommendations (5 min) 
 Do you have any recommendations for how the DI project could be improved? 
 How would you improve DI’s trainings?
8. Conclusion / Close (5 min) 
 Summarize the session 
 Ask if everyone agrees to what was discussed, offer any final comments and thank all those 
who participated. 
62
Focus Group Discussion Guide 
Bloggers 
Time: 90 minutes 
Be sure to:
 Set up the room to facilitate a participatory discussion 
 Introduce yourselves
 Introduce SI and explain why you are conducting the focus group session
 Ask the participants to do the same
 Give a verbal agenda and length of the meeting
 Set goals
 Clarify your role as a facilitator
Evaluation’s objective
To better understand what the DI program has accomplished, document any challenges and identify opportunities 
for program improvement. 
Purpose of discussion
To hear from bloggers about their experience in the Development Innovation Program. 
Focus Group facilitator’s guidelines for effective discussions:
 Everyone is clear on the topic
 Everyone participates; no one dominates the discussion - No speeches!
 One person talks at a time
 Comments and discussion stay on the topic
 Comments should be to the whole group - no side discussions
 Respect time limits
 Write down unanswered questions
 No divisive or confrontational language or tone
 Take notes of discussion, comments and observations so that you can write report
1. Introduction (10 min) 
Reaffirm points of the meeting 
Welcome participants 
Set 90 minute timeline 
Introduce the evaluation, give verbal agenda, objectives (as stated above) 
2. Setting rules (5 min) 
Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define 
how a group will function and how the participants will interact. 
Sample rules 
Each group member has the right to participate. 
The opinion of each group member is important and should be respected. 
Group members should be tolerant of different ideas. 
Each group member is important. 
3. Tell us about your involvement with the DI project (35 min) 
Ask participants: 
 Describe your knowledge/interaction with the DI program. 
63
4. CSO/TSP Collaborations (20 min) 
Ask participants: 
 How could a project such as DI better support CSO/TSP collaboration? 
 Can you identify any entry points where collaboration could be strengthened? 
 Can you identify any barriers to collaboration? 
5. Cross-cutting issues (10 min) 
Ask participants: 
 How could the DI project encourage more women to become involved with technology 
and seek out technology solutions? 
6. Contextual Factors (15 min) 
Ask participants: 
 Are there issues in the blogging community or in Cambodia that may impact your ability 
to continue working and utilizing ICT? 
7. Recommendations (5 min) 
 Do you have any recommendations for how the DI project could be improved? 
 How would you improve DI’s trainings?
8. Conclusion / Close (5 min) 
 Summarize the session 
 Ask if everyone agrees to what was discussed, offer any final comments and thank all those 
who participated. 
64
Focus Group Discussion Guide 
Developers/Programmers 
Time: 90 minutes 
Be sure to:
 Set up the room to facilitate a participatory discussion 
 Introduce yourselves
 Introduce SI and explain why you are conducting the focus group session
 Ask the participants to do the same
 Give a verbal agenda and length of the meeting
 Set goals
 Clarify your role as a facilitator
Evaluation’s objective
To better understand what the DI program has accomplished, document any challenges and identify opportunities 
for program improvement. 
Purpose of discussion
To hear from developers/programmers about their experience in the Development Innovation Program. 
Focus Group facilitator’s guidelines for effective discussions:
 Everyone is clear on the topic
 Everyone participates; no one dominates the discussion - No speeches!
 One person talks at a time
 Comments and discussion stay on the topic
 Comments should be to the whole group - no side discussions
 Respect time limits
 Write down unanswered questions
 No divisive or confrontational language or tone
 Take notes of discussion, comments and observations so that you can write report
1. Introduction (10 min) 
Reaffirm points of the meeting 
Welcome participants 
Set 90 minute timeline 
Introduce the evaluation, give verbal agenda, objectives (as stated above) 
2. Setting rules (5 min) 
Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define 
how a group will function and how the participants will interact. 
Sample rules 
Each group member has the right to participate. 
The opinion of each group member is important and should be respected. 
Group members should be tolerant of different ideas. 
Each group member is important. 
3. Tell us about your involvement with the DI project (40 min) 
Ask participants: 
 Describe your interaction with the DI program. How long have you been coming to the 
lab? 
 Can you describe the type of services or programs the lab offers? 
65
o How have they been received by CSOs and TSPs? 
 Has the lab encountered any challenges? If so, please describe. 
o If challenges did occur, how did they affect the lab’s work?
o How did DI address and overcome the challenges? 
 What do you think the lab should do more of? 
 What do you think the lab should do less of? 
 What types of support/training have you requested from the lab? 
4. CSO/TSP Collaborations (20 min) 
Ask participants: 
 Can you describe the collaborations that you observed between CSOs and TSPs that 
come to the lab? 
 How do you think those collaborations could be further strengthened or expanded? 
5. Cross-cutting issues (10 min) 
Ask participants: 
 What efforts does the lab take to promote women’s involvement in program activities?
 What could the lab do differently to encourage more women and youth to become 
involved in technology and civil society? 
6. Recommendations (5 min) 
 Do you have any recommendations for how the DI project could be improved? 
 How would you improve DI’s trainings?
7. Conclusion / Close (5 min) 
 Summarize the session 
 Ask if everyone agrees to what was discussed, offer any final comments and thank all those who participated. 
66
Focus Group Discussion Guide 
5D Lab/DI Advisory Board 
Time: 90 minutes 
Be sure to:
 Set up the room to facilitate a participatory discussion 
 Introduce yourselves
 Introduce SI and explain why you are conducting the focus group session
 Ask the participants to do the same
 Give a verbal agenda and length of the meeting
 Set goals
 Clarify your role as a facilitator
Evaluation’s objective 
To better understand what the DI program has accomplished, document any challenges and identify opportunities 
for program improvement. 
Purpose of discussion
To hear from 5D Lab members and DI Advisory Board about their experience in the Development Innovation 
Program. 
Focus Group facilitator’s guidelines for effective discussions:
 Everyone is clear on the topic
 Everyone participates; no one dominates the discussion - No speeches!
 One person talks at a time
 Comments and discussion stay on the topic
 Comments should be to the whole group - no side discussions
 Respect time limits
 Write down unanswered questions
 No divisive or confrontational language or tone
 Take notes of discussion, comments and observations so that you can write report
1. Introduction (10 min) 
Reaffirm points of the meeting 
Welcome participants 
Set 90 minute timeline 
Introduce the evaluation, give verbal agenda, objectives (as stated above) 
2. Setting rules (5 min) 
Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define 
how a group will function and how the participants will interact. 
Sample rules 
Each group member has the right to participate. 
The opinion of each group member is important and should be respected. 
Group members should be tolerant of different ideas. 
Each group member is important. 
3. Tell us about your involvement with the DI project (20 min) 
Ask participants: 
 What is your role as an advisory board member? Please describe. 
4. CSO/TSP Collaborations (30 min) 
Ask participants: 
67
 Do you think the DI project is having a positive impact within the CSO and TSP 
communities? 
 In relation to the DI project, what do you see as the primary needs of both CSOs and 
TSPs? 
5. Cross-cutting issues (10 min) 
Ask participants: 
 How can more women and youth be included in CSO/ICT collaborations? 
6. Contextual Factors (15 min) 
Ask participants: 
 What are some of the challenges you have seen for CSOs use of ICTs? 
 The ICT sector is growing in Cambodia. Do you see organizations using ICT solutions 
for communications in their activities? 
 Do you have any concerns about security in relation to using ICTs in CSO work? 
7. Recommendations (5 min) 
 Do you have any recommendations for how the DI project could be improved? 
 How would you improve DI’s trainings?
8. Conclusion / Close (5 min) 
 Summarize the session 
 Ask if everyone agrees to what was discussed, offer any final comments and thank all those who participated. 
68
USAID/Cambodia DI Evaluation 
Observation Protocol 
Date of Observation: Activity ID: 
Team: Time Start: Time End:
Observer (s): Location:
Purpose of Observation: 
To witness DI events, activities or labs in order to better understand what the DI program has accomplished, 
document any challenges and identify opportunities for program improvement.
Before Observation: 
 Explain: 
o the purpose of the observation to activity facilitators/staff 
o that no information will be used to evaluate a particular person’s performance
o that the results of the observation will not affect the facilitators/organizations/TSPs 
ability to receive support now or in the future 
 Obtain consent from activity facilitators/staff to do the observation 
 Clarify the focus of each evaluation team members’ observation
General Guidelines: 
 Do not, in any way, interfere with the activity or event 
 Observe quietly and take notes 
Observation Notes: 
Facilitator/Organizer Name: 
Name of participating organizations/TSPs: 
1.
2.
3.
Number of speakers/facilitators: 
M/F: M _________ F_____________ 
Number of adults in attendance/participating: ______________ 
 M/F: M _________ F_____________ 
Subject being discussed/taught: ________________________________________________ 
Describe the topic being discussed/taught: 
Describe facilitation/teaching style (inclusive, participatory, etc.):
69
Describe engagement between participants and facilitators (between CSOs and TSPs? during Q&A? during coffee 
breaks?): 
Describe main outcomes of the activity/event: 
Describe how (and if) the facilitator/organizer will follow-up with participants post-event: 
 
70
ANNEX IV: SOURCES OF INFORMATION
List of Respondent Affiliations 
The evaluation team, as part of the KII and FGD protocol, told all respondents that their names would 
be kept confidential in order to encourage open and frank feedback. The evaluation team did, however, 
inform respondents that their organization would be included in the final report. All organizations that 
were represented by a respondent in a KII or FGD are detailed below. Numbers in parenthesis 
represent the number of individuals interviewed from a particular organization/company/office. 
1. Current USAID Staff (7: 5 female, 2 men) 
2. Former USAID Staff (1: male) 
3. Current DI Staff (18: 6 female, 12 male) 
4. Former DI Staff (2: 1 female, 1 male) 
5. Key CSOs (participating and non-participating) 
a. Transparency International-Cambodia (1 male) 
b. Open Development Cambodia (2 male) 
c. Community Legal Education Center (CLEC) (1 male) 
d. Saakum Teamng Ta Naut (STTN) (1 male) 
e. Returnee Integration Support Center (RISC) (1 male) 
f. SILAKA (1 female, 2 male) 
6. Key TSPs (participating and non-participating) 
a. EZECOM (1 male) 
b. CellCard (1male) 
7. Subgrantees 
a. Advocacy and Policy Institute (API) (1 female, 2 male) 
b. Open Institute (OI) (1 male) 
c. The Documentation Center of Cambodia (DC-CAM) (1 male) 
d. Cambodian Center for Independent Media (CCIM) (1 female) 
e. ADHOC (2 male) 
f. People In Need (2 female, 1 male) 
g. PACT-Cambodia (1 female, 2 male) 
h. ALiEN Dev (1 male) 
i. Action IEC (2 male) 
j. World Education (1 female) 
k. Cambodian Center for Human Rights (CCHR) (1 female) 
8. Rejected Subgrantees 
a. Voluntary Services Overseas (VSO) (1 male) 
b. Cooperative Committee for Cambodia (CCC) (2 male) 
9. Training Beneficiaries (2) (2 male) 
10. Bloggers (1) (1 male) 
11. 5D Lab Members (6) (2 female, 4 male) 
12. University/Technology Vocational Schools (5) 
a. Institute for Technology of Cambodia (1 male) 
b. National University of Management (1male) 
c. Liger Learning Center (2 male) 
d. Passarelles Numeriques Cambodia (1 male) 
13. Local Lab/Hub Community Members (1) 
a. Impact Hub (1 male) 
71
Bibliography of Documents Reviewed 
The evaluation team conducted an in-depth review of all program documents provided by USAID and 
DAI. Documents provided included: 
 Program Request for Application (RFA) 
 USAID Social Media Assessment 2012 
 Program cooperative agreement and modifications (SILK/DI) 
 Quarterly program reports 
 Quarterly financial reports 
 Annual work plans 
 Project monitoring and evaluation plan 
 Program grants documentation. 
o Approved and rejected subgrant applications, 
o Subgrantee program and financial reports, 
o Subgrant tracking pipeline 
 USAID Cambodia Country Development Cooperation Strategy (CDCS) 
 DI Business Plan 
 DI Sustainability Assessment 
 Grant Program Announcements and guidelines from spring 2015 
 Promotional DI program literature 
 Contact information for stakeholder and potential evaluation respondents 
72
ANNEX V: DATA COLLECTION SCHEDULE
Weeks (as in Contract) Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13
Week of 8/17 8/24 8/31 9/7 9/14 9/21 9/28 10/5 10/12 10/19 10/26 11/2 11/9
Award Contract Award (August 17)
Kick Off Meeting with USAID/Cambodia
Consultant on-boarding
Document Review
Team Planning Meeting (Webex)
Development of data collection tools and 
evaluation work plan
Logistics
Submit Draft EWP + tools
x
10
Travel to Cambodia (TL): Arrive on Sept 13
Team Planning Meeting (Cambodia)
Inbrief with USAID/DG
x
15
Submit Final EWP
x
17
Data Collection
Data Analysis
Out-brief with USAID
x
8
Out-brief with Partners
Travel from Cambodia (TL): Depart on October 9
Data Analysis and report writing
Draft Report Submission (10 working days for USAID 
review)
x
23
Address comments from USAID
Revise/Finalize Evaluation Report and submit all 
records from the evaluation
x
13
Key
Completion of tasks
Fieldwork Red outline
Submission of Deliverable x
November
Mid-term Evaluation of Cambodia Development Innovations Project: August 17 - November 16, 2015
August Analysis and Reporting
September October Start-Up and Coordination Field Work
73
ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST 
Name Kourtney Pompi
Title Team Leader
Organization Social Impact
Evaluation Position? X Team Leader Team member
Evaluation Award Number
(contract or other instrument)
AID-442-A-13-00003
USAID Project(s) Evaluated
(Include project name(s), 
implementer name(s) and award 
number(s), if applicable)
Development Innovations
I have real or potential conflicts 
of interest to disclose.
 Yes X No
If yes answered above, I disclose 
the following facts:
Real or potential conflicts of interest may 
include, but are not limited to:
1. Close family member who is an employee 
of the USAID operating unit managing the 
project(s) being evaluated or the 
implementing organization(s) whose 
project(s) are being evaluated.
2. Financial interest that is direct, or is 
significant though indirect, in the 
implementing organization(s) whose 
projects are being evaluated or in the 
outcome of the evaluation.
3. Current or previous direct or significant 
though indirect experience with the 
project(s) being evaluated, including 
involvement in the project design or 
previous iterations of the project.
4. Current or previous work experience or 
seeking employment with the USAID 
operating unit managing the evaluation or 
the implementing organization(s) whose 
project(s) are being evaluated.
5. Current or previous work experience with 
an organization that may be seen as an 
industry competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, 
groups, organizations, or objectives of the 
particular projects and organizations 
being evaluated that could bias the 
evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will 
update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished. 
Signature
Date October 20, 2015
74
x
x
Name Panhavuth LONG
Title Mr. 
Organization Social Impact
Evaluation Position? Team Leader X Team member
Evaluation Award Number 
(contract or other instrument)
AID-442-A-13-00003
USAID Project(s) Evaluated 
(Include project name(s), 
implementer name(s) and 
award number(s), if applicable)
Social Innovation Lab-Kampuchea (SILK)/
Innovations Development
I have real or potential conflicts 
of interest to disclose.
 Yes NNo
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of 
interest may include, but are 
not limited to:
7.Close family member who is 
an employee of the USAID 
operating unit managing the 
project(s) being evaluated or 
the implementing 
organization(s) whose 
project(s) are being 
evaluated.
8.Financial interest that is 
direct, or is significant 
though indirect, in the 
implementing organization(s) 
whose projects are being 
evaluated or in the outcome of 
the evaluation.
9.Current or previous direct or 
significant though indirect 
experience with the project(s) 
being evaluated, including 
involvement in the project 
design or previous iterations 
of the project.
10. Current or previous 
work experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the 
implementing organization(s) 
x
75
whose project(s) are being 
evaluated.
11. Current or previous 
work experience with an 
organization that may be seen 
as an industry competitor 
with the implementing 
organization(s) whose 
project(s) are being 
evaluated.
12. Preconceived ideas 
toward individuals, groups, 
organizations, or objectives 
of the particular projects and 
organizations being 
evaluated that could bias the 
evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my 
ability and (2) that I will update this disclosure form promptly if relevant 
circumstances change. If I gain access to proprietary information of other 
companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the 
information for any purpose other than that for which it was furnished. 
Signature
Date
16 October 2015
76
U.S. Agency for International Development 
1300 Pennsylvania Avenue, NW 
Washington, DC 20523