1
A SYSTEMS-BASED EVALUATION OF THE BUILDING 
HEALTH CITIES PROGRAM IN MAKASSAR, INDONESIA:
CORE FINDINGS AND RECOMMENDATIONS
January 2023
Strategic Program for Analyzing Complexity and Evaluating Systems (SPACES)
2
A Systems-Based Evaluation of the Building Healthy Cities 
Program in Makassar, Indonesia:
Core Findings and Recommendations
JANUARY 2023
Report courtesy of SPACES team based at Johns Hopkins University and City University New York 
(Marie F. Martinez, MSPH, Kevin Chin, MPH, Jessie Heneghan, MCP, Colleen Weatherwax, MS, Sarah 
M. Bartsch, MPH, Sheryl A. Scannell, MS, Timothy H. Moran, PhD, Bruce Y. Lee, MD, MBA)
Data collection conducted by SPACES team based at Global Knowledge Initiative (Megan Scanlon, MSW, 
PPA, Seema Patel, MALD, Boris Ristovski, MA, and consultants Nelti Anggraini, MA, Karno B. Batiran, 
MS, Levriana Yustriani, MC-GMCOM, and Lila Sari)
Evaluation Mechanism Number: AID-OAA-A-15-00064 
From July 2021 – January 2023, the Strategic Program for Analyzing Complexity and Evaluating Systems
(SPACES) Monitoring, Evaluation, Research and Learning Innovations Program (MERLIN) Consortium
piloted a systems diagnostic approach in partnership with USAID Asia Bureau to answer exploratory
learning questions related to the Building Health Cities program (BHC).
DISCLAIMER
The authors’ views expressed in this publication do not necessarily reflect the views of the United States 
Agency for International Development or the United States Government
This is not an impact evaluation; therefore, it is classified as a performance evaluation, however, the 
intent of this study was not to assess program performance, but rather assess systems and networks. 
3
ABSTRACT
This evaluation sought to understand BHC’s approach to engaging stakeholders, how stakeholders 
conceptualize resilience and how the BHC model can inform future urban project design. The SPACES 
team collected data via key informant interviews and surveys and conducted qualitative data analysis as 
well as social network analysis. This study found that BHC’s approach may have fostered a stronger, 
more resilient network, it may have helped increase stable partnerships and may have resulted in a more 
inclusive network. Additionally, it found that none of the stakeholders represented in interviews were 
aware of their own institutions’ definitions of resilience. Further, we found that stakeholders support 
resilience as a programming direction and want greater city government engagement with communities.
Recommendations specific to BHC include targeting engagements to specific organizations (e.g., 
organizations with 2 connections or less) such as identifying joint funding opportunities. BHC can also 
connect organizations that may have ongoing work that could synergize. BHC can also protect against 
overreliance on one actor by creating a structure that has different organizations managing and 
coordinating different cores of the project. Lessons gleaned that can be applied by USAID to future
programming similar to BHC include suggestions to conduct both pre-program and post-program 
network surveys/analyses and proactively monitor the network to identify potentially vulnerable areas of 
the network (e.g., organizations with one connection). When trying to strengthen the networks, it’s 
important to focus on diversifying stakeholders represented in the network and including community 
members.
4
TABLE OF CONTENTS
ACRONYMS ...........................................................................................................................................5
EXECUTIVE SUMMARY........................................................................................................................6
INTRODUCTION ..................................................................................................................................9
CORE FINDINGS & CONCLUSIONS ................................................................................................10
Evaluation Question 1. BHC’s Stakeholder Engagement Approach ...............................................10
Evaluation Question 2. BHC Stakeholder Conceptualizations of Resilience .................................20
Evaluation Question 3. How BHC’s approach can inform future urban project design................21
LIMITATIONS ......................................................................................................................................22
RECOMMENDATIONS ......................................................................................................................23
LESSONS LEARNED............................................................................................................................26
ANNEXES .............................................................................................................................................29
A – Purpose, audience and anticipated uses of the evaluation........................................................29
B – Evaluation questions.....................................................................................................................29
C – Activity context and background information............................................................................31
D – Methodology .................................................................................................................................32
E - Documents reviewed......................................................................................................................39
F – Data collection instruments..........................................................................................................41
G– Statement of Work ........................................................................................................................85
H – Summary information about SPACES team members.............................................................100
I – Signed Disclosures of Conflicts of Interests................................................................................104
5
ACRONYMS
ABHT Asia Bureau Health Team
BHC Building Health Cities program
CSO Civil society organization
EI Engaging Inquiry
INGO International Non-governmental Organization
JSI John Snow, Inc.
KII Key informant interview
MERLIN Monitoring, Evaluation, Research and Learning Innovations Program
MOU Memorandum of understanding
NGO Non-governmental organization
SNA Social network analysis
SPACES Strategic Program for Analyzing Complexity and Evaluating Systems
USAID United States Agency for International Development
6
EXECUTIVE SUMMARY
Purpose and evaluation questions
The purpose of this evaluation is to answer exploratory learning questions related to the Building Health 
Cities program (BHC). Specifically, the evaluation aims to understand: 
1.) How BHC’s approach to engaging stakeholders affected BHC implementation, 
2.) How stakeholders conceptualize resilience and determine how these conceptualizations affect 
how city leaders act, and 
3.) How BHC’s model can inform future urban project design.
Methodology
The SPACES team conducted both qualitative and quantitative data collection and analyses. Social 
network analysis involved developing pre and post BHC network diagrams for the 14 responding 
organizations as well as the overall network. We also developed EgoNets, which represent 2 degrees of 
connections to the focal actor, to be able to see how BHC affected the surrounding networks of each of 
the responding organizations.
Conclusions
Evaluation Question 1. BHC’s 
Stakeholder Engagement
Approach
BHC’s approach may have fostered a
stronger, more resilient network
We found that in general, BHC may 
have helped generate a stronger, more 
resilient network; the number of 
connections and network density 
doubled from before BHC launched to
the end of BHC, 5 years later. 
BHC’s approach may have helped 
increase stable partnerships
In particular, BHC may have helped to 
create more stability for each of the 
partners by converting one-sided 
relationships to two-way relationships.
BHC may have helped expand the 
stability and reach of each of the 
partners in the network. as the average 
number of connections per organization 
in the network increased. 
BHC’s approach may have resulted in a more inclusive network
Additionally, BHC may have created a more inclusive network; the number of organizations not
connected to the network went from 3 prior to BHC to 0. BHC may have reduced 2 specific 
KEY LIMITATION: The robustness of analysis and 
subsequent findings in some instances may be limited 
by the fact that we did not receive the following 
from the organization conducting the data collection 
(e.g., survey and key informant interviews) and 
qualitative data analysis: 
• specific processes followed for survey 
validation (e.g., survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, and 
how the analysis of qualitative data was 
validated and checked by our partner
Callout boxes like this one highlight where in the 
report this missing information has impacted the 
findings, conclusions, or recommendations.
7
vulnerabilities in the network, which were 2 organizations that were loosely connected to the rest of 
the network and at threat of becoming disconnected if anything happened to their one existing 
connection. The pre-BHC network had 2 sets of pendants (i.e., an element that is only connected to the 
network by one degree, or one edge/connection) and both are now better connected, and no longer 
connected by just one degree in the post-BHC network. Lastly, BHC may have helped reduce the 
overreliance on particular partners in the network as the number of brokers, or organizations that act 
as the sole connector for another organization to the rest of the network, decreased from 4 to 1.
BHC may have helped expand the individual reach of every organization. The EgoNets, or the portion of 
the network formed around a given organization, for each of the 14 organizations increased in size. BHC 
appears to have helped all but one of the 14 organizations strengthen their existing connections: 92% of 
these networks with direct connections to a focal organization had an increase in reciprocity. BHC 
appears to have had the biggest positive effect on private sector organizations’ networks, and the 
second biggest positive effect on government organizations.
Evaluation Question 2. BHC Stakeholder’s Conceptualizations of Resilience
None of the stakeholders represented in interviews were aware of their own institutions’ formal or 
working definitions of resilience as none of the stakeholders were able to provide their definitions in 
response to the direct question. 
Probing questions conveyed that stakeholders conceived of institutional approaches to resilience as 
bound up in budget allocations and planning for disaster response and management governed by national 
regulations, rather than within the purview of city government decision makers. However, further 
probing and survey results reveal openness to a broader conception of resilience. 
Evaluation Question 3. How BHC’s approach can inform future urban project design
Stakeholders support resilience as a programming direction and want greater city government 
engagement with communities
BHC helped foster citizen engagement, with 25% of survey respondents reporting that they “increased 
citizen feedback and empowerment structures” as a result of BHC. Stakeholders also want to see more 
engagement with the community members. Additionally, 83% of survey respondents supported 
resilience as being a focus of future programming. Stakeholders represented in key informant interviews 
were not aware of their own institutions’ formal or working definitions of resilience as none were able 
to provide definitions in response to the direct question.
Limitations
The conclusions are based on a small sample size because of low response rates. Additionally, there was 
insufficient data to make a conclusion about how the respondents define resilience or in some cases "a 
relationship" with another organization.
Recommendations
The following recommendations are specific to the BHC project.
BHC should conduct targeted engagements that would increase the connections of organizations with 
degrees of 2 or less, organizations connected in a single loops (e.g., a ring of single connections without 
any relationships connecting across the loop and with limited connections to the rest of the network), 
as well as non-government organizations. Targeted engagement strategies can involve ensuring these 
groups are included in events and participate in activities, conducting follow-up focus groups with 
8
systems mapping participants, identifying funding opportunities for organizations to jointly pursue, and 
spotlighting the work of these organizations in communications that reach across the network.
Additionally, BHC can consider strategies to further improve reciprocity among organizations. Example 
strategies include delineating specific roles for different organizations when working to achieve a specific 
goal or complete a task, and identifying and connecting organizations that may have ongoing work that 
could synergize. Related, BHC can protect against over relying on one actor by helping to create a 
structure that has different organizations managing and coordinating different cores of the project.
BHC should increase the diversity of stakeholders in the network, because currently, government 
organizations make up the lion’s share of the network actors. This may involve empowering 
organizations other than local government agencies to more frequently share information, engage in 
joint activities and build trust in order to foster a stronger network. Another way to increase diversity 
amongst BHC stakeholders could involve additional efforts to integrate community members and 
citizens’ voices into their activities and engagements (e.g., include citizen representatives in future 
participatory systems mapping workshops).
Lessons Learned
The following are lessons gleaned from this evaluation that can be applied by USAID to future
programming similar to BHC. One lesson learned is to conduct both pre-program and post-program 
network surveys/analyses. This will allow USAID to monitor the evolving strength and morphology of 
the network proactively and progressively, by periodically administering the survey and updating the 
network map/analyses. The proactive monitoring of the network will allow USAID to identify potential 
network brokers and in turn potentially vulnerable areas of the network such as pendants (e.g., 
organizations connected to the network by only one tie and tend to “dangle” off of the rest of the 
network) and establish connections to bolster these. When trying to strengthen networks, it’s 
important to focus on diversifying stakeholders represented in the network (e.g,, beyond government, 
private sector and NGOs), and including community members, as that will result in a more robust and 
resilient network. In addition to hosting events that bring together multiple stakeholders, a 
program/initiative could continue to identify opportunities for organizations to share responsibility in 
joint projects/efforts. Formalizing relationships through written agreements such as data sharing
agreements, charters, and memorandums of understanding is valuable for solidifying and strengthening 
connections throughout the network. Going forward, it will be important to encourage existing brokers 
within the system to foster interconnections between different stakeholders. To ensure continuity 
among stakeholders, even when staff and leadership turnover occurs, programs/initiatives can create a 
system that can collect, store, and readily transmit information on the history of each stakeholder’s 
engagement with the program/initiative. Finally, it will be valuable to clearly define what resilience means 
to the program and develop stakeholder’s understanding of the concept of resilience. 
9
INTRODUCTION
The purpose of this evaluation is to answer exploratory learning questions related to the Building Health 
Cities program (BHC). These questions investigate what worked, what did not work (and why), the 
conditions under which the BHC approach is likely to be effective, and what lessons on urban resilience 
exist for future programming.
The first line of inquiry of this evaluation is to analyze the impact of BHC’s model, approach and tools 
for stakeholder engagement. In answering this, USAID wishes to 1. understand the feasibility of achieving 
project goals via the stakeholder engagement tools, 2. support the ‘harmony’ that resulted from the 
well-functioning engagement structures, processes, and functions, and 3. determine how to better 
address the ‘disharmony’ resulting from obstacles to stakeholder cooperation. The second set of 
evaluation questions focus on how BHC defines the concept of resilience and what impact that 
conceptualization has had, as well as how city leaders better responded to community needs as a result 
of BHC programming. In analyzing this, USAID looks to 1. understand if and how the concept of 
resilience has evolved and differed across the implementing cities, 2. know whether and the extent to 
which the technical focus of BHC’s approach was integrated into this conceptualization of resilience, 3. 
determine to what extent the conceptualizations of resilience affected activity design and 
implementation, and if these activities contributed to the resiliency of the city, and 4. understand how 
the city leaders were enabled or empowered to improve their preparation and response to future 
shocks and how BHC supported these improvements. The last line of inquiry, the third evaluation 
question, explores how BHC’s cross-sectoral models in the different country contexts can be applied 
and sustained in the future. For this last question, USAID wishes to 1. better understand BHC’s 
operating context to determine the applicability of the cross-sectoral model for other country contexts, 
2. equip future projects with the tools to learn, adapt, and improve quickly and effectively to improve 
chances of sustainability, and 3. understand the validity of the theory of change. 
The resources available for this evaluation necessitated focusing on only one city, thus the following is a
summary of some of the major findings for the Building Healthy Cities (BHC) program in Makassar, 
Indonesia. The data for the findings reported here came from responses to a survey sent initially to 56
organizations identified from the BHC Engagement Tracker. Of those sent the survey, 24 provided 
usable responses. However, not all organizations who responded completed the entire survey. Of the 
respondents, 14 completed the social network portion of the survey (see Annex D – Methodology).
Unfortunately, this evaluation was not able to inform the follow-on activity because of delays in data 
collection due to COVID and further delays in trying to get information from a partner that left the 
consortium before this analysis was finalized.
10
CORE FINDINGS & CONCLUSIONS
Evaluation Question 1. BHC’s 
Stakeholder Engagement 
Approach 
Overall Network Analysis
Many of the 14 organizations that completed 
the social network portion of the survey did 
mention connections with organizations that 
did not respond. Since we could not confirm 
the connections that non-respondents had to 
other organizations, our initial network 
analysis was limited to the 14 respondents. 
We developed pre and post BHC network 
diagrams for these 14 responding 
organizations as well as the overall network. 
The pre-BHC network diagrams show 
connections that were reported, in the 
survey, to have existed prior to the start of 
the BHC program. The post-BHC network 
diagrams show connections that existed 
prior to the BHC program as well as 
connections that were reported to have formed 
as a result of BHC. Some of the stakeholders reported connections but were uncertain when these 
relationships formed. In our analysis, we mapped the connections of unknown origin in both the pre￾and post-BHC diagrams to show the different possibilities (relationships existing prior to BHC and as a 
result of BHC). Figures 2 and 4 show the relationships of uncertain origin in the post-BHC diagrams.
1. In general, Building Healthy Cities (BHC) may have helped generate a stronger, more 
resilient network with a 2-fold increase in the total number of connections and network 
density.
Pictured in Figures 1 and 2 are the network diagrams for the 14 organizations that responded to the 
survey, demonstrating their connections to other organizations prior to BHC as well as those 
connections that may have been formed as a result of BHC. In each diagram, each node is depicted by a 
circle, representing an organization in the network, and each line, i.e. edge, represents a connection 
between the nodes. After BHC, the network of 14 survey respondent organizations expanded by 17 
additional connections (11 when comparing to relationships formed prior to BHC and relationships that 
were reported to be unknown if they existed prior to or as a result of BHC). An increase in 
connections often means that organizations are exposed to more diverse information. As a result of 
these added connections, the network density (i.e., the ratio of the number of actual connections to the 
number of potential connections) went from 0.08 (0.12 if including the relationships of unknown origin) 
to 0.18. When considering the broader network beyond the 14 organizations that responded (including 
respondents’ reporting connections to organizations who did not fill out the survey themselves), the 
total connections went from 81 to 171 (a 37% increase) if including connections of unknown origin. 
Network density increased from 0.03 to 0.06 (i.e., by 50%).
KEY LIMITATION: The robustness of analysis and 
subsequent findings related to BHC fostering 
citizen engagement may be limited by the fact that 
we did not receive the following from the 
organization conducting the data collection (e.g., 
survey and key informant interviews) and 
qualitative data analysis: 
• specific processes followed for survey 
validation (e.g., survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, 
and how the analysis of qualitative data 
was validated and checked by our partner.
11
2. BHC may have helped create more stability for each of the partners by converting one￾sided relationships to two-way relationships as the network reciprocity increased from 
0 to 0.14.
Reciprocity measures the likelihood that a reported connection from one organization to another is 
reciprocated. Figure 2 shows that as a result of BHC, the network reported four confirmed 
relationships, where both organizations reported having a relationship with the other one. Three of the 
four mutual relationships were confirmed by the City Planning and Development Agency. This suggests 
that post-BHC, the organizations may have seen a stronger or more formal partnership with the agency 
than they perceived prior to BHC. Through the collaborations, work, and joint activities that BHC 
hosted, the organizations may have established a more reciprocal relationship with the City Planning and 
Development Agency. 
Figure 1: Pre-BHC Network Diagram of the 14 Organizations that Responded to the Survey
12
Figure 2: Post-BHC Network Diagram of the 14 Organizations that Responded to the Survey
Figure 3: Pre-BHC Network Diagram of All Organizations
13
Figure 4: Post-BHC Network Diagram of All Organizations
Reciprocity is not only important for verifying and validating the relationships in a network, but also for 
determining how different organizations view one another. If a relationship is unreciprocated, this could 
be problematic. For example, an organization could think they have buy-in from multiple organizations, 
but in reality, they may not have the support and partnership they expect if there is no reciprocity. In 
general, for a relationship to be considered strong, it requires reciprocity. The post-BHC network’s 
increased reciprocity points to a strengthening and stabilizing of relationships as a result of BHC. 
3. BHC may have helped expand the stability and reach of each of the partners in the 
network as the average number of connections across all organizations in the network
increased from 2.14 to 4.57.
The average degree calculates the average number of connections across all elements in the network. In 
Figure 1, the network diagram includes only the connections listed as formed “prior to BHC”, and 
shows the average degree for this diagram as 2.14. When including the connections listed as “not sure if 
connections existed before or as a result of BHC”, the average degree was 3. In the post-BHC network 
diagram, shown in Figure 2, the average degree was 4.57, indicating an increase of 1.57 in average degree 
as a result of BHC. 
14
BHC contributed to increasing the average number of connections above three, which is a critical 
threshold for significantly increasing stability and decreasing vulnerability in a network. With only two 
connections, an organization is vulnerable to being cut out from the network or isolated if just one 
organization disappears from the network. However, having an average of three or four connections 
across elements means that multiple organizations have to disappear in order for an organization to be 
isolated or disconnected from the network. 
When looking at the broader network that includes both survey respondents and non-respondents to 
the survey, BHC may have more than doubled the average degree from 2.95 to 6.22 from pre-BHC to 
post-BHC, and by a relative 37% if including connections that are of uncertain origin (i.e. whether or not 
they existed prior to BHC) in the pre-BHC diagram, with the average degree of 4.55 increasing to 6.22 
as a result of BHC. This points to BHC contributing to an increase in stability across the network, 
where organizations are less likely to be greatly impacted by negative changes to the network.
4. BHC may have led to a more inclusive network as the number of organizations not 
connected to the network went from 3 prior to BHC to 0.
Disconnected organizations refer to organizations that do not have any connections to any other 
element or organization in the network. As seen in Figure 1, the network diagram of relationships prior 
to BHC, DHIS2 Indonesia, Makassar Ecobrick, and the Institute for Public Policy are disconnected 
organizations. Of the three organizations that are not connected, two of them, the Institute for Public 
Policy and Makassar Ecobrick, were local non-governmental organizations (NGOs)/CSOs/community 
groups. The third organization that was disconnected prior to BHC, DHIS2, was from the private 
sector. None of the organizations disconnected from the network prior to BHC were local or national 
government entities, which supports findings that suggest that the network was centered around local 
government, specifically the City Planning and Development Agency, prior to BHC. 
In the post-BHC network diagram, shown in Figure 2, you can see that the previously disconnected 
organizations are now connected to the rest of the network. While it is possible that BHC played a role 
in integrating these organizations, a limitation of the survey design and data collection worth 
acknowledging is that one of the organizations, DHIS2, was not included in the list of organizations that 
other survey respondents had the option of selecting with whom they have a relationship. 
The greater inclusivity of the network generated by BHC, however, was confirmed in the broader 
network diagram, which includes the 14 responding organizations as well as those organizations with 
which the survey respondents indicated connections, even if those organizations did not fill out the 
survey, shown in Figure 4. In the pre-BHC network, nine organizations were isolated or disconnected 
(two were disconnected when including connections that are uncertain of whether they existed prior to 
BHC) and there were zero disconnected or isolated organizations in the post-BHC diagram. This finding 
suggests that BHC contributed to creating a more inclusive and better connected network. 
5. BHC may have helped reduce two specific vulnerabilities in the network as the pre￾BHC network had 2 sets of pendants and both are eliminated in the post-BHC network. 
A pendant vertex is an element that is only connected to the network by one degree, or one 
edge/connection. When looking at Figure 1, the network diagram prior to BHC, Cita Sehat Clinic is 
shown to be a pendant. Pendants are very vulnerable because the elimination of any of the connections 
results in completely removing an element or set of elements from the network. With the existence of 
pendants, the network is vulnerable to isolation and is also less likely to receive information shared 
within it, as the information can only be shared through one pathway.
15
Pendants in a network may also indicate the existence of organizations whose efforts are siloed from the 
rest of the network, without a sense of what other organizations are doing or how their efforts may 
synergize. In the diagram showing relationships prior to BHC, Figure 1, Cita Sehat Clinic, Indonesian 
Breastfeeding Mother’s Association, and the Data and Information Center Ministry of Health are all 
pendants. A new pendant formed post-BHC as well, when connecting Makassar EcoBrick, a previously 
disconnected element, to the network. Because of the vulnerability inherent to the existence of 
pendants in a network, BHC could potentially play a role in proactively identifying pendant organizations 
and specifically targeting them in their outreach efforts when hosting activities and events in the future. 
This proactive engagement of pendants would help to establish and reinforce connections between the 
pendant and other organizations. 
6. BHC may have helped reduce the overreliance on particular partners in the network as 
the number of brokers decreased from 4 to 1.
A broker is an actor that connects other actors who would otherwise be unconnected. Thus, brokers 
serve important roles. Networks that rely too heavily on single brokers are vulnerable because if that 
broker were to no longer serve its role or exist then major gaps/holes would result. Brokers can also 
have outsized roles in controlling information flow, which could lead to misinformation or information 
gaps if the broker doesn’t serve its role properly. Specific types of brokers include gatekeepers (where 
the broker connects one actor of its own group to another actor of a different group), liaisons (where 
the broker connects two actors who are of two different groups different from the broker), and 
coordinators (where all three actors belong to the same group). The pre-BHC network (shown in Figure 
1) had four brokers (Rumah Zakat as both a gatekeeper and a coordinator in relation to Cita Sehat 
Clinic; the City Planning and Development Agency as both a gatekeeper and a liaison between JALIN and 
the Indonesian Breastfeeding Mother’s Association pendant; JALIN as a liaison broker between the City 
Planning and Development Agency and the Indonesian Breastfeeding Mother’s Association; and lastly, 
the Public Health Center - Barrang Lompo is both a gatekeeper and liaison to the Data and Information 
Center of the Ministry of Health.) The post-BHC network (Figure 2) had only one broker (the Indonesia 
National Slum Upgrading Project serving as a coordinator to Makassar Ecobrick). We also acknowledge 
that the pre-BHC network including relationships of unknown origin does not have any brokers.
EgoNet Analysis
An "ego-network" is a network for the set of actors who are directly connected to a focal actor, termed 
ego. Egonets allow us to focus in on one organization to understand that organization’s network of 
direct connections as well as how those direct connections are connected to each other, providing a 
more focused view of a portion of the BHC network for a given organization. We developed EgoNets 
for each of the 14 responding organizations and represented 2 degrees of connections to be able to see 
how BHC affected the surrounding networks of each of the responding organizations. EgoNets are 
valuable because they help us understand the opportunities and constraints facing specific actors, which 
helps us understand the role of BHC in the “local” social structures of specific organizations and groups 
of organizations.1
1 Hanneman, Robert A., and Mark Riddle. "Introduction to social network methods." (2005).
16
1. BHC may have helped expand the individual reach of every organization as the 
EgoNets for each of the 14 organizations increased in size.
Each of the 14 EgoNets showed an increase in the number of connections from the relationships 
reported prior to BHC (when also including the relationships of unknown origin) and post BHC, ranging 
from 1 to 79 additional connections, with a median increase in 34 connections. Additionally, all of the 
egos increased their reach from the relationships reported prior to BHC (when also including the 
relationships of unknown origin) and post BHC, ranging from an absolute 2.9% to 73.1% increase. On 
average, across the EgoNets, reach increased by a median of 37%, with 71% of EgoNets showing an 
increase in reach between 0 and 50% and 39% experiencing an increase between 50% and 73.1%. 
Additionally, 85% of EgoNets demonstrated an increase in the average degree with 71.4% of those that 
reported an increase adding an average degree between 0 and 2, 14.3% increasing their average degree 
between 3 and 4, 7% showing no change in average degree and 7% showing a decrease. Average degree 
increased by a median of 0.98. The degrees, specific to the ego in the EgoNets also shifted from pre to 
post BHC networks. 71% of the egos in each of the EgoNets had an increase in degree resulting from 
BHC. The median increase in degrees for the egos across the EgoNets was 2. 42% of egos experienced 
a change in indegree, and 50% of egos experienced a change in outdegree. On average across all of the 
EgoNets, the indegree increased by a mean of 0.8 and median 0, and outdegree increased by a mean of 
3.2, and median 0.5. 21.4% of egos experienced an increase in both indegree and outdegree (City 
Planning and Development Agency, Lembaga Institute for Public Policy, Indonesia National Slum 
Upgrading project). Interestingly, 2 of the organizations, DHIS 2 and Makassar EcoBrick started with 0 
degrees in their EgoNet prior to BHC (excluding relationships of unknown origin) and after BHC, DHIS 
2 had 88 connections and Makassar EcoBrick had 31 connections. These additional connections across 
the EgoNets contribute to more stable networks that improve the ease of information sharing and 
therefore, increase the influence of the ego within their network. 
2. BHC may have helped all but one of the 14 organizations strengthen their existing 
connections as 92% of the EgoNets had an increase in reciprocity. 
Prior to BHC, all EgoNets had a reciprocity of 0. The increases in reciprocity ranged from 0.03 to 0.05 
with 57.1% of all EgoNets seeing an increase of 0.03, 28.6% seeing an increase of 0.04 and 7.1% seeing an 
increase of 0.05. The only organization that did not have an increase in reciprocity was the Indonesian 
Breastfeeding Mother’s Association.
3. BHC may have had the biggest effect on the single private sector organization’s 
network with greater than the median increases in connections (>2.5 times the overall 
median increase), reach (>1.8 times greater), and average degree (> 3 times greater).
One private sector organization, DHIS2 experienced the greatest increase in connections, 79, compared 
to the median increase in connections for all organizations at 34. For comparison, the median increase in 
connections for city government organizations was 50. Additionally, the private sector organization 
experienced an increase in reach amounting to 0.685, which was greater than the overall median 
increase in reach, at 0.376 (when including the connections of unknown origin in the pre-BHC diagram).
DHIS2 as the sole private sector organization also had the greatest change in average degree at 3.43, 
compared to city government organizations, which had a median change of 1.24, national government 
organizations which had a median change of 0.98, and local NGOs which had a median change of 0.96. 
The median change in average degree for all organizations was 0.98. 
17
These changes were also evident by the increases in closeness centrality and degree for the ego (when 
including connections of unknown origin in the pre-BHC diagram). DIHS2 had the largest increase in 
closeness centrality, 0.545, which is greater than the median increase in closeness centrality at 0.207. 
DIHS2 also had an increase in degree of 11, compared to the local NGO group which had a median 
increase in degree of 1, and city government organizations which had a median increase in degree of 3. 
The median increase in degree for all organizations was 2.
4. BHC may have had the second biggest effect on government organizations, which 
increased connections up to 1.5 times more than the overall median increase, with city 
government organizations increasing their average degree over 1.3 times greater than 
the overall median.
The second largest changes within the network were among government organizations, which had a 
greater median increase in connections than the overall median increase of 34 connections. City 
government organizations had a median increase of 50 connections and national government 
organizations had a median increase of 36.5 connections. The second largest increase in number of 
connections was with the City Planning and Development Agency, which had 65 new connections. City 
governments also had a median change in average degree of 1.24, greater than the median change in 
average degree for all organizations, 0.98 (when including relationships of unknown origin in the pre￾BHC diagram). 
These changes within the network for government organizations are also demonstrated by the increase 
in degree and reach for these organizations. City government organizations had a median increase in 
degree of 3 and national government organizations had a median increase in degree of 2, greater than 
the median increase in degree for local NGOs (1) and increase in degree for the donor program (0). 
The median increase in degree for all organizations was 2. City government organizations had a greater 
median increase in reach, 0.395, compared to the median increase in reach across all organizations, 
0.376 (when including relationships of unknown origin in the pre-BHC diagram). 
Two of the three biggest resulting EgoNets post-BHC are local government departments, with City 
Planning and Development Agency (159) and Social Department (147).
Deeper Analysis of Relationship Changes
In addition to understanding how BHC affected the connections in the network, we also conducted
deeper analysis on the nature of the 32 relationships within the respondent network (connections 
between the 14 responding organizations). This includes aspects such as whether and how the 
relationship changed from informal to formal, the change in trust in the relationship, as well as the 
perception of change in the frequency, quality and quantity of information sharing, joint activities, social 
connections and networking as a result of BHC programming and activities. 
1. Over 1/3 of connections between organizations under BHC increased trust; this was 
not directly related to transitioning from an informal to formal relationship. 
When considering the nature of the relationships reported among organizations, 6% of the relationships 
were reported to have changed from informal to formal, 50% stayed formal, 19% stayed informal, and 
25% were reported as unsure of the nature of the relationship.
When reporting on trust in the relationships, 38% of the connections reported an increase in trust (13% 
increased from low to medium, 6% increased from low to high, 19% increased from medium to high). 
18
28% of the connections maintained the same level of trust (6% stayed low, 3% stayed medium, and 19% 
stayed high). Lastly, 35% of the connections were reported as being unsure of the nature of trust in the 
relationships with the other organizations. Nine out of the twelve organizations that others reported an 
increase in trust with were government agencies.
2. BHC may have contributed to a more responsive and adaptable network as there was 
improved information sharing among the network, with connections reporting 41% 
increased quantity of information sharing, 28% increased frequency of information 
sharing, and 38% increased quality of information sharing.
41% of the connections reported an increase in the quantity of information sharing (28% increased from 
low to medium, 13% increased from medium to high). 28% of connections maintained the same level in 
the quantity of information sharing (6% stayed low, 9% stayed medium, and 13% stayed high). Finally, 
31% of the connections did not know the quantity of information sharing that took place. Nine out of 
the thirteen organizations that others reported an increase in quantity of information sharing with were 
government agencies, and six out of the thirteen reported increases in the quantity of information 
sharing were between local NGOs and government.
For frequency of information sharing, 28% of the connections reported an increase (9% increased from 
low to medium, 9% increased from low to high, and 9% increased from medium to high). 38% of the 
connections maintained the same frequency of information sharing (16% stayed medium, and 22% stayed 
high). 34% of the connections did not know the nature of the frequency of information sharing. 
Finally, for quality of information sharing, 38% of connections reported an increase (9% increased from 
low to high, 13% increased from low to medium, and 16% increased from medium to high), 28% stayed 
the same in terms of quality of information sharing (16% stayed high, 3% stayed low, 9% stayed medium), 
and lastly, 31% did not know about the nature of the quality of their organization’s information sharing. 
3. BHC may have contributed to a more united network that can capitalize on existing 
synergies as they saw increases in number, frequency and quality of joint activities, a 
positive improvement to the network’s strength, with reports of 25% increase in the 
number of joint activities, and 31% increase in the frequency and 25% increase in the
quality of these joint activities.
For the quantity of joint activities, 25% reported an increase in the number of joint activities (13% 
increased from low to medium, 13% increased from medium to high), while 41% stayed the same in 
amount of joint activities (22% stayed low in quantity, 3% stayed medium, and 16% stayed high). 
Additionally, 31% did not know if there was a change in quantity of joint activities. Seven out of the eight 
organizations that others reported an increase in joint activities with were government entities.
In terms of frequency of joint activities, 31% reported an increase in the frequency of joint activities(19% 
increased from low to medium, 13% increased from medium to high), 38% stayed the same in frequency 
of joint activities (16% stayed high in frequency, 16% stayed low, and 6% stayed medium), whereas 31% 
did not know if there was a change in the frequency of their organization’s joint activities.
When looking at the quality of joint activities undertaken with other organizations, 25% of the 
connections reported an increase in the quality of joint activities (12.5% reported an increase in quality 
from low to medium, and 12.5% reported an increase in quality from medium to high), 41% reported the 
quality of joint activities remained the same (3% reporting the quality remaining low, 19% stayed 
19
medium, and 19% remained high ), and lastly, 34% of reported connections did not know the change in 
quality of their joint activities with other organizations in the network. 
4. BHC may have contributed to a more interactive and harmonious network with not 
only the number of social connections increasing (among 47%), but also the frequency 
(among 41%) and quality of social connections improving. 
When asked about the quantity of social connections, 47% reported an increase in the quantity of social 
connections (25% increased from low to medium, 3% increased from low to high, and 19% increased 
from medium to high). 19% maintained the same quantity of social connections (6% of connections 
stayed medium, and 13% remained high), whereas 34% reported not knowing the nature of the change 
in the quantity of social connections.
Considering the frequency of social connections, 41% reported an increase in the frequency of social 
connections (19% increased from low to medium, 9% increased from low to high, and 13% increased 
from medium to high). 28% of reported connections maintained the same frequency of social 
connections (6% remained low, 9% remained medium, and 13% remained high), and 31% of reported 
connections did not know the nature of change in the frequency of social connections.
Survey respondents were also asked about the quality of social connections, where 41% reported an 
increase in the quality of their social connections (9% increasing from low to medium, 9% increasing 
from low to high, and 22% increasing from medium to high). 25% of reported connections maintained 
social connection quality levels (3% reported the quality remained low, 9% reported staying medium, and 
for 13% reported the quality remaining high). 34% of reported connections did not know the nature of 
change to the quality of their social connections.
5. BHC’s approach positively impacted government engagement the most, elevating their 
role in the network, with government entities experiencing the greatest increases in all 
of these indicators, with notable increases between government and local 
NGOs/CSO/community groups.
Nine out of thirteen organizations that others reported an increase in the quantity of information 
sharing with were government agencies. Six of the thirteen organizations that reported increases in the 
quantity of information sharing were between local NGOs and government (5/13 for local government, 
3/13 government to government, and 3/13 were NGO to NGO.
Seven out of eight that others reported an increase in joint activities were with government, with eight 
out of eight of these connections including at least one government organization (9/12 organizations that 
others reported an increases in quality of information sharing were with government agencies, and 
10/12 that showed an increase in quality of information sharing included a government agency). 
6. BHC helped foster engagement between different stakeholder types, contributing to 
the diversity in the network.
Ten of fifteen that reported increases in social connections were between organizations of different 
types. Six out of eight organizations that reported an increase in the quantity of joint activities were 
between organizations of different types.
20
Evaluation Question 2. BHC 
Stakeholder 
Conceptualizations of 
Resilience
Key Informant Interviews 
1. None of the stakeholders 
represented in interviews were
aware of their own institutions’ 
formal or working definitions of 
resilience as none of the 
stakeholders asked were able to 
provide their definitions in 
response to the direct question. 
Probing questions conveyed that 
stakeholders conceived of institutional 
approaches to resilience as bound up in 
budget allocations and planning for disaster 
response and management governed by 
national regulations, rather than within the 
purview of city government decision makers. 
However, further probing and survey results reveal openness to a broader conception of resilience. As 
one informant noted:
"...Actually there already are resilience regulations, so for example, [when] I said earlier that for financing our 
[annual regional government] budget - whether it's a matter of economic shocks and so on - they are there 
already, there are already regulations, there is already even Government Regulation No. 12 regarding the 
management of regional finances. It is already extremely clear, the Ministry of Home Affairs is also 
extremely clear. What’s left then is for us to put down in the Mayor's Regulation how to take advantage of 
those indirect funds.” (Key Government Department)
KEY LIMITATION: The robustness of analysis and 
subsequent findings related to how stakeholders 
conceptualize resilience as well as the impact of 
that conceptualization may be limited by the fact 
that we did not receive the following from the 
organization conducting the data collection (e.g., 
survey and key informant interviews) and 
qualitative data analysis: 
• specific processes followed for survey 
validation (e.g., survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, 
and how the analysis of qualitative data 
was validated and checked by our partner.
21
Evaluation Question 3. How BHC’s approach can inform future urban 
project design
Additional Survey Results 
1. While BHC helped foster 
citizen engagement, with 25% 
of survey respondents (6 out of 
24) reporting that they 
“increased citizen feedback 
and empowerment structures” 
as a result of BHC, 
stakeholders want to see more 
engagement with the 
community members.
Increasing citizen feedback and 
empowerment structures was on the 
lower half of most commonly reported 
benefits realized through BHC (6th 
highest out of 10). It was, however, 
listed in the top half of the list of most 
commonly reported priorities (4th 
highest out of 10). 
Respondents emphasized finding and 
improving approaches to engaging and including community members. They provided the following 
suggestions:
- “Approaching communities for involvement, so that they are more aware of BHC activities”
- “An approach more to win over the hearts of the community, especially mothers and school-age 
children, so that at an early age they understand more about the impact of pollution on the 
environment”
- “Data collection on community needs related to environmental sanitation must be massively 
increased”
- “Community training”
- “Integrated data management that can be accessed by the general public.” 
2. Resilience is a promising programming direction among stakeholders; 83% of survey 
respondents supported resilience as being a focus of future programming (20 out of 
24). 
When asked “What future impact(s) do you hope Makassar City gains from future engagement with 
programs like BHC?,” more selected “increases resilience/capacity to respond effectively to shocks and 
stressors (such as natural disasters, economic shocks, food security, health pandemics such as currently 
in COVID, etc.” than other options.
KEY LIMITATION: The robustness of analysis and 
subsequent findings related to how BHC’s model 
can inform future urban project design may be 
limited by the fact that we did not receive the 
following from the organization conducting the 
data collection (e.g., survey and key informant 
interviews) and qualitative data analysis: 
• specific processes followed for survey 
validation (e.g. survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, 
and how the analysis of qualitative data 
was validated and checked by our partner.
22
LIMITATIONS
We requested the following information from the partner that administered the survey and KIIs: specific 
processes involved in the validation process of the survey instrument and KII tool, final list of 
organizations that received the survey instrument and details around their method for selection, as well 
as the dissemination process, including how recipients were asked to share the survey to colleagues, 
what snowball sampling occurred, and what additional follow up was conducted. We also requested 
details on how qualitative data was coded, who performed the qualitative analysis, and how the analysis 
of qualitative data was validated and checked by our partner. We are also missing information about 
specifically when the survey instrument changed and the potential impact this mid-course change may 
have on results. However, the partner did not provide the requested information.
1. The overall survey response rate for the social network survey was 25.5% with different 
response rates among various sectors/groups. Therefore, we were unable to make 
robust conclusions about all of the organizations in the BHC network.
The response rate was 12.5% among city government organizations, 20% among donor programs, 63.6% 
among local NGOs, 66.7% among national government organizations, and 100% with the one private 
sector organization that was contacted. No International NGOs (INGOs), universities, state owned 
enterprises, or hospitals responded to the survey. This raises the possibility that results were skewed 
towards particular types of organizations and not fully representative. However, there was no clear 
indication of bias or skew among the respondents. 
2. For 26% of the connections, the respondents indicated: “My organization has a 
relationship with this organization/government department, but I do not know the 
nature of the relationship (whether it was before or as a result of BHC)”. Therefore, 
we were unable to make robust conclusions about BHC’s role in facilitating these 
relationships.
Forcing survey respondents to research and identify whether the relationship was established before or 
due to BHC would improve the validity of the end analysis. We performed additional analyses with 
these responses when determining our pre and post networks in our best attempt to address this 
limitation. 
3. Since survey respondents were asked to identify up to 20 organizations that they have 
relationships with, there is the possibility that some may not have indicated all of the 
organizations to which they were connected. This means that there may be additional 
organizations that are important to the resilience of the network that do not appear in 
this analysis.
Having an upper bound on the number of organizations could lead to undercounts of the true number 
of relationships. However, all organizations listed connections to fewer than 20 organizations, making it 
less likely that any were inhibited by the limit. 
4. Seven of the 24 survey respondents did not complete the social network portions of the 
survey. Therefore, the social networks were constructed using incomplete data and the 
actual network likely contains more individuals/organizations than are currently 
represented in our findings.
23
This could have resulted in response bias. However, there was no clear systematic skew of non￾responders (i.e., the non-responders had similar distributions of characteristics as the responders.) 
 
5. Respondents may have interpreted the definition of ‘a relationship with another 
organization’ differently. This means that nature of the relationships (e.g., the 
formality, and extent to which they engage) reported may vary.
Respondents may have different ideas about what constitutes a relationship or interaction with other 
organizations, as there were no constraints on these definitions. This may contribute to the overall low 
reciprocity in the network, since organizations on either side of a connection may have differing ideas of 
whether they have a relationship. 
RECOMMENDATIONS
The following recommendations are specific to 
the BHC project.
1. While network connections increased
BHC should consider additional 
opportunities (e.g., focus groups with 
systems mapping participants, 
identifying funding opportunities for 
organizations to jointly pursue) to 
further increase connections and 
network density. 
While connections and density both doubled in 
the network of responding organizations, the 
density is still only 6%, which represents a fairly 
sparse network, and therefore, BHC should 
target a higher overall network density, ideally 
over 50%, the threshold over which networks 
have significant protective reciprocity and 
redundancy. Increasing the connections and 
density can increase the speed and extent to which 
information is shared and diffused across the network. 
2. BHC should conduct targeted engagements that would increase the connections of 
partners with degrees of 2 or less. 
Having fewer than two connections leaves an organization extremely vulnerable since it means that the 
organization becomes heavily reliant on a single other organization. Having two connections offers at 
least one other connection preventing isolation. However, three or more connections would offer at 
least some level of protective redundancy. 
 
3. While there was an increase in reciprocity, BHC should consider strategies (e.g., 
delineate specific roles for different organizations when working to achieve a specific 
goal/complete a task, identify and connect organizations that may have ongoing work 
that could synergize) to further improve reciprocity among organizations. 
KEY LIMITATION: The robustness of 
recommendations may be limited by the fact that 
we did not receive the following from the 
organization conducting the data collection (e.g., 
survey and key informant interviews) and 
qualitative data analysis: 
• specific processes followed for survey 
validation (e.g., survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, 
and how the analysis of qualitative data 
was validated and checked by our partner.
24
While reciprocity did increase between the pre and post BHC networks, the overall reciprocity is low 
(14% in the network of responding organizations). To foster reciprocity in the network, BHC can 
identify opportunities for partners to share responsibility in joint efforts, thereby creating 
interdependent relationships. 
4. BHC should invest in identifying other potential pendants in the network and including 
them in ongoing events and engagement opportunities (e.g., ensure Makassar EcoBrick 
is invited to events with other organizations besides the Indonesia National Slum 
Upgrading Project). 
A new pendant was formed when Makassar Ecobrick, a previously disconnected element, was added to 
the network post-BHC. The Indonesia National Slum Upgrading Project is the only organization that 
Makassar Ecobrick is now connected to. 
5. BHC should invest in identifying single loops (e.g., a ring of single connections without 
any relationships connecting across the loop and with limited connections to the rest of 
the network) and reinforcing them with cross-connections (e.g., targeting these 
organizations with strategies to engage with other organizations across the network)
to reduce vulnerabilities to breakage resulting in isolation. 
In the post-BHC network of responding organizations, there are multiple loops (between Cita Sehat, 
JALIN and Indonesian Mother’s Breastfeeding Association as well as Rumah Zakat, Sekolah Tanpah and 
Indonesia National Slum Upgrading Project) which tend to be weaker aspects of networks, and more 
prone to breaking resulting in further disconnections from the network. BHC could focus on identifying 
organizations within these loops and creating opportunities for cross-collaboration in order to eliminate 
these vulnerabilities.
6. BHC should encourage the use of other means (e.g., historical records) to clarify when 
some of the connections were formed. 
For 26% of connections, organizations reported that they were unsure when connections with other 
organizations formed (pre- or post- BHC). This suggests that members of these organizations were 
unaware of the specific effect of BHC on their relationships. BHC could work to clarify their goals and 
accomplishments with these organizations. 
7. BHC should help create a structure that has different organizations managing and 
coordinating different cores of the project to protect against over relying on one actor.
Currently, the highest number of connections go through the City Planning and Development Agency 
(Bappeda), creating a more centralized network could be problematic if something were to disrupt their 
work. BHC could help create a structure that has different organizations managing and coordinating 
different project elements to protect against over relying on one actor, and increase connections among 
organizations currently only connected through Bappeda. 
8. BHC should target engaging non-government organizations, (e.g., spotlight their work
in communication that reaches a diverse set of organizations, ensure NGOs are 
included in events and participate in activities) such as local NGO/CSO/community 
groups, universities, state-owned enterprises, private sector organizations, hospitals 
and donor programs in order to increase their number of connections.
25
Most organizations that experienced increases in quality and quantity of information sharing, joint 
activities, and trust were city or national governments. After DHIS2, government organizations also 
experienced the biggest increases in the number of connections. BHC could work to foster connections 
and the quality of connections among non-government organizations, including local NGOs and other 
types of organizations of which none responded to the survey (universities, hospitals, INGOs). 
9. Currently, government organizations make up the lion’s share of the network actors 
(27 of the 55 actors), indicating an opportunity to increase the diversity of stakeholders 
in the network. 
The local NGOs/CSOs/community group organizations make up 24% of the network, and the private 
sector organization represents just 2% of the overall network. The network would be stronger, more 
resilient, and the information sharing would be of potentially higher quality if the network was made up 
of a diverse set of stakeholders more representative of the ecosystem the network exists within. 
10. BHC should continue to integrate community members and citizens’ voices into BHC 
activities and engagements (e.g., include citizen representatives in future participatory
systems mapping workshops). 
While it is critical to get the range of different types of organizations, there are many valuable 
perspectives among community members who often have firsthand knowledge of their community 
needs and may be motivated to orchestrate community-led responses supported by the BHC network. 
Continuing to offer opportunities such as the Musrengbang and also considering inviting community 
members to additional systems mapping workshops could foster these key connections. 
11. BHC should consider ways to formalize relationships and existing connections (e.g., 
establish memorandums of understanding [MOUs], identify funding opportunities for 
joint collaboration) in order to increase reliability in information sharing and stability 
across the network.
Post-BHC, 19% of the relationships stayed informal, whereas only 6% of the relationships changed from 
informal to formal. By formalizing the relationships in the network, the connections are strengthened 
and there’s a greater amount of confidence in the reliability of information sharing, improving stability 
across the network. 
12. BHC should help empower organizations other than local government agencies to 
more frequently share information, engage in joint activities and build trust to foster a 
stronger network. 
Among the stakeholders other than government entities, there is additional room for growth under the 
nature of their relationships with each other. Only 31% reported an increase in the quantity of 
information sharing and 12.5% reported an increase in joint activities outside of government entities. 
Part of this could be related to #11 in the sense that these relationships remain informal. Continued 
investments to build these relationships can include more frequent opportunities for interaction, data 
sharing, and strategizing. 
26
13. BHC should consider additional efforts to better understand their role in engaging
private sector organizations (other than DHIS2 Indonesia) to see if there are other 
lessons to be learned that can be applied to the rest of the network. 
DHIS2 Indonesia is the only private sector organization included in the Social Network Analysis (SNA) 
and it experienced large benefits overall thanks to BHC. Additional analysis should be done to see if 
DHIS2 Indonesia was unique in that manner or if there were other private sector players that are 
underrepresented in the network. 
LESSONS LEARNED
The following are lessons gleaned from this 
evaluation that can be applied by USAID to 
future programming similar to BHC. 
1. Conduct both pre-program and 
post-program network 
surveys/analyses, make it clear 
that these are occurring, and 
compare the findings. 
The social network analyses conducted 
with the data from the single point-in-time 
survey did show important changes in the 
connections among the organizations. 
However, several organizations did report 
that they were unsure of relationship origin 
or unsure of the nature of changes that 
occurred in these relationships. A survey 
conducted before the launch of the 
program would have offered a clearer set 
of comparison data to improve the 
consistency and completeness of the 
program evaluation.
2. Proactively and progressively monitor the evolving strength and morphology of the 
network, by periodically administering the survey and updating the network 
map/analyses.
Iteratively adding to the social network over time can also help demonstrate which activities and 
engagements are working or not working to engage different types of stakeholders. 
3. Identify potential network brokers and in turn potentially vulnerable areas of the 
network such as pendants and establish connections to bolster these. 
Pendants are clear indicators of vulnerability in a system. By proactively identifying and engaging with 
pendant organizations with a clear plan for their improved integration into the network, avoiding the 
overreliance on a broker, you limit the amount of vulnerability the system is exposed to. Pendants can 
become more fully integrated into the network by increasing the amount of and quality of their 
connections to other organizations. It is in this process that you concretely remove vulnerable areas and 
increase resilience in the network. 
 
KEY LIMITATION: The robustness of lessons 
learned may be limited by the fact that we did not 
receive the following from the organization 
conducting the data collection (e.g., survey and 
key informant interviews) and qualitative data 
analysis: 
• specific processes followed for survey 
validation (e.g., survey changes occurring 
mid-course, lack of test runs, etc.)
• sampling and dissemination (e.g., what 
snowball sampling occurred, and what 
additional follow up was conducted)
• details on how qualitative data was coded, 
who performed the qualitative analysis, 
and how the analysis of qualitative data 
was validated and checked by our partner.
27
4. When trying to strengthen the network, target groups beyond government, private 
sector and NGOs to diversify the stakeholders represented in the network. 
By targeting the inclusion of under-represented groups, you enable the exchange of a richer and more 
diverse set of perspectives and information, which, if they are well connected, should improve the 
overall quality of the information shared throughout the network. This can have positive rippling effects 
over time by helping organizations to grow and learn from one another through the sharing of unique 
perspectives and lessons learned. 
5. Community members are a key stakeholder group that organizations are eager to 
engage. 
In addition to activities that are specifically geared towards citizen-engagement, finding ways to 
incorporate diverse groups of community members, from a range of demographics, into events that 
include other stakeholders is important, especially to provide leaders with an opportunity to hear 
community members’ voices.
6. In addition to hosting events that bring together multiple stakeholders, a 
program/initiative could continue to identify opportunities for organizations to share 
responsibility in joint projects/efforts.
By creating opportunities for organizations to engage in joint projects, BHC could increase the number 
of connections among organizations and interdependence among these organizations, which would 
increase the number of mutual relationships and decrease the number of one-way connections. This 
could increase the value of these relationships to each organization.
7. Formalize relationships through written agreements such as data sharing agreements, 
charters, and MOUs. 
Implementing partners can help facilitate a process of turning informal relationships between 
organizations in the network into formal ones. BHC did this through their Charter of Joint Commitment 
which was signed by 19 different organizations from a range of different sectors ranging from health, 
education, food security, sanitation, etc. It is important for organizations to periodically revisit the 
commitments and allow new stakeholders to join the agreements as the network grows.
8. Convert existing brokers within the system into “matchmakers” that foster 
interconnections between different stakeholders. 
As brokers are critical elements in the system, who have power over the spread of information to parts 
of the network, it is important to leverage their influence and encourage them to facilitate connections 
between their different partners and relationships so they do not always have to serve as the “go￾between.” 
9. For a program/initiative, create a system that can collect, store and readily transmit 
information on the history of each stakeholder’s engagement with the 
program/initiative.
Developing a system to document the ongoing progression of the program, and each organization’s
engagement, can help ensure awareness and collaboration continuity among stakeholders, even when 
staff and leadership turnover occurs.
28
10. Clearly define what resilience means to the program and develop stakeholder’s 
understanding of the concept of resilience through formal and informal channels. 
Stakeholders are receptive to learning what resilience means for broader efforts as well as their 
organization-specific efforts. Formally reflecting resilience in a program’s theory of change and also 
providing concrete examples of how they can build resilience can support their understanding.
29
ANNEXES
A – PURPOSE, AUDIENCE AND ANTICIPATED USES OF THE EVALUATION
The purpose of this evaluation was to answer exploratory learning questions related to the Building 
Health Cities program (BHC). These questions investigate what worked, what did not work (and why), 
the conditions under which the BHC approach is likely to be effective, and what lessons on urban 
resilience exist for future programming. Specifically, the evaluation aims to understand: 
1. How BHC’s approach to engaging stakeholders affected BHC implementation, 
2. How stakeholders conceptualize resilience and determine how these conceptualizations affect 
how city leaders act, and 
3. How BHC’s model can inform future urban project design.
Annex Table 1. Audience and anticipated uses of the evaluation
User Use
ABHT
● Understand which approaches/interventions were effective in 
achieving project goals and why
● Inform design/implementation of future Asia Bureau projects 
related to urban development
BHC Implementing 
Partners
● Integrate/reference relevant evaluation findings from the 
Evaluation in PMP and reporting
The ABHT determined that the following intended uses are outside the purview of the Evaluation:
● Manage and/or modify activities being implemented in years 4-5 of the project
● Assess the performance of the IP
● Assess the impact of activities (as a baseline was not conducted)
The goal of this evaluation was not to judge the success of BHC, but rather determine what elements 
might be useful going forward. The resources available for this evaluation necessitated focusing on only 
one city: Makassar, Indonesia.
B – EVALUATION QUESTIONS
In addition to the uses stated above, the SPACES team discussed some of the following, overarching 
motivations for the Evaluation with the ABHT. Based on the common themes and priority interests, 
SPACES and ABHT arrived at the following preliminary research questions:
Note: The term “stakeholders” below is inclusive of USAID implementing partners under BHC, local 
partners, municipal government, Smart City Boards, and beneficiaries (i.e., community members and 
those involved in co-creation activities) engaged in each city.
Annex Table 2. Evaluation Questions
Questions Motivation
30
1. Model/Approach/Tools
a. Stakeholder engagement: how, 
who, failures, successes. How 
did engagement between all 
stakeholders facilitate or hinder 
implementation and 
achievement of project goals?
b. How successful were the 
tools/assessments in achieving 
project goals (i.e., what worked 
and what didn’t work)?
USAID wishes to:
● understand feasibility/viability of achieving 
project goals via the stakeholder engagement 
tools
● support the “harmony” that resulted from 
the well-functioning engagement 
structures/processes/functions 
● determine how to better address the 
“disharmony” resulting from obstacles to 
stakeholder cooperation 
● understand how BHC uses evidence/findings 
● understand whether the stakeholders are 
acting upon findings and experiences of BHC 
tools/assessments and recommendations
2. Resilience
a. How does BHC define the 
concept of resilience, measure 
it, and contribute to achieving 
more resilient cities? 
b. How did city leaders better 
respond to community needs as 
a result of BHC programming?
USAID wishes to: 
● understand if/how the concept of resilience 
has evolved over the course of the project 
and differed across the implementing cities 
● know whether and the extent to which the 
technical focus (i.e., the six areas) of BHC’s 
approach was integrated into this 
conceptualization of resilience.
● determine to what extent these 
conceptualizations of resilience affected 
activity design/implementation and if these 
activities in turn contributed to the resiliency 
of the city (if so - how was this measured)
● understand how the city leaders were 
enabled/empowered to improve their 
preparation and response to future shocks 
and how BHC supported these 
improvements
3. Way forward: How could the BHC 
cross-sectoral models in the different 
country contexts be applied and 
sustained in the future?
USAID wishes to: 
● better understand BHC’s operating context 
in order to determine the 
appropriateness/applicability of the cross￾sectoral model for other country contexts
● equip future projects with the tools to learn, 
adapt, and improve quickly and effectively to 
improve chances of sustainability
● understand the validity of the theory of 
change
Each of these questions will be examined through the lenses of “What? Why? What does this mean for 
31
the future?” with an eye towards understanding what improvements can be made in a successor project.
C – ACTIVITY CONTEXT AND BACKGROUND INFORMATION
The USAID-funded Building Healthy Cities (BHC) project is partnering with Smart Cities in Indore, 
India, Makassar, Indonesia, Kathmandu, Nepal, and Da Nang, Vietnam with the goals of increasing 
equitable access to healthy lifestyles, improving access to health-related services, strengthening the 
community voice, decreasing environmental and lifestyle risk factors for chronic diseases, decreasing the 
burden of infectious diseases, and enabling wider use of data that supports community decision making 
to make different choices which then better includes health outcomes in city planning. 
Planning for a Smart City is intrinsically linked to health: transportation, natural resources, sanitation, 
education, recreation, technology, and the built environment all influence the health of an urban 
population. BHC employs a participatory systems mapping approach to clearly articulate these linkages 
in each context, engages a wide range of stakeholders across sectors, and formulates a set of priority 
actions for each city. BHC works in partnership with Smart City initiatives and urban sector 
departments to build this new vision for healthy urban planning. In addition, BHC engages Smart City 
citizens of every demographic with a focus on marginalized populations so that they have a voice (via 
citizen reporting systems and participatory research) and are empowered to advocate for the needs of 
their communities. 
BHC recently completed Year 3 of their 5-year cooperative agreement. BHC has been working with 
three Smart Cities including Da Nang, Indore, and Makasssar, and in fiscal year 2020 began an operation 
in Kathmandu. To date, the project has focused on establishing and deepening relationships in each city 
(in the community and within municipal government), facilitating sustainable connections across sectors, 
completing the systems mapping process, supporting the execution of city-level interventions, and 
making recommendations for multi-sectoral health activities. 
32
D – METHODOLOGY
The SPACES team conducted both qualitative and quantitative data collection and analyses. This section 
describes the mixed-methods approach used, starting with the development of Engagement Tracker 
which helped identify the different BHC activities, as well as the organizations involved and served as the 
basis of the sampling framework. The next section describes the development, deployment, and use of 
the survey, including the process for sampling. The following section describes the development and use 
of the key informant interview guide, as well as how the interviews were conducted and coded for use 
in qualitative analysis. The last section describes the social network analyses (quantitative analyses) 
performed to show the impact of the BHC program on organizations and their relationships.
Engagement Tracker
Determining BHC Activities
SPACES developed an engagement tracker to help measure the different interactions between key 
organizations and BHC organization engagement tools/activities which reflect the different BHC events 
or “touchpoints.” We identified the different BHC activities via a combination of background literature 
(see Annex E Documents Reviewed) and the BHC implementation team. We included 39 different 
activities in the engagement tracker, ranging from system thinking and mapping workshops to trainings, 
forums, community town halls, peer-to-peer study tours, and webinars. 
Determining the Organizations Involved
The organizations included in the tracker were based on a combination of participant and outreach lists 
used by the BHC team to track participation and engagement in the different activities they facilitated, 
organizations those who signed the commitment charter or memorandum of understanding (MOU), the 
organizations involved in the City Action Plan as well as input from the BHC implementation team. The 
BHC implementation team was consulted to verify the organizations included in the sample, provide 
contact information for institution representatives and recommendations for sampling.
As a result, we included 79 different organizations/institutions in the tracker including donor programs, 
state owned enterprises, local NGOs/CSOs/community groups, INGOs, national and subnational 
government organizations, hospitals, academia and private sector organizations. To note, this number 
does not include specific puskesmas or specific sub-district/village offices, or governmental offices of 
districts nearby Makassar or from other provinces consulted or invited to join in certain activities. While 
numerous names of such institutions were suggested for the tracker due to their involvement in one 
activity (for instance, 42 puskesmas participated in one sole data training), it was difficult to determine if 
this was a complete list. In addition, consultations with the BHC implementation team confirmed they 
do not have a system-level role in public health in Makassar.
Survey
Survey Instrument
The survey was designed to capture descriptive data to develop the network diagrams and conduct the 
social network analysis (SNA). The initial survey was constructed through collaboration with BHC and 
further refined with input from JSI EI and ABHT before a final version, with ABHT approval, was 
deployed. Information about additional validation of the survey has not been provided by our partner.
The final survey utilized an online survey tool (Alchemer) and consisted of 30 questions with a set of 15 
questions specific to each relationship identified and was available in both English and Bahasa. The survey 
questions were grouped into the following sections:
33
1. Information on respondent and their organization (12 questions): demographic questions about 
the individual respondent and the organization they represent helped establish the respondent's 
background and experience with the BHC program.
2. Understanding the level of participation the organization had with BHC programs (7 questions): 
questions helped assess the respondent’s ability to answer questions about BHC programs.
3. Adoption of BHC principles in their organization (3 questions): questions about the 
organizations’ engagement with the BHC program activities. This information was collected to 
help determine the impact of BHC activities on their organization.
4. Impact of BHC on resilience (2 questions): questions about readiness for urban resilience.
5. Effectiveness of BHC programs and opportunities for improvement (6 questions): questions to 
understand the organization’s perspective on the various BHC activities in which they 
participated to identify value and areas for improvement.
6. Relationships with other organizations, their nature, activities, and how they have changed 
through BHC (15 questions): questions on the organizations’ relationships, the nature of those 
relationships, and how they may have changed as a result of the BHC program. This included 
collecting information on sharing information/dialogue, coordinating on joint activities, social 
connection and networking. This information was used to generate the diagrams and conduct 
the SNA. 
The Engagement Tracker informed the development of survey questions on BHC activities and the list 
of organizations/organizations from which respondents could select those with whom they had a 
relationship with. The organizations list consisted of 50 organizations based on the most highly relevant 
of the 79 in the tracker. Respondents had the option to add missing organizations under a “puskesmas,” 
“sub-city/district government,” and general “other” categories. Due to issues with survey dropout, the 
questions were re-ordered mid-course (i.e., questions on the readiness for urban resilience and 
opportunities for the future were moved before the relationship identification section.) Details on the 
timing and impact of this change mid-course was not provided by the data collection partner. 
Survey Sample
The survey sample consisted of organizations or government departments, given the focus in the 
evaluation on understanding organizational relationships and approaches. The Engagement Tracker 
served as the basis for the sampling and the survey was initially sent directly to 56 contacts in the 
Engagement Tracker, all of which contact information (i.e., email, phone number, WhatsApp number) 
was available for. Surveys were emailed/messaged to participants for which emails/phone numbers were 
available. Additionally, the team conducted snowball sampling, in which the current respondents helped 
recruit additional respondents who have knowledge of BHC. Our partner did not provide the details as 
to how the snowball sampling occurred or what proportion of respondents were identified this way.
The survey period ran from July 5, 2022 to August 12, 2022. Both invitations and reminders to 
participate in the survey were sent by email using an email marketing service (Mailchimp). The response 
rates, both full and partial, were reviewed on a daily basis, with special attention being made to 
encourage partial respondents to complete their surveys. This monitoring showed that the 56 contacts 
had under a 10% open rate and none of them clicked the link to the survey. Therefore, further follow up 
was conducted including phone calls, text messages and emails.
Overall, there were a total of 26 responses: 17 responses were complete with responses to both survey 
questions and the SNA questions, representing 14 unique organizations. Of the 26 individual responses 
to the survey, 2 responses were removed during the data cleaning stage as they were unusable for 
analysis, leaving a total of 24 individual responses to be analyzed for the non-SNA questions. Key 
informant interviewees contributed 11 of the 26 responses submitted. The rest of the responses were 
34
collected by sending an email link after a phone call. In most cases (22), responses were entered by the 
participants in the online tool after a link was emailed and/or messaged to them, in 3 cases, responses 
were populated by the enumerators over a phone call, and in 1 case the responses were partially 
entered by the respondent and completed by an enumerator via a phone call. These responses 
represent 7 local NGO/CSO/community group organizations, 3 city government entities, 2 national 
government entities, 1 private sector company, and 1 donor program.
Key Informant Interviews (KIIs) 
KII Instrument
The key informant interviews (KIIs) were developed to provide additional context and deeper insight 
into the interviewee’s respective organizations. Similar to the survey, the KII guide was developed in 
collaboration with ABHT and JSI EI. Once the tool was finalized, it was translated into Bahasa 
Indonesian.
The KII guide consists of six sections and a total of 13 questions. These questions not only asked about 
the uptake of tools and insights learned as a result of the BHC program, but also how the strengthening 
of systems leads to collaboration and impact. Specifically they sought to garner additional insights into 
participants adoption/uptake of BHC outcomes (e.g., uptake of BHC priorities, approaches, and 
concepts), organizational changes to enable action on urban health challenges and community needs, and 
how relationships have changed, how resilience is defined and how it may have changed due to BHC, 
and about how they felt about the BHC program and what additional actives may be valuable to include 
in in BHC in the future. Additionally, participants were asked to provide specific examples of changes, 
activities, and actions (e.g., specific examples of how BHC practices were adopted, and how these have 
impacted decisions or practices in the organization).
Selection of KII Participants 
Interviewees were selected using multiple criteria. First, the interviewees needed to be from the BHC 
Engagement Tracker which included relevant BHC stakeholders. Next, the majority of the interviewees 
were marked as “recommended for interview” by the BHC implementation team. We also reviewed the 
interview sample to ensure it reflected the sample of BHC stakeholders and also represented a broad 
range of viewpoints, experiences, and organizations. We scheduled interviews between May 15, 2022 
and July 16, 2022. A total of 23 participants were included in the key informant interview sample. These 
23 interviewees represent 18 different organizations. Of these, 11 respondents were from city 
government entities, 7 from local NGO/CSO/community group organizations, 2 were from Indonesian 
universities, 1 from national government and 1 from the private sector. 
How KIIs were Conducted
All interviews were conducted remotely via Zoom and recorded, between May 15-July 16, 2022. Each 
interview was recorded and transcribed word for word in Indonesian by a human transcriptionist. In 
addition, the interviewer maintained rapid summary notes, stored in a word document, following each 
interview regarding any important context to the interaction/data or significant insights that emerged. 
The interviews were conducted by an Indonesian national social researcher with 15+ years in 
development research and advisory experience. She has extensive field experience, including conducting 
semi-structured interviews at the district level and below on topics including frontline health services, 
community-based health and education programs, water and sanitation, social protection, and village 
governance.
How KII Results were Coded
To identify initial qualitative findings, the interviewer along with another analyst conducted the 
qualitative data coding of the full interview transcripts in Indonesian applying a coding framework 
35
developed based on anticipated themes in the interviews and related keywords, and also allowed for 
unexpected themes to emerge from the interview data. Themes include: organizations’ motivation for 
engaging BHC; BHC and community needs; integration of BHC approaches into organizational practices; 
organizational changes due to organization engagement and barriers to organizational changes that might 
be prompted by organization engagement; opportunities to support organization engagement and 
improve weak relationships and barriers to organization engagement; and resilience.
In the case of themes related to resilience, the coding framework included terminology related to 
various shocks and stresses in addition to other resilience-related keywords. The framework also 
included a list of BHC tools and approaches. The interviewer also looked for unexpected themes 
grounded in the interview data. The interviewer and another analyst (also a native Indonesian speaker 
with experience in public health governance) then prepared briefs in Bahasa Indonesian summarizing 
emergent findings, based on the major themes in the data. (These were subsequently translated into 
English to aid in evaluation report preparation.) Following this, another member of the team (a native 
English speaker with Indonesian language skills) prepared a summary presentation in English for ABHT 
regarding the qualitative findings. It is unclear as to the level of involvement that our partner had in the 
analysis of the qualitative data (e.g., the extent to which our partner validated and provided inputs and 
checks on the analysis), as this information was not provided. 
Social Network Analyses (SNA)
Study Population
Of the survey respondents, only 14 were included in the SNA because there were 14 unique 
organizations. To note, there were multiple complete responses from the same organization but were 
from different representatives. The specific reason as to why we had multiple responses from one 
organization has not been provided by our partner. Since the SNA is focused on organization to 
organization relationships, the duplicated responses were cleaned by keeping the answer from the 
respondent with the highest rank/title within the organization as well as the most familiarity with the 
BHC program, per their response to the question: How familiar personally are you with the BHC 
program (goals, objectives, and activities)?.
The 14 organizations include 7 local NGO/CSO/community group organizations, 3 city government 
entities, 2 national government entities, 1 private sector company, and 1 donor program. The 
organizations represented across the full network in the SNA were the organizations with which the 14 
respondents reported a connection. This included 28 city government organizations, 11 local 
NGO/CSO/community group organizations, 5 donor programs, 3 national government entities, 2 
Indonesian universities, 2 international nongovernmental organizations, 1 hospital, 1 private sector 
company, 1 state owned enterprise, and 1 unknown. 
Definition of Nodes and Edges
Each organization in the BHC network is included as a node which is visually represented with a circle. 
Each connection is included as an edge which is visually represented with directional lines that connect 
nodes in the network diagram, indicating where a relationship exists between organizations. For 
example, a directional line was drawn from Organization A to Organization B if Organization A 
reported a relationship with Organization B. Relationships were categorized as relationships that existed 
prior to the BHC program (i.e., ‘pre-BHC’), relationships that formed as a result of the BHC program 
(i.e., ‘post-BHC'), or relationships for which the respondents indicated that they were unsure if the 
relationships occurred prior to or as a result of BHC (i.e., ‘unknown origin’).
Network Diagrams
36
The research team generated network diagrams using Kumu (Kumu Inc., Oahu, Hawaii). We explored 
three different networks: 1) network among just the respondents, 2) network among all organizations 
reported and 3) ego networks of respondents. This first network consisted of only the 14 respondents 
and was done to understand the ‘verifiable’ network, that is a network in which the only connections 
displayed are those between the responding organizations, which ensures we have obtained the 
perspective/responses of each element in the network. The second network consisted of all 55 
organizations/organizations, including both the survey respondents and all of the organizations that the 
14 survey respondents noted having a relationship with (but who did not themselves respond to the 
survey). This allowed us to understand the effects of BHC on a broader set of organizations, however, it 
is not a verified network and there are limitations to the conclusions we can draw since it does not 
include the perspective of each of the organizations included in the diagram. Lastly, in order to see how 
BHC affected the surrounding networks of each of the responding organizations we developed EgoNets 
for each of the 14 responding organizations. EgoNets also allow for comparison across organizations in 
the network to see how influential certain organizations are as compared to others. EgoNets help us 
understand the opportunities and constraints facing specific actors, which helps us understand the role 
of BHC in the “local” social structures of specific organizations and groups of organizations,2 We 
characterized each respondents’ two-step ego network, which consisted of a respondent organization 
(ego) and all the other organizations directly connected to that ego as well as the ties among those 
organizations directly connected to the ego.
For each network we generated four sets of diagrams. The first network diagram (pre-BHC) showed all 
relationships that existed prior to BHC in order to understand the relationships and connections 
between organizations and organizations in Makassar before BHC was implemented. The second 
diagram we generated showed all of the relationships that occurred pre-BHC, as well as all of the 
relationships of unknown origin (e.g., the respondent was unable to report when the relationship 
formed). The third diagram (post-BHC) showed both the relationships that existed prior to the 
implementation of BHC as well as the relationships that formed following the implementation of BHC. 
The fourth diagram showed all of the reported connections including those that existed prior to BHC, 
those that developed as a result of BHC as well as all of the relationships of unknown origin. We 
developed separate diagrams to include the relationships of unknown origin in both the pre-BHC 
diagram as well as the post-BHC diagram as it is unclear when these relationships formed. The type of 
relationship: developed prior to BHC, as a result of BHC or relationships of unknown origin, are color 
coded in the network diagrams created using the Kumu software as blue, red and gray, respectively. 
Social Network Measures
For each of the diagrams we developed, including pre-BHC, pre-BHC + unknown origin, post-BHC and 
post-BHC + unknown origin diagrams for each of the networks (network among just the respondents, 
network among all organizations reported and ego networks of respondents) we conducted social 
network analyses in Kumu, calculating network metrics including the number of elements, number of 
connections, density, reciprocity, and average degree, as well as metrics specific to an organization (e.g., 
egos) including size, degree centrality, indegree, outdegree, closeness centrality, betweenness centrality, 
and reach. Table 1 below provides a description for each of these measures as well as their 
interpretation.
For each network, we compared the measures between the pre and post diagrams (pre-BHC, pre-BHC 
+ unknown origin and post-BHC and post-BHC + unknown origin) to quantify any changes that may 
have occurred as a result of BHC. We compared changes from the pre-BHC diagram with and without 
2 Hanneman, Robert A., and Mark Riddle. "Introduction to social network methods." (2005).
37
relationships of unknown origin to show the range of BHC’s possible impact. Comparisons between the 
post-BHC diagrams including relationships of unknown origin to the diagrams including both the pre￾BHC connections and the connections of unknown origin may be more conservative as it assumes that 
all the relationships of unknown origin existed prior to BHC, when in fact, some may have formed 
during the course of BHC. However, it may also underrepresent changes due to BHC. Therefore, in the 
Core Findings and Recommendations, we report comparisons between both sets of diagrams (including 
and not including the relationships of unknown origin in the pre-BHC diagram), and specified which 
connections were represented.
Annex Table 3. Description and Interpretation of Social Network Measures
Social Network 
Measure
Description Interpretation
Total elements A count of all the elements (i.e., organizations) 
in the network.
Networks with more elements = larger network
Number of 
ties/connections
Total number of organization relationships in 
the network.
Networks with more ties = more interconnected
Density Number of existing ties divided by the 
total number of possible ties in a network. 
Thus, compares the number of actual 
connections to the number of potential 
connections in a network.
Networks with a lower density = sparser network
Reciprocity Number of organization pairs with 
bidirectional ties divided by the number of 
connected organization pairs. It is the extent 
to which two organizations share/confirm each 
other’s relationship.
Lower reciprocity = more unidirectional ties
Average degree Average number of connections across all 
organizations in the network.
Higher degree = network is more interconnected
Size Number of other organizations directly 
connected to a given ego facility (plus the ego 
organization itself).
Larger size = more organizations than an ego directly 
interacts with
Degree Centrality Total number of connections an organization 
has.
High degree centrality = organization has a high 
number of connections with others; organizations may 
be local connectors / hubs, but may not necessarily be 
the best connected to the wider network
Indegree Measures the number of incoming 
connections for a given organization (total 
number of other organizations with a 
relationship with a given organization).
High indegree = organization is more looked to by 
others (e.g., ‘popular’); organizations may serve as a 
source of advice, expertise, or information
Outdegree Measures the number of outgoing connections 
for a given organization (total number of 
different organizations that have a relationship 
with a given organization).
High out-degree = organization can affect other 
organizations (e.g., high influence); organizations may be 
able to reach a high number of other organizations and 
serve as a source of information flow 
Closeness 
centrality
Measures the distance each organization is 
from all other organizations (is a probability 
metric, with a value between 0 and 1).
High closeness = organization is more closely 
connected to others; organization may be able to 
spread information to the rest of the network most 
easily, and may have high visibility into what is 
happening across the network
38
Betweenness 
centrality
Probability that a given organization is part 
of the shortest path between two other 
organizations (i.e., degree to which a given 
organization serves as an intermediary 
between other organizations).
High betweenness = an organization serves as an 
intermediary between many pairs of organizations; 
organizations may have more control over the flow of 
information and act as key bridges within the network 
(can serve as brokers and be bottlenecks)
Reach (two-step 
out)
Portion of the network within two steps of an 
organization. 
. 
High reach = organization can spread information through 
the network through close friend-of-a-friend contact
Limitations
We requested the following information from the partner that administered the survey and KIIs: specific 
processes involved in the validation process of the survey instrument and KII tool, final list of 
organizations that received the survey instrument and details around their method for selection, as well 
as the dissemination process, including how recipients were asked to share the survey to colleagues, 
what snowball sampling occurred, and what additional follow up was conducted. We also requested 
details on how qualitative data was coded, who performed the qualitative analysis, and how the analysis 
of qualitative data was validated and checked by our partner. We are also missing information about 
specifically when the survey instrument changed and the potential impact this mid-course change may 
have on results. However, the partner did not provide the requested information.
39
E - DOCUMENTS REVIEWED
● BHC Annual Report Year 1
● BHC Annual Report Year 2
● BHC Annual Report Year 3
● Final BAA Concept Note
● BHC Makassar Infographic
● BHC Da Nang Infographic
● BHC Indore Infographic
● BHC Project Business Card English
● Building Healthy Cities - 2 pager
● Urbanization in Asia Lit Review
● Urban Poverty and Health PRB
● Livable Cities IDB exec summ 2019 (Creating Livable Cities: Regional Perspectives)
● Future of AP Cities Report
● Evaluating Urban Quality: Indicators and Assessment Tools for Smart Sustainable Cities
● C40 Cities and Mayors Migration Council (2021) Cities, Climate and Migration
● System of Cities World Bank
● Workplan Year 1 Makassar
● Workplan Global Year 1
● Workplan Year 2, 3
● Workplan Year 4, 5
● Makassar Summary Presentation January 2021 (Makassar – Summary of City Activities)
● Defining Priorities for Urban Health Makassar Presentation
● PMP Revised
● BHC Evaluation Document Summary
● USAID 2020 Urban Learning Brief for Mission Staff
● USAID Urban Resilience Measurement Training Guide
● Data Validation & Systems Mapping Workshop-Makassar
● Building Healthy Cities - Makassar Leverage Workshop 2019
● Makassar Healthy Alley Knowledge Exchange Story
● Cooperative Agreement
● Cataloguing Unintended Benefits, Consequences, and External Shocks during City Project Period 
(2018-2021)
● Improving Urban Health in Asia (BAA Co-Creation Workshop Participant Program)
● Indore Study Tour Zee News Coverage
● Indore Systems Map Review Notes
● Healthy City Action Plan – Proposed Multisector Action Plan: Makassar
● BHC PMP Makassar Year 4 Q3
● BHC Timelines_evaluation_20210723
● BHC Timelines_Makassar_20220120
● Makassar Journey Maps Year 1, 2, 3, 4
● Quarterly Presentation Year 1 Q1 to Year 4 Q2
● How Community Participation Can Impact Air Pollution Mitigation Efforts in Indore: A Systems 
Thinking Perspective
● City Planning in Makassar City, Indonesia
● System Mapping to Support City Planning in Makassar City, Indonesia
● Digital Implementation for Healthy City: Case Study in Makassar City, Indonesia
● Building Healthy Cities – Urban Planning as a Tool for Health (Fifth Global Symposium on Health 
Systems Research)
● Urban Planning for Health
40
● System Mapping and Visualization Tools for Urban Social Determinants of Health Approaches
● Fostering vibrant urban health systems: Results of participatory action inquiry in three Asia cities
● Summary of Makassar Smart City Master Plan
● Synthesizing Investment Opportunities for Strengthening Urban Health Systems
● How Can Pipe Cleaners and Post-Its Strengthen Urban Health Systems?
● Makassar Workshop Report: Training of Call Center 112 Call Takers
● Makassar Workshop Report: Call Center 112 Reporting and Data Management Training
● Joint Commitment Charter English Signatures List
● Links_Participants Presentations and Materials
41
F – DATA COLLECTION INSTRUMENTS 
Key Informant Interview Guide - English
SPACES/ Building Healthy Cities (BHC)
Stakeholder KII Protocol 
Please read to the respondent before starting the interview:
My name is [NAME]. I am working with a research team conducting interviews with organizations/government 
departments involved in the Building Healthy City (BHC) project in Makassar, Indonesia. This study, which is 
funded by the United States Agency for International Development (USAID) focuses on how stakeholders worked 
together under this project to address key urban health issues. You have responded to the SNA Survey and 
agreed to participate in the additional qualitative key informant interview, and we appreciate you taking the time 
to share your perspectives, experiences and examples, and insights.
This interview usually takes about 60 minutes to complete. 
Your participation in this interview is entirely voluntary. I will record your answers to each interview question via 
the Zoom platform (an online teleconferencing platform), which will only be shared with the research team for 
this project. While the research team will see your individual responses, our reports will not connect your 
responses to your name and or other related demographic information. Knowing this, do I have your permission 
to record this interview? 
Do you have any questions before we begin?
Section 1. BHC Activities 
1. What were the main motivations for your organization/government department's engagement in 
the BHC activities? 
a. Probe - internal and external motivations / incentives? 
b. Did they change over time? 
c. If so, why? 
Section 2. BHC Outcomes (Including uptake of Priorities, Approaches and Concepts)
2. You mentioned that [fill in from SNA] were strategic priority objectives that your organization 
focused on as part of your engagement in BHC activities. 
a. Can you give an example of how you acted on one of these priority objectives and how 
the insights, decisions, learning or relationships you gained through BHC activities 
contributed? 
3. You mentioned [fill in from SNA] as outcomes your organization as a result of engaging in BHC 
activities. 
a. Probe into the outcome to develop a clearer sense – ask for concrete examples. 
b. How did the insights, decisions, learning, or relationships you gained through BHC 
activities contributed to achieving this outcome? 
4. What new approaches, concepts or tools was your organization/government department exposed 
to through the BHC activities? 
a. Have you applied any of these new approaches and concepts to your day-to-day routine 
work? 
■ If yes, how are they impacting decisions or practices in the organization? 
■ If no, do you hope to apply them to your work in the future? What would make 
that possible? 
Section 3. BHC Engagement/Stakeholder Collaboration Model
42
5. What changed within your organization – as a result of your relationship to other organizations 
– that enabled you to take action on urban health challenges and community needs because of 
your relationships with others?
a. Can you share an example of new actions your organization was able to take because of 
improved relationships with other stakeholders? 
b. What specific value did stakeholders provide to you to take this action? 
c. What might support your organization to continue to collaborate with others in the 
city? 
6. In your experience, how have stakeholder relationships as a whole changed across the City of 
Makassar as a result of BHC program activities? 
a. In your opinion, what enabled effective stakeholder interactions and relationships, and 
why? 
b. In your opinion, what inhibited effective stakeholder interactions and relationships, and 
why?
7. [Optional, if time] How might BHC better address the weak relationship, for example, to support 
increased information sharing, coordination, and/or collaboration between stakeholders in the 
future?
Section 4: Readiness for Urban Resilience 
8. Resilience was not a specific goal of BHC but it is of interest to USAID, and we recognize that 
work to support resilience can happen even if it is not defined as such. We describe resilience as 
‘the capacity to respond effectively to urban community needs in the aftermath of impending and 
unforeseen shocks and stressors such as natural disasters, economic shocks, food security, 
health pandemics such as currently in COVID, etc”. ? [display the definition in writing] Can you 
think of ways that your organization/government department plans and budgets for activities 
that strengthen resilience? 
a. Has this changed because of BHC programming? If so, how? 
b. Does your organization have an internally approved definition or framework for urban 
resilience that it uses for relevant internal planning and decision making? If not, is there a 
commonly understand definition to which staff of the organization refer? [If no - move to 
#9 ; if yes - ask if they can share with you as follow up] 
c. [optional] How was your organization's formal or commonly understood definition 
influenced by BHC? Please provide a specific example. 
9. Do you think that BHC program activities have increased Makassar City's resilience to recent 
shocks like COVID-19 and also to future shocks and stressors?
a. Can you provide specific examples of activities or actions through which organizations 
across the City of Makassar are working together to improve the city’s resilience as a 
result of BHC? 
b. Where do gaps continue to exist, and why?
c. What type of support (financial, capacity building, etc.) is still necessary to ensure a 
more resilient city?
Section 5: General Project Perceptions and Way forward
10. Would you continue to participate in future BHC activities? 
a. Why or why not? 
b. What would motivate greater participation?
11. What activities do you think would be most valuable for BHC to undertake in the future to 
enable a more effective, responsive, and resilient city?
12. Are you aware of other similar projects or initiatives occurring in the city that you think BHC 
should collaborate with or learn from? 
43
Section 6: Additional Comments 
13. Do you have any additional and related comments or concerns that were not addressed by 
these questions that you would like to share with us?
Key Informant Interview Guide – Bahasa, Indonesian
SPACE/ Building Healthy City (BHC)/Membangun Kota Sehat 
Protokol Wawancara Informan Kunci Pemangku Kepentingan
Mohon dibacakan kepada informan sebelum memulai wawancara:
Nama saya [NAMA]. Saya bekerja dengan tim peneliti yang melakukan wawancara dengan 
organisasi/OPD/Kantor yang terlibat dalam proyek Building Healthy City (BHC) /membangun Kota Sehat di 
Makassar, Indonesia. Studi ini didanai oleh Badan Pembangunan Internasional Amerika Serikat (USAID) berfokus 
pada bagaimana pemangku kepentingan bekerja sama dalam proyek ini untuk mengatasi masalah utama 
kesehatan perkotaan. Bapak/Ibu sebelumnya sudah mengikuti Survei SNA dan bersedia untuk berpartisipasi 
dalam wawancara informan kunci kualitatif tambahan, dan kami sangat menghargai kesediaan Bapak/Ibu 
meluangkan waktu untuk berbagi perspektif, pengalaman dan contoh, serta wawasan.
Wawancara ini akan membutuhkan waktu sekitar 90 menit.
Partisipasi Bapak/Ibu dalam wawancara ini sepenuhnya bersifat sukarela. Saya akan merekam jawaban 
Bapak/Ibu untuk setiap pertanyaan wawancara melalui platform Zoom (platform telekonferensi online), yang 
hanya akan dibagikan dengan tim peneliti untuk proyek ini. Meskipun tim peneliti akan melihat jawaban￾jawaban Bapak/Ibu, laporan kami tidak akan menghubungkan jawaban-jawaban Bapak/Ibu dengan nama 
Bapak/Ibu dan atau informasi demografis terkait lainnya. Dari penjelasan tersebut, apakah Bapak/Ibu 
mengizinkan saya untuk merekam wawancara ini?
Sebelum kita mulai apakah Bapak/Ibu ada pertanyaan?
Bagian 1. Latar Belakang Informan/Kegiatan BHC
1. Mohon Bapak/Ibu memperkenalkan diri secara singkat (nama, jabatan/posisi saat ini, 
peran/keterlibatan dalam program/kegiatan terkait BHC).
2. Apa motivasi utama keterlibatan organisasi/OPD/Kantor Bapak/Ibu dalam kegiatan BHC?
○ Cari tahu lebih lanjut - motivasi / insentif internal dan eksternal?
○ Apakah berubah seiring waktu?
○ Jika demikian, mengapa?
Bagian 2. Hasil BHC (Termasuk penyerapan Prioritas, Pendekatan dan Konsep)
3. [Pertanyaan ini khusus informan yang belum mengisi SNA] Apa saja tujuan prioritas strategis BHC 
yang menjadi fokus organisasi Bapak/Ibu sebagai bagian dari keterlibatan Bapak/Ibu dalam 
kegiatan BHC?
○ Dapatkah Bapak/Ibu memberikan contoh upaya atau langkah apa yang bapak/ibu lakukan 
untuk mencapai tujuan prioritas tersebut?
○ Bagaimana Bapak/Ibu membuat keputusan untuk melakukan hal tersebut dan bagaimana 
itu dilakukan?
○ Bagaimana pengetahuan/wawasan, keputusan, pembelajaran atau hubungan yang 
bapak/ibu peroleh melalui kegiatan BHC berkontribusi?
44
4. [Pertanyaan ini khusus informan yang sudah mengisi SNA] Bapak/Ibu menyebutkan bahwa [isi dari 
SNA] adalah tujuan prioritas strategis BHC yang menjadi fokus organisasi Bapak/Ibu sebagai 
bagian dari keterlibatan Bapak/Ibu dalam kegiatan BHC.
○ Dapatkah Bapak/Ibu memberikan contoh upaya atau langkah apa yang Bapak/Ibu lakukan 
untuk mencapai tujuan prioritas tersebut? 
○ Bagaimana Bapak/Ibu membuat keputusan untuk melakukan hal tersebut dan bagaimana 
itu dilakukan?
○ Bagaimana pengetahuan/wawasan, keputusan, pembelajaran atau hubungan yang 
Bapak/Ibu peroleh melalui kegiatan BHC berkontribusi?
5. [Pertanyaan untuk informan yang belum mengisi SNA] Apa saja hasil/manfaat yang organisasi/kantor 
Bapak/Ibu peroleh, jika ada, dari keterlibatan dalam kegiatan BHC?
○ Cari tahu lebih lanjut hasil (outcome) yang disampaikan. Untuk memperjelas minta 
contoh konkret. 
○ Bagaimana pengetahuan/wawasan, keputusan, pembelajaran, atau hubungan yang 
Bapak/Ibu peroleh melalui BHC berkontribusi untuk hasil tersebut?
6. [Pertanyaan untuk informan yang sudah mengisi SNA] Bapak/Ibu menyebutkan [isi dari SNA] 
sebagai hasil/manfaat organisasi Bapak/Ibu yang merupakan hasil dari keterlibatan dalam kegiatan 
BHC.
○ Cari tahu lebih lanjut hasilnya (outcomenya) untuk memperjelas – mintalah contoh 
konkret.
○ Bagaimana pengetahuan/wawasan, keputusan, pembelajaran, atau hubungan yang 
Bapak/Ibu peroleh melalui kegiatan BHC berkontribusi untuk mencapai hasil tersebut?
7. Pendekatan, konsep, atau alat baru apa yang diperkenalkan kepada organisasi/OPD/Kantor 
Bapak/Ibu melalui kegiatan BHC? [Lihat daftar pendekatan BHC.] 
○ Sudahkah Bapak/Ibu menerapkan salah satu dari pendekatan dan konsep baru tersebut 
pada pekerjaan rutin Bapak/Ibu sehari-hari?
■ Jika ya , bagaimana hal itu diterapkan? Apakah ada pengaruhnya terhadap 
keputusan atau praktik dalam organisasi? Jika iya, bagaimana? Jika tidak, 
mengapa?
■ Jika tidak , apakah Bapak/Ibu berharap dapat menerapkannya pada 
pekerjaan Bapak/Ibu di masa mendatang? Apa yang membuat itu mungkin?
 
Bagian 3. Model Keterlibatan/Kolaborasi Pemangku Kepentingan BHC
8. Dengan siapa/organisasi mana saja Bapak/Ibu berkolaborasi dalam kegiatan BHC?Apa saja yang 
dilakukan dengan mereka?
9. Apakah ada perubahan dalam organisasi Bapak/Ibu yang memungkinkan Bapak/Ibu mengambil 
tindakan atas tantangan kesehatan perkotaan dan kebutuhan masyarakat sebagai hasil dari 
hubungan Bapak/Ibu dengan pemangku kepentingan lain?
○ Mohon Bapak/Ibu menyebutkan contoh tindakan baru yang bisa dilakukan organisasi 
Bapak/Ibu sebagai hasil dari hubungan yang lebih baik dengan pemangku kepentingan 
lainnya?
○ Menurut Bapak/Ibu nilai spesifik apa yang diberikan pemangku kepentingan lain kepada 
Bapak/Ibu untuk mengambil tindakan tersebut?
○ Menurut Bapak/Ibu apa yang mungkin mendukung organisasi Bapak/Ibu untuk terus 
berkolaborasi dengan pemangku kepentingan lain di Kota Makassar?
○ Cari tahu lebih lanjut-Jika informan mengatakan tidak melihat ada perubahan, cari tahu apa 
yang menjadi alasan.
10. Menurut pengalaman Bapak/Ibu, bagaimana hubungan pemangku kepentingan secara keseluruhan 
berubah di Kota Makassar sebagai akibat dari kegiatan program BHC? [Use image if helpful] 
45
○ Menurut Bapak/Ibu, apa yang memungkinkan interaksi dan hubungan pemangku 
kepentingan yang efektif, dan mengapa? 
i. Interaksi pemangku kepentingan yang efektif memungkinkan organisasi mengambil 
tindakan/merespons tantangan kesehatan dan kebutuhan masyarakat perkotaan 
sebagai hasil dari berbagi informasi, koordinasi, dan/atau kolaborasi di antara 
pemangku kepentingan.
○ Menurut Bapak/Ibu, apa yang menghambat interaksi dan hubungan pemangku 
kepentingan yang efektif, dan mengapa?
11. [ Opsional, jika ada waktu] Bagaimana BHC dapat dengan lebih baik memperbaiki hubungan yang 
lemah, misalnya, untuk mendukung peningkatan berbagi informasi, koordinasi, dan/atau 
kolaborasi antar pemangku kepentingan di masa mendatang?
Bagian 4: Kesiapan untuk Ketangguhan/Ketahanan Perkotaan
12. Ketangguhan/Ketahanan/daya lenting bukanlah tujuan khusus BHC tetapi menjadi perhatian 
USAID, dan kami menyadari bahwa upaya untuk mendukung ketangguhan/ketahanan dapat 
terjadi meskipun tidak ditentukan. Kami mendefinisikan ketangguhan/ketahanan sebagai 'kapasitas 
untuk merespon secara efektif kebutuhan masyarakat perkotaan setelah mengalami guncangan 
dan stresor yang datang dan tidak terduga seperti bencana alam, guncangan ekonomi, ketahanan 
pangan, pandemi kesehatan seperti COVID19 saat ini, dll. ? [tunjukkan definisi secara tertulis] 
Dapatkah Bapak/Ibu memikirkan cara-cara organisasi/OPD/Kantor Bapak/Ibu merencanakan dan 
menganggarkan kegiatan yang memperkuat ketangguhan/ketahanan?
○ Apakah hal tersebut berubah karena program BHC ? Jika demikian, bagaimana?
○ Apakah organisasi Bapak/Ibu memiliki definisi atau kerangka kerja yang disetujui secara 
internal mengenai ketangguhan/ketahanan perkotaan yang digunakan untuk perencanaan 
internal dan pengambilan keputusan yang relevan? Jika tidak, apakah ada definisi yang 
dipahami secara umum yang dirujuk oleh staf organisasi? [ Jika tidak - pindah ke #13 ; jika 
ya - tanyakan apakah bisa dilihat/dibagikan sebagai tindak lanjut]
○ [opsional] Bagaimana definisi formal atau yang dipahami secara umum oleh organisasi 
Bapak/Ibu dipengaruhi oleh BHC? Berikan contoh spesifik.
13. Menurut Bapak/Ibu, apakah kegiatan program BHC telah meningkatkan ketangguhan/ketahanan 
Kota Makassar terhadap goncangan yang dialami saat ini seperti COVID-19 dan juga guncangan 
dan stresor di masa depan?Jika iya, bagaimana?
○ Dapatkah Bapak/Ibu memberikan contoh spesifik kegiatan atau tindakan di mana 
organisasi-organisasi di Kota Makassar bekerja sama untuk meningkatkan 
ketangguhan/ketahanan kota sebagai hasil dari program BHC?
○ Apakah masih ada kesenjangan, dan mengapa?
○ Jenis dukungan apa (keuangan, pengembangan kapasitas, dll.) yang masih diperlukan untuk 
memastikan kota yang lebih tangguh?
Bagian 5: Persepsi Umum Proyek dan Langkah ke Depan
14. Apakah Bapak/Ibu akan terus berpartisipasi dalam kegiatan BHC di masa mendatang?
○ Mengapa atau mengapa tidak?
○ Apa yang dapat memotivasi partisipasi yang lebih besar?
15. Kegiatan apa yang menurut Bapak/Ibu paling berharga untuk dilakukan BHC di masa depan untuk 
memungkinkan kota yang lebih efektif, responsif, dan tangguh?
16. Apakah Bapak/Ibu mengetahui proyek atau inisiatif serupa lainnya yang dilaksanakan di Kota 
Makassar yang menurut Bapak/Ibu/Saudara/Saudari harus dikolaborasikan dengan atau dipelajari 
oleh BHC?
Bagian 6: Komentar Tambahan
46
17. Apakah Bapak/Ibu memiliki komentar atau hal lain terkait yang belum tercakup dalam 
pertanyaan-pertanyaan di atas yang ingin Bapak/Ibu sampaikan kepada kami?
Survey (English)
SPACES/ Building Healthy Cities (BHC) Social Network Analysis
Introduction
This is a survey for organizations involved in the Building Healthy City (BHC) program in Makassar, Indonesia. 
The survey is designed to understand how stakeholder capacities and relationships changed as a result of
engagement in the BHC program activities. This survey usually takes [less than 45 minutes] to complete.
The Building Healthy Cities Project (BHC) is a 5-year (2017-2022) project, grounded in a systems approach, with a
focus on the social and environmental determinants of health, which span areas well beyond the health sector.
BHC’s goal is to test feasible options for healthy urban planning that reflect the following core values: 
Develop a common understanding across multiple sectors and stakeholders of ways to improve overall
health and quality of life in Makassar. 
Strengthen community engagement in municipal decision-making for urban health-related services. 
Support the use of urban health-related data for planning and decision-making. 
The SPACES consortium (Strategic Program for Analyzing Complexity and Evaluating Systems) is the research
team administering and analyzing the survey. JSI, as the lead organization on the BHC Program, is assisting in
coordination but is not part of the research team. This research is funded by USAID’s Asia Bureau, Office of Heath
(USAID/Asia/Health).
Your organization has been identified as a valuable stakeholder in this program. This survey is intended for
respondents with familiarity with your organization's participation in the BHC program as well as in urban health
systems in Makassar. If someone else is better suited to respond, please ask them to respond instead.
Your participation in this survey is voluntary and your responses will only be visible to the research team for this
program. The reports stemming from this research will not connect your responses to your name or other related
demographic information. We appreciate you taking the time to share your perspectives and insights.
Please use the button above to save and continue your responses later. In this case, your email address will only be used
for purposes of resuming the survey. (If you do not save using your email address, you will need to repeat answers that you
have already provided.) 
Respondent Background Information (Respondent Attributes)
Page description:
In this section, we will capture basic demographic and background information on you. Your personal information will not be
shared outside of the research team and will be used for the purposes of being able to contact you for any additional follow-up.
Please check "Yes" if we have your permission.*
Yes
No
1. Please state your full name *
2. Please select the name of your organization/government department from the following list*
If you don't see your organization's name in the list, please use the 'Other' option at the end of the list.
Asian Venture Philantropy Network (APVN)
Asosiasi Ibu Menyusui Indonesia (AIMI)
Badan Perencanaan Pembangunan Daerah
(BAPPEDA) Kota Makassar
Bagian Kerjasama Sekretariat Daerah Kota Makassar
BAKTI (Yayasan Bakti)
BPJS Kesehatan Makassar
Celebes Organic
Dinas Kearsipan Kota Makassar
Dinas Kependudukan dan Pencatatan Sipil
Dinas Kesehatan Kota Makassar
Dinas Komunikasi dan Informatika (Kominfo)
Dinas Komunikasi dan Informatika (Kominfo) Kota
Makassar
Dinas Koperasi dan UMKM
Dinas Lingkungan Hidup
Dinas Lingkungan Hidup Kota Makassar
Dinas Pariwisata
Dinas Pekerjaan Umum
dinas pemberdayaan perempuan dan perlindungan
anak (DP3A) Kota Makassar
Dinas Pendidikan
Dinas Pengendalian Penduduk dan Keluarga
Berencana
Dinas Pertanian dan ketahanan pangan
Dinas Sosial
Dukcapil (Dinas Kependudukan dan Pencatatan Sipil)
Kota Makassar
Forum Jurnalis Sanitasi Makassar
Garden Alley Research Project (US National Science
Foundation)
Helen Keller International
HRH2030 (USAID-funded program)
IUWASH / Indonesia Urban Water, Sanitation and
Hygiene Plus (USAID-funded program)
JALIN (USAID-funded program)
Kantor Balai Kota Makassar
Kantor Smart City
Kecamatan Barrang Lompo
Kementerian Kesehatan
Klinik Cita Sehat
Kompak (Kolaborasi Masyarakat dan Pelayanan untuk
Kesejahteraan)
Kotaku** (Kota Tanpa Kumuh) program
Lembaga Studi Kebijakan Publik (LSKP)
Madani Civil Society Support Initiative (USAID-funded)
Makassar Ecobrick
PDAM (Perusahaan Daerah Air Minum)
Politeknik Kesehatan Makassar
PPDI/Perkumpulan Penyandang Disabilitas Indonesia)
Pusdatin (Pusat Data dan Informasi), Kementerian
Kesehatan
RSUD Kota Makassar
Rumah Zakat
Save the Children
Sekolah Tanpa Batas
Telkom (PT. Telkom Indonesia)
Tri Arta Medika (TAM)
UNICEF
Universitas Hasanuddin (UNHAS)
War Room Makassar
Yayasan Kalla
Other - Write In (Required)
Organizational Background Information (Organizational Attributes)
Page description:
In this section, we will capture a few basic questions about your organization. Please select the response or responses that best
match your organization.
3. Please state the position that best describes your current role in the organization/government department.
*
4. How long have you been working with this organization/government department?*
Less than 1 year
1-3
5-10
10+
5. How familiar personally are you with the BHC program (goals, objectives, and activities)?*
Not at all
A small amount
A fair amount
A great deal
Not sure
6. The SPACES team may follow up with select survey respondents regarding some of their responses to the
survey, for additional clarity or detail. Do we have your consent to follow up with you on your responses if
necessary? *
Yes
No
7. How many employees does your organization have?
0-5
6-25
26-50
51+
8. What category best describes your organization?*
Only select ONE answer
National government
Local government
Local NGO
International NGO
Development Partner (Bilateral or Multilateral agency that provides funding that comes from another
government to address social and economic issues in Indonesia. For example, the World Bank, USAID, or UN
agency)
Development Implementing Partners (IPs)
Academic Institution/Student
Private Sector (including formal companies & MSMEs)
Community- based organization (CBO)
Faith-Based Organization
Other - Write In (Required)
9. At what level does your organization operate?*
Select the PRIMARY level of operations
Global
Regional
National
Provincial
District/Municipal
Other - Write In (Required)
BHC Activities Engagement and Outcomes
Page description:
In this section, we will capture questions about your organization’s engagement and participation in BHC activities and the
outcomes. By engagement, we mean both active contributions as well as passive attendance. Please select the response or
responses that best match your understanding of your organization’s experience.
10. What category best describes your organization/government department's primary sectoral focus?*
Select ONE only
Health
Urban Planning & Development
Information & Communication
Women & Children
Nutrition & Food Safety
Education
Water & Sanitation
Waste Management
Environment & Natural Resources
Transport
Tourism & Commerce
Other - Write In (Required)
11. What category best describes your organization/government department’s primary type of activity?*
Please select only ONE answer
Direct service provision
Regulation of service provision
Monitoring and evaluation
Policy setting and lawmaking
Coordination and planning
Community mobilization/organizing
Public education and/or awareness-raising
Defense of constituent group rights
Information dissemination and reporting
Data collection/ research
Advocacy
Other - Write In (Required)
12. In which year did your organization/government department begin to engage with the BHC program? *
2018
2019
2020
2021
2022
I don’t know
13. For how long did your organization/government department engage with the BHC program?*
<1 year
1-2 years
2-3 years
3-4 years
4-5 years
I don’t know
14. Approximately how many other individuals from your organization engaged in BHC activities other than
yourself? *
0
1-3
4-5
6 or more
I don’t know
15. Which of the following BHC activities did your organization engage in?*
You can choose multiple options.
Systems Mapping Workshop: Defining Contexts
Systems Mapping Workshop: Identifying Leverage
Points City
Systems Mapping Workshop: Action Planning
Journey Mapping Interviews
Needs Assessment(s) (various)
Musrenbang/Prioritization Training(s) using Systems
Mapping/Kumu techniques
Data and Data Integration Training(s)
Citizen Complaint Reporting System & Trainings (112
call center trainings)
Barrang Lompo island nutrition or school environmental
health trainings
Cross City Learning, for example Study Tour(s) or Field
Visits
Healthy City Forum Meetings
Technical Webinars or meetings (ie. Land Use
Implications, strengthening waste management, etc)
Other Partner Meetings on specific technical activities
My organization did not attend any of the above BHC
activities
Other - Write In (Required)
*
Other - Write In (Required)
*
Other - Write In (Required)
*
16. If your organization/government department attended BHC activities, which of the following outcomes
were achieved for your organization? *
You can choose multiple options
Better understanding urban health challenges and
community needs
Better understanding of opportunities for change within
the City of Makassar
New and stronger relationships with other organizations
Improvements to data systems and data-driven
decision making
Increased citizen feedback and empowerment
structures
New knowledge and technical skills gained
Changes in strategic objectives or goals
Changes in organizational policies, processes and/or
practices
Changes to budgets, workplans or spending
None of the previous
17. Which of the following priority objectives did your organization/government department focus on as part
of your engagement in BHC activities? *
You can choose multiple options
Improved multi-sector coordination and collaboration across sectors
Increasing quality, accessibility, and timeliness of data for decision-making.
Ensuring equity and transparency in policy implementation to ensure equitable community access to services.
Maximizing community participation in programs to improve community awareness of health-promoting
practices and resources.
None of the previous
Organizational Self-Assessment within the City of Makassar
Page description:
This Section will capture how you perceive your organization as an actor in the city of Makassar related to BHC activities. Please
select the response or responses that best match your organization’s experience.
18. Which of the following impacts do you believe were achieved in the City of Makassar as a result of BHC?
*
You can choose multiple options
Improved urban health planning, related to changes in policy, regulations, decision making, and/or financing
structures
Improved health and quality of life in Makassar, related to improved health services, improved living conditions
for the urban poor, improved sanitation conditions, reduced pollution, etc.
Increased resilience; the capacity to respond effectively to shocks and stressors such as natural disasters,
economic shocks, food security, health pandemics such as currently in COVID, etc.
None of the above
19. How open do you perceive your organization/ government department to working with others around
BHC-identified priorities and outcomes? *
For example, is your organization/ government department is willing to engage in frank, open and civil
discussion (especially when disagreement exists), and/or communicates in an open, trusting manner around
shared priorities
Not at all
A small amount
A fair amount
A great deal
Not sure
20. How committed do you perceive your organization/ government department to be in coordinating
activities with other organizations/government departments to achieve BHC-identified priorities and
outcomes? *
For example, shows a willingness to share information and plans, to stop activities that may be duplicative or
counter-productive, to shift resources to better aligned to shared goals
Not at all
A small amount
A fair amount
A great deal
Not sure
Readiness for Urban Resilience
Page description:
In this section, we will capture your perception of how your organization responds to urban resilience as a result of engagement in BHC
activities and initiatives. By resilience, we mean the capacity to respond effectively to urban community needs in the aftermath of
impending and unforeseen shocks and stressors (such as natural disasters, economic shocks, food security, health pandemics such as
currently in COVID, etc).
21. How influential do you perceive your organization/government department to be in achieving BHC￾identified priorities and outcomes? *
For example having formal power to influence resourcing and planning, the ability to influence other
organizations and department's decisions, well-connected networks to influence perceptions and actions,
significant human/resources capital to contribute
Not at all
A small amount
A fair amount
A great deal
Not sure
It
stayed
low
It
stayed
medium
It
stayed
high
It
increased
from low
to
medium
It
increased
from low
to high
It
increased
from
medium
to high
It
decreased
from high
to medium
It
decreased
from high
to low
It
decreased
from
medium to
low
I
don’t
know
How has the importance
your
organization/government
department places on
resilience/being
prepared to respond to
shocks and stressors
changed as a result of
BHC programming?
How has the amount of
time your
organization/government
department allocates to
resilience/being
prepared to respond to
shocks and stressors
changed as a result of
BHC programming?
Has the amount of
resources your
organization/government
department allocates to
resilience/being
prepared to respond to
shocks and stressors
changed as a result of
BHC programming?
How has your overall
capacity to respond to
future shocks and
stressors changed as a
result of BHC
programming?
22. Please provide an answer for each row:*
It
stayed
low
It
stayed
medium
It
stayed
high
It
increased
from low
to
medium
It
increased
from low
to high
It
increased
from
medium
to high
It
decreased
from high
to medium
It
decreased
from high
to low
It
decreased
from
medium to
low
I
don’t
know
How has your ability
to have
conversations
across
organizations about
preparing for shocks
and stressors
changed as a result
of BHC
programming?
How has the degree
of collaboration with
other organizations
on responding to
shocks and
stressors changed
as a result of BHC
activities?
To what degree did
engagement in BHC
activities impact
your organization’s
ability to respond in
a timely way to the
COVID-19
pandemic?
To what degree did
engagement in BHC
activities impact
your organization’s
ability to respond
more effectively to
the COVID-19
pandemic?
Opportunities for the Future
Page description:
In this section, we will capture your recommendations for future BHC activities.
23. Please provide an answer for each row:*
Keep Improve Abandon
Systems Mapping Workshops
City Action Planning
Journey Mapping Workshops
Needs Assessment(s) (various)
Systems Mapping/Kumu Training(s)
Data and Data Integration Training(s)
Cross City Learning, for example Study Tour(s) or
Field Visits
Healthy City Forum Meetings
Other Partner Meetings on specific technical
activities
Technical Webinars (ie. Land Use Implications)
Name of organizations/government departments Why you believe they play an important role
1:
2:
3:
4:
5:
24. For each of the following activities and approaches state your opinion if they should be kept, improved,
or abandoned in the potential future BHC activities? *
25. Are there any other organizations/government departments NOT on this roster that you believe play an
important role in improving urban planning efforts and access to health-related services in Makassar and
should be included in future BHC activities? *
Yes
No
26. Please provide up to 5 additional and for each of them state why you believe they play an important role.
*
Please ensure that you provide the organizations/government departments’ commonly recognized name or
acronym. If you are unable to name a specific organization/government department, individuals may be entered
here. Please be confident that the individual represents an organization, even if you cannot recall the name of
the organization.
Relationship Identification
27. What future outcome(s) do you hope your organization/government department achieves to gain from
future engagement with programs like BHC? *
You can choose multiple options
Better understanding urban health challenges and community needs
Better understanding of opportunities for change within the city of Makassar
New and stronger relationships with other organizations
Improvements to data systems and data-driven decision making
Increased citizen feedback and empowerment structures increased
New knowledge and technical skills gained
Changes in strategic objectives or goals
Changes in organizational policies and practices
Other - Write In (required)
*
Other - Write In (Required)
*
28. What future impact(s) do you hope Makassar City gains from future engagement with a programs like
BHC? *
You can choose multiple options
Increased resilience/capacity to respond effectively to shocks and stressors (such as natural disasters,
economic shocks, food security, health pandemics such as currently in COVID, etc)
Improved urban health planning, related to changes in policy, regulations, decision making, and/or financing
structures
Improved health, related to improved health services, improved living conditions for the urban poor, improved
sanitation conditions, reduced pollution, etc.
Other - Write In (Required)
*
29. Please suggest any new activity or approach that was not mentioned in the previous question that you
think is crucial/critical to be introduced in future programming.
Page description:
This section relates to relationships and interactions that members of your organization have in participating in and
advancing multisectoral approaches to urban health in Makassar.
30. Please try to identify at least 10 and ideally 20 organizations with which members of your organization
interact with for participating in and advancing multisectoral approaches to urban health in Makassar, and
answer subsequent questions related to those relationships to be best of your knowledge. *
If none of the organization names on the list are relevant to relationships members of your organization has,
please respond using the "other" options at the end of the list. (The survey will require at least 1 response to this
question, including an "other" response, in order to proceed).
Asian Venture Philantropy Network (APVN)
Asosiasi Ibu Menyusui Indonesia (AIMI)
Badan Perencanaan Pembangunan Daerah
(BAPPEDA) Kota Makassar
Bagian Kerjasama Sekretariat Daerah Kota Makassar
BAKTI (Yayasan Bakti)
BPJS Kesehatan Makassar
Celebes Organic
Dinas Kearsipan Kota Makassar
Dinas Kependudukan dan Pencatatan Sipil
Dinas Kesehatan Kota Makassar
Dinas Komunikasi dan Informatika (Kominfo)
Dinas Komunikasi dan Informatika (Kominfo) Kota
Makassar
Dinas Koperasi dan UMKM
Dinas Lingkungan Hidup
Dinas Lingkungan Hidup Kota Makassar
Dinas Pariwisata
Dinas Pekerjaan Umum
dinas pemberdayaan perempuan dan perlindungan
anak (DP3A) Kota Makassar
Dinas Pendidikan
Dinas Pengendalian Penduduk dan Keluarga
Berencana
Dinas Pertanian dan ketahanan pangan
Dinas Sosial
Dukcapil (Dinas Kependudukan dan Pencatatan Sipil)
Kota Makassar
Forum Jurnalis Sanitasi Makassar
Garden Alley Research Project (US National Science
Foundation)
Helen Keller International
Kantor Desa - Write In which ones (Required)
*
Kantor Smart City
Kecamatan Barrang Lompo
Kementerian Kesehatan
Klinik Cita Sehat
Kompak (Kolaborasi Masyarakat dan Pelayanan untuk
Kesejahteraan)
Kotaku** (Kota Tanpa Kumuh) program
Lembaga Studi Kebijakan Publik (LSKP)
Madani Civil Society Support Initiative (USAID-funded)
Makassar Ecobrick
PDAM (Perusahaan Daerah Air Minum)
Pemerintah daerah (kecamatan, kabupaten) - Write In
which ones (Required)
Politeknik Kesehatan Makassar
PPDI/Perkumpulan Penyandang Disabilitas Indonesia)
Pusdatin (Pusat Data dan Informasi), Kementerian
Kesehatan
Puskesmas - Write In which ones (Required)
*
RSUD Kota Makassar
Rumah Zakat
Save the Children
Sekolah Tanpa Batas
Telkom (PT. Telkom Indonesia)
Tri Arta Medika (TAM)
UNICEF
Universitas Hasanuddin (UNHAS)
War Room Makassar
Assessment of BHC Key Stakeholder Relationships
Page description:
In this section, we will capture the relationship you have with other organizations/government departments and how those
interactions may have been impacted by engagement in the BHC program. It is important that your answers honestly reflect the
nature of your organizations’ interactions. Please respond regarding puskesmas, kantor desa, and pemerintah daerah in general
rather than any one specific entity.
Relationship Questions
HRH2030 (USAID-funded program)
IUWASH / Indonesia Urban Water, Sanitation and
Hygiene Plus (USAID-funded program)
JALIN (USAID-funded program)
Kantor Balai Kota Makassar
Yayasan Kalla
Other - Write In the name (Required)
*
Other - Write In the name (Required)
*
Other - Write In the name (Required)
*
31. How open and engaged in the discussion do you perceive each organization/ government department to
be around BHC-identified priorities and outcomes? *
For example, this organization/ government department is willing to engage in frank, open and civil discussion
(especially when disagreement exists), and/or communicates in an open, trusting manner around shared
priorities
31. To what degree does each organization/ government department seem committed to coordinating
activities they are leading on with other organizations/government departments to achieve BHC-identified
(through participatory systems mapping process) priorities and outcomes? *
For example, showing a willingness to share information and plans, to stop activities that may be duplicative or
counter-productive, to shift resources to better aligned to shared goals
31. How influential do you perceive each organization/government department to be in achieving BHC￾identified (through participatory systems mapping process) priorities and outcomes? *
For example, having formal power to influence resourcing and planning, ability to influence other organizations
and departments decisions, well-connected networks to influence perceptions and actions, significant
human/resources capital to contribute
31. For each organization/government department selected, select all that apply *
Sharing Information/Dialogue
Coordinating on Joint Activities
Social Connection and Networking
31. How, if at all, has the nature of the relationship with each organization/government department changed
as a result of the BHC program? *
by nature of the relationship, we mean, informal or formal, such as having agreements signed
contract(s)/MOUs/charter(s) in place
31. How, if at all, has the trust in your relationship with each organization changed as a result of the BHC
program? *
31. Has the quantity of information sharing/dialogue with each organization/department improved as a result
of the BHC program? *
31. Has the frequency of information sharing/dialogue with each organization/department improved as a
result of the BHC program? *
31. Has the quality of information-sharing/dialogue with each organization/department improved as a result
of the BHC program? *
31. Has the quantity of joint activities with each organization/department improved as a result of the BHC
program? *
31. Has the frequency of joint activities with each organization/department improved as a result of the BHC
program? *
31. Has the quality of joint activities with each organization/department improved as a result of the BHC
program? *
31. Has the quantity of social connection and networking with each organization/department improved as a
result of the BHC program? *
Additional Comments
Page description:
In this section, we will capture additional questions, comments, and recommendations
Thank You!
Thank you for taking our survey. Your response is very important to us.
31. Has the frequency of social connection and networking with each organization/department improved as
a result of the BHC program? *
31. Has the quality of social connection and networking with each organization/department improved as a
result of the BHC program? *
31. Do you have any additional and related comments or concerns that were not addressed by the survey
questions that you would like to share with us?
Survey (Bahasa Indonesian)
SPACES/ Building Healthy Cities (BHC) Social Network Analysis
Pengantar
Page description:
Survei ini adalah survei untuk organisasi-organisasi yang terlibat dalam program Building Healthy City (BHC)/(Membangun
Kota Sehat) di Makassar, Indonesia. Survei ini dirancang untuk memahami bagaimana perubahan kapasitas dan hubungan
para pemangku kepentingan sebagai hasil dari keterlibatannya dalam kegiatan-kegiatan program BHC. Dibutuhkan waktu
sekitar kurang lebih 45 menit untuk menyelesaikan survei ini.
Proyek Building Healthy City (BHC) adalah proyek berdurasi 5-tahun (2017-2022). Proyek ini menggunakan pendekatan sistem
(systems approach), yang berfokus pada faktor-faktor penentu sosial dan lingkungan atas kesehatan, yang terhubung dengan
area-area di luar sektor Kesehatan itu sendiri. Tujuan BHC adalah untuk melihat kemungkinan-kemungkinan pilihan yang
cocok untuk perencanaan kota yang sehat yang merefleksikan nilai-nilai inti berikut:
Mengembangkan pemahaman bersama di berbagai sektor dan pemangku kepentingan tentang cara meningkatkan derajat
kesehatan dan kualitas hidup secara keseluruhan di Makassar.
Memperkuat keterlibatan masyarakat dalam pengambilan keputusan di level Kota Makassar untuk layanan terkait
kesehatan.
Mendukung penggunaan data terkait kesehatan perkotaan untuk perencanaan dan pengambilan keputusan.
Konsorsium SPACES (Strategic Program for Analyzing Complexity and Evaluating Systems) adalah tim peneliti yang
melaksanakan dan menganalisis data survei. JSI, sebagai organisasi utama dalam Program BHC, membantu dalam koordinasi
tetapi tidak menjadi bagian dari tim peneliti. Penelitian ini didanai oleh USAID’s Asia Bureau, Office of Health
(USAID/Asia/Health).
Organisasi Bapak/Ibu/Saudara/Saudari diidentifikasi sebagai pemangku kepentingan yang penting dalam program ini. Survei ini
ditujukan bagi responden yang mengetahui partisipasi organisasi Bapak/Ibu/Saudara/Saudari dalam program BHC serta
mengetahui bagaimana sistem kesehatan perkotaan di Makassar. Jika Bapak/Ibu/Saudara/Saudari mengetahui orang lain yang
lebih cocok untuk survey ini, dimohon bantuan Bapak/Ibu/Saudara/Saudari meminta orang tersebut untuk menggantikan
mengikuti survei ini.
Partisipasi Bapak/Ibu/Saudara/Saudari dalam survei ini bersifat sukarela dan jawaban Bapak/Ibu/Saudara/Saudari hanya akan
dilihat oleh tim peneliti program ini. Laporan yang berasal dari penelitian ini tidak akan menghubungkan jawaban
Bapak/Ibu/Saudara/Saudari dengan nama Bapak/Ibu/Saudara/Saudari dan atau informasi demografis terkait lainnya. Kami
menghargai kesediaan Bapak/Ibu/Saudara/Saudari meluangkan waktu untuk berbagi perspektif dan pengetahuan
Bapak/Ibu/Saudara/Saudari.
Silakan klik di atas untuk menyimpan tanggapan Bapak/Ibu dan melanjutkan survei nanti. Alamat email Ibu/Bapak hanya akan digunakan
untuk tujuan melanjutkan survei. (Jika Bapak/Ibu tidak menyimpan dengan alamat email, Bapak/Ibu akan harus mengulang jawaban
yang telah diberikan.)
Informasi Latar Belakang Responden (Atribut Responden)
Centang “Ya” jika Bapak/Ibu/Saudara/Saudari bersedia. *
Ya
Tidak
Page description:
Pada bagian ini, kami ingin mengetahui informasi dasar demografi dan latar belakang Bapak/Ibu/Saudara/Saudari. Informasi pribadi
Bapak/Ibu/Saudara/Saudari akan dirahasiakan dan tidak akan disebarluaskan/dibagikan di luar dari tim peneliti dan hanya akan digunakan
untuk tujuan agar kami dapat menghubungi Bapak/Ibu/Saudara/Saudari kembali bila kami memerlukan tindak lanjut atau informasi lebih lanjut.
1. Mohon sebutkan nama lengkap Bapak/Ibu/Saudara/Saudari. *
2. Silakan pilih nama organisasi/OPD Bapak/Ibu/Saudara/Saudari dari daftar berikut *
Jika nama organisasi/OPD Bapak/Ibu/Saudara/Saudari tidak ada dalam pilihan yang disediakan mohon pilih “lainnya”
Asian Venture Philantropy Network (APVN)
Asosiasi Ibu Menyusui Indonesia (AIMI)
Badan Perencanaan Pembangunan Daerah (BAPPEDA) Kota Makassar
Bagian Kerjasama Sekretariat Daerah Kota Makassar
BAKTI (Yayasan Bakti)
BPJS Kesehatan Makassar
Celebes Organic
Dinas Kearsipan Kota Makassar
Dinas Kependudukan dan Pencatatan Sipil
Dinas Kesehatan Kota Makassar
Dinas Komunikasi dan Informatika (Kominfo)
Dinas Komunikasi dan Informatika (Kominfo) Kota Makassar
Dinas Koperasi dan UMKM
Dinas Lingkungan Hidup
Dinas Lingkungan Hidup Kota Makassar
Dinas Pariwisata
Dinas Pekerjaan Umum
dinas pemberdayaan perempuan dan perlindungan anak
(DP3A) Kota Makassar
Dinas Pendidikan
Dinas Pengendalian Penduduk dan Keluarga Berencana
Dinas Pertanian dan ketahanan pangan
Dinas Sosial
Dukcapil (Dinas Kependudukan dan Pencatatan Sipil) Kota Makassar
Forum Jurnalis Sanitasi Makassar
Garden Alley Research Project (US National Science
Foundation)
Helen Keller International
HRH2030 (USAID-funded program)
IUWASH / Indonesia Urban Water, Sanitation and Hygiene
Plus (USAID-funded program)
JALIN (USAID-funded program)
Kantor Balai Kota Makassar
Kantor Smart City
Kecamatan Barrang Lompo
Kementerian Kesehatan
Klinik Cita Sehat
Kompak (Kolaborasi Masyarakat dan Pelayanan untuk
Kesejahteraan)
Kotaku** (Kota Tanpa Kumuh) program
Lembaga Studi Kebijakan Publik (LSKP)
Madani Civil Society Support Initiative (USAID-funded)
Makassar Ecobrick
PDAM (Perusahaan Daerah Air Minum)
Politeknik Kesehatan Makassar
PPDI/Perkumpulan Penyandang Disabilitas Indonesia)
Pusdatin (Pusat Data dan Informasi), Kementerian Kesehatan
RSUD Kota Makassar
Rumah Zakat
Save the Children
Sekolah Tanpa Batas
Telkom (PT. Telkom Indonesia)
Tri Arta Medika (TAM)
UNICEF
Universitas Hasanuddin (UNHAS)
War Room Makassar
Yayasan Kalla
Lainnya - Tuliskan (Wajib)
Informasi latar belakang organisasi (Atribut organisasi)
Page description:
Pada bagian ini, kami ingin mengetahui informasi-informasi dasar tentang organisasi Bapak/Ibu/Saudara/Saudari. Silakan pilih jawaban￾jawaban yang paling sesuai menggambarkan organisasi Bapak/Ibu/Saudara/Saudari.
3. Sebutkan posisi yang paling sesuai menggambarkan peran Bapak/Ibu/Saudara/Saudari saat ini di
organisasi/OPD/Kantor pemerintah tempat Bapak/Ibu/Saudara/Saudari bekerja. *
4. Sudah berapa lama Bapak/Ibu/Saudara/Saudari bekerja di organisasi/OPD/Kantor pemerintah tempat
Bapak/Ibu/Saudara/Saudari saat ini? *
kurang dari 1 tahun
1-3
5-10
10+
5. Secara personal, seberapa akrab Bapak/Ibu/Saudara/Saudari dengan program BHC (sasaran, tujuan, dan kegiatan
BHC)? *
Tidak akrab Sama sekali
Sedikit akrab
Cukup akrab
Sangat akrab
Tidak yakin
6. Tim SPACES mungkin akan menghubungi Bapak/Ibu/Saudara/Saudari untuk mengklarifikasi atau untuk penjelasan
lebih lanjut. Apakah Bapak/Ibu/Saudara/Saudari bersedia jika kami menghubungi Bapak/Ibu/Saudara?saudari kembali
jika diperlukan? *
Ya
Tidak
7. Berapa banyak staf yang organisasi Bapak/Ibu/Saudara/Saudari imiliki?
0-5
6-25
26-50
51+
8. Dari kategori berikut mana yang paling sesuai menggambarkan organisasi Bapak/Ibu/Saudara/Saudari?*
Pilih salah SATU
Pemerintah nasional/pusat
Pemerintah lokal (misalnya, Provinsi/Kabupaten/Kota)
LSM lokal
LSM Internasional
Lembaga Donor Pembangunan (Badan Bilateral atau Multilateral yang menyediakan dana yang berasal dari pemerintah lain
untuk mengatasi masalah sosial dan ekonomi di Indonesia, misalnya Bank Dunia, USAID, atau badan PBB)
Mitra Pelaksana Pembangunan/Mitra Lembaga Donor
Institusi Akademik/Mahasiswa
Sektor Swasta (termasuk perusahaan formal & UMKM)
Organisasi Kemasyarakatan/Organisasi berbasis masyarakat
Organisasi Berbasis Keyakinan/Organisasi keagamaan
Lainnya, tuliskan (wajib)
9. Level organisasi Bapak/Ibu/Saudara/Saudari bekerja? *
Pilih level yang paling utama dimana organisasi beroperasi
Global
Regional
Nasional
Provinsi
Kabupaten/Kota
Lainnya - Tuliskan (Wajib)
Keterlibatan dengan Kegiatan BHC dan Hasilnya
Page description:
Pada bagian ini, kami ingin mengetahui tentang keterlibatan dan partisipasi organisasi Bapak/Ibu/Saudara/Saudari dalam kegiatan BHC dan
hasilnya. Yang kami maksud keterlibatan adalah baik berupa kontribusi aktif maupun kehadiran pasif. Silahkan pilih jawaban yang paling
sesuai dengan yang Bapak/Ibu/Saudara/Saudari ketahui tentang keterlibatan organisasi Bapak/Ibu/Saudara/Saudari.
10. Kategori apa yang paling sesuai menggambarkan fokus sektoral utama organisasi/OPD/Kantor
Bapak/Ibu/Saudara/Saudari *
Pilih SATU saja
Kesehatan
Perencanaan & Pembangunan Kota
Informasi & Komunikasi
Perempuan & Anak-anak
Nutrisi & Keamanan Pangan
Pendidikan
Air & Sanitasi
Penanganan limbah
Lingkungan & Sumber Daya Alam
Transportasi
Pariwisata & Perdagangan
Lainnya, tuliskan (wajib)
11. Kategori apa yang paling menggambarkan jenis kegiatan utama organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari?*
Pilih salah SATU
Penyedia langsung layanan publik
Mengatur/mengawasi penyediaan layanan
Monitoring dan evaluasi
Pembuat kebijakan dan legislasi
Koordinasi dan perencanaan
Mobilisasi/pengorganisasian masyarakat
Pendidikan publik dan/atau peningkatan kesadaran
Pembelaan hak kelompok konstituen
Pelaporan dan Penyebaran Informasi
Pengumpulan data/penelitian
Advokasi
Lainnya, tuliskan (wajib)
12. Pada tahun berapa organisasi/OPD Bapak/Ibu/Saudara/Saudari mulai terlibat dengan program BHC?*
2018
2019
2020
2021
2022
Tidak tahu
13. Berapa lama organisasi/ OPD Bapak/Ibu/Saudara/Saudari terlibat dalam program BHC?*
<1 bertahun-tahun
1-2 tahun
2-3 tahun
3-4 tahun
4-5 tahun
I don’t know
14. Kira-kira berapa banyak orang lain dari organisasi Bapak/Ibu/Saudara/Saudari yang terlibat dalam kegiatan BHC
selain Bapak/Ibu/Saudara/Saudari sendiri? *
0
1-3
4-5
6 atau lebih
Tidak tahu
15. Dari Kegiatan BHC berikut mana yang organisasi Bapak/Ibu/Saudara/Saudari ikut terlibat?*
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Lokakarya Pemetaan Sistem: Mendefinisikan Konteks
Lokakarya Pemetaan Sistem: Mengidentifikasi Kota dengan
Poin Leverage
Lokakarya Pemetaan Sistem: Perencanaan Aksi
Wawancara “Pemetaan Perjalanan”/Journey Mapping
Penilaian Kebutuhan
Musrenbang/Pelatihan Pemerioritasan menggunakan teknik
Pemetaan Sistem/Kumu
Pelatihan Integrasi Data dan Pelatihan Data
Sistem Pelaporan Pengaduan Warga dan Pelatihan sistem
pengaduan warga (112 pelatihan call center)
Pelatihan gizi pulau Barrang Lompo atau Pelatihan kesehatan
lingkungan sekolah
Pembelajaran Antar Kota, misalnya Study Tour atau
Kunjungan Lapangan
Pertemuan Forum Kota Sehat
Webinar atau pertemuan teknis (mis. Implikasi Penggunaan
Lahan, penguatan pengelolaan sampah, dll)
Pertemuan Mitra lainnya tentang kegiatan teknis terten
Organisasi saya tidak menghadiri salah satu kegiatan BHC di
atas
Lainnya, Tuliskan (wajib)
*
Lainnya, Tuliskan (wajib)
*
Lainnya, Tuliskan (wajib)
*
Penilian Mandiri Organisasi di Kota Makassar
Page description:
Bagian ini ingin mengetahui bagaimana Bapak/Ibu/Saudara/Saudari menilai organisasi Bapak/Ibu/Saudara/Saudari sebagai salah satu aktor di
kota Makassar terkait dengan kegiatan BHC. Pilih jawaban yang paling sesuai dengan pengalaman organisasi Bapak/Ibu/Saudara/Saudari.
16. Jika organisasi/OPD Bapak/Ibu/Saudara/Saudari menghadiri kegiatan BHC, manakah dari hasil berikut yang dicapai
oleh organisasi Bapak/Ibu/Saudara/Saudari? *
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Lebih memahami tantangan kesehatan perkotaan dan
kebutuhan masyarakat
Pemahaman yang lebih baik tentang peluang perubahan di
Kota Makassar
Hubungan baru dan lebih kuat dengan organisasi lain
Perbaikan sistem data dan pengambilan keputusan
berdasarkan data
Peningkatan struktur pemberdayaan warga dan umpan balik
dari warga
Memperoleh pengetahuan baru dan keterampilan teknis
Perubahan pada tujuan atau sasaran strategis
Perubahan dalam kebijakan, proses, dan/atau praktik
organisasi
Perubahan anggaran, rencana kerja, atau pengeluaran
Tidak ada dari salah satu yang disebutkan diatas
17. Manakah dari tujuan prioritas berikut yang menjadi fokus organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari sebagai
bagian dari keterlibatan Bapak/Ibu/Saudara/Saudari dalam kegiatan BHC? *
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Peningkatan koordinasi multi-sektor dan kolaborasi lintas sektor
Meningkatkan kualitas, aksesibilitas, dan ketepatan waktu tersedinya ( ( ) data untuk pengambilan keputusan.
Memastikan kesetaraan dan transparansi dalam implementasi kebijakan untuk memastikan akses masyarakat terhadap
layanan yang adil.
Memaksimalkan partisipasi masyarakat dalam program untuk meningkatkan kesadaran masyarakat atas praktik dan sumber
daya yang mempromosikan kesehatan.
Tidak ada dari salah satu yang disebutkan diatas
18. Manakah dari dampak berikut yang Bapak/Ibu/Saudara/Saudari yakini dicapai oleh Kota Makassar sebagai hasil dari
program BHC? *
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Peningkatan perencanaan kesehatan kota, terkait dengan perubahan kebijakan, peraturan, pengambilan keputusan, dan/atau
struktur pembiayaan
Peningkatan kesehatan dan kualitas hidup di Makassar, terkait dengan peningkatan pelayanan kesehatan, perbaikan kondisi
kehidupan masyarakat miskin perkotaan, perbaikan kondisi sanitasi, pengurangan polusi, dll.
Peningkatan ketangguhan/ketahanan; kapasitas untuk merespon secara efektif terhadap guncangan dan stresor seperti
bencana alam, guncangan ekonomi, ketahanan pangan, pandemi kesehatan (seperti COVID19 saat ini), dll.
Tidak ada dari salah satu yang disebutkan diatas
Kesiapan untuk Ketangguhan Perkotaan
Page description:
Pada bagian ini, kami ingin mengetahui persepsi Bapak/Ibu/Saudara/Saudari tentang bagaimana organisasi
Bapak/Ibu/Saudara/Saudari merespon terhadap ketangguhan/ketahanan perkotaan sebagai hasil dari keterlibatan dalam
kegiatan dan inisiatif BHC. Yang kami maksud dengan ketangguhan/ketahanan adalah kemampuan untuk merespon secara
efektif kebutuhan masyarakat perkotaan setelah guncangan dan stresor yang datang dan tidak terduga (seperti bencana alam,
guncangan ekonomi, ketahanan pangan, pandemi seperti COVID19 saat ini, dll).
19. Seberapa terbuka Bapak/Ibu/Saudara/Saudari menilai organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari untuk
bekerja dengan orang lain untuk mencapai prioritas dan hasil BHC? *
Misalnya, apakah organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari bersedia untuk terlibat dalam diskusi yang jujur,
terbuka, dan diskusi untuk mencapai kesepahaman (terutama jika ada ketidaksepakatan), dan/atau berkomunikasi secara
terbuka dan saling percaya atas prioritas bersama
Sangat tidak tebuka
Sedikit terbuka
Cukup terbuka
Sangat terbuka
Tidak yakin
20. Seberapa berkomitmen organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari dalam mengoordinasikan kegiatan
dengan organisasi/OPD/Kantor lain untuk mencapai prioritas dan hasil yang diidentifikasi BHC? *
Misalnya, menunjukkan kesediaan untuk berbagi informasi dan rencana, untuk mencegah kegiatan yang mungkin
sama/tumpang tindih atau kontra-produktif, untuk mengalihkan sumber daya agar lebih selaras dengan tujuan bersama
Sangat tidak berkomitmen
Sedikit berkomitmen
Cukup berkomitmen
Sangat berkomitmen
Tidak yakin
21. Menurut Bapak/Ibu/Saudara/Saudari seberapa berpengaruh organisasi/OPD/Kantor Bapak/Ibu/Saudara/Saudari dalam
mencapai prioritas dan hasil yang diidentifikasi BHC ? *
Misalnya memiliki kekuatan formal untuk memengaruhi pengalokasian sumber daya dan perencanaan, kemampuan untuk
memengaruhi keputusan organisasi dan OPD/Kantor lain, jaringan yang terhubung dengan baik untuk memengaruhi persepsi
dan tindakan, modal manusia/sumber daya yang signifikan untuk berkontribusi
Sangat tidak berpengaruh
Sedikit berpengaruh
Cukup berpengaruh
Sangat berpengaruh
Tidak yakin
Tetap/masih
rendah
Tetap/masih
sedang
Tetap/masih
tinggi
Meningkat
dari
rendah ke
sedang
Meningkat
dari
rendah ke
tinggi
Meningkat
dari
sedang ke
tinggi
Menurun
dari
sedang
ke
rendah
Menurun
dari
tinggi ke
rendah
Menurun
dari
tinggi ke
sedang
Saya
tidak
tahu
Bagaimana posisi penting
organisasi/OPD/Kantor
Bapak/Ibu/Saudara/Saudari
terhadap
ketangguhan/ketahanan/kesiapan
untuk merespon guncangan dan
stresor berubah sebagai akibat
dari program BHC?
Bagaimana jumlah waktu yang
dialokasikan
organisasi/OPD/Kantor
Bapak/Ibu/Saudara/Saudari untuk
Ketangguhan/ketahanan/kesiapan
untuk merespon guncangan dan
stresor berubah sebagai akibat
dari program BHC?
Apakah jumlah sumber daya yang
dialokasikan
organisasi/OPD/Kantor
Bapak/Ibu/Saudara/Saudari untuk
Ketangguhan/ketahanan/kesiapan
untuk merespon guncangan dan
stresor berubah sebagai akibat
dari program BHC?
Bagaimana kapasitas
Bapak/Ibu/Saudara/Saudari
secara keseluruhan untuk merespon guncangan dan stresor
yang akan datang berubah
sebagai akibat dari program
BHC?
22. Mohon berikan jawaban untuk setiap kolom: *
Tetap/masih
rendah
Tetap/masih
sedang
Tetap/masih
tinggi
Meningkat
dari
rendah ke
sedang
Meningkat
dari
rendah ke
tinggi
Meningkat
dari
sedang ke
tinggi
Menurun
dari
sedang
ke
rendah
Menurun
dari
tinggi ke
rendah
Menurun
dari
tinggi ke
sedang
Saya
tidak
tahu
Bagaimana kemampuan
Bapak/Ibu/Saudara/Saudari
untuk melakukan dialog
dengan seluruh organisasi
tentang persiapan menghadapi goncangan
dan stresor berubah
sebagai akibat dari
program BHC?
Bagaimana tingkat
kolaborasi dengan
organisasi lain dalam
merespon guncangan dan
stresor berubah sebagai
akibat dari kegiatan BHC?
Sejauh mana keterlibatan
dalam kegiatan BHC memengaruhi kemampuan
organisasi
Bapak/Ibu/Saudara/Saudari
untuk merespon pandemi
COVID-19 dengan tepat
waktu ?
Sejauh mana keterlibatan
dalam kegiatan BHC memengaruhi kemampuan
organisasi
Bapak/Ibu/Saudara/Saudari
untuk merespon pandemi
COVID-19 secara lebih
efektif ?
Peluang ke depan
Page description:
Pada bagian ini, kami ingin mengetahui rekomendasi Bapak/Ibu/Saudara/Saudari untuk kegiatan BHC di masa mendatang.
Dipertahankan Ditingkatkan dibuang
Lokakarya Pemetaan Sistem
Perencanaan Aksi Kota
Lokakarya Pemetaan Perjalanan/Journey Mapping
Penilaian Kebutuhan
Pelatihan Pemetaan Sistem/Kumu
Data and Data Integration Training(s)
Pembelajaran Antar Kota, misalnya Study Tour atau
Kunjungan Lapangan
Pertemuan Forum Kota Sehat
Pertemuan Mitra lainnya tentang kegiatan teknis tertentu
Webinar Teknis (mis. Implikasi Penggunaan Lahan)
23. Mohon berikan jawaban untuk setiap kolom: *
24. Untuk setiap kegiatan dan pendekatan berikut/di bawah ini, berikan pendapat Bapak/Ibu/Saudara/Saudari apakah
kegiatan dan pendekatan tersebut harus dipertahankan, ditingkatkan, atau dibuang dalam kegiatan BHC di masa
mendatang? *
Nama Organisasi/OPD mengapa Bapak/Ibu/Saudara/Saudari menganggap mereka memainkan peran penting.
1:
2:
3:
4:
5:
25. Apakah ada organisasi/OPD/Kantor lain yang TIDAK masuk dalam daftar ini yang Bapak/Ibu/Saudara/Saudari yakini
memiliki peran penting dalam meningkatkan upaya perencanaan kota dan akses ke layanan terkait kesehatan di
Makassar dan harus dimasukkan dalam kegiatan BHC di masa mendatang? *
Ya
Tidak
26. Harap sebutkan hingga 5 nama organisasi dan masing-masing dari organisasi tersebut mohon berikan alasan
mengapa Bapak/Ibu/Saudara/Saudari menganggap mereka memainkan peran penting. *
Harap [Pastikan Bapak/Ibu/Saudara/Saudari memberikan nama atau akronim organisasi/OPD/Kantor tersebut. Jika
Bapak/Ibu/Saudara/Saudari tidak dapat menyebutkan organisasi/OPD/Kantor tertentu, anda dapat memasukkan perorangan.
Harap diyakinkan bahwa individu tersebut mewakili sebuah organisasi, walaupun Bapak/Ibu/Saudara/Saudari tidak dapat
mengingat nama organisasinya.
27. Apa hasil yang Bapak/Ibu/Saudara/Saudari harapkan dapat dicapai oleh organisasi/OPD/Kantor
Bapak/Ibu/Saudara/Saudari di masa depan dari keterlibatan dengan BHC di masa yang akan datang? *
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Lebih memahami tantangan kesehatan perkotaan dan kebutuhan masyarakat
Pemahaman yang lebih baik tentang peluang perubahan di kota Makassar
Hubungan baru dan lebih kuat dengan organisasi lain
Perbaikan sistem data dan pengambilan keputusan berdasarkan data
Peningkatan umpan balik warga dan peningkatan struktur pemberdayaan warga
Memperoleh pengetahuan baru dan keterampilan teknis baru
Perubahan tujuan atau sasaran strategis
Perubahan dalam kebijakan dan praktik organisasi
Lainnya, tuliskan (Wajib)
*
Lainnya, tuliskan (Wajib)
*
Identifikasi Hubungan
Page description:
Bagian ini berkaitan dengan hubungan dan interaksi yang dimiliki oleh anggota organisasi Bapak/Ibu/Saudara/Saudari dalam
berpartisipasi dan memajukan pendekatan multisektoral untuk sektor kesehatan perkotaan di Kota Makassar.
28. Apa dampak yang Bapak/Ibu/Saudara/Saudari harapkan diperoleh Kota Makassar di masa depan dari keterlibatan
dengan BHC di masa yang akan datang? *
Bapak/Ibu/Saudara/Saudari bisa memilih lebih dari satu pilihan.
Peningkatan Ketangguhan/ketahanan/kapasitas untuk merespon secara efektif terhadap guncangan dan stresor (seperti
bencana alam, guncangan ekonomi, ketahanan pangan, pandemi kesehatan seperti COVID19 saat ini, dll)
Peningkatan perencanaan kesehatan kota, terkait dengan perubahan kebijakan, peraturan, pengambilan keputusan, dan/atau
struktur pembiayaan
Peningkatan derajat kesehatan, terkait dengan peningkatan pelayanan kesehatan, perbaikan kondisi kehidupan masyarakat miskin perkotaan, perbaikan kondisi sanitasi, pengurangan polusi, dll.
Lainnya, tuliskan (Wajib)
*
29. Silakan usulkan kegiatan atau pendekatan yang tidak disebutkan dalam pertanyaan-pertanyaan diatas yang menurut
Bapak/Ibu/Saudara/Saudari penting/kritikal untuk diperkenalkan pada fase berikutnya.
30. Mohon identifikasi setidaknya 10 dan idealnya 20 organisasi yang berinteraksi dengan anggota organisasi/OPD
Bapak/Ibu?saudara/Saudari dalam berpartisipasi dan memajukan pendekatan multisektoral dalam sektor kesehatan
perkotaan di Kota Makassar, dan mohon jawab pertanyaan berikut yang terkait dengan hubungan tersebut sesuai
dengan yang Bapak/Ibu/Saudara/Saudari ketahui. *
Jika nama-nama organisasi dalam daftar tidak ada yang relevan dengan hubungan yang dimiliki anggota organisasi
Bapak/Ibu/Saudara/Saudari, harap memilih jawaban "lainnya". (diharuskan setidaknya memilih satu pilihan jawaban untuk
dapat lanjut ke pertanyaan berikutnya).
Asian Venture Philantropy Network (APVN)
Asosiasi Ibu Menyusui Indonesia (AIMI)
Badan Perencanaan Pembangunan Daerah (BAPPEDA) Kota Makassar
Bagian Kerjasama Sekretariat Daerah Kota Makassar
BAKTI (Yayasan Bakti)
BPJS Kesehatan Makassar
Celebes Organic
Dinas Kearsipan Kota Makassar
Dinas Kependudukan dan Pencatatan Sipil
Dinas Kesehatan Kota Makassar
Dinas Komunikasi dan Informatika (Kominfo)
Dinas Komunikasi dan Informatika (Kominfo) Kota Makassar
Dinas Koperasi dan UMKM
Kantor Desa/Kelurahan - Tuliskan nama desa/Kelurahan
(Wajib)
*
Kantor Smart City
Kecamatan Barrang Lompo
Kementerian Kesehatan
Klinik Cita Sehat
Kompak (Kolaborasi Masyarakat dan Pelayanan untuk
Kesejahteraan)
Kotaku** (Kota Tanpa Kumuh) program
Lembaga Studi Kebijakan Publik (LSKP)
Madani Civil Society Support Initiative (USAID-funded)
Makassar Ecobrick
PDAM (Perusahaan Daerah Air Minum)
Penilaian Hubungan Pemangku Kepentingan Utama BHC
Page description:
Di bagian ini, kami ingin mengetahui bagaimana hubungan Bapak/Ibu/Saudara/Saudari dengan organisasi/OPD lain dan bagaimana interaksi
tersebut dipengaruhi oleh keterlibatan Bapak/Ibu/Saudara/Saudari dalam program BHC. Penting agar jawaban Bapak/Ibu/Saudara/Saudari
secara jujur mencerminkan sifat interaksi organisasi Bapak/Ibu/Saudara/Saudari tersebut. Mohon memberi tanggapannya tentang puskesmas,
kantor desa/lurah, dan pemerintah daerah secara umum, bukan satu entitas tertentu.
Dinas Lingkungan Hidup
Dinas Lingkungan Hidup Kota Makassar
Dinas Pariwisata
Dinas Pekerjaan Umum
dinas pemberdayaan perempuan dan perlindungan anak
(DP3A) Kota Makassar
Dinas Pendidikan
Dinas Pengendalian Penduduk dan Keluarga Berencana
Dinas Pertanian dan ketahanan pangan
Dinas Sosial
Dukcapil (Dinas Kependudukan dan Pencatatan Sipil) Kota Makassar
Forum Jurnalis Sanitasi Makassar
Garden Alley Research Project (US National Science
Foundation)
Helen Keller International
HRH2030 (USAID-funded program)
IUWASH / Indonesia Urban Water, Sanitation and Hygiene
Plus (USAID-funded program)
JALIN (USAID-funded program)
Kantor Balai Kota Makassar
Pemerintah daerah (kecamatan, kabupaten) - Tuliskan nama
Kecamatan/Kabupaten (Wajib)
Politeknik Kesehatan Makassar
PPDI/Perkumpulan Penyandang Disabilitas Indonesia)
Pusdatin (Pusat Data dan Informasi), Kementerian Kesehatan
Puskesmas - Tuliskan nama Puskesmas (Wajib))
*
RSUD Kota Makassar
Rumah Zakat
Save the Children
Sekolah Tanpa Batas
Telkom (PT. Telkom Indonesia)
Tri Arta Medika (TAM)
UNICEF
Universitas Hasanuddin (UNHAS)
War Room Makassar
Yayasan Kalla
Lainnya - Tuliskan (Wajib)
*
Lainnya - Tuliskan (Wajib)
*
Lainnya - Tuliskan (Wajib)
*
31. Seberapa terbuka dan seberapa terlibat dalam diskusi menurut Bapak/Ibu/Saudara/Saudari organisasi/OPD/kantor ini
terhadap prioritas dan hasil yang diidentifikasi BHC? *
Misalnya, organisasi/OPD ini bersedia terlibat dalam diskusi yang jujur, terbuka, dan diskusi untuk mencapai kesepahaman
(terutama jika ada ketidaksepakatan), dan/atau berkomunikasi secara terbuka dan saling percaya terkait prioritas bersama
31. Sejauh mana organisasi/OPD ini tampaknya berkomitmen untuk mengoordinasikan kegiatan yang mereka pimpin
dengan organisasi/OPD/Kantor lain untuk mencapai prioritas dan hasil yang diidentifikasi BHC (melalui proses pemetaan
sistem partisipatif)? *
Misalnya, menunjukkan kesediaan untuk berbagi informasi dan rencana, untuk mencegah kegiatan yang mungkin
sama/tumpang tindih atau kontra-produktif, untuk mengalihkan sumber daya agar lebih selaras dengan tujuan bersama
Pertanyaan Hubungan
Page description:
Berbagi Informasi/Dialog
Page description:
Koordinasi Kegiatan Bersama
Page description:
31. Menurut Bapak/Ibu/Saudara/Saudari seberapa berpengaruh organisasi/OPD/Kantor ini dalam mencapai prioritas dan
hasil yang diidentifikasi BHC ? *
Misalnya memiliki kekuatan formal untuk memengaruhi pengalokasian sumber daya dan perencanaan, kemampuan untuk
memengaruhi keputusan organisasi dan OPD/Kantor lain, jaringan yang terhubung dengan baik untuk memengaruhi persepsi
dan tindakan, modal manusia/sumber daya yang signifikan untuk berkontribusi
31. Untuk organisasi/OPD yang Bapak/Ibu/Saudara/Saudari pilih, pilih semua pernyataan yang sesuai *
31. Bagaimana sifat hubungan dengan organisasi/OPD ini berubah sebagai akibat dari program BHC?*
yang kami maksud dengan sifat hubungan adalah informal atau formal, seperti memiliki perjanjian resmi yang ditandatangani
misalnya Perjanjian/Kontrak/MoU/Akta
31. Bagaimana, jika ada, kepercayaan dalam hubungan Bapak/Ibu/Saudara/Saudari dengan organisasi ini berubah
sebagai akibat dari program BHC ? *
31. Apakah kuantitas pertukaran informasi/dialog dengan masing-masing organisasi/OPD/kantor tersebut meningkat
sebagai hasil dari program BHC? *
31. Apakah frekuensi berbagi/dialog informasi dengan masing-masing organisasi/OPD/kantor tersebut meningkat
sebagai hasil dari program BHC? *
31. Apakah kualitas pertukaran informasi/dialog dengan setiap organisasi/OPD/Kantor tersebut meningkat sebagai hasil
dari program BHC? *
31. Apakah kuantitas kegiatan bersama dengan masing-masing organisasi/OPD/Kantor tersebut meningkat sebagai hasil
dari program BHC? *
Hubungan Sosial dan Jaringan
Page description:
Komentar Tambahan
Page description:
Pada bagian ini, kami ingin mengetahui jika Bapak/Ibu/Saudara/Saudari memiliki pertanyaan, komentar, dan rekomendasi tambahan.
Terima kasih!
Terima kasih telah berpartisipasi dalam survei kami. Jawaban Anda sangat penting bagi kami.
31. Apakah frekuensi kegiatan bersama dengan masing-masing organisasi/OPD/Kantor tersebut meningkat sebagai hasil
dari program BHC? *
31. Apakah kualitas kegiatan bersama dengan masing-masing organisasi/OPD/kantor tersebut meningkat sebagai hasil
dari program BHC? *
31. Apakah kuantitas hubungan sosial dan jaringan dengan masing-masing organisasi/OPD/Kantor tersebut meningkat
sebagai hasil dari program BHC? *
31. Apakah frekuensi hubungan sosial dan jaringan dengan masing-masing organisasi/OPD/Kantor tersebut meningkat
sebagai akibat dari program BHC? *
31. Apakah kualitas hubungan sosial dan jaringan dengan masing-masing organisasi/OPD/Kantor tersebut meningkat
sebagai hasil dari program BHC? *
31. Apakah Bapak/Ibu/Saudara/Saudari memiliki komentar atau hal-hal terkait lainnya yang tidak tercakup dalam
pertanyaan-pertanyaan survei ini yang ingin Bapak/Ibu/Saudara/Saudari sampaikan kepada kami?
Engagement Tracker
Organizations are listed in alphabetical order
# Institution (English cleaned name) Institution (INDONESIAN 
NAME)
Activity 
"score"
TYPE OF 
INSTITUTION KII SNA 
Survey
SNA 
MAP Type_Binary
1
Academy of Occupational Health and Safety 
(Makassar) Akademi Hiperkes Makassar 1
Private Higher 
education institution 
(Academy) No No No
2 Among Tani Foundation Among Tani Foundation 1 Local NGO/CSO No No No
3
Asian Venture Philantropy Network 
(APVN)
Asian Venture Philantropy 
Network (APVN) 0 INGO No No No
4
Indonesian Breastfeeding Mothers' 
Association (AIMI)
Asosiasi Ibu Menyusui Indonesia 
(AIMI) 1
Local 
NGO/CSO/community 
group Yes Yes Yes Other actors
5 City Planning and Development Agency
Badan Perencanaan 
Pembangunan Daerah 
(BAPPEDA) Kota Makassar 23
Government (city 
level) Yes Yes Yes
Sub-national 
government
6
Collaboration Department- Secretary of 
Makassar City
Bagian Kerjasama Sekretariat 
Daerah Kota Makassar 0 Government (city) Yes No Yes
Sub-national 
government
7 BAKTI (Yayasan Bakti) BAKTI (Yayasan Bakti) 2
Local 
NGO/CSO/community 
group No No Yes Other actors
8
Geophysical, Climatology, Meteorology 
Agency
BMKG (Badan Metereologi, 
Klimatologi dan Geofisika) 1 Government (level?) No No No
9
BPJS Kesehatan Makassar (health insurance 
agency) BPJS Kesehatan Makassar 1 Government (city) No No Yes
Sub-national 
government
10
BPKS (Badan Pengusahaan Kawasan Sabang) 
governmental services oversight body
BPKS (Badan Pengusahaan 
Kawasan Sabang) 1 Government (regional) No No No
11 Celebes Organic Celebes Organic 1
Local 
NGO/CSO/community 
group No Partial Yes Other actors
12 Archives Department Dinas Kearsipan Kota Makassar 0 Government (city) No No No
13 Marine and Fisheries Department Dinas Kelautan dan Perikanan 1 Government (city) No No No
14
Dukcapil (Population and Civil Registration 
Agency)
Dukcapil (Dinas Kependudukan 
dan Pencatatan Sipil) Kota 
Makassar 1 Government (city) No No Yes
Sub-national 
government
15 Health Department Dinas Kesehatan Kota Makassar 6 Government (city) Yes No Yes
Sub-national 
government
16 Workforce Department Dinas Ketenagakerjaan 1 Government (city) No No No
17 Communications & Informatics Department
Dinas Komunikasi dan 
Informatika (Kominfo) Kota 
Makassar 11 Government (city) Yes No Yes
Sub-national 
government
18
Cooperative Department (Small enterprises 
department) Dinas Koperasi dan UMKM 3 Government (city) No No Yes
Sub-national 
government
19 Environmental Department
Dinas Lingkungan Hidup Kota 
Makassar 5 Government (city) Yes No Yes
Sub-national 
government
20 Tourism Department Dinas Pariwisata 1 Government (city) No No Yes
Sub-national 
government
21 Public Works Department Dinas Pekerjaan Umum 5 Government (city) Yes Partial Yes
Sub-national 
government
22 Fire Department DInas Pemadam Kebakaran 1 Government (city) No No No
23
Department of Women's Empowerment 
and Child Protection
dinas pemberdayaan perempuan 
dan perlindungan anak (DP3A) 
Kota Makassar 3 Government (city) No No Yes
Sub-national 
government
24 Educational Department Dinas Pendidikan 2 Government (city) No No Yes
Sub-national 
government
25 Population Control and Family Planning
Dinas Pengendalian Penduduk 
dan Keluarga Berencana 1 Government (city) No Partial Yes
Sub-national 
government
26 Trade Department Dinas Perdagangan 1 Government (city) No No No
27 Library Department DINAS PERPUSTAKAAN 1 Government (city) No No No
28
Department of Agriculture and Food 
Security
Dinas Pertanian dan ketahanan 
pangan 2 Government (city) No No Yes
Sub-national 
government
29 Housing and Residential Areas Department
Dinas Perumahan Kawasan 
Pemukiman Dan Pertanahan Sul￾sel 1
Government 
(Provincial) No No No
30 Housing and Residential Areas Department Dinas Perumahan Kota Makassar 1 Government (city) No No No
31 Social Department Dinas Sosial 2 Government (city) No Yes Yes
Sub-national 
government
32 Directorate of Immigration General Direktorat Jendral Imigrasi 1 Government (level?) No No No
33
Indonesia Association of Person with 
Diasabilities
Disability Organization 
(PPDI/Perkumpulan Penyandang 
Disabilitas Indonesia) 1
Local 
NGO/CSO/community 
group No Yes Yes Other actors
34 EWINDO EWINDO 1 Private company No No No
35 Makassar Sanitation Journalist Forum Forum Jurnalis Sanitasi Makassar 3
Local 
NGO/CSO/community 
group No No Yes Other actors
36
Health Polytechnic (Politeknik Kesehatan 
Makassar) - Kementerian Kesehatan
Health Polytechnic, MoH, 
Makassar (Politeknik Kesehatan 
Makassar) 1
University (public 
polytechnic) No No Yes
Sub-national 
government
37 Helen Keller International Helen Keller 1 INGO No No Yes Other actors
38 HRH2030 (USAID program name) HRH2030 2 Donor program No No No
39 Humanitarian Forum Indonesia Humanitarian Forum Indonesia 1 Local NGO/CSO No No No
40 Regional Inspectorate Inspektorat Daerah 1 Government (city) No No No
41 Office of the Mayor Makassar City Kantor Balai Kota Makassar 0 Government (city) No No Yes
Sub-national 
government
42 Smart City Office Kantor Smart City 0 ? No No No
43 Health Ministry Kementerian Kesehatan 1 Government (national) No No Yes Other actors
44 Cita Sehat Clinic Klinik Cita Sehat 4 Local NGO/CSO Yes Yes Yes Other actors
45
Kompak (Kolaborasi Masyarakat dan 
Pelayanan untuk Kesejahteraan)
Kompak (Kolaborasi Masyarakat 
dan Pelayanan untuk 
Kesejahteraan) 1 Donor program No No Yes Other actors
46 Indonesia National Slum Upgrading Project
Kotaku** (Kota Tanpa Kumuh) 
program 8 Government Yes Yes Yes Other actors
47 Institute for Public Policy Studies
Lembaga Studi Kebijakan Publik 
(LSKP) ** 4 Local NGO/CSO Yes Yes Yes Other actors
48 Makassar Ecobrick Makassar Ecobrick 2
Local 
NGO/CSO/community 
group Yes Yes Yes Other actors
49
National Science Foundation (Garden Alley 
Research Project)
National Science Foundation 
(Garden Alley Research Project) 1 Donor program No No No
50 City enterprise for clean water
PDAM (Perusahaan Daerah Air 
Minum) 2
State owned 
enterprise No No Yes
Sub-national 
government
51 State-owned enterprise, oil & natural gas Pertamina 1 State-owned company No No No
52
Blindness Association (Indonesia Blind 
Association)
Pertuni (Persatuan Tunanetra 
Indonesia) 0
Local 
NGO/CSO/community 
group No No No
53
Data and Information Center of Ministry of 
Health
Pusdatin (Pusat Data dan 
Informasi) 1 Government (national) No Yes Yes Other actors
54 RNS MSM RNS MSM** 1 ? No No No
55 General Hospital - PMC indramayu RSU PMC indramayu 1 Hospital No No No
56
Local General Hospital - H.A. Sulthan Dg. 
Radja
RSUD H.A. Sulthan Dg. Radja 
Kabupaten Bulukumba 1
District General 
Hospital of Bulukumba No No No
57 City Hospital RSUD Kota Makassar 3 Hospital No Partial Yes
Sub-national 
government
58 Rumah Zakat Rumah Zakat** 7
Local 
NGO/CSO/community 
group Yes Yes Yes Other actors
59 Save the Children Save the Children** 5 INGO No No Yes Other actors
60
Sekolah Tanpa Batas (School without 
Borders) Sekolah Tanpa Batas ** 7 Local NGO/CSO Yes Yes Yes Other actors
61 Telkom - phone services Telkom (PT. Telkom Indonesia) 1
State owned 
enterprise No No Yes Other actors
62 Tri Arta Medika (TAM) Tri Arta Medika (TAM) 3 Private company No No No
63 Dayanu Ikhsanuddin University Universitas Dayanu Ikhsanuddin 1 University No No No
64 Esa Unggul University Universitas Esa Unggul 1 University No No No
65 Gadjah Mada University Universitas Gadjah Mada 1 University No No No
66 Hasanuddin University (UNHAS)
Universitas Hasanuddin 
(UNHAS) 8 University (Indonesian) Yes No Yes Other actors
67 Indonesia University Universitas Indonesia 1 University No No No
68 Mega Buana Palopo University Universitas Mega Buana Palopo 1 University No No No
69 Sriwijaya University Universitas Sriwijaya 1 University No No No
70 Tanjungpura University Universitas Tanjungpura 1 University No No No
71 University of Oslo University of Oslo 13 University (Foreign) Yes No No
72
School in Bone District three hours from 
Makassar UPT SMAN 5 (BONE) 1 Government (city) No No No
73 USAID Indonesia USAID Indonesia 1 Donor No No No
74
USAID Indonesia Urban Water, Sanitation 
and Hygiene (IUWASH) Plus
USAID Indonesia Urban Water, 
Sanitation and Hygiene 
(IUWASH) Plus 2 Donor program No No Yes Other actors
75 USAID JALIN Project USAID JALIN Project 0 Donor program No Yes Yes Other actors
76
USAID Madani Civil Society Support 
Initiative
USAID Madani Civil Society 
Support Initiative** 5 Donor program No No Yes Other actors
77 UNICEF Wash UNICEF 2 Donor program No No Yes Other actors
78 Yayasan Kalla Yayasan Kalla 3
Local 
NGO/CSO/community 
group No Partial Yes Other actors
79 Yayasan Mazarina Yayasan Mazarina 1 Local NGO/CSO No No No
Were not on the tracker
* Barrannglompo Island Village Desa Pulau Barrannglompo 4 Government (village) Yes No No
* Community Health Clinic Barrang Lompo Puskesmas Barrang Lompo 1
Community Health 
Clinic Yes Yes Yes
Sub-national 
government
** DHIS2 Indonesia DHIS2 Indonesia No Yes Yes Other actors
USAID MPHD USAID MPHD No Partial No
Village Office - Barrang Lompo Kantor Desa - Barrang Lompo No No Yes
Sub-national 
government
Kulurahan Kaluku Bodoa Kulurahan Kaluku Bodoa No No Yes
Sub-national 
government
Village Office - Maccini Sombala Kantor Desa - Maccini Sombala No No Yes
Sub-national 
government
Barrang Lompo District Kecamatan Barrang Lompo No No Yes
Sub
-national 
government
Kelurahan Barrang Caddi Kelurahan Barrang Caddi No No Yes
Sub
-national 
government
Regional Government (District, Regency)
Pemerintah daerah (kecamatan, 
kabupaten) No No Yes
Sub
-national 
government
Public health center Puskesmas No No Yes
Sub
-national 
government
Public health center 
- Maccini Sombala Puskesmas 
- Maccini Sombala No No Yes
Sub
-national 
government
Village Office 
- Rappokalling Kantor Desa 
- Rappokalling No No Yes
Sub
-national 
government
Residence shelter Residence shelter No No Yes [Unknown]
War Room Makassar War Room Makassar No No Yes
Sub
-national 
government
* Were in the tracker master list 
** Related to University of Oslo (#71)
85
G– STATEMENT OF WORK
Joint Partnership Plan for SPACES Monitoring, Evaluation, Research, and Learning
BETWEEN
USAID MERLIN Team (Office of Evaluation, PPL/LER)
AND 
USAID/ASIA/Health
This joint partnership plan (“partnership plan”) and any attachments or annexes thereto, is made and 
entered into by and between the United States Agency for International Development (USAID) MERLIN 
Team (Office of Evaluation) (“the MERLIN Team”), and USAID/ASIA/Health on this XX day of February, 
2021 (“Effective Date”).
The purpose of this partnership plan is to set the terms and understanding between the USAID -
MERLIN Team and the USAID Asia Bureau Health team (ABHT) under the SPACES approach, focused 
on systems, methods, and tools.
Background 
SPACES is an initiative of the Monitoring, Evaluation, Research and Learning Innovations (MERLIN) 
Program. SPACES aims to bring a variety of systems tools and methodologies to USAID Operating 
Units, missions and bureaus. 
All development efforts involve working in complex systems. Unaided, understanding and addressing 
such systems presents a number of risks. Without an understanding of the dynamics of the system, 
interventions can be unsustainable, resulting in multiple secondary, tertiary and reverberating effects, 
and can even lead to unintended consequences. As Operating Units go through the process of designing 
development programs, it is critical to use an approach that engages with and addresses systems 
relevant to the development efforts to avoid these pitfalls.
Tools to help better understand and address systems have transformed numerous other fields such as 
transportation, manufacturing, aerospace engineering, and meteorology. Currently, there is a need for a 
suite of tools to help decision makers dealing with complex systems in international development. 
Simply translating tools from other fields is not enough, since international development has unique 
elements and challenges. There is a need for a suite of systems tools tailored appropriately for
international development.
Systems approaches to program design and implementation for USAID and its partners is an operational 
innovation championed and pioneered by the PPL and being tested throughout Operating Units within 
the Agency. 
The SPACES approach includes an array of systems tools and methods to assist USAID to address the 
many challenges to multiple development objectives at multiple stages of the project cycle, including 
design, monitoring, evaluation and learning. For this scope of work, SPACES will conduct a network 
analysis - SPACES will develop a network diagram of BHC stakeholders before BHC and at present to 
demonstrate how relationships have changed over time due to BHC tools and further how the change 
in relationships is affecting the achievement of project goals. The diagram will be based on data collected 
during key informant interviews (KIIs) and quantitative surveys which will be conducted with ABHT 
stakeholders, BHC staff, local stakeholders (e.g., municipal government) who worked with BHC, 
86
members of stakeholder organizations who both did and did not work directly with BHC (as speaking 
with actors who did not work directly with BHC can provide key information about the system and its 
dynamics) and citizens. To understand how BHC conceptualizes resilience the SPACES team will 
conduct KIIs to determine how different stakeholders in different contexts define resilience. SPACES 
will then visually represent (e.g., concept map) these different definitions of resilience, highlighting how
this varies between stakeholders and will also draw out how these different understandings of resilience 
and BHC’s role in affecting this understanding ultimately changed city leadership’s preparation and 
response to future shocks. The team will also identify contextual factors, via surveys and KIIs that have 
either facilitated or hindered effective implementation of the BHC model, and analyze survey and 
interview responses to identify enabling conditions for BHC to better understand the validity of the 
BHC theory of change which can inform how to improve the sustainability of future approaches.
Purpose
This Joint Partnership Plan (JPP) outlines a scope for partnership between the MERLIN Team and USAID 
ABHT. An overview of the partnership scope of engagement through SPACES is outlined below. 
SPACES Scope of Engagement
The MERLIN and SPACES Commitment to Partnership:
The MERLIN Team is committed to providing resources that will ensure the best possible environment 
for successful application of SPACES. The MERLIN Team is committed through the SPACES Consortium 
to conduct an evaluation of BHC (see SOW):
ABHT Commitment to Partnership:
ABHT is committed to working in partnership with the SPACES Consortium through the MERLIN Team 
to:
● Identify a single point of contact to coordinate with SPACES on the evaluation
● Provide necessary staff time and resources to support the involvement of the SPACES 
Consortium, including regular phone calls with SPACES Consortium.
● Involve the SPACES Consortium (or representatives within) in key decision-making processes as 
related to MERL opportunities.
● Provide resources, if needed and as agreed to, to support activities as related to SPACES 
activities.
● Commit to data sharing based on information collected that can support SPACES activities and 
learning agenda.
● Participate in qualitative interviews and quantitative surveys as related to SPACES activities and 
the MERLIN learning agenda.
● Provide SPACES with contact information and introductory emails for other stakeholders 
involved in BHC.
● Commit to collaborative learning and transparent decision-making with SPACES Consortium to 
ensure a joint partnership for the SPACES engagement.
● Review identified deliverables before finalization
BHC’s role in the evaluation:
● Identify a single point of contact to coordinate with SPACES on the evaluation
● Provide necessary staff time and resources to support the involvement of the SPACES 
Consortium, including regular calls with SPACES Consortium and coordinating with SPACES’ 
data collection 
● Commit to data and document sharing to support SPACES’ evaluation
● Participate in qualitative interviews and quantitative surveys as related to SPACES activities
87
● Send introductory emails with stakeholders about KIIs/survey being conducted by SPACES
● Provide list of stakeholders and contact information
Learning Agenda for SPACES
Each SPACES pilot engagement is designed to meet the monitoring, evaluation, research and learning 
needs of its activity, while also contributing to a larger MERLIN learning agenda. This includes testing the 
various MERLIN approaches to contribute to a broader learning agenda with SPACES as an approach. 
Each engagement will contribute to a set of core questions about using SPACES approaches in USAID, 
as well as questions specific to the approach used and the activity being monitored. To facilitate these 
dual learning objectives, each SPACES engagement will involve a series of M&E approaches to assess 
whether SPACES is effective as an approach and in generating better program outcomes.
Periodically, the SPACES consortium or the Policy Planning and Learning Bureau’s office of Learning, 
Evaluation and Research (PPL/LER) will solicit feedback and document the learning process and its 
outcomes with USAID/ASIA/Health participants. Data sources may include program/extant data, rapidly 
conducted survey data, key informant interviews, and/or focus groups. This may involve participation by 
representatives from the MERLIN Team (EIA), SPACES, and other implementing organizations. 
Agreement
The Partnership between the MERLIN Team and USAID/ASIA/Health will begin on the date of 
agreement to this document, via email, between the MERLIN Team and the USAID/ASIA/Health, and 
last a duration of 6 months. These dates are conditional on the USAID/ASIA/Health and SPACES
implementation timelines, and as such are subject to change following discussion among all parties 
involved. The JPP can be updated with a new end-date should USAID/ASIA/Health decide to 
incrementally fund this project. 
Timeline and deliverables for SPACES: Undertaken for six months following the transfer of 
funding, the evaluation is anticipated to deliver presentations of findings and an evaluation report, both 
to include analysis of the results of data collection.
Key Contacts
USAID/ASIA/Health
Name: Lucy Mize
Organization: USAID
Title: Health Team Lead/AOR for BHC
Email: lmize@usaid.gov
Phone: 802-685-2296 or 703-258-4113
Name: Jean Paul Petraud
Organization: USAID
Title: Senior Evaluation Specialist
Email: jpertraud@usaid.gov
Phone: 202 322 3467.
MERLIN Lab Team points of contact:
Name: Sophia van der Bijl
Organization: USAID
Title: Senior Impact Assessment Advisor
Email: svanderbijl@usaid.gov
88
Phone: 571-228-3836
SPACES Consortium Contacts:
Name: Bruce Y. Lee
Organization: PHICOR/JHU
Title: SPACES PI, Executive Director, PHICOR
Email: bruceleemdmba@gmail.com
Phone: (646) 364-9523
Building Healthy Cities Evaluation 
SPACES
Scope of Work 
Purpose and Use
The purpose of this evaluation is to answer exploratory learning questions related to the Building Health 
Cities program (BHC). These questions investigate what worked, what did not work (and why), the 
conditions under which the BHC approach is likely to be effective, and how lessons on urban resilience 
for future programming. Specifically, the evaluation aims to understand: 
4. How BHC’s approach to engaging stakeholders affected BHC implementation, 
5. How stakeholders conceptualize resilience and determine how these conceptualizations affect 
how city leaders act, and 
6. How BHC’s model can inform future urban project design.
User Use
ABHT
● Understand which approaches/interventions were effective in 
achieving project goals and why
● Inform design/implementation of future Asia Bureau projects 
related to urban development
BHC Implementing 
Partners
● Integrate/reference relevant evaluation findings from the 
Evaluation in PMP and reporting
The ABHT determined that the following intended uses are outside the purview of the Evaluation:
● Manage and/or modify activities being implemented in years 4-5 of the project
● Assess the performance of the IP
● Assess the impact of activities (as a baseline was not conducted)
Background
The USAID-funded Building Healthy Cities (BHC) project is partnering with Smart Cities in Indore, 
India, Makassar, Indonesia, Kathmandu, Nepal, and Da Nang, Vietnam with the goals of increasing 
equitable access to healthy lifestyles, improving access to health-related services, strengthening the 
community voice, decrease environmental and lifestyle risk factors for chronic diseases, decrease the 
burden of infectious diseases, and enable wider use of data that supports community decision making to 
make different choices which then better includes health outcomes in city planning. 
89
Planning for a Smart City is intrinsically linked to health: transportation, natural resources, sanitation, 
education, recreation, technology, and the built environment all influence the health of an urban 
population. BHC employs a participatory systems mapping approach to clearly articulate these linkages 
in each context, engages a wide range of stakeholders across sectors, and formulates a set of priority 
actions for each city. BHC works in partnership with Smart City initiatives and urban sector 
departments to build this new vision for healthy urban planning. In addition, BHC engages Smart City 
citizens of every demographic with a focus on marginalized populations so that they have a voice (via 
citizen reporting systems and participatory research) and are empowered to advocate for the needs of 
their communities. 
BHC recently completed Year 3 of their 5-year cooperative agreement. BHC has been working with 
three Smart Cities including Da Nang, Indore, and Makasssar, and in fiscal year 2020 began an operation 
in Kathmandu. To date, the project has focused on establishing and deepening relationships in each city 
(in the community and within municipal government), facilitating sustainable connections across sectors, 
completing the systems mapping process, supporting the execution of city-level interventions and 
making recommendations for multi-sectoral health activities. However, this evaluation is going to focus 
on only one country: Indonesia (Makassar).
Evaluation Questions
In addition to the uses stated above, the SPACES team discussed some of the following, overarching 
motivations for the Evaluation with the ABHT. Based on the common themes and priority interests, 
SPACES and ABHT arrived at the following preliminary research questions:
Note: The term “stakeholders” below is inclusive of USAID implementing partners under BHC, local 
partners, municipal government, Smart City Boards, and beneficiaries (i.e., community members and 
those involved in co-creation activities) engaged in each city.
Questions Motivation
4. Model/Approach/Tools
a. Stakeholder engagement: how, 
who, failures, successes. How 
did engagement between all 
stakeholders facilitate or hinder 
implementation and 
achievement of project goals?
b. How successful were the 
tools/assessments in achieving 
project goals (i.e., what worked 
and what didn’t work)?
USAID wishes to:
● understand feasibility/viability of achieving 
project goals via the stakeholder engagement 
tools
● support the “harmony” that resulted from 
the well-functioning engagement 
structures/processes/functions 
● determine how to better address the 
“disharmony” resulting from obstacles to 
stakeholder cooperation 
● understand how BHC uses evidence/findings 
● understand whether the stakeholders are 
acting upon findings and experiences of BHC 
tools/assessments and recommendations
90
5. Resilience
a. How does BHC define the 
concept of resilience, measure 
it, and contribute to achieving 
more resilient cities? 
b. How did city leaders better 
respond to community needs as 
a result of BHC programming?
USAID wishes to: 
● understand if/how the concept of resilience 
has evolved over the course of the project 
and differed across the implementing cities 
● know whether and the extent to which the 
technical focus (i.e., the six areas) of BHC’s 
approach was integrated into this 
conceptualization of resilience.
● determine to what extent these 
conceptualizations of resilience affected 
activity design/implementation and if these 
activities in turn contributed to the resiliency 
of the city (if so - how was this measured)
● understand how the city leaders were 
enabled/empowered to improve their 
preparation and response to future shocks 
and how BHC supported these 
improvements
6. Way forward: How could the BHC 
cross-sectoral models in the different 
country contexts be applied and 
sustained in the future?
USAID wishes to: 
● better understand BHC’s operating context 
in order to determine the 
appropriateness/applicability of the cross￾sectoral model for other country contexts
● equip future projects with the tools to learn, 
adapt, and improve quickly and effectively to 
improve chances of sustainability
● understand the validity of the theory of 
change
Each of these questions will be examined through the lenses of “What? Why? What does this mean for 
the future?” with an eye towards understanding what improvements can be made in a successor project. 
1. The Approach
1.1 Overview of Design and Methodology
The SPACES approach includes an array of systems tools and methods to assist USAID to address the 
many challenges to multiple development objectives at multiple stages of the project cycle, including 
design, monitoring, evaluation and learning. For this scope of work, SPACES will conduct a network 
analysis - SPACES will develop a network diagram of BHC stakeholders before BHC and at present to 
demonstrate how relationships have changed over time due to BHC tools and further how the change 
in relationships is affecting the achievement of project goals. The diagram will be based on data collected 
during key informant interviews (KIIs) and quantitative surveys which will be conducted with ABHT 
stakeholders, BHC staff, local stakeholders (e.g., municipal government) who worked with BHC, 
members of stakeholder organizations who both did and did not work directly with BHC (as speaking 
with actors who did not work directly with BHC can provide key information about the system and its 
dynamics) and citizens. To understand how BHC conceptualizes resilience the SPACES team will 
conduct KIIs to determine how different stakeholders in different contexts define resilience. SPACES 
91
will then visually represent (e.g., concept map) these different definitions of resilience, highlighting how 
this varies across cities and will also draw out how these different understandings of resilience and 
BHC’s role in affecting this understanding ultimately changed city leadership’s preparation and response 
to future shocks. The team will also identify contextual factors, via surveys and KIIs that have either 
facilitated or hindered effective implementation of the BHC model, and analyze survey and interview 
responses to identify enabling conditions for BHC to better understand the validity of the BHC theory 
of change which can inform how to improve the sustainability of future approaches.
1.2 Geographic scope
The ABHT intends to focus the Evaluation on Makassar as implementation in this city is mature enough 
to provide sufficient answers to the evaluation questions. 
1.3 Phases and Timeline
PHASE 1: Pre-Evaluation and Start-up (estimated Start of evaluation - Month 1.5)
Step 1. Evaluation Team Recruitment: In the time prior to implementation, SPACES will 
identify/recruit as many of the team members as possible, and work on various 
administrative tasks that will prepare the team for a prompt launch upon ABHT’s funding 
obligation. Each team member’s qualifications and job description will be shared with ABHT 
for review. 
Step 2. Assemble existing data on key outcomes of interest for all evaluation questions
a. This step will involve systematically gathering intelligence and data from a wide range of 
sources. This includes but is not limited to the ABHT literature review, BHC project 
documents3 (e.g., project description, annual and quarterly reports, BHC workplans, 
agendas, presentations, and other materials used for their stakeholder engagements and 
workshops), and relevant background information from quantitative indicators, Smart 
Cities documents and, literature on citizen-government relations in Makassar (e.g., 
Smart City Workplans - as city include BHC activities in their work plans)
b. Identify any data gaps and determine how additional information will be gathered (e.g., 
via surveys/interviews)
c. Identify key informants to interview and respondents for survey
Step 3. Establish Evaluation design, workplan, and launch
d. During this step SPACES will host a kick off meeting with our SPACES team and 
relevant USAID and BHC representatives. This session will focus on clarifying the SOW 
ensuring alignment around the purpose of the evaluation; identifying how findings will be 
used; and confirm roles, responsibilities (outlined in section 6 and annex) and norms for 
stakeholder outreach. 
e. Produce evaluation design and workplan 
Step 4. Propose and implement a communication strategy to stakeholders and 
beneficiaries as appropriate for the methodology proposed
f. The communication strategy should be cleared by USAID with input from BHC/JSI
g. USAID PoC should be copied for main communications to stakeholders
PHASE 2: Implementation (estimated Month 1.5 - Month 5)
The steps outlined below outline the steps necessary to answer the evaluation questions, organized by 
Evaluation question. We will provide a chronological plan for the entire evaluation in the evaluation 
design and workplan. 
Step 1. Develop and test data collection tools
a. SPACES will co-design the KII protocol, the survey, and data collection tools along with 
a set of Trainer of Trainers modules with key stakeholders. We will hold a virtual 
3 All BHC project documents have been compiled in a spreadsheet with links to a google folder.
92
trainer of trainers workshop to build the capacity of the local enumerators in systems 
sensing, systems mapping, and data collection. This will also include plans for ensuring 
willing participation in surveys and KIIs. We will tap into our local partner networks in 
India and Indonesia to identify enumerators in each country. The interview and survey 
will be conducted with stakeholders including ABHT stakeholders, BHC staff, local 
stakeholders (e.g., municipal government) who worked with BHC, members of 
stakeholder organizations who both did and did not work directly with BHC (as 
speaking with actors who did not work directly with BHC can provide key information 
about the system and its dynamics) and citizens. 
Note: KIIs will be conducted virtually and recorded, transcribed, and coded using a 
platform like NVivo so that common themes can be identified and the quality of 
evidence can be assessed, including external validation of evidence where possible and 
applicable.
To ensure we are getting a representative and comprehensive understanding of the 
network we will select a diverse sample (e.g., balance of gender, age, socioeconomic 
status, role in the system). This sample criteria will be co-created with the ABHT and 
BHC team and could target those that are intimately involved with BHC, those that are 
affected by BHCs initiatives, and those that are tangentially affected. 
KIIs and surveys will gather information necessary to answer evaluation questions. The 
specific outline and plan for each data collection tool will be included in the evaluation 
design and workplan. 
Evaluation Question 1: Stakeholder Engagement Social Network Analysis
Step 2. Define what is a partnership/relationship between stakeholders
a. This step will involve determining a definition for what a partnership is between 
stakeholders that will serve as the basis for the network diagrams developed in Steps 4 
and 5. This will be done by conducting KIIs. Based on these, we will use the best/most 
appropriate definition. 
Step 3. Identify the stakeholders and how they are connected pre- BHC and now that 
BHC is in progress and determine the level and quality of the connection (some 
measure of the relationships & how they are defining relationships)
b. This step will involve identifying all of the various stakeholders that are relevant and 
determining how these stakeholders are connected prior to implementing BHC, using 
the definition determined in Step 2. Further, this step will also include determining how 
stakeholders are connected with BHC (e.g., new connections). We will also determine 
the level of the connections between all the stakeholders identified in this step. This will 
entail determining some measure of the strength and quality of the relationships, as 
defined in Step 2. This step will be done by reviewing documents about BHC (e.g., 
program documents, reports) and will build and iterate on Step 2. Once we further 
refine the network diagram of key stakeholders, and the other actors they engaged with 
to achieve the desired project goals, we will be able to flag any other types of 
respondents that would be useful to survey or interview. The network diagram 
refinement described in the steps below, as well as data collection (via KIIs /surveys) will 
be an iterative process.
Step 4. Develop a network diagram pre-BHC and conduct network analyses
93
c. This step will entail developing a network diagram of the stakeholders identified in Step 
3 as well as their connections and the level of the connection prior to the 
implementation of BHC. The pre-BHC diagram will use information gathered from key 
informant interviews on whether connections existed pre-BHC, what were the extent 
of those connections, etc. In order to limit recall bias we will establish a retrospective 
baseline through an extensive data review from a range of sources. Pulling from grant 
lists, website communications, outreach lists and other BHC project documents such as 
event/webinar registrations lists will help us establish existing connections to build up a 
baseline that is not overly dependent on recall. We will also look at the current nature 
of the relationship and if recent interactions align with the kinds of principles and actions 
advocated under the BHC activity. We will then conduct social network analyses of this 
network to establish how these stakeholders were connected to each other pre-BHC. 
This will include various measures, for example, degree, reciprocity, in-degree, out￾degree and betweenness between various stakeholders, as well as measures such as 
subgroups and cliques analyses looking at the groups of stakeholders. Each of these 
measures will be outlined using standard definitions in the evaluation design and 
workplan. This step and Step 5 will be informed by and build on existing systems maps 
and network analyses where appropriate. 
Step 5. Develop a network diagram now that BHC is in progress and conduct network 
analyses
d. This step will entail developing a second network diagram with BHC and will include the 
new connections and how connections have changed using the information gathered in 
the steps above. The social network analyses performed in Step 4 will be conducted on 
this network. 
Step 6. Compare the network with and without BHC and conduct network change 
analyses 
e. This step will compare the resulting network diagrams with and without BHC and 
consist of a network change analysis. The analysis can include determining the 
differences and evaluating the effect of BHC tools on the various social network 
analyses from Steps 4 and 5. This can be used to determine and quantify how various 
relationships and strengths of the connections have changed with BHC. The method for 
quantifying relationships will be further outlined in the evaluation design and workplan. 
Step 7. For each of these changed relationships, identify tools that may have been 
involved in altering these relationships
f. The BHC initiative used multiple tools (e.g., participatory systems mapping workshop) 
to connect various stakeholders. This step will identify each of the tools used by BHC 
via document review and KIIs to determine how specific tools led to changes in specific 
relationships that are identified and analyzed in Steps 2-6 above. This step will provide a 
foundation to assess specifically how these tools affected each relationship. This step will 
also help understand when and where the tool intended to change a relationship but did 
not. 
Step 8. For each tool, identify and list the mechanisms that led to the relationship being 
changed. This includes representing the relationship without BHC, each of the steps 
involved in the BHC tool and how it connects partners, and the eventual end￾relationship. 
94
g. Building off of Step 7, this step aims to identify, list, and ultimately represent the 
mechanisms involved in each relationship changed with BHC. For each relationship 
affected by BHC, this will include identifying the mechanisms that led to the relationship 
being changed. For example, while the participatory mapping workshop is one tool that 
BHC used to connect stakeholders, what were the specific and stepwise processes that 
occurred as a part of this that led to changes? This should be done for each altered 
relationship. 
Step 9. For each of these tools and relationships, represent these mechanisms identified 
in step 7 as a layer in the diagram 
h. Once the mechanisms and processes that affected each relationship are identified, the 
next step will be to layer in these mechanisms and processes to the diagram. The 
diagram will show the relationships of key stakeholders without BHC, the mechanisms 
and processes of each tool that connected these stakeholders, and the relationships of 
stakeholders with BHC. 
Step 10. Incorporate into the diagram, the mechanisms for how these connections 
affected the implementation of the BHC project and project goals/outcomes. 
i. Building on the diagram layer developed in Step 9, this step will add the changed 
mechanisms and processes that resulted from the formation of new relationships. While 
these mechanisms will begin to be identified and sketched out during the initial phases of 
the diagram (e.g. Steps 4 and 5), this step will build on the network diagram to include a 
layer focused on how these relationships affected implementation. This step should 
include representations of not just project goals and outcomes that resulted from new 
relationships, but the specific steps that were taken by connected stakeholders to 
achieve these outcomes for all Activity Areas. 
Evaluation Question 2: Resilience
Step 11. Conduct landscaping review of the literature on frameworks for understanding 
resilience.
j. The literature will inform a framework to help frame and focus interviews, for example, 
identifying different aspects or components of resilience that can be explored in more 
depth. This framework will include ABHT’s conceptualization of resilience: strengthening 
the capacity of the actors to be responsive to community needs.
k. Once the literature review is complete, the framework will be reviewed with ABHT and 
JSI, refined, and then used as a basis for KIIs.
Step 12. Collect definitions of resilience from BHC stakeholders in Makassar
l. Use the surveys and KIIs to collect definitions of resilience from BHC stakeholders. 
Probe about their understanding of resilience. Use prompts in KIIs to see if their 
understanding includes dimensions such as “plan,” “absorb,” “recover,” and “adapt”. 
Ask if they feel resilience is a concept that applies to individuals, households, 
communities, institutions, nations. Ask if they think it can apply to value chains and even 
systems. Ask if they have adopted any specific framework such as USAIDs Resilience 
Framework? We will use USAID’s Resilience measurement framework to inform our 
questioning in the KIIs.
m. Ask if they think resilience, as they have defined it, has been operationalized as part of 
the BHC programs. In other words, do they feel that BHC programs increased 
resilience? Ask if their household, community, or institutions have an emergency 
response plan.
95
n. Ask if they can give an example of when they observed resilience in action in their 
locality/community in relation to the work of BHC (at any level). This will produce 
stories we can use in our reporting. Examples could be from the health sector (natural 
disaster, mass casualty incident, COVID-19 response) or non-health sector. 
Step 13. Visualize and conduct analyses to interpret the collected definitions of 
resilience and determine how they may vary by stakeholder and context (e.g., concept 
maps). 
o. Concept mapping provides a visual illustration that clarifies relationships between 
concepts and ideas. In this case, concept maps will be used to depict stakeholders’ 
understanding of “resilience” and how these conceptualizations of resilience relate to 
each other. The process of constructing concept maps of resilience for each city using 
the data collected in Step 12 will help draw out the various ways that different 
stakeholders conceptualize resilience and illustrate how these vary between cities. 
Illustrating these different conceptualizations of resilience and determining how they are 
related to one another will also help elucidate how these definitions and understandings 
may have engaged with each other and evolved over the course of the BHC project.
Step 14. Conduct KIIs to determine whether and how these conceptualizations affected 
BHC programming and how city leaders better responded to community needs as a 
result of BHC programming.
p. By using a combination of KII interview guides developed in Phase 2 and definition maps 
generated in Step 13, SPACES will conduct KIIs reflecting conceptualizations of 
resilience back to BHC and stakeholders to seek out any connections between these 
ideas and BHC programming. KIIs will also investigate whether stakeholders are aware 
of any evidence that BHC programming across all Activity Areas ultimately affected 
cities’ resilience and how city leaders responded to community needs. As noted above 
KIIs will be conducted virtually and recorded, transcribed, and coded so that common 
themes and the quality of evidence can be identified and assessed, including external 
validation of evidence where possible and applicable. 
Evaluation Question 3: Way Forward
Step 15. Identify barriers/facilitators to the BHC program to inform how USAID could 
implement programs that sustainably address urban challenges in the future
a. This step will use the findings from evaluation question 1 that explore the changes in 
relationships between stakeholders that occurred over the course of the BHC 
project and how BHC tools contributed to those changes as a basis for identification 
of factors that either facilitated or hindered the implementation of the BHC program. 
Specifically, this will involve understanding how the context impacted steps involved in 
establishing relationships between stakeholders and how these steps changed over 
time. Contextual factors may or may not begin to be identified from these analyses; 
either way, these findings will provide important points of focus and direction for 
further exploration. 
b. Use the survey and KIIs to identify or further explore factors that have either 
facilitated or hindered effective implementation of the BHC agenda. 
c. Ask for examples of why things worked or why they did not work to understand 
what are deeply rooted and context-constrained factors. 
Analyze
96
d. Consider enabling conditions, external and internal, that advanced or hindered BHC’s 
approach. Considering institutional factors, infrastructure, cost, policy, willingness to 
act, competing priorities, etc.
e. Identify which factors required multisectoral approaches (as opposed to those that 
are siloed and operate in one sector)
f. Formal structures and systems set up to move work forward map these processes, 
systems and structure and assess the effectiveness of these
Understand systems and approaches to equip future projects to learn, adapt, and 
improve quickly and effectively to improve chances of sustainability
a. Ask questions in KIIs about what practices they have developed AND 
institutionalized to capture learnings for future programming
b. Ask for examples of learning practices adopted (formal, informal, information sharing, 
use of IT platforms, etc.)
Theory of Change
a. Ask questions in the KIIs about the extent to which the theories of change have 
informed BHC activities and whether there is evidence to support the theories of 
change. 
b. Use the systems map in each city to look at the leverage hypotheses and ask 
questions in the KIIs about whether these interventions and/or outcomes will be 
sustained and whether the leverage hypotheses have guided actions and produced the 
benefits to the system that were anticipated. 
Evaluation Matrix
Questions Data to Collect Methodology
Model/Approach/Tools
● Stakeholder engagement: 
how, who, failures, 
successes. How did 
engagement between all 
stakeholders facilitate or 
hinder implementation and 
achievement of project 
goals?
● How successful were the 
tools/assessments in 
achieving project goals (i.e. 
what worked and what 
didn’t work)?
● Identifying stakeholders and their 
connections before and after/during 
BHC
● Characteristics of relationships that 
exist before and after BHC (e.g., 
strength, frequency of 
communication, formal vs. informal, 
effective)
● Steps involved in establishing 
relationships between stakeholders, 
how the context impacted these 
steps and how these steps changed 
over time
● Reasons for effective vs. ineffective 
relationships or relationships that 
didn’t form
● How connections between 
stakeholders affect different 
organizational processes/program 
implementation goals and to what 
extent these relationships affected the 
process, efficiency of BHC activities
● Conduct key informant 
interviews and surveys
● Develop network 
diagrams
● Conduct social 
network analyses 
(SNAs). SNAs are a 
highly useful tool for 
representing 
relationships and 
networks, including the 
number of 
connections, strength 
of connections, and 
other useful metrics. 
● Develop systems 
map(s). 
97
Resilience
● How does BHC define the 
concept of resilience, 
measure it, and contribute 
to achieving more resilient 
cities? 
● How did city leaders 
better respond to 
community needs as a 
result of BHC 
programming?
● How BHC stakeholders in 2 different 
cities conceptualize resilience and 
how this has changed over time
● Are the six areas of BHC included in 
the definition/ conceptualization of 
resilience
● Steps involved in BHC activity design/ 
implementation
○ Was resilience considered in this 
process? If so, when (before, 
during, after)?
● Steps city leaders took to respond to 
prepare for shocks. Where are city 
leaders at now in preparing for 
shocks?
○ Was BHC involved in any of these 
steps?
● Conduct KIIs and 
surveys
● Develop concept map
Way forward
● How could the BHC 
cross-sectoral models in 
the different country 
contexts be applied and 
sustained in the future?
● Steps involved in establishing 
relationships between stakeholders, 
how the context impacted these 
steps and how these steps changed 
over time
● Determine which contextual factors 
support establishing connections 
between stakeholders, and which 
factors inhibit and how to leverage 
facilitators and navigate around 
inhibitors
● Review theory of change and 
determine if changes are correlational 
and may be due to other factors as 
opposed to mechanistic causal 
changes that were a result of BHC
○ Validate this during data collection
● Conduct KIIs and 
surveys
● Look at leverage 
hypotheses and ask 
questions in KII about 
whether interventions 
have produced 
anticipated benefits
PHASE 3: Review Findings and prepare final deliverables for dissemination (Month 4 -
Month 6)
B. Determine the robustness of results, key drivers and how the above findings may 
differ by circumstance/context: This will also involve identifying key drivers such as specific 
players, specific connections and/or processes that were established that were crucial for 
facilitating project implementation. 
C. Present and discuss findings and implications:
a. This step is to ensure uptake and application of the evaluation. SPACES will include 
opportunities to discuss what the data is showing in the data collection and model, the 
interpretation of the findings and what the data are suggesting, determining the 
implications for USAID/BHC work and discussing next steps and how to apply these 
findings to future work. 
98
b. Specifically, SPACES will present preliminary results to ABHT, USAID Missions, and 
BHC before drafting any written deliverables. 
D. Prepare final deliverables
a. Final written deliverables consist of an evaluation report, two-page briefs, and a 
presentation of the findings. The briefs and the summary presentation (and possibly 
additional infographics) should be targeted to specific audiences: USAID decision-makers 
and external stakeholders (these will be identified in coordination with ABHT).
b. Draft versions of the Evaluation Report and the briefs will be shared with ABHT, BHC, 
and India and Indonesia Missions. 
c. First Draft Evaluation Report and briefs: On a date agreed in the workplan, SPACES will 
provide a draft version of the evaluation report. ABHT and BHC may provide 
comments on this draft through USAID. The report should meet the criteria outlined in 
USAID’s Evaluation Policy, and include all elements described in ADS 201mah, USAID 
Evaluation Report Requirements. 
d. Second Draft Evaluation Report and briefs: On a date agreed in the Workplan, the 
SPACES will share a second draft evaluation report that considers comments received 
from USAID based on the first draft report. ABHT will provide comments on this draft. 
e. Final Evaluation Report and briefs: On a date agreed upon in the Work Plan, the 
SPACES will share a final evaluation report in both Microsoft Word and Adobe 
Document compatible formats whose content and format reflects comments received 
from USAID on the second draft report. 
f. Final Presentation: SPACES will share and clear a draft of the presentation for J(b) below
E. Disseminate 
a. All deliverables that are in written format will be in plain, grammatically correct English 
language; be submitted in an appropriate electronic format (i.e. Microsoft Word, Excel, 
PowerPoint Presentation, or PDF compatible); and meet all the requirements.
b. SPACES should prepare a presentation and disseminate the briefs to BHC stakeholders 
(to be determined with ABHT).
c. Other communications and publications of the research by SPACES will be cleared by 
USAID
d. Upload to the DEC
2. Schedule of Activities and Deliverables
All deliverables will be shared with ABHT within the agreed upon timeline. SPACES will promptly notify 
the ABHT of any problems, delays, or adverse conditions which materially impair the ability to meet the 
requirements (see section 7: Risks). All modifications to the required elements of the workplan, 
whether in evaluation questions, design and methodology, deliverables and reporting, evaluation team 
composition, schedule, and/or other requirements should be agreed upon in writing by ABHT. Any 
revisions made will be noted in the SOW annexed to the final Evaluation Report. Timeline below is 
illustrative and actual dates will be finalized in the work plan and will be mutually agreed upon between 
SPACES and USAID ABHT. 
Phase Tasks/Deliverables Timeline
Phase 1-2: Pre Eval and Start up Kick-Off Meeting Week 1
Evaluation Design and workplan Week 2
Phase 3: Implementation Data collection tools Ongoing from Month 1.5 -
Month 4
Results and iterative presentations (e.g.
preliminary findings/midpoint)
Ongoing from Month 3-5
99
Analysis Ongoing from Month 2 - 5
Phase 4: Findings & Final 
Deliverable submission 
Presentation of final version of findings to 
ABHT 
Month 5.5
Report and Brief Writing Month 5 - 6
Submission of First Draft Evaluation Report 
and Briefs
Month 5
Submission of Second Draft Report and 
Briefs
Month 5.5
Submission of Final Draft Report and Briefs Month 6
Final Presentation Month 6
Dissemination activities Month 6
3. SPACES Team Composition, Roles and Responsibilities 
● SPACES Team
Dr. Bruce Y. Lee and Dr. Tim Moran have collaborated together for over 5 years and are Co￾Principal Investigators of the SPACES consortium. They will share responsibilities in overseeing 
the project as they both bring different and complementary systems expertise to the team. 
Bruce Y. Lee, MD MBA, Co-Principal Investigator (Co-PI)
Dr. Lee is Co-Principal Investigator for the USAID SPACES consortium and has been the 
Principal Investigator of the USAID-SPACES cooperative agreement since its inception in 2015. 
Dr. Lee is Executive Director of PHICOR (Public Health Computational and Operations 
Research), and Professor of Health Policy and Management at the CUNY Graduate School of 
Public Health and Health Policy. With over two decades of expertise in systems approaches to 
help decision makers in medicine and public health, Dr. Lee has authored over 240 scientific 
publications (including over 105 first author and over 85 last author) with well over 90% of the 
publications on mathematical and computational modeling as well as three books. An 
international leader in systems science and mathematical and computational modeling, Dr. Lee 
has considerable experience leading system science consortium, projects and evaluations. His 
previous positions include serving as Associate Professor of International Health at the Johns 
Hopkins Bloomberg School of Public Health, Executive Director of the Global Obesity 
Prevention Center (GOPC), Director of Operations Research at the International Vaccine 
Access Center (IVAC), Associate Professor at the University of Pittsburgh, Senior Manager at 
Quintiles Transnational where he led teams that developed economic and operational models 
for a variety of clients in the pharmaceutical, biotechnology, and medical device industries, 
working in biotechnology equity research at Montgomery Securities, and co-founding Integrigen, 
a biotechnology/bioinformatics company. He's worked with a wide range of decision makers 
[e.g., Centers for Disease Control and Prevention (CDC), World Health Organization (WHO), 
UNICEF, GAVI Alliance, country Ministries of Health, Bill and Melinda Gates Foundation, State 
and County Health Departments, Department of Health and Human Services (DHHS), and 
Department of Homeland Security]. He will oversee all phases of the project.
Tim Moran, PhD, Co-Principal Investigator (Co-PI)
100
Dr. Moran is Co-Principal Investigator for the USAID SPACES consortium and Paul R. McHugh 
Professor of Motivated Behaviors and Executive Vice Chair in the Department of Psychiatry and 
Behavioral Science at the Johns Hopkins University School of Medicine. As Co-Principal 
Investigator he will also oversee all aspects of the project. Dr. Moran has led academic 
consortiums and centers dedicated to bringing systems methods and approaches to a range of 
different fields in public health. Dr. Moran has received extensive NIH and foundation funding. 
His work is reflected in over 375 publications. He contributes behavioral expertise to the 
system science work of the SPACES Consortium.
4. Risks
There are a few potential risks that may lead to problems, delays, or adverse conditions which materially 
impair the ability to meet the requirements. One risk is the ongoing COVID-19 pandemic, which is likely 
to continue to limit in-person meetings and workshops, suspend or delay travel, and impact 
stakeholders’ time and accessibility. A second risk is that BHC staff may be occupied with other 
engagements and responsibilities and, as such, may not be available to work with the SPACES team in 
the timeline indicated above. A third risk in working with stakeholders across multiple countries and 
government agencies is that changes in political climates, structures, and personnel may prevent or 
reduce SPACES’ ability to interface with key stakeholders and gather information. As mentioned above, 
SPACES will work to actively report on and mitigate any delays or issues that may arise as a result of 
these or other risks. 
5. Budget Notes
● The budget incorporates the anticipated costs of all staff (outlined in section 6 and annex). it is 
expected that the evaluation will be conducted remotely. 
● The budget assumes:
○ 6 Month project, start date pending on receipt of contract and funding
○ All Activity Areas (0-6) are included in the evaluation 
○ The geographic locations for the evaluation are maintained to Makassar.
○ No travel
● Budget table with line items and corresponding details are provided in separate attachment
H – SUMMARY INFORMATION ABOUT SPACES TEAM MEMBERS
Bruce Y. Lee, MD MBA, Co-Principal Investigator (Co-PI)
Dr. Lee is Co-Principal Investigator for the USAID SPACES consortium and has been the Principal 
Investigator of the USAID-SPACES cooperative agreement since its inception in 2015. Dr. Lee is 
Executive Director of PHICOR (Public Health Computational and Operations Research), and Professor 
of Health Policy and Management at the CUNY Graduate School of Public Health and Health Policy. 
101
With over two decades of expertise in systems approaches to help decision makers in medicine and 
public health, Dr. Lee has authored over 240 scientific publications (including over 105 first author and 
over 85 last author) with well over 90% of the publications on mathematical and computational 
modeling as well as three books. An international leader in systems science and mathematical and 
computational modeling, Dr. Lee has considerable experience leading system science consortium, 
projects and evaluations. His previous positions include serving as Associate Professor of International 
Health at the Johns Hopkins Bloomberg School of Public Health, Executive Director of the Global 
Obesity Prevention Center (GOPC), Director of Operations Research at the International Vaccine 
Access Center (IVAC), Associate Professor at the University of Pittsburgh, Senior Manager at Quintiles 
Transnational where he led teams that developed economic and operational models for a variety of 
clients in the pharmaceutical, biotechnology, and medical device industries, working in biotechnology 
equity research at Montgomery Securities, and co-founding Integrigen, a biotechnology/bioinformatics 
company. He's worked with a wide range of decision makers [e.g., Centers for Disease Control and 
Prevention (CDC), World Health Organization (WHO), UNICEF, GAVI Alliance, country Ministries of 
Health, Bill and Melinda Gates Foundation, State and County Health Departments, Department of 
Health and Human Services (DHHS), and Department of Homeland Security]. He oversaw all phases 
of the project.
Tim Moran, PhD, Co-Principal Investigator (Co-PI)
Dr. Moran is Co-Principal Investigator for the USAID SPACES consortium and Paul R. McHugh 
Professor of Motivated Behaviors and Executive Vice Chair in the Department of Psychiatry and 
Behavioral Science at the Johns Hopkins University School of Medicine. As Co-Principal Investigator 
he also oversaw all aspects of the project. Dr. Moran has led academic consortiums and centers 
dedicated to bringing systems methods and approaches to a range of different fields in public health. Dr. 
Moran has received extensive NIH and foundation funding. His work is reflected in over 375 
publications. He contributes behavioral expertise to the system science work of the SPACES 
Consortium.
Sheryl Siegmund, MS, Director of Operations
Ms. Siegmund is Director of Operations for the USAID SPACES consortium and PHICOR at the CUNY 
Graduate School of Public Health and Health Policy. Ms. Siegmund has directed operations for the 
SPACES consortium since its inception in 2015, including all SPACES pilot awards. She has over 15 years 
of experience managing the operations for complex multi-level projects. She has over 14 peer-reviewed 
scientific publications in systems approaches. Ms. Siegmund assisted the PI with managing 
project operations, developing and maintaining timelines, coordinating partners, 
maintaining the project resources and infrastructure, documentation, communicating 
among the team members, and producing reports and presentations. 
Marie Martinez, MSPH, Project Manager
Ms. Ferguson is the coordinator for the USAID SPACES consortium and PHICOR Project Director at 
the CUNY Graduate School of Public Health and Health Policy. She has a MSPH from the Johns Hopkins 
Bloomberg School of Public Health. Ms. Ferguson has coordinated the SPACES consortium since its 
inception, coordinating 11 pilot projects in diverse regions and sectors ranging from market systems, 
political economy, health supply chains, and education. She also co-authored the seminal SPACES MERL 
Systems and Complexity White Paper . She has extensive experience in project leadership and expertise 
in applying systems approaches. She has over 15 peer-reviewed scientific publications in the field of 
systems science including systems models. She assisted the PI with coordinating the overall 
project such as establishing and maintaining timelines, facilitating communications within 
the SPACES team and between SPACES, ABHT and BHC, documentation and 
communications among team members, and managing all phases of the project.
102
Sarah Bartsch, MPH, Senior Researcher
Ms. Bartsch is PHICOR Project Director at the CUNY Graduate School of Public Health and Health 
Policy. She has experience with and expertise in social network analysis and helped guide 
these aspects of the project. She also helped analyze, interpret, and present these findings.
She has published over 85 scientific publications (including over 25 first author) in the field of systems 
science in peer-reviewed scientific journals and has over a decade developing mathematical and 
computational models to assist a wide range of decision makers in public health. These publications span 
work focused on the United States, internationally, including several developing countries, as well as 
globally. Her work primarily focuses on various infectious diseases and healthcare-associated infections, 
the effects of patient sharing on hospital networks and supply chains for vaccines. 
Kevin Chin, MPH, Research Assistant
Kevin L. Chin, MPH, is a Research Assistant and Senior Analyst with Public Health Informatics, 
Computational, and Operations Research (PHICOR), the Artificial Intelligence, Modeling, and 
Informatics for Nutrition Guidance and Systems (AIMINGS) Center, and the Center for Advanced 
Technology and Communication in Health (CATCH), headquartered at the City University of New 
York (CUNY) Graduate School of Public Health and Health Policy. He has been a part of the team since 
January of 2022 and brings over a decade of management consulting and legislative analysis experience 
with some of the nation’s leading healthcare providers. Mr. Chin supported the social network 
analysis and write up of the findings. 
Jessie Heneghan, MCP, Research Associate
Jessie Heneghan is a Research Associate and Senior Analyst for Public Health Informatics, 
Computational, and Operations Research (PHICOR), the Center for Advanced Technology and 
Communication in Health (CATCH), and the Artificial Intelligence, Modeling, and Informatics, for 
Nutrition Guidance and Systems (AIMINGS) Center at the City University of New York (CUNY) 
Graduate School of Public Health and Health Policy. Jessie has a background in applying systems thinking 
approaches to urban planning, design, and climate resilience research projects that she applies to her 
work at PHICOR. Ms. Henneghan supported the social network analysis and write up of the 
findings. 
Colleen Weatherwax, MS, Research Assistant
Colleen Weatherwax, MS is a Programmer and Senior Analyst for Public Health Informatics, 
Computational, and Operations Research (PHICOR), the Artificial Intelligence, Modeling, and 
Informatics for Nutrition Guidance and Systems (AIMINGS) Center, and the Center for Advanced 
Technology and Communication in Health (CATCH), headquartered at the City University of New 
York (CUNY) Graduate School of Public Health and Health Policy. Her current research focuses on 
designing and implementing computational models of communicable and non-communicable diseases, 
with a focus on COVID-19 and employee burnout. Ms. Weatherwax supported the social 
network analysis and write up of the findings.
Megan McGlynn Scanlon, Chief Operating Officer / Project Lead 
Ms. Scanlon has nearly 20 years of experience across nine countries focused on civil society/nonprofit 
organizations, networks, and systems and the development and execution of international and 
community development programs. Between 2010-2015 she led a series of World Bank and Australian 
Aid research and development initiatives in Indonesia, including work with the World Bank’s flagship 
support facility for sub-national development. She has undertaken and overseen numerous mixed 
methods action-oriented research projects, including field research throughout Indonesia. Ms. Scanlon 
103
led the GKI research team starting in March 2022 and oversaw all aspects of data 
collection and analysis.
Seema Patel, Chief Executive Officer / Technical Adviser
Ms. Patel, the CEO of the Global Knowledge Initiative, is a highly experienced systems design specialist 
with decades of work in the global development and social impact sectors. She has expertise in systems 
strategy, multi-stakeholder partnerships, innovation mobilization, participatory design, organizational 
development and change management, and network mobilization. Ms. Patel provided oversight to the 
GKI research team, particularly during the first round of informational interviews and 
resilience framework compilation, and informed design of data collection instruments within 
the program constraints.
Boris Ristovski, MA Systems & Network Data Analyst
Mr. Ristovski is an experienced IT specialist and researcher conducting quantitative and desk research
(design and data collection, analysis, graphical and interactive visualization). He has extensive experience 
designing and conducting network analyses for multi-stakeholder groups. He was responsible for 
building databases, structuring data for network analyses and setting up interactive 
network visualizations in Kumu. Boris led the quantitative data analysis and assisted the 
design of data collection instrument
Nelti Anggraini, Social Researcher
Ms. Anggraini is an Indonesian national social researcher with 15+ years in development research and 
advisory experience. She has extensive field experience at the district level and below on topics 
including frontline health services, community-based health and education programs, water and 
sanitation, social protection, and village governance. She has extensive experience in qualitative research 
techniques including facilitating focus group discussions, semi-structured interviews, immersion, and 
participant observation. Ms. Anggraini led on key informant interviews for the evaluation as 
well as initial qualitative data coding and analysis. 
Karno B. Batiran 
Mr. Batiran has 18 years of experience of community organizing and facilitating at village level in the 
Makassar, South Sulawesi, Indonesia region. He also has extensive field research experience. Mr. 
Batiran supported translation and quantitative data enumeration on the evaluation. 
Levriana Yustriani
Ms. Yustriani is an Indonesian national social researcher in the areas of media, youth, political 
participation, higher education and knowledge sector, and government and institutional reform. She also 
has experience in Monitoring and Evaluation. Ms. Yustriani supported development of 
Indonesian-language KII instruments and testing, and initial qualitative data frameworks.
Lila Sari, Social Researcher
Ms. Sari is an Indonesian national PhD candidate in political science at the Australian National University, 
with research focused on the variations of clientelism politics in the health sector at the subnational level 
in Indonesia. She also has extensive experience in donor project management having worked for the 
AusAID/ Australian Embassy, DANIDA, and DfID/ British Embassy Jakarta. Ms. Sari supported 
qualitative data coding and analysis.
104
I – SIGNED DISCLOSURES OF CONFLICTS OF INTERESTS
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Bruce Y Lee
Title Executive Director, Professor 
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
I – Signed Disclosures of Conflicts of Interests
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date
Signature
12/16/2022
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Marie F. Martinez
Title Project Director
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Sheryl Scannell
Title Director of Operations
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Colleen Weatherwax
Title Programmer Analyst
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose. 
 Yes 
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature Colleen Weatherwax
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Kevin L Chin
Title Research Assistant
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose. 
 Yes 
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
7. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
8. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
9. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
10. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
11. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
12. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Sarah Bartsch
Title PHICOR Project Director
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose. 
 Yes 
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
13. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
14. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
15. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
16. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
17. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
18. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Timothy H. Moran 
Title Professor 
Organization Johns Hopkins Bloomberg School of Public Health
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose. 
 Yes 
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
19. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
20. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
21. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
22. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
23. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
24. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature Tim Moran
USAID Disclosure of Real or Potential Conflict of Interest for External
Evaluation Team Members
Name Jessie Heneghan
Title Research Associate
Organization PHICOR (Public Health Informatics, Computational, and 
Operations Research) at the City University of New York 
(CUNY) Graduate School of Public Health and Health Policy
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument) AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
The Building Healthy Cities (BHC) Program in Makassar, 
Indonesia 
I have real or potential 
conflicts of interest to 
disclose. 
 Yes 
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
25. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
26. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
27. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
28. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
29. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
30. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/16/22
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Megan McGlynn Scanlon
Title Chief Operating Officer / Program Lead
Organization Global Knowledge Initiative 
Evaluation Position x Team Leader
Team member
Evaluation Award Number 
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
Building Healthy Cities
JSI
I have real or potential 
conflicts of interest to 
disclose.
Yes
x No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
2. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
3. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
5. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
6. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 12/19/2022
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Boris Ristovski
Title Consultant (DataVisualization and Analysis)
Organization Global Knowledge Initiative
Evaluation Position
✗Team Leader
✓Team member
Evaluation Award Number
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s)
Evaluated (Include activity
name(s), implementer name(s)
and award number(s), if
applicable)
Building Healthy Cities
JSI
I have real or potential
conflicts of interest to
disclose.
✗Yes
✓ No
If yes answered above, I
disclose the following facts:
Real or potential conflicts of
interest may include, but are not
limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations of
the activity.
CONTINUED
If yes answered above, I
disclose the following facts:
Real or potential conflicts of
interest may include, but are not
limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will
update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary
information of other companies, then I agree to protect their information from unauthorized use or
disclosure for as long as it remains proprietary and refrain from using the information for any purpose
other than that for which it was furnished.
Date
12/19/2022
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Karno B. Batiran
Title Consultant (Translation & Enumeration) 
Organization Global Knowledge Initiative
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
Building Healthy Cities
JSI
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
2. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
3. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
5. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
6. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 20 December 2022
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Levriana Yustriani
Title Consultant (Qualitative Analyst) 
Organization Global Knowledge Initiative
Evaluation Position Team Leader
XXTeam member
Evaluation Award Number 
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
Building Healthy Cities
JSI
I have real or potential 
conflicts of interest to 
disclose.
Yes
Xx No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
2. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
3. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
5. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
6. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 20 December 2022
Signature
Levriana Yustriani
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Nelti Anggraini
Title Consultant (Interviewer/Analyst)
Organization Global Knowledge Initiative (GKI)
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
Building Healthy Cities
JSI
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
2. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
3. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
5. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
6. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date December 20th, 2022
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Lila Sari
Title Consultant (Qualitative Analysis)
Organization Global Knowledge Initiative
Evaluation Position Team Leader
Team member
Evaluation Award Number 
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s) 
Evaluated (Include activity 
name(s), implementer name(s) 
and award number(s), if 
applicable)
Building Healthy Cities
JSI
I have real or potential 
conflicts of interest to 
disclose.
Yes
No
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
1. Close family member who is 
an employee of the USAID 
operating unit managing the 
activity(s) being evaluated or 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
2. Financial interest that is direct, 
or is significant though 
indirect, in the implementing 
organization(s) whose 
activities are being evaluated 
or in the outcome of the 
evaluation.
3. Current or previous direct or 
significant though indirect 
experience with the activity(s) 
being evaluated, including 
involvement in the activity 
design or previous iterations 
of the activity.
CONTINUED
If yes answered above, I 
disclose the following facts:
Real or potential conflicts of interest 
may include, but are not limited to:
4. Current or previous work 
experience or seeking 
employment with the USAID 
operating unit managing the 
evaluation or the implementing 
organization(s) whose 
activity(s) are being evaluated.
5. Current or previous work 
experience with an 
organization that may be seen 
as an industry competitor with 
the implementing 
organization(s) whose 
activity(s) are being evaluated.
6. Preconceived ideas toward 
individuals, groups, 
organizations, or objectives of 
the particular activities and 
organizations being evaluated 
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I 
will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary 
information of other companies, then I agree to protect their information from unauthorized use or 
disclosure for as long as it remains proprietary and refrain from using the information for any purpose 
other than that for which it was furnished.
Date 21 December 2022
Signature
USAID Disclosure of Real or Potential Conflict of Interest for External Evaluation Team Members
Name Seema Patel
Title CEO
Organization Global Knowledge Initiative
Evaluation Position
☐Team Leader
☐Team member
Evaluation Award Number
(contract or other instrument)
AID-OAA-A-15-00064
USAID Activity(s)
Evaluated (Include activity
name(s), implementer name(s)
and award number(s), if
applicable)
Building Healthy Cities
JSI
I have real or potential
conflicts of interest to
disclose.
☐Yes
☐ No
If yes answered above, I
disclose the following facts:
Real or potential conflicts of
interest may include, but are not
limited to:
1. Close family member who is
an employee of the USAID
operating unit managing the
activity(s) being evaluated or
the implementing
organization(s) whose
activity(s) are being evaluated.
2. Financial interest that is direct,
or is significant though
indirect, in the implementing
organization(s) whose
activities are being evaluated
or in the outcome of the
evaluation.
3. Current or previous direct or
significant though indirect
experience with the activity(s)
being evaluated, including
involvement in the activity
design or previous iterations of
the activity.
CONTINUED
If yes answered above, I
disclose the following facts:
Real or potential conflicts of
interest may include, but are not
limited to:
4. Current or previous work
experience or seeking
employment with the USAID
operating unit managing the
evaluation or the implementing
organization(s) whose
activity(s) are being evaluated.
5. Current or previous work
experience with an
organization that may be seen
as an industry competitor with
the implementing
organization(s) whose
activity(s) are being evaluated.
6. Preconceived ideas toward
individuals, groups,
organizations, or objectives of
the particular activities and
organizations being evaluated
that could bias the evaluation.
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will
update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary
information of other companies, then I agree to protect their information from unauthorized use or
disclosure for as long as it remains proprietary and refrain from using the information for any purpose
other than that for which it was furnished.
Date
Signature
12/20/2022
135
U.S. Agency for International Development
1300 Pennsylvania Avenue, NW
Washington, DC 20523