HAPP- Infernal Evaluation Report 
A report on the HAPP "Lessons Learned was launched in September 2000. In response to 
several requests for a clear explanation of how the "Lessons Learned" report was produced, 
FHI compiled much of the material produced during the course of the evaluation. These 
materials include the information gathered during the course of the evaluation, either regarding 
HAPP itself or regarding the evaluation method. In order to reflect the process used in the internal 
evaluation, some working notes have been deliberately retained and differing points of view have 
been left unedited. This is essentially a working document that enhances its value for those involved 
in the process or interested in attempting a similar style of evaluation. 
These materials were produced by the members of the Evaluation Team. An index is to be found at 
the back of this book to help with reference. 
The first part contains an explanation of the evaluation method and the questionnaires used during 
the evaluation. 
The second part contains samples of the tables containing the responses to the evaluation questions 
and a report on individual interviews. 
The third part contains the reports drafted based on the field interviews with organizations and 
individuals. These reports were the basis for "Lessons Learned". 
The fourth part contains various materials developed during the process of preparing to disseminate 
the evaluation findings. 
HAFP- htemd Evaluation Reporf 
A. First Stage - Collecting the Data 
Evaluation Method 
Questions for NGOs 
Questions for GOI/Donors/Managers 
Questions for Komisi Penanggulangan Aids Daerah (KPAD) 
Questions for Partners, PVOs And Others 
Questions for Beneficiaries 
Questions for The NGO Forum 
Questions for GOI/Donors/Managers/Partners 
N4PP - Internal Evaluation Report 
A. First Stage - Collecting the Data 
The first document in this section is an explanation of the method used to conduct the evaluation. 
This section also contains the questions asked during the site visits, both to organizations and to 
individuals. 
I. Evaluation Method 
Introduction 
HAPP is a bilateral partnership between the Government of Indonesia (GOI) and the USAID, being 
implemented by the Government through the Project Implementation Letter (FIL), by Family Health 
International (FHI) and by a Participating Agencies Service Agreement (PASA) with the CDC. 
A Completion Report and an Evaluation of HAPP (1996-2000) were stipulated in the original Project 
Contract. The Completion Report covers both the technical and non-technical standards. This 
evaluation focuses on Lessons Learned from the entire process of the Project's implementation and 
incorporates the perspectives of the different stakeholders, including the GOI, Department of Health, 
FHI, HAPYs various partners (NGOs and PVOs) and the different target audiences 'n the 
community) and does so within the religious and cultural norms of Indonesia. 
An attempt was made to make the evaluation process itself user-friendly, practical, participatory and 
consultative. It was primarily a self-evaluation being undertaken by FHI in association with HAPP 
staff, for a project planned and implemented by them. The evaluation was managed internally by FHI 
Jakarta, coordinated by the Country Director, using an Evaluation Team. The evaluation process 
consisted of a number of task-oriented teams who represented end users and stakeholders, including 
the GOI, HAPP Provincial staff, the Implementing Agencies (IAs), the community (end users), 
Komseni - the public relations company, universities and other research agencies and facilitators. 
The Evaluation TOR 
The process of evaluation began in January 2000 with the development of an Evaluation Terms of 
Reference (TOR). This specified the process to be followed, the tasks to be undertaken, including field, 
PVO, GO1 and donor consultations, a time plan for these activities and the end report production and 
dissemination plan. The TOR was revised after in-house consultations and formal and informal 
meetings with the Technical Advisory Group, consisting of HAWS PVO parhers and USAID. 
Information and data were gathered in two ways. as described below. The present qualitative and 
participatory evaluation was conducted by integrating the findings of qualitative analysis of two 
types of data: (1) field-interview based data and (2) secondary data. 
Secondary Data Assessment 
One Evaluation Team member scanned and perused all available reports, documents, letters, minutes 
of meetings, etc. and carried out qualitative analysis of the secondary data. The secondary data 
comprised aU documents pertinent to technical and non-technical aspects of the Project, including 
minutes of meetings, letters, progress reports (e.g. quarterly and annual reports) and interim 
evaluation reports. 
HAPP - lntemaJEvaJuation Report 
The analysis was conducted by reading all the above documents to identify common themes or 
recurrent issues and other critical issues. The latter could include issues that had a great impact on 
different aspects of the Project. The next step was to draw Lessons Learned from the issues identified. 
Site Visits 
The three HAPP sites, Manado/Bihmg, Surabaya and North Jakarta, were visited in the months of 
April and May 2000 (Manado: April 9 - 13,2000, Surabaya: ~~ril.23 - 27,2000 and Jakarta: May 15 - 19,2000). This was done to gain a firsthand impression of the actual existing situation at the sites and 
to meet the people involved in the implementation of the Project. 
A set of questions (See Annexes) was developed over a period of two weeks in March 2000, by a 
consultant in the Evaluation Team, in close coordination with the senior staff of the HAPP Jakarta 
Office and with inputs from the Provincial Offices. These were a set of broad questions for the 
following target audiences, to be used as a guideline in order to ensure complete information and 
data, rather than to be strictly adhered to: 
GO1 
NGOs 
Provincial HAPP Office staff 
KPADIKPAD Program Advisor 
Beneficiaries 
NGO Forum 
PVOs 
Donor Agencies 
A set of questions was also prepared for the GO1 and donors and for PVOs and other partners. 
Each visit was conducted by a team consisting of independent consultants who comprised the 
Evaluation Team, including a facilitator as well as a member of the HAPP Jakarta Office staff to 
formalize some of the meetings and to introduce the team and briefly summarize the objectives and 
process of the evaluation. 
The HAPP Jakarta Office and Provincial Office staff helped organize the site visits and set up 
meetings and focus group discussions with the KPAD, IAs and the Beneficiaries. However, they were 
not present at these meetings in order to avoid bias and to encourage openness and honesty among 
participants. 
- KPAD Meetings - These were conducted by the Evaluation Team members and the 
facilitator/interpreter, following the guideline questions described above. Meetings were held 
separately with the members of the KPAD Core Group and the KPAD Program Advisors in each 
site. - HAPP Provincial Office Staff - The Staff at each site were sin~ilarly consulted for information and 
their perceptions of the Project. 
- The Implementing Agencies - All three HAPP-supported NGOs in Manado, Surabaya and Jakarta 
met separately with the Evaluation Team and the facilitator/interpreter. An attempt was made to 
ensure the presence of all the staff from each NGO at the meetings so as to make the process as 
honest and participatory as possible. - Beneficiaries - Each group of Beneficiaries involved in the HAPP-supported NGOs in every site 
participated in focus group discussions with the Evaluation Team, the interpreter and a staff 
member from the HAPP Jakarta Office or another HAPP site. The meetings with the Beneficiaries 
were organized a little differently as each group was different (CSWs, sailors, dock workers, 
HAFP - Infernal Evaluation Report 
transvestites, youth, midwives and traditional medicine vendors). The focus group discussions 
were begun in innovative ways with games and other ice breakers. 
Each of the above meetings took place at sites and times convenient for the group being consulted. 
HAPP staff organized and provided snacks for the IAs and snacks and gifts for the Beneficiaries 
during their respective consultations. 
An international consultant, who was part of the Evaluation Team, carried out the consultations with 
the various Project partners (PVOs), concerned officials of the GOI and a number of relevant donor 
organizations based on the pre-prepared set of broad questions. 
It is important to mention at this stage that, apart from the aforementioned interviews, a number of 
other formal and informal meetings and dialogues were also held, for example, meetings with FHI's 
capacity building partners. 
Compilation and Analysis of the Qualitative Data 
The results of the above interviews were used by the evaluation team to gain a qualitative assessment 
of the main issues and Lessons Leamed from the implementation of HAPP at the provincial level 
governmental (KPAD) and non-governmental (HAPP IAs) sectors. This was done following 
sequential organized steps. 
After detailed deliberation on the methodologies of compiling qualitative data, a method was evolved 
where the information was first set in a tabular format - target group from each site by key topics 
discussed (see Annexes). The objective of developing and utilizing this method was to ensure the 
inputting of all the data collected (and there were vast volumes of it after all the consultations took 
place) in a format that was feasible, easily do-able and easy to read at a glance. It was then easier to 
seek out and use the relevant data for actual inclusion in the evaluation report. Information about 
each group's perceptions (site-wise) about the Project, their activities, partners, capacity building, etc. 
was thus systematically entered. Key themes and issues emerging from the tables were then 
highlighted. These were discussed in detail among the core evaluation team and main key issues - both cross-cutting and particular to specific target groups, areas or topics - were then identified. 
The next step was to develop a framework within which to work with respect to writing up the key 
issues and the main Lessons Learned that arose from the analysis. This was once again done keeping 
in mind that everything of relevance was to be included. The core HAPP Evaluation Team - most of 
whom had been involved in all the previous steps (from developing the questions, to doing the field 
consultations to doing the data analysis) to ensure the flow and continuity - then divided the writing 
to fit within the framework developed earlier. A broad outline was discussed and decided upon to 
keep the writing style as uniform as possible. 
Challenges 
It is to be kept in mind that a number of limitations were faced and Lessons Learned during the actual 
process of the evaluation. HAPP was a large project that spanned a period of five years, three years of 
which were the focus for the evaluation. Much was accomplished and many lessons were learned. 
Some points of importance had to be kept in mind, including the following: 
- There were many personalities and parties, playing many parts, each with their own 
characteristics. Since one of the main elements of this evaluation was that it was participatory, it 
was of the utmost inrportartce to include, during data collection and report writing, each player's 
and group's perceptions and views. A sincere attempt was made to do so. Confidentiality of the 
people interviewed, formally as well as informally, was also ensured. 
HAPP - Infernal Evaluation Report 
At every stage of the evaluation, the bias of those involved was kept out of the interpretation and 
analysis as much as possible. 
There were large volumes of data and, therefore, the development of a suitable data compilation 
method (in particular, for qualitative data) was time-consuming and labor-intensive. 
The report itself included all relevant information, perceptions and Lessons Learned for 
consideration in similar future projects and programs. 
It was important to plan well ahead of time for the production and dissemination of the findings, 
particularly as these were tailor-made for specific target groups. 
It was felt important to document in detail the entire evaluation process so as to make it user-friendly 
and to be to able to replicate it in the future for similar projects. 
11. Questions for NGOs 
1. Outreach 
a. How useful did you find the outreach activities? 
b. Why? 
c. What were your best results? 
d. What was special about your outreach activities? 
e. How did these 'special' things contribute to your achievements? 
f. What was the process of reaching these results? 
g. If you had to repeat it, what changes would you make to your outreach process? 
2. Peer Education 
a. How useful did you find the peer educator activities? 
b. Why? 
c. What were your best results? 
d. What was special about your peer educator activities? 
e. How did these 'special' things contribute be to your achievements? 
f. What was the process of reaching these results? 
g. If you had to repeat it, what changes would you make to your peer educator process? 
3. BCC Material Development 
a. What were the materials you developed? 
b. Which were the most useful? 
c. Why were these useful? 
d. How did you utilize the different materials? 
e. What was the training you received to develop this material? 
f. Was it useful? 
g. Why?/Why not? 
h. If you had to do it again, what changes would you make? 
i. Did you receive feedback from the beneficiaries on the material you developed and 
distributed? 
j. If so, what? 
k. Did you pre-test your materials? 
4. Coordination and Networking (WAD, Private Sector, HAPP Partners, Provincial Office) 
a. With whom did you collaborate? 
b. In what ways did you collaborate? 
HAP' - Internal Evaluation Report 
c. How useful was this collaboration to your organization in achieving your organization's 
objectives? 
d. How useful was this collaboration to your organization in achieving the Project objectives? 
e. What were the challenges you faced during your collaboration? 
5. Sustainability 
a. What are the main factors that contributed to sustainability? 
b. Do you think these factors were included in the HAP Project design? 
c. How have inputs towards proposal development contributed to the sustainability of your 
organization? 
d. How have inputs towards resource mobilization contributed to the sustainability of your 
organization? 
e. How have inputs towards monitoring and reporting contributed to the sustainability of your 
organization? 
f. How have inputs towards strengthening management contributed to the sustainabiity of 
your organization? 
6. Capacity Building 
a. What were the capacity building activities you were involved in? 
b. Which of these approaches did you find useful? 
c. Why?/Why not? 
d. What content was useful? 
e. Why?/Why not? 
f. What other capacity building activities would you like to have been involved in? 
7. Management Issues 
a. What were the main challenges you faced in the area of management? 
b. Why? 
c. How did you monitor/use data for your management decisions? 
d. How have your skills in monitoring/using data improved over time? 
e. What have been the challenges you faced in recruiting, supporting and retaining staff? 
f. Did you overcome then? 
g. How? 
8. Project Start-up 
a. What are your views on the "Jump-Start" process? 
b. If you were to start a project again, what do you think would be the best possible way to do 
it? 
9. STD Service Provision 
a. Which STD clinic in your opinion was good? 
b. Why? 
c. What do you think a good STT) clinic should provide? 
d. What do you think of the quality of the STD service provision supported by HAPP? 
10. Condom Promotion 
a. Is your collaboration with condom manufacturers useful? 
b. If yes, how? 
NAPP - Internal Evaluation Report 
c. If no, why not? 
111. Questions for GOI/Donors/Managers 
1. Project Start-up (JW -Jump-start) 
a. What would be the quickest/sirnplest transition to keep on-going projects continuing 
successfuIly without disturbing the flow of the Project? 
2. Coordination and Networking 
a. What does partnership mean to you? (quality and quantity) 
b. Does your partnership with HAPP satisfy your definition of partnership? 
c. If yes, how? 
d. If no, why not? 
e. Given your experience with partnership, how importantly, do you think, would similar 
partnership benefit a new project? 
3. Surveillance - Behavioral and Biological 
a. How do you think the surveillance process couId be simplified? 
b. What forms of capacity building are required for its implementation? 
c. What forms of capacity building are required for analysis? 
d. What forms of capacity building are required for interpretation of data? 
e. How do you think that surveillance can be made sustainable? 
f. In what way has the surveillance affected your decision-making? 
4. STD Service Provision 
a. Which STD clinic in your opinion was good? 
b. Why? 
c. What do you think a good STD clinic should provide? 
d. What do you think of the quality of the STD senrice provision supported by HAPP? 
e. How would you improve the quality of STD service provision? 
5. Condom Promotion 
a. Is the strategy being followed for condom promotion sustainable? 
b. If yes, in what way? 
c. If no, why not? 
6. Management of Sub Offices 
a. What do you expect from the sub offices? 
b. Do your sub offices fulfill these expectations? 
c. If no, what are the gaps? 
d. In what way can the sub offices be made more efficient? 
MPP - htemal Evaluation Report 
N. Questions for Komisi Penanggulangan Aids Daerah WAD) 
1. Capacity Building and Support 
a. What were the capacity building activities you were involved in? 
b. Which of these approaches did you find useful? 
c. Why?/Why not? 
d. What content was useful? 
e. Why?/Why not? 
f. What other capacity building activities would you like to have been involved in? 
g. What were the outcomes of capacity building inputs provided to you? 
2. Their Role 
a. How did you perceive your role in the life of the Project? 
b. Has it changed over time? 
c. If so, how? 
d. What is your perception of your coordination/interaction with NGOs? 
e. What is your perception of the PAD Advisor/Working Group)? 
3. Sustainability 
a. What are the key factors that contribute to sustainability? 
b. How have inputs towards monitoring and reporting contributed to your sustainabity? 
c. How have inputs towards strengthening management contributed to your sustainability? 
4. Decentralization 
a. How do you see your role changing in a decentralizing system? 
b. What did you have that could be useful during decentralization? 
V. Questions for Partners, PVOs And Others 
Making partnerships productive and satisfying 
a. What does partnership mean to you? 
b. What is your perception of the Provincial Office? 
c. How did you perceive your role in the implementation of the Project? 
d. Was your involvement and partnership in the Project satisfying? 
e. If so, how? 
f. If not, why not? 
g. Given your experience with partnership, in what way would a similar partnership benefit a new 
project? 
h. How could your perception of the 'ideal partnership' be operationaked? (mechanism) 
VI. Questions for Beneficiaries 
1. Project Goals and Activities 
a. Which part of the Project that you were involved in did you find most useful/relevant? 
b. Why? 
HAPP - Internal Evaluation Report 
c. Which part was the most interesting? 
d. Why 
e. Which part was the most fun? 
f why? 
2. Perceptions of the Project 
a. Did the Project help you in solving any of your problems? 
b. If so, how? 
c. Did you think you were at risk before your involvement with the Project? 
d. After your involvement with the Project, what were the safe practices you adopted? 
e. Would you like to have been involved in the Project in a greater or different way than you 
were? How?. 
f. In what way do you think the Project could have been better with respect to you? 
g. Would you continue in the future what you have learned with the Project? 
h. What were your most useful'gains' from the Project? 
i. What did you feel was not useful? Why? 
VII. Questions for The NGO Forum 
1. Capacity Building and Support 
a. What werethe capacity building activities you were involved in? 
b. Which of these approaches did you find useful? 
c. Why?/Why not? 
d. What content was useful? 
e. Why?/Why not? 
f. What other capacity building activities would you like to have been involved in? 
g. What were the outcomes of capacity building inputs provided to you? 
2. Their Role 
a. How did you perceive your role in the life of the Project? 
b. Has it changed over time? 
c. If so, how? 
d. What is your perception of your coordination/interaction with WAD? 
3. Sustainability 
a. What are the key factors that contribute to sustainability? 
b. How have inputs towards monitoring and reporting contributed to your sustainability? 
c. How have inputs towards strengthening management contributed to your sustainability? 
a. How do you see your role changing in a decentralizing system? 
b. What did you have that could be useful during decentralization? 
hXPP - Internal Evaluation Reporf 
VIII. Questions for GOI/Donors/Managers/Partners 
1. Project Start-up (JW - Jump-start) 
a. What would be the quickest/simplest transition to keep on going projects continuing 
successfully without disturbing the flow of the Project? 
2. Coordination and Networking 
a. What does partnership mean to you? (quality and quantity) 
b. Does your partnership with HAPP satisfy your definition of partnership? 
c. If yes, how? 
d. If no, why not? 
e. How significantly would similar partnerships benefit a new project? 
3. Surveillance - Behavioral and Biological 
a. How do you think the surveillance process could be siniplified? 
b. What forms of capacity building are required for its implementation? 
c. What forms of capacity building are required for analysis? 
d. What forms of capacity building are required for interpretation of data? 
e. How do you think surveillance can be made sustainable? 
f. In what way has the surveillance affected your decision-making? 
4. STD Service Provision 
a. In your opinion, which STD chic is most functional? 
b. why? 
c. What do you think a good STD clinic should provide? 
d. What do you think of the quality of the SSTC service provision supported by HAPP? 
e. How would you improve the quality of STD service provision? 
5. Condom Promotion 
a. Is the strategy being followed for condom promotion sustainable? 
b. If yes, in what way? 
c. If no, why not? 
6. Management of Sub Offices 
a. What do you expect from the sub offices? 
b. Do your sub offices fulfill these expectations? 
c. If no, what are the gaps? 
d. In what way can the sub offices be made more efficient? 
NAPP - Internal Evaluation Report 
B. Second Stage - Compiling the Data 
I. Beneficiary table 
II. KPAD Table 
III. Jakarta Consultations 
HAPP- Infernal Evaluation Report 
B. Second Stage - Compiling the Data 
The data from the site visits was transferred into matrices by the Evaluation Team, as was the 
secondary data. This section contains examples of the matrices. It is not possible to include all the 
matrices but they are available on request from the FHI office in Jakarta. 
During this compilation process, the critical points were highlighted and common themes identified. 
Items of interest which were particularly unique were also marked. 
HAPP - Internal Evaluation Report 
I. Beneficiary Table 
CS W) I Usefulness: 
Increased self-esteem, especially 
because YKB treats them with respect 
(which they don't receive from the 
community) and YKB gives them the 
opportunity to become Outreach 
Workers, facilitators and resource 
persons in training. 
~"ildin~ networks with people from all 
levels 
Able to make contact with other CSWs 
in other areas tluough Study Tours 
(National and International) and went 
to international forum. 
Condom negotiation training 
Knowledge of how to prevent oneself 
from getting STD/HIV/AIDS and 
ability to disseminate that information 
and skill to their peers (domino effect) 
Being able to be part of the community, 
not marginalized. 
Perceived problems solved: 
Their STD treatment seeking behavior (e.g. 
seek , medical treatment for vaginal 
discharge) was increased and the referral 
system developed by YKB gave them the 
chance to have the same treatment and 
services as other patients (non-CSIVs). 
They could afford adequate STD 
treatment. 
After becoming PEs at the YKB project, 
they were able to change their behavior by 
using condoms with their clients. 
CSWs were given the chance to become 
ORWs and to change their occupation. 
After becoming ORWs, CSWs quit their 
previous job as CSWs (e.g. one of the 
CSWs said that "being a SW is not what I 
want to be). 
Notable behavior changes: 
Giving up smoking and alcohol. 
No longer working as CSW. 
More mature. 
Able to get along better with people. 
Self-awakening to help others. 
Learned how to budget. 
Futureplans: 
Because of YKB support, BW is now an 
independent organization. The members 
would like to be supported and funded to 
provide services and to disseminate 
HIV/AIDS prevention information and 
skills to the CSWs themselves, 
independently from YKB. 
With their high motivation and 
commitment, BW members would like to 
continue OR and PE programs to 
disseminate information to SWs. 
According to them, this is important 
because, after Kramat Tunggak was 
banned, SW were spread throughout the 
community and STD prevalence increased 
(information from BSS). 
BW would like to build a network with the 
government to minimize stigmatization. 
Pespondent ( Proiect Goals and Activities I Perceotion of the Proiect 
RECOMMENDATIONS 
akarta 
XB I 
YKB should provide Bandungwangi with 
Capacity Building and Technical Assistance in 
rsing computers, income-generating programs 
xganizational management and human 
:esource developn~ent. 
, 
HAPP - Internal Evaluation Report 
riespondent 
fIK 
'CSW) 
'utri Mandiri 
Proiect Goals and Activities I Perception of the Project 
Usefulness: 
Increased STD/HIV/AIDS preve was previousiy the name a 
knowledge. which led to 
condoms with clients. 
available at that particular red are lishment, PM took over YIK activities such 
Vocational training such as d 
organizations. 5 SCVs have c has to participate in PM discussion 
profession and become dancers s at least 2 times and must always use a 
and discos. ndom. Belonging to PM was valuable to the 
3 Durex and Simplex are prefe ause "everybody is there", they could 
rvaria (for oral sex); Sum is not popular consult about their problems and they could 
(because its cheapness is perceived as participate in income-generating programs 
denoting low cpality); Artika is the i such as selling peanuts (with seed money from 
most popular condom in that particular i YIK). 
area. 
Perceivedproblems solved 
Realization of their own risk of STD and 
HIVIAIDS infection has led to them 
starting to use condoms with clients. 
SmaU-scale income generating activities 
Future plan: I , PM wants to obtain the support and funds 
to broaden the scope of programs with 
SWs, not only STD/HW information 
dissemination but also vocational/income￾generating programs. 
HAPP - Internal Evaluahon Report 
'espondent 
4KMI 
Maria) 
'00s (MSM) 
?roiect Goals and Activities 
Usefu fness: 
The clinic is useful because waria can 
get free STD treatment and health 
assistance. 
Regular discussions are useful because 
they increase STD/HIV/AIDS 
prevention knowledge, which leads to 
their buying condoms for their own 
safety and they can change their 
behavior by persuading clients to use 
condoms (although this still depends 
on the client's decision). Condon~s are 
not, however, used with their parhers. 
Through the IAKMI clinic they were 
able to make friends and new contacts 
Transportation money, free lunches at 
regular weekly meetings and free STD 
check-ups motivate waria to come to 
the IAKMI clinic and encourage their 
friends to come along. 
They became resource persons at 
training and seminars. 
Domino effect for the clinic and STD 
check-ups. 
A good relationship between ORWs 
and waria is a must. 
Usefulness: 
The MSMs became resource persons at 
the request of other 
NGOs/organizations and Government 
institutions. 
They used condoms during sexual 
intercourse with partners. 
Domino effect in countering the myth 
that HN is a gay disease. 
Broadened their views. 
Made friends and new contacts and 
built network with the government 
~eople as well. 
Increased self-esteem and greater self￾confidence 
Increased income through 
entertainment at Discos and Pubs and 
became known by people. 
+:+ A solid organization with waria and 
iMSMs as members is advantageow 
especially because they have the same 
attitude and behavior and can use the 
organization to support each other for 
behavior change. 
Perception of the Proiect 
Percejvedprobfems solved: 
Realization that they are at risk of STD and 
HIVIAIDS infection has led waria to have 
regular STD check-ups and to use 
condoms regularly. 
They were able to get STD examination 
and treatment free from a clinic which was 
not judgmental. They felt that the health 
providers at this clinic accepted them and 
were friendly. 
RECOMMENDATIONS 
Clinics for waria and gay men such as the 
IAKMI clinic should continue to operate. 
otherwise waria will start self-treatmenl 
again because they cannot go to general 
clinics. 
HAPP could set up other clinics and 
HIV/AIDS prevention programs for waria 
in another parts of Jakarta such as Easl 
Jakarta and Bekasi because the waria 
populations there are high. 
. IAKMI should rotate persons appointed tc 
represent IAKMI as resource persons il 
requested by other NGOs or institutions. 
Notable change: 
Realization that they are at risk of STD and 
HIV/AIDS infection has led to thei~ 
regularly using condoms and being aware 
of how to have sex with a parher if they 
get HIV. 
. Self-awakening to build empathy with 
people who are HIV+ and becoming 
assured that they would still be accepted 
even if they got HIV 
Futue pfans: . To reach MSMs outside IFOOS. 
To disseminate HIV/AIDS idormatior 
through entertainment (such as operetta) 
quizzes and games. 
Outreach and Basic Counseling training. 
HAPP - Internal Evaluation Report 
Tespondent 1 Proiect Goals and Activities 1 Perception of the Proiect 
iurabaya 
(AA 
CSWs anc 
)imps) 
Usefulness: 
Decrease in STD cases due to condom 
use. 
Greater knowledge about STDs led to 
dissemination of that inforniation ta 
clients (domino effect) and to using that 
inforniation to negotiate condom use 
with clients. 
Were able to refer STD cases to PKBI or 
Putat Jaya. 
Training and seminars organized by 
YAA gave them new experience and 
knowledge and enabled them to make 
friends and network. 
Were able to participate in 
entertainment (edutainment) events 
organized by YAA. 
Becoming a PE for YAA increased their 
self esteem and status in the 
community. 
Becoming PEs and having regular 
meetings organized by YAA was useful 
because this was a starting point for 
organizing theniselves into 'Sudi Asih' 
for Mami and 'Hampan Sejati' for SWs 
and these new organizations were used 
for income-generating programs. 
One wisma owner expressed her 
Notable change: 
Perception that condom use is better for 
STD prevention than regular check-ups or 
penicillin injections. 
. Changes in STD treatment seeking 
behavior, from self-treatment to adequate 
examination and treatment. 
Perceivedproblents solved: 
Realization that they are at risk of STD and 
HIV/AIDS infection has led to regular STD 
check-ups and condom use. 
They were able to get STD examinations 
and treahnent free at the PKBI clinic, 
which was a relief because paying a doctor 
is burdensome for them. 
Using condoms and getting adequate STD 
treatment led to feeling safer and getting 
more money from their trade &cause the 
clients see them as"clean"). 
Future plans: 
To continue STD/HN information 
dissemination and adequate STD treatment 
even with their own money. 
Could form an NGO among themselves to 
continue STD/HN prevention program 
work ("It is a just a matter of 
nioney/funding") because they already 
have the knowledge and information. 
satisfaction with the Project and said : RECOMMENDATIONS: 
she feels no regret that the SWs working 
in her house are involved in the Project. , . Training needed: vocational training (hair 
In fact, they have benefited since the / dressing, salon, tailoring) for income 
clients see her SWs as clean. This project j generating. (Success stor).: after tailor 
has brought respect to their house, 1 training and getting a sewing machine 
which has then increased their income. : from the Dept. of Social Welfare, one SW 
! was able to change profession. ! 
/ J Story from one SW: She asked a client to 
I use a condom but he refused saying it 
would lessen the pleasure (although he 
1 had never tried one). He was drunk and 
1 did not realize when she put the condom i on him anyway. After intercourse, she told 
him about the condom and he could not 
believe that he had not felt it. 
9 Almost all SWs interviewed said that there 
is a time limit for being an SW. After they 
become old, no one will give them money 
for sex transactions and they will not be 
able to compete with the young SWs; that's 
why almost all said they need to think 
about another job to anticipate 
"retirement". 
HAPP - Internal Evahation Report 
Yespondent 
(Pros 
Sailors, Dock 
Vorkers, 
3Ws, Waria) 
Proiect Goals and Activities 
UsefuLness: 
Training is useful because they realize 
their own risk of STD and HIV. 
For dock workers: they would like to 
disseminate information regarding the 
danger of STDs (these are common 
diseases among them) and the fact that 
5TD is an entry point for HIV. 
D For waria: condom negotiation is 
useful because it can be used to 
negotiate condom use with clients. 
hey felt good when disseminating 
STD and HIV prevention information 
to peers and other people. 
Street Theater is a good way to 
disseminate information and to refer 
illiterate STD patients. 
For dock workers: Y.Pros clinic is useful 
because, for those who have health 
insurance but whose health insurance does 
not cover STDs, this service is a relief. 
Perception of the Proiect 
Notable chanzes: - 
Realization of their own risk of STD and 
HN infection has led to strong motivation 
to disseminate that information and 
especially to make their peers and other 
people have the same awareness of the 
risk. 
More controlled sex behavior by using 
condoms. 
People who have been treated at the Y.Pros 
C~NC have a lot of faith in it and wanted to 
be Pes. 
The Y.Pros STD clinic is a place that helps 
them to change their behavior from self￾h.eatment to more adequate STD treatment 
seeking behavior because it gives free 
examinations and treatment (affordable). 
Only for advanced diseases will they be 
referred to the hospital. 
RECOMMENDATIONS: 
PE SWs could go to other areas 
(Balikpapan and Makasar) to teach SWs 
there about condom use and STDs. 
. The people interviewed depend too much 
on the Y.Pros !3TD clinic and wish to 
continue receiving services free of charge. 
Y.Pros could open branches in other 
provinces. 
.:. Stoly from one SW (PE): when she first 
became aware of STDs and HIV she was 
afraid because she realized how risky her 
work was. This realization encouraged her 
to educate and disseminate this 
information to her peers and she became a 
Peer Educator. In her experience, it took 
time for her peers to believe her but after 
that time they listened to her and even 
sailors listen to her now. There are times 
when there is denial, so persistence is 
required for PEs. In addition to providing 
STD/HIV prevention information, she 
refers peers, sailors and dock-workers she 
has contacted to the Y.Pros clinic. She 
needs to persuade them to go to that clinic 
HAPP - Internal Evaluation Report 
G'espondent I Project Goals and Activities Perception of the Project 
before they believe that Y.Pros gives free 
STD examinations and lreahnent. 
According to her, those people like the 
Y.Pros clinic because it provides free 
services and guarantees confidentiality. 
She sometimes gets mistaken for a condom 
saleswoman, because, as a PE, she talk so 
much about condoms and carries condoms 
all the time. J Another story re. condom use: one dock 
worker (PE) offered to pay for his peer to 
have sex with an SW if he would use a 
condom. This is one way to prove that 
condoms don't lessen the pleasure. 
.:. Challenges: Educating people who don't 
speak Indonesian. To overcome this 
problem, collaboration with local 
authorities is useful. 
- 
HAPP - Internal Evafuation Report 
Zespondent 
'KBI 
Midwives) 
'KBI 
rraditional 
Ierbs 
'endor) 
Usefulness: 
Training was useful because it increased 
their knowledge especially re. 
Counseling and discussions, STD 
syndromes) and helped a lot in their 
daily work and information 
dissemination (e.g. to youth, 
housewives and wives of government 
officials). 
Training and meetings were useful 
beiause they were able to make friends 
and build networks. 
B The referral system developed by PKBI 
was perceived as good because 
midwives could either refer patients or 
send vaginal smears. 
B IEC material distributed by PKBI was 
useful because it was colorful, had 
pictures and was easy to understand. 
RECOMMENDATIONS: 
The PKBI clinic could continue giving STD 
laboratories testing and treatment for free 
for patients referred by midwives. 
More IEC material should be distributed to 
through midwives, including 
slides and OHF to be used when 
disseminating information during 
discussions. 
+:. Midwives were unable to make follow-up 
visits to their patients since time was 
limited. .:. Older patients are more interested in HIV 
information, while younger patients don't 
care. 
/ RECOMMENDATIONS: 
Usefulness: 
STD and counseling training is useful 
because they become more confident in 
giving advice to clients, they expand 
their services and, as a result, generate 
income. 
Being able to refer clients to PKBI 
generates more client respect for them. 
Networking and making friends. 
Training and discussion on STD 
syndrome, counseling and small 
enterprise management increase their 
knowledge. 
Further business management and 
marketing skills training. 
Notable changes.. 
. For assumed STD clients, they did not just 
give traditional herbs but also referred 
them to the PKBI clinic (they sometimes 
even brought clients to PKBI). 
. Knew more about hygiene especially in the 
preparation of traditional herbs. 
. Became more confident in giving advice to 
clients. 
. Generated income. 
Established a "Traditional Herbs" 
Organization as a place to share 
information. 
HAPP - Internal Evaluation Report 
Youth) 
Teka (CSW) 
Respondent I Proiect Goals and Activities I Perception of the Proiect 
Manado 
YMM 
I - 
\1 
I Ysefulness: Notable changes: 
Condom negotiation training and Realization of their own risk of STD and 
discussions were useful because they HIV/AIDS infection because of their work 
got in-depth information which was (both to them from a client and to the 
relevant to their dailv work. Thev client's wife at home) has motivated them 
disseminated this knowledge and skills to persuade clients to use condoms (by 
to their peers as well. saying this is to protect his loved one al 
Discussions were useful because they home). However, the bargaining power is 
increased their knowledge and still in the man's hands; if he does not 
understanding about STD/HIV agree to use one, they can do nothing. 
prevention and condom negotiation. Futureplans: 
They were also able to share this Small-scale income generating activities 
info&nation. 
Regular STD examinations were the 
most useful because they were able to 
get free exanrinations and treatment and 
even transportation money. 
Wisnla owners were very positive about 
the Project and felt no regrets at the SWs 
working in their houses being involved 
in the Project. In fact, one benefited 
since her house was seen as the cleanest 
brothel in that area, always active in 
health sector activities and was the 
center of NGO activities. This project 
brought respect to her house, which J 
then increased its income. 
A good relationship between ORWs and 
wisma owners and SWs is a must, so as 
to build a good foundation for the 
outsider to build a good rapport his/her 
audience. 
J 
such as kiosks. 
Becoming PEs to teach other S\\'s in other 
places. There have been occasions when 
SWs from Kotamobagu, Mariri and Jalan 
Siswa came to their place and asked for 
condom. They used this occasion to teach 
them about STD/HIV issues and the 
importance of using condoms and to 
motivate them to always use condoms 
with clients. However, they do not want to 
be 'open' in front of the general public 
since they still have to protect their family 
from being exposed to their profession. 
Story about condoms: Some of their clients 
always use condoms because of the 
suggestions made by the SWs. -4s result of 
OR, the clients realize the importance of 
protecting their wife. They do not want 
their wife to get infected with an STD or 
HIV, so they always use condoms. 
More free condoms from DepKes are 
needed compared to more Sutra or 
'Consortium' (Durex, Simplex and Xrtika) 
condoms because of the their low income. 
So far, stocks from DepKes are adequate as 
checked by Y.Peka ORWs. They have hied 
to sell consortium brands but were out of 
stock, the price being far too high for their - . 
clients. 
-- 
HAPP - Internal Evaluation Report 
:espondent 
HS (Sailors) 
'roiect Goals and Activities 
Ilsetulness: 
HIV information dissemination through 
IEC material and Traditional Drama 
Performance is useful because it give 
them the skill to protect their own 
health by using condom properly and 
got free condom. 
Traditional Drama Performance by LHS 
is good because it generates discussion 
among them and as a means 01 
reinforced each other to use condom. 
Perception of the Project 
Notable changes: 
. Realization of their own risk for STD and 
HN/AIDS infection has led them to use 
condoms with SWs put not with regular 
partner). 
Not using any penis accessories anymore 
(penisaccessories such as pearls implanted 
in the head of the penis. 
Willing to buy the condom themselves if 
they run out of it because they know this 
will help them. 
Recommendation: 
Antibiotic (such as 500mg of Amphicilline) 
being provided on ships. This is for 
temporary relief until they can see the 
doctor as they are often far out in the sea 
when they get STD syndrome and it is 
often days before they can see a doctor. 
The other reason is that doctors are not 
always available on every harbor they 
come into port. 
. Use Family Health Campaign shategy for 
AIDS Campaign and make it nationalized 
it. 
. Continue providing free condoms on the 
ships. 
Sailor story: 
Getting STD for very common for a sailor and 
most self-heat with 500mg of Amphicilline as 
that was available on the ship. They are 
ashamed when they get STDs since their 
HAPP - Internal Evaluation Report 
11. KPAD Table 
I Capacitv Buildina and I Their Role I Sustainabilitv I Decenhalization 
therefore gets more NGOs can be more 
attenhon & respect flexible - timings, etc. 
from government the role of NGO is 
officials. b~gger and therefore 
Implementation of their role is that of 
achvities not his facihtator. 
sheneth because The~r role is multi￾List of CB to be inclu￾ded in the introduction 
KPAD - Program Ad￾visor Role as perceived 
by WAD: 
- Gives advice on 
progran~s/activities 
/selection of NGOs 
. Coordinates with 
KPAD members 
. Coordinates & sup￾ports PAD activi￾HAP: Jakarta is the 
final decision maker. 
They perceived their 
role & involvement in 
HI activities as well￾coordinated and par￾ticularly important & 
deep with respect to 
HAPP activities be￾cause of their per￾ceived sense of res￾ponsibilities (sp. To 
Most useful ones: 
. Training for officials 
from different sec￾tors very useful to 
integrate HI/AIDS 
ties I identify health center 
. Facilitates conmmhi- ' with STD clinic) 
cation with exe- They can see than in 
cutive staff of some situation (e.g. 
Government dealing with high risk 
Seen as HAPP staff behavior groups) al+o 
activity better. 
Study tours very 
good because: 
Broaden participants 
view 
Has domino effect 
specially to sectors 
other than health & 
to executive level of 
govemment 
Leads to good colla￾boration between 
KPAD & NGOs 
Leads to increase in 
HIV/AIDS budget 
allocation 
. Leads to HIV/AIDS 
being included in 
other programs. 
sector in nature. 
The activities of the 
health sector have 
incurred in frequency 
and quality: 
Became integrated: 
counseling. STD clinic 
(at health centers), 
IEC, treatment, lab. 
Exam, condom infor￾mation and dishi￾bution, m/HN 
information 
HAPP coordination 
activities between 
KPAD and NGOs 
Who does what acti￾vities 
Collaboration for joint 
activities (e.g. WAD) 
Shared role of the 
different government 
sector & NGO 
Representation of 
NGOs in WAD 
through the NGO fo￾rum representative to 
improve communica￾tion and hust. 
KPAD has 
develop a think 
tank in which the 
NGO are repre￾sented, should be 
more professio￾nal &comn~itted. 
w Important 
factor. 
Continue to 
support the 
Advisor & se￾cretariat 
through other 
funding. 
Advocacy to 
members of the 
provincial par￾liaments to en￾sure their sup￾port for budget 
allocation for 
HN/AIDS 
activity, both 
through WAD 
& the sectors 
A mufti-sec￾toral approach 
should be con￾tinued. 
Increase the 
capacity of 
district & lower 
levels of KPAD 
Recommendations: 
Joint decisions bet￾ween the KFAD & 
donor should be taken 
on pla~ed acti\ rlhes ' ' to 
avoid confusion 6 
changes. 
District level KPAD 
should be made capa￾ble of implementing 
more activity (capacity 
building) 
Preparation for: 
* Included lower level 
staff in capacity buil￾ding activities. 
HAPP- Internal Evaluation Report 
Also conducted 100% 
Gpacitv Buildine and I Their Role 
condom workshop 
funded by HAPP 
All were very useful 
because: 
All were new 
All were conducted 
differentlv 
Sustainabilitv 
All had ' different 
purposes 
. IDF training tough 
them to develop 
their mission & 
mission & budget 
development for 
more allocations of 
funds. 
Decentralization 
KPAD - Program 
Advisor & Secretary 
Role as perceive by 
WAD 
Program Advisor & 
secretary regard them￾selves as HAPP Jakarta 
staff and largely follow 
HAPP Jakarta ins￾tructions. 
. They perceived 
their role as a 
support & coor￾dination body for 
HAPP activities 
and between the 
government & 
NGOs at all levels. 
. HAPP 
involvement led 
to: 
The Governor & 
other sector be￾coming more a￾ware of and more 
involved in HIV 
prevention. 
Money from the 
Governor & 
HAPP resulting in 
conducting acti￾vities and inviting 
participation even 
from the district 
level NGOs 
Good relationship 
with NGOs 
Improved activity 
reporting from 
NGOs would lead to 
better coordination 
by the WAD 
To avoid 
duplication & 
overlap of 
activities, 
coordination 
between WAD, 
NGO & Donor 
should be 
strengthened. 
Recommendations: 
. Increase the capa￾city of district & 
lower levels of 
KPAD 
. District level KPAD 
should also have 
better interaction 
with NGO at that 
level 
. Creating a network 
of comprehensive 
health services for 
HIV+ people at all 
levels. 
. Empowering HIV + 
people 
. KPAD core group 
recommends a sala￾ly to increase moti￾vation and incen￾tives. 
KPAD has develop 
a core group in 
which the head of 
NGO forum is re￾presented. 
Important factors: 
. Political will of 
policy makers 
including the 
governor 
Money 
. Commitment of 
KPAD members. 
iecommendations: 
District level 
KPAD should be 
made capable of im￾plementing more ac￾tivity (capacity buil￾ding) e.g. leadership, 
IEC de-velopment & 
net working. 
(PA, KPAD & District 
eve1 KPADs all 
rutonomous & so no 
lecentralization issues. 
XecommendatIons: 
VGOs should work 
rnd coordinate 
lirectly with the 
WAD at their level. 
HAPP - Infernal Evaluation Report 
'akarta 
I 
CavaciN Buildine I Their Role 
and Suuvort 
Ongoing proposal 
develo~ment skills 
HAPP' required each 
activity to be based on 
a proposal) Office 
equipment from HAPP 
Operational hds 
kom HAPP 
SustainabiliQ 
Usefulness . (their 
perceptions) 
. Leadership 'aining 
had no relevance for 
these because as go￾vernment officials 
they get leadership 
training for their 
level. 
Study tours are only 
useful if they are 
abroad because they 
feel that Jakarta is 
more advanced than 
other regions in 
Indonesia. 
. They feel that after 
having received IEC 
development 
training (from 
HAPP), they should 
received funding 
from HMP for their 
one IEC material 
development. 
DecenWtion 
WAD - PA 
Role as perceived by 
KPAD 
+ Program Advisor 
and secretary per￾ceived themselves as 
HAPP Jakarta slaff 
and foUowed Jakarta 
instructions. 
. PA should be either 
part of KPAD or a 
KPAD consultant. 
Salary of Program 
Advisor 61 Secretary 
should be channeled 
through WAD 
. Their recruihnent 
process should be in 
consultation with 
KPAD. 
Further support re￾quired: 
Financial 
management 
. PR TOT about 
HIV, drugs, He- 
. They see their role as 
a facilitator for NGOs 
to implement their 
activities. 
They would Like to 
strengthen their role 
as a coordinator 
behveen WAD, 
NGOs and Donors. 
bey are self suffident 
md not necessarily 
.quire donors. 
HAPP- Internal Evaluation Report 
11. Jakarta Consultations 
A. Details of Takarta-Based Organizations' Individual Consultations 
World Health Organiztion (WHO) 
Coordination and Networking - Partnerships should be built upon respective strengths and be 
focussed on reaching a common goal. The IDU Rapid Assessment has been a good example of a 
collaborative effort. WHO should be seen as having technical expertise with high level Government 
assistance and USAID/HAPP should be seen as having technical expertise with the ability to 
implement in the field. 
Surveillance - Behavioral and Biological - In relation to sero-suweillance, WHO felt that the 
guidelines have already been simplified. They felt there is a need for improved capacity in some areas 
to implement guidelines and an overall need for funding. In particular, the need for funds for test kits 
remains a critical gap. In behavioral surveillance, WHO feels that the BSS guidelines need to be 
simplified to provide a sustainable strategy to be implemented within the provinces. At the moment, 
WHO feels the BSS is unsustainable. Overall, WHO felt that there should be a link between the sero 
and behavioral surveillance implemented at a provincial level with governmental and non￾governmental involvement. 
STD Service Provision - WHO believes that the most sustainable system for STD service provision is 
integration with the Puskesmas or at least linkages of STD service provision for sex workers with the 
local Puskesmas. WHO believes that an integrated approach to STD service provision is sustainable. 
Condom Promotion - WHO believes there needs to be a coordinated approach to review the national 
policy for condom promotion. It seems that the approach across the nation is a bit confusing and 
condom promotion could be enhanced with a coordinated and clear policy. 
Project Start-up - WHO hopes that this internal evaluation could be used as a baseline for the next 
project. Their hope is to see enhanced efforts at a provincial level. 
Ahna jaya 
Atma Jaya's role was described as assessing the capacity for monitoring and evaluation (M&E) among 
NGOs and providing technical assistance to NGOs for M&E. The original needs assessment showed 
that most NGOs were using quantitative means to monitor their programs and they required greater 
skills in qualitative analysis and data management. Atma Jaya developed a 5-day workshop and 
several manuals for the NGOs to use in M&E. 
In addition, Atma Jaya has been involved in technical assistance for the IDU Rapid Assessment and 
Response (RAR). 
Overall, Atma Jaya has been pleased to be a part of HAPP and welcomes continued collaboration and 
partnership. They felt HAPP provided them with their own authority and responsibility in 
developing and implementing their work. 
Issues Raised: Atma Jaya felt that M&E was initiated too late in the development of the NGO 
program. They felt that an earlier start was required in the new project. In addition, Atma Jaya felt 
HA PP - htemai Evaluabon Report 
that their efforts could have a greater impact if all capacity building assistance was coordinated under 
HAPP. 
In the area of RAR, Atma Jaya felt that it was very good that donors were coordinating to improve an 
overall understanding of IDU in Indonesia. However, donor coordination could be improved by 
better conlmunication among the donors on rules and regulation (e.g. per diem) and for the donors to 
speak in a single 'voice' to provincial partners. In addition, their role in technical assistance to the 
provinces is constrained by a limited involvement in decisions about who locally will be involved and 
by the lack of not also having the financial management responsibilities of the Project. 
Overall Recommendation: A clearly articulated capacity development plan that identifies the roles 
and responsibilities of each partner organization could improve the efficiency and effectiveness of the 
capacity building efforts for the NGOs. 
Business Dpamics 
Business Dynamics' role was described as offering capacity development in the areas of leadership 
and participatory management practices to HAPP partners, specifically, the KPADs and the NGOs. 
Although they were hired somewhat late in the Project, Business Dynamics worked with the WADS 
and NGOs to describe baseline assessments of the organizations, retrospectively and follorved up 
with current self-assessments. They offered several capacity building workhops with these partners. 
Details can be found in the "Institutional Developnient" report by HAPP. 
Partnership: Partnership was defined as 'working together and drawing on each other's strengths to 
provide maximum benefit to the end users' and was also described as: 'mutual work towards a goal 
to be achieved, a collaborative effort of pooled strength with a team spirit.' 
They perceived their partnership with HAPP as high quality with a focus on producing results for the 
end users. In addition, they believed HAPP was sensitive to cultural issues and provided many 
opportunities for NGOs to improve themselves. Business Dynamics was very satisfied with their 
partnership with HAPP. 
Lessons Learned Leadership training in association with institutional development was critical for 
HAPP partners in the field. 
1. Future assistance under the new project to local organizations must be better coordinated 
by FHI. 
2. Business Dynamic can assist by developing and implementing training in core 
competency for management. 
3. In the future, it is required that resources across HAPP that are focussed on NGO 
capacity building are planned ahead of time as a well-coordinated strategy with clear 
linkages between the various HAPP partners who are offering TA. 
Recommendation: A strategy for NGO capacity building is developed early in the new project. The 
new project should have 'targeted capacity building' efforts based upon the assessed needs of 
individual partners. 
HAPP - Internal Evaluation Report 
A USAID HIV/STLJ Project 
Project Start-up - AusAID is undertaking a situational assessment for the next stage of their new 
project and will field a team for new project design in October 2000. Their current project will most 
likely run into the year 2001. The AusAID project suggests that the start-up of the new project can be 
assisted by a regional exchange between provinces of the old and new projects. AusAID sees the two 
projects as complementary as USAID's approach offers greater focus on high risk groups and 
AusAID's approach is a greater focus on institutional strengthening. 
It was generally thought that a 'jump-start' approach will not prove to be 'quicker' for the new 
project. It is thought a that a careful phased approach with a hierarchy of provinces based upon 
epidemiological criteria and provincial capacity in relation to responding to HIV/AIDS would be the 
most appropriate strategy. 
Coordination - Coordination according to AusAID should be done for a common goal or through a 
common activity. AusAID felt that Issue-Based coordination has been the most successful, i.e. IDU 
Rapid Assessments. In the future, AusAID sees that the majority of the work should be with a 
provincial focus with locally tailored strategies and responses. In addition, AusAID sees that there 
has been very good collaboration between AusAID and CDC on STDs and with HAPP on the BSS. 
Overall, AusAID suggested that collaboration and coordination between the Project improved with 
time and effort. The lesson learned was that collaboration was best operationalized around a common 
goal or activity. In the future, collaboration and coordination between AusAID and USAID projects 
are essential. He suggested that the model of collaboration should be on issued-based training and 
coordinated geographic coverage. 
Recommendation: Coordination between the two projects should be both in geographical placement 
of the Projects (e.g. Irian Jaya & Jakarta) and in issue-based collaboration in assessments, training and 
policy discussions. 
National AIDS Commission &PA) 
Overall, the KPA would like to see greater communications and coordination of HAPP with the KPA. 
The KPA said that their few human resources and the lack of equipment constrains their ability to be 
completely effective within their role. Support directly for the KPA was requested. 
The Futures Group 
Overall, TFG had a very close collaboration with the communications advisors under HAPP and 
appreciated their partnership under the Project. They felt the condom social marketing component 
was constrained by the slow start-up of HAPP and the slow approval process for subagreements. TFG 
felt the partnerships across the Project could have been improved by more proactively-initiated 
communications and coordination of the partners by the leader of HAPP/FHI. In addition, in the 
future, where provincial offices may exist, they could be given greater authority to make decisions so 
that their role is more programmatic and not only administrative. 
NAPP - htemal Evaluation Report 
Concerning the sustainability of condoms supply and promotion, TFG felt that the current strategy 
within Indonesia to supply subsidized condoms will undermine the market share of Indonesian 
condom manufacturers. This strategy was deemed unsustainable. 
PVO Partners -Pa PAZYand PACT 
PCI's focus has been to provide TA to HAPP NGOs giving a community health focus. They have 
worked in M&E, financial assistance, sustainability and facilitation of HAPP workshops. 
PATH has worked in Irian Jaya under a separate agreement with USAID. They have worked with 
local KGOs, developed drop-in centers, worked with students, high risk groups, clinics and also 
carried out cross-border activities. 
PACT has worked on IEC materials development, nehvorking, training for youth, capacity 
assessments and integrating HIV/AIDS into NGO programs. 
Overall, this group of international PVOs felt that the 'partnership' with HAPP was unsatisfactory. In 
general, it was recommended that the follow-on project improve its understanding of partnership and 
ability to coordinate. Planning and continuity should be given special attention to address the roles 
the PVOs expect to play and improved commu~cations across the Project and particularly with its 
partners could increase the effectiveness of the program in the field. 
Universiq Partners -JEN/A fma Jaya and (Iniversiiy of hdonesia ((I4 
JEN has been responsible for implementing and publishing an analysis of the quality of STD 
management (PI6&7) and an STD prevalence study. 
UI has been responsible for implementing and publishing the BSS for HAPP. 
Su~eiUance - In general, for sero-surveillance, it was felt that the procedure has been fairly 
simplied. The overall requirement is to decentralize and how to accomplish decentralization. In 
many places the local capacity to implement is already there; however, in some places equipment and 
overall funding is needed. 
Analysis/Interpretation of Data - There needs to be continual training in how to analyze the 
serosurveillance data at a local level. JEN has been working with local universities and NGOs to 
improve this capacity. In the future, more local involvement is required to improve the perceptions 
and use of research data. 
Decentralization - To improve the involvement of local organizations, the involvement of "Urban 
Forums" is required in the response to HIV/AIDS. 
STD Service Provision - JEN feels that the clinics worked with do not know what the HAPP strategy 
is to improve STD case management. It is not worth measuring PI6&7 if there is no clear and 
attainable strategy to improve STD case management. Due to the lack of availability of drugs, there 
will be limited improvements in the measurement of PI6&7. 
BSS Simplification - Although, there is potential to simplify the BSS, several issues must be taken into 
account while planning for 'local' implementation and use of data. 
HAPP - Infernal Evaluation Report 
. The role and responsibility of every organization must be very clear. Increasing the number 
of partners will increase the complexity and increase the requirements associated with quality 
control. 
. A plan for Capacity Building of local institutions must be articulated, funded and provided 
sufficient time and follow-up must be budgeted for. 
Overall, the following questions need to be answered prior to simplifying and implementing the BSS 
with local partners: 
1. What data is required at a local level? 
2. Which questions are critical in the BSS in relation to local needs? 
3. What is-the minimum criteria/experience of organizations to implement the BSS or a 
similar tool? 
4. What are the capacity development needs? 
5. How will the process and findings be linked to both policy development and the end 
users? 
Government of hdonesia (Minisstry of Health) 
Project Start-up - In the original project, the Jump-start method did not allow for enough time for 
GO1 to assess the NGOs to be funded. At the beginning of the next project, the GO1 would like to 
have an appropriate amount of time to assess the NGOs. In addition, the GO1 would also like to be 
involved in monitoring project implementation especially through NGO activities. The NGOs need to 
know that the GO1 is the ultimate authority with Indonesia and that they are the lead partners for the 
new project. Finally, under the new decentralized authority within the GOI, the workplan for the new 
project will have to be approved by the local Government. 
Under the new project, there is a need for early discussions and clarification of the relationship 
between the GOI/Depkes and the Project. A clear terms of reference for the work to be done in 
collaboration needs to be articulated early in the development of the Project. This is especially true for 
the collaborative work in surveillance, but also in terms of local GO1 involvement in the Project at 
provincial/district levels. 
The new project should be tailored specifically to needs at the provincial/district level. In addition, 
the new project should clearly articulate its own limitations to.the 11 provinces so that expectations 
are in line with these limitations. Finally, service delivery at a local level should have a greater 
emphasis in the next project. 
Overall, there was a concern that, within the next project, there are not enough funds to cover all of 
the provinces/districts proposed. In addition, they felt that there should be more emphasis on the 
outcomes not only process indicators. However, there was concern that the level of indicators 
described in the new project (i.e. STD service provision) cannot be measured through NGOs. 
Finally, it was unclear to the GO1 how the GO1 will access funds through the new project. The 
overarching question was how the new partnership with the GO1 will be operationalized. The 
important recommendation that the GO1 has is that the new project be proactive about early and 
continued GO1 involvement and not reactive. 
Surveillance - The GO1 feels that HAPP's biggest contribution has been in the area of Behavioral 
Surveillance. In the future, to reduce the costs associated with the BSS, priority areas should be 
HAPP - Internal Evaluation Report 
identified where it is implemented. In addition, it was requested that a 'second generation' BSS 
should be developed that can be implemented locally and the data used locally. There was a 
suggestion that perhaps the BSS is not the best tool for the local assessment of programs. 
Condoms -There was disconsensus about whether 100% condom use policies should be implemented 
through a law enforcement strategy. It was agreed that teaching female sex workers negotiation skills 
was insufficient and that involvement of the client is critical to increase condom use. Discussions 
about promoting condoms as a means of inexpensive contraception ensued. 
Management of Sub-offices - The overriding question was: How will the FHI procedure for approval 
processes be decentralized? In addition, it was felt that the Beneficiary should be the focus in the next 
project, but, given GO1 decentralization, local advocacy and involvement of the local GO1 will be 
critical to ensure the agreement for NGO participation and local project success. 
Project Start-up - The new project should not use a 'jump-start workshop' as was done previously. 
There is no need to rush into subagreements prior to orienting the staff and project partners. There 
should be an introductory workshop that will orient everyone to USAID's strategy, the goals, 
objectives and framework of the new project, FHI and FHI's subagreement process. This workshop 
should include opportunity for the partners to organize themselves and to develop a plan of how they 
will collaborate under the new project for joint objectives. In this way, a clear definition of partnership 
can be articulated early and potential cooperative plans articulated that will operationalize the 
partnership approach. Only after this orientation, should the process for subagreement development 
begin. 
The new project should be organized through a phased approach - this includes a phased approach to 
engaging partners and a phased approach to geographical coverage of the provinces. 
Coordination and Partnerships - Overall, USAID felt that they had a good partnership with HAPP. 
They found that collaboration with some of the donors improved over time - particularly with WHO 
and AusAID. The IDU Rapid Assessment was an example of this collaboration. In the new project, 
cooperation with the GO1 is both under the USAID larger umbrella within the 'SOAG' as weU as 
through FHI. However, USAID sees that few of their partners understand the SOAG completely. 
Under the new project, the GO1 will need to see FHI as a resource for technical assistance. 
Surveillance - Sun-eillance will be the 'branches of the tree' under the new project. Surveillance will 
need to be intimately connected to the central and local level for decision-making. Teambuilding with 
Depkes will be required to implement the surveillance program properly. A TOR between FHI and 
GO1 will need to be developed to implement this component. Overall, the central authority will be 
involved in strategy and policy decisions while the local GO1 will be involved in implementation. 
Where the KPADs don't exist, capacity building will need to occur. Technical assistance will need to 
be tailored specifically to the needs of the local province and district. 
STD Service Provision - USAID sees STD service provision as an entry point for HN/AIDS 
prevention. There needs to be continued support to Putra Jaya to develop Lessons Learned from this 
site. In general, the new project will have to develop a strategy to supply STD services close to sex 
workers and their clients, whether this is in Puskesmas or through NGOs or private clinics. 
Condoms -The sustainability of the current strategy of supplementing the cost of foreign condoms 
and aggressively promoting them was questioned. This strategy, funded by KFW seems to be 
H4PP - InternalEvahation Report 
undermining Indonesian condom manufacturers' market share. The concern is that the Indonesian 
condom manufacturers could go out of business under this current strategy. 
Sub-office Management - USAID suggested that, under the new project, an efficient structure for the 
sub-offices is imperative. This includes hiring one project manager and one 'WAD advisor' per 
province. These three provincial offices should work as 'regional' offices within Indonesia. 
Dissemination of the Final Evaluation Document - Finally, USAID would like to see a workshop for 
the dissemination of the final evaluation document of HAPP. This kind of workshop is not within the 
current workplan. 
CDC Center for Disease Conirol USA 
Coordination - In general, CDC found that it was a difficult to coordinate and collaborate within the 
framework of HAPP. For a number of reasons, it was difficult for a US Government employee to 
collaborate with a contractor from a different branch of the US government. Nonetheless, the role of 
the CDC advisor was focussed on the provision of technical assistance, analysis and technical 
guidance in relation to several areas across HAPP and in relation to requests from USAID. Technical 
guidance in STD service provision was a priority focus for the CDC advisor. 
STD Service Provision - In the area of a future !3'D strategy, the CDC advisor suggested that primary 
prevention of STD/HlV through condom promotion should be the most important strategy for the 
new project in the area of STD services. 
B. Overall Summarv of Iakarta Consultations 
Projt-rt Start-up 
It was universally agreed that the 'Jump-start' approach used under HAPP should not be repeated for 
a number of reasons. Rather, the Project start-up should be seen as a phased approach with some 
provinces considered 'priority' under phase one. Priority provinces could be chosen based upon 
epidemiological significance and level of current development of a local response. Within provinces, 
obviously certain districts would also be considered priority. In relation to program development, 
there could also be some technical areas that could be given initial priority under phase I. (i.e. 
interventions with sex workers and their clients). There was not a detailed discussion about all the 
necessary components of phase I. 
It was axreed that an initial I~~~o~uc~oN workshop is essential that would allow proiect partners to: 
1. Learn about USAID and their SOAG for Indonesia; 
2. Learn about the Goals, Objectives, Design/Focus and the Structure/Management of the new 
project; 
3. Learn about FHI and their subagreement process; and 
4. Allow the partners to meet to discuss respective strengths, define their roles in relation to the 
structure of the Project and to identify strategies for cooperation/collaboration prior to 
subagreement development. 
The purpose of the meeting should be both to orient project partners, but also to begin to develop 
plans to operationalize a partnership approach for the new project. Sub-Agreement development 
NAP'- Internal Evaluation Report 
should not occur at this initial meeting. Rather, TA should be provided for subagreement 
development with cooperative planning behveen the organizations in mind. 
In addition, it was reported that the Provincial/District level GO1 partners need to be involved early 
on in planning. Therefore, it was discussed that the annual workplan for the new project will need to 
reflect the need for provincial/district level assessments, planning and approvals. 
Coordination, Networking and Parhrship 
Partnership was generally defined as a collaborative effort where the individual strengths of 
organizations are pooled to provide maximum benefit for the 'end user'. 
Overall, a wide variation of satisfaction among the various organizations existed concerning their 
experience of 'partnership' under HAPP. Some organizations were 'very satisfied' and others were 
'completely unsatisfied'. 'Satisfaction' was generally correlated with the expectations of the 
organization's role and subsequent involvement in the Project. Overall, most partners reported that 
all components of the Project, but especially the NGO strengthening efforts, could be improved by 
increased internal/external HAPP communications and a strategic and well articulated plan for 
collaboration early on in the new project. 
The need for a strateec plan for NGO capacitv building and NGO related technical assistance efforts 
for the new project cannot be understated. 
Serological Surveillance 
Although the majority of respondents in this consultation felt the sero-surveillance system was 
generally simplified, several key issues still need to be addressed. A recently conducted review of the 
national response to HIV/AIDS including the sero-surveillance system revealed several issues 
relating to strengthening the surveillance system in Indonesia. Specifically, the surveillance guidelines 
nationally need to be strengthened to ensure that implementation is clear concerning: 
. the type of target group to be sampled, 
the appropriate sampling procedures and 
lines of reporting from district to province to the national level. 
It is critical that enough funds are available to ensure basic surveillance with a minimum coverage of 
sex workers and blood donors. However, surveillance within additional high-risk groups is required, 
especially with transvestites and injecting drug users. In addition, the implementation of linked 
serological surveillance with behavioral surveillance should be considered. 
Behavioral Surveillance 
Several informants within this review suggested that the BSS results may be the largest contribution 
of HAPP. However, the utilization of the BSS data at local level has been minimal. In addition, under 
HAPP there has been little capacity development locally to implement or utilize the data. It was 
generally believed that either a short version of the BSS or a different tool for collecling and analyzing 
local data is required under the new project. In addition, it was discussed at length that if collection of 
data with a tool like the BSS is to be done by local organizations then several key issues need to be 
addressed including: 
Increasing the number of partners will increase the complexity and increase the requirements 
associated with quality control. Therefore, the role, responsibility and fmancial/technical 
authority of every organization must be very clear. 
HAPP - Internal Evaluation Report 
A plan for Capacity Building of local institutions must be articulated, provided with sufficient 
time, provided with appropriate follow-up and properly funded. 
In conclusion, it was not clear whether the BSS is the most appropriate tool for provincial and district 
level authorities for decision-making in association with programs, primarily due to the complexity of 
implementing it. Other tools, including qualitative research, need to be a part of locally appropriate 
evaluation methods. It was suggested that the new project consider the model of implementation 
used under the AusAID project for the BSS in their provinces where there was a combination of 
quantitative and qualitative data collected and local organizations were involved in the collection of 
this data. 
STD Service Provision 
There generally was no consensus on the best way to strengthen the service delivery of STD case 
management among the organizations reviewed. Responses varied from mass treatment, stand alone 
private clinics for high risk groups, integrated private sector approaches, integrated public sector 
approaches or by focusing greater efforts on promoting primary prevention through condom use. It 
was agreed that, if the drugs are not available, then there is no need to assess improvements in senrice 
delivery. Furthermore, it was generally agreed that if STD service delivery was to be prioritized 
under the new project, then services that will be accessed by the female sex workers and their clients 
should be a priority. (See: External HIV/AIDS Assessment, Indonesia 11/99). 
Condom Promotion 
It was generally believed that the condom promotion strategy could be improved by a stronger focus 
on male clients of sex workers. However, the current strategy within the country to allow foreign 
subsidized condoms to be aggressively promoted and to gain market share over Indonesian 
manufactured condoms was considered unsustainable. Overall, it was reported that there is not a 
clear national strategy in condom promotion. 
Management of HAPP induding Sub-offices 
Overall, the management of HAPP was constrained by a number of issues; however, an overriding 
theme was the length of time it took to get subagreements approved. It was reported by several 
partners that at times it could take several months for a subagreement to be approved and for funds 
to flow, even when the Project time was only a few months long. This was a major constraint and it 
hampered project progress. 
The need for decentralization of authority to the FHI Country Office was raised as an overarching 
issue. In addition, the utilization of the provincial office in more than an administrative role was also 
cited as critically important. However, under the new project, the provincial offices are suggested to 
have minimal staff and also have a 'regional role'. There seems to be a contradiction. Many persons 
interviewed suggested the need for increased decentralization to the provinces, however, under the 
new project there could be fewer personnel in the provinces with an increased role across a region. It 
seems unlikely that 'decentralization to the provincial offices' will be able to take place. Nonetheless, 
overall, the roles, responsibilities and authority of HQ, the ARO, the local Jakarta office and the 
provincial offices need to be clearly defined and appropriate authority given to each to ensure both 
timely responses and high quality work. 
CtAPP - lnternal Evaluation Report 
C. Summary of People Contacted 
ATMA JAYA: 
AusAID Project: 
Business Dynamics: 
CDC, USA: 
Government of Indonesia: 
KPA: 
PACT: 
PATH: 
PCI: 
The Futures Group: 
University of Indonesia: 
USAID: 
WHO: 
Drs. Clara Ajisuksmo, Yustina Rostiawati, Charles Su qadi 
Drs. David Elkins, Abby Ruddick 
Diena Trigg and Andrew Trigg 
Dr. John Moran 
Drs. Runizar Roesin, Arwati, Haikin Rahmat, Drs. Saiful Jazan, Sigit 
Priohutomo, Rita Priohutomo. 
Djoko Prayitno 
Imam Mulyadi 
Silvy Wantania 
Carol Carp and Paiupi Widjajanti 
Reed Ranilow and Danny Yatin~ 
Dr. Budi Utomo 
Rafna Kurniawati and Joy Pollock 
Dr. Bing and Dr. Amaya 
hXPP - htemal Evaluation Reporf 
C. Third Stage - Analyzing the Data 
Outline of Evaluation Report 
I. 
11. 
In. 
N. 
v. 
VI. 
VII. 
VIII. 
IX. 
X. 
XI. 
XI. 
XIII. 
HAP Project Context 
HAP Project Design 
HAPP Internal Management 
HAPP Intra-Office Management 
Funding Mechanism 
Working with KPADs 
Working with Parhers 
Working with NGOs 
Building Support: Networking and Coordination 
Beneficiary Involvement 
Technical Issues 
Surveillance 
Policy Support 
HAPP- htemaI Evaluation Report 
C. Third Stage - Analyzing the Data 
A framework was then developed for writing the evaluation report, which was expected to take the 
form of a large, narrative-style document. The Evaluation Team wrote within this framework, each 
concentrating on an area in which they had previously gained considerable experience. 
During this process, it was then decided that it would be more manageable to produce the book in 
bullet points and some of the reports in this section reflect this intention. It is interesting to compare, 
for example, the development of the sections on internal management and intra-office management 
from report style to bullet points. Similarly, one can see the way in which text was reallocated to 
different sections in the bullet point version; this becomes evident, for example, upon close 
examination of some of the text in "Subproject Management Issues", which was moved to "Funding 
Mechanisms Bullet Points". 
This section contains the original framework for the report as well as the detailed reports in their 
correct order according to the original framework. These provide a complete and thorough picture of 
HAPP and its activities. The report on the evaluation method is not included here as it has been 
included in Section 1 on "Collecting the Data". 
Outline of Evaluation Report 
The main part of the evaluation report, which was to cover all information gathered during the first 
stage, was originally expected to cover the following matters in the following order. 
HAPProject Contexf 
a. Project timeline - major project and environmental events 
b. GOI/central, provincial. FHI: HQ, ARO, USAID, NGOs, PVOs - requirements, agenda 
and 'culture' of all players 
c. Economic crisis 
HAP Project Desig 
(issues: shift in project conception projections; audience data; broaden view of intervention; 
better surveillance; fragmented agenda/locations) 
E-IAPPlnternal Management 
a. HAPP internal management 
- finance, 
- administration, 
- human resources - staffing, recruitment, training, personnel policy; internal 
monitoring/ 
- project documentation (ME?), submission of complex documents in English)- Intra￾office communication 
- infrastructure development - management style of HAPP (participator): openness, commitment to change) - 
hXPP - Internal Evaluation Report 
b. Review activities: 
- Program Feedback - formal - ongoing 
- Internal Review 
- ExternaI Review 
- USAID 
- Retreats 
- Daichi Meeting 
- Acacia Meeting 
- Note: unwritten assumptions - Senior staff preparation by FHI 
I HAPPInfra-Office Management 
- FHI - HQ/ ARO/HAPP - InternaI/ProvinciaI - HAPP Internal Review 
- HAPP - IA (Intra project communication) 
Points: - Provincial offices - Finance - internal + to IA 
V. Funding Mechanisms 
VI. Working with PADS 
- cross cutting Capacity Building 
VII. Working with Parlners 
- (include as capacity building also) 
- Intra partner communication 
- Networking/NGO Forum - Relationship with HAPP, KPAD, other IAs, local authorities, PVOs, NGO Forum - Working with Partners: (a) PVOs; (b) TAGS 
- Capacity Building 
VII. Working With NGOs 
- Capacity Building - cross cutting for IAs, 
IX Building Support: Networking and Coordination 
X Beneficiary Involvement 
XI. Technical Issues: 
a. BCC - (IEC to BCC - capacity building) 
- Material (DKT, HAPP, IAs, TFG) 
Issues: availability, timelines, target specific, capacity building 
- Media 
- Channels for BCC - 
NAPP- Internal EvaIuation Report 
Interpersonal -Outreach and PE 
Mass Media (radio and TV) - mass media, press capacity - reporter training (Jogja, 
GOI, IPM) 
c. CSM 
- BCC - Promotion - Logistic/Task Force 
- DKT 
- Service delivery strengthening (information, HRD) 
- Sewice promotion: 
referrals 
messages 
XII. Surveillance 
Issues: local capacity building, dissemination and use of data, connection behveen biological 
and BSS surveillance; local ownership 
a. Biological 
b. BSS 
XI11 PoIicy Support 
GO1 - Central 
Coordination and Collaboration - WAD 
As far as possible, this order is the order in which the reports compiled from the initial data are 
presented in this book, although the final order used in the 'Lessons Learned' report is markedly 
different. Some sections, such as 'Internal Management' have more than one report. Some, such as 
'Project Design', are covered in just one page. 
HAPP - Internal Evaluation Report 
I I. HAP Project Context 
Evaluation Report 
1 HAPP Project 
Context 
11. HAPP Project 
design 
111. HAPP Internal 
Management 
IV. HAPP intra-office 
management 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
A. Managine Expectations -Multiple Stakeholders 
Recognition needs to be given to the enormous accomplishments and 
progress made by all involved in this project; all of whom contributed to 
the overall building of HAPP's capacity. All of the problems, gaps and 
remaining needs should be viewed in the context of a very large, 
complex and relatively 'young' project which sought to bring together a 
number of partners and stakeholders, to enable learning while 
implementation was taking place and to build on that learning. 
Stakeholders in HAPP included: the Government of Indonesic - MOH 
and KPA/D at central and provincial levels; the USAID Mission and, 
indirectly, USAID/DC; Family Health International- HQ and Asia 
Regional Office; local offices of international PVOs - but which must also 
be responsive to their own headquarters; local NGOs; religious-based 
organizations; universities; various private sector businesses; and the 
communities in which activities were to be implemented. One of the 
greatest challenges of the HAP Project was creating a technical and 
management 'marriage' among the many institutional and individual 
stakeholders involved in the Project. 
Each of these had its own continuously evolving needs, perceptions, 
agendas, priorities and cultures which affected expectations and 
working relationships with the HAP Project. Not unexpectedly, these 
inherent differences produced friction or conflict within the Project. The 
management of outcomes of this 'friction' required continuous attention 
and effort. 
In most cultures, there is an unspoken perception of conflict as 'failure' - usually on the part of one or 
more parties. This perception can prevent effective management of conflict or disagreements. 
Though, in retrospect, the potential for discord appears predictable, this was not a significant 
management role and, therefore, no mechanisms or strategies were in place to deal with or to take 
advantage of, conflict. 
The evaluation activities identified many valid recommendations from a variety of sources, ranging 
from 'shoulds' to 'musts'. Implementing any of the 'one-line' recommendations will require time and 
resources. Some of the recommendations, even from the same source, create opposition. For example: 
the recommendation that bureaucratic processes be streamlined and delays decreased will have to be 
balanced with a recommendation that there be greater consultation and involvement of 'partners' in 
decision-making. Another source of conflicting recommendations is the perceived need by NGOs and 
others for a quick start-up and flow of funding versus the recommendation and perceived need of 
FHI, USAID and the GO1 to start in an orderly, phased manner. 
Each recommendation needs to be evaluated in relation to overall project priorities and requirements, 
project design and the resources available: particularly staff and budget. To a certain extent, each 
group making recommendations or voicing requirements may be in competition with others as 
resources available within the Project are not adequate to meet all expectations. 
HAPP- Infernal Evaluation Repod 
Participation and consultation of parties involved is essential - particularly to negotiate or to obtain 
agreement from the various stakeholders regarding the degree to which their expectations can be met. 
The geographic spread and the number and range of stakeholders have shown that this component of 
the Project is one that will require significant funding and staff time. 
It is clear that management of expectations will be a priority concern: both during start-up and 
throughout the next phase of the HAP Project. 
Lessons Learned : 
Greater information sharing and consultation with partners and stakeholders tended to decrease 
friction. The process and level of resources to be used to ensure this happens needs to be 
consultatively negotiated and planned as much as possible. 
Consultation takes a significant commitment of time and human resources. The amount needed is 
underestimated by all involved. 
Successful management of the tensions inherent in this project wit1 require the cooperation of all 
stakeholders and an understanding that the differences in the needs and priorities of aU 
stakeholders will require compromise among all. If possible, an agreement to 'disagree' should be 
negotiated from the beginning, along with an agreed process for managing negotiation and 
friction. Such an agreement and process may function to decrease the negative aspects of the 
intrinsic differences. 
Actual or potential conflict can provide an opportunity for exploring differing needs and 
perceptions which will affect the success of the Project, but this expectation and the skills and 
processes need to be in place from the beginning. 
Viewing a certain level of conflict as inevitable in a complex development project may allow for 
more positive management approaches. Project managers and staff may benefit from basic 
training in conflict management or mediation. 
Multiple start-up 'orientation' activities will be needed in order to manage expectations and to 
negotiate mutually acceptable processes for information sharing and consultation. Ongoing 
updating and negotiating activities will be necessary throughout the life of the Project. 
Requirements and Procedures of FHI/USAID/GOI 
A collaboration among FHI/USAID/GOI means the obligation to meet the requirements and 
regulations of concerned parties. This was compounded by requirements for project management to 
be conducted in a consultative manner and to meet requirements of relevant parties. Examples of 
practical issues included setting and meeting deadlines, following procedures and regulations, 
reporting, maintaining quality control, meeting expectations of MOH and other relevant minish-ies, 
provincial government and USAID. 
Effective implementation of the Project was further hampered by a lack of understanding of the 
processes/procedures of these parties (MOH, FHI, including HQ, ARO and USAID) and inadequate 
orientation/briefiing/training for HAPP staff with little experience of working with FHI/USAID. 
More intensive orientation was provided to staff of HAPP in line with the recommendations of the 
Internal Review. 
Improvement of the management skills of the HAPP staff is required to enhance their ability to 
understand each HAPP team member's roles and responsibilities and to communicate/ coordinate 
among staff about routine HAPP activities, such as: (a) responsibilities for determining timetables and 
deadlines for subagreement finalization and submission of reports; @) responsibility for regular 
contact with PVO partners; (c) responsibility for monitoring and support to local NGOs (IAs); and (d) 
responsibilities for and differentiation between administration and program responsibilities. 
p~ 
HAPP - Internal Evaluahon Report 
Attention to USAID regulations and requirements is needed to better support FHI's ability to respond 
to necessary program actions. 
C. Economic Crisis 
Zntroduction 
The economic crisis, which be~an in - July 1997, rapidly showed sociopolitical ramifications in addition 
to the economic/financial consequences such as substantial depreciation of the rupiah, inflation and 
extreme fluctuation of exchange rates. Between July 1997 and the end of the HAP Project, there was a 
succession of three different governments, while political instability and insecurity were consistent 
major concerns throughout the period, albeit at varying degrees of severity. Among others, the 
sociopolitical impact of the crisis took the form of sharp increases in rates of unemployment and/or 
under-employment owing to financial constraints in the private sector and an increase in the 
prevalence of poverty. By and large, people's income and purchasing power declined severely as did 
the public sector revenue. Low morale among people in general was not uncommon in the face of 
uncertainty surrounding the future of the country in terms of the volatile political situation and 
financial difficulties. 
Issues: 
. The Indonesian econonlic crisis led to significant changes in USAID's development response 
in Indonesia. The initial approach involved the identification of 'Social Safety Net' responses 
by all USAID contractors. The inflexible nature of the Performance-Based Contract prevented 
changes to the existing contract and the CPO required consistency over time in the technical 
approach. Consequently, in addition to all deliverables stated in the original contract, the 
HAP Project was responsible for integrating the social safety net feature in its intervention. In 
the context of an emergency situation, time and efforts invested by HAPP staff in assessing 
and documenting possible social safety net applications within the Project components were 
also intensive. This dilemma was ultimately resolved in the new USAID SOAG and in the use 
of more flexible funding mechanisms, such as RRF. 
. Depletion of public sector revenue undermined the Government's ability to meet the people's 
needs during the crisis and to sustain components of the Project that the GO1 had planned to 
take over. In connection with this, financial difficulties experienced by the private sector and 
a drop in people's purchasing power hindered reliance on their contribution. One illustration 
was that the Government's fiscal constraints led to a request to USAID for FHI to purchase 
HIV test kits for the Red Cross. The package of support to KPAD was designed to decrease 
gradually over the life of the Project, but the economic crisis caused maintenance of the same 
Ievel of support to the end of the contract. 
. The crisis underscored the needs for HIV/AIDS prevention intervention in light of the 
anecdotal observation, suggesting an increase in the number of migrant CSWs in the HAPP 
demonstration sites due to sociopolitical unrest in other parts of Indonesia. Moreover, a 
higher number of factory workers turned to sex work and a higher demand for free condoms 
was also noted. The rise in the number of younger and less experienced CSWs after the crisis 
was found in a survey conducted by the University of Indonesia. 
. At the same time, the economic crisis also impeded implementation of the Project and 
ultimately, the attainment of some of the contract performance objectives (00). Among 
others, some implications of the crisis were: (a) a decrease in the number of clients of CSWs 
which further lowered the bargaining power of CSWs to negotiate condom use; (b) an 
HAPP - Internal Evaluation Report 
increase in the mobility of CSWs and waria, caused by open hostility of the community 
against them in the era of 'reform' and/or a decline in the number of clients; (c) the 
disruption of outreach workers' activities because of public disturbances (e.g. general election 
and presidential campaigns, riots); and (d) the relocation of some condom outlets, resulting 
from the riots. Solutions adopted by IAs included: (a) re-directing their activities to different 
red light districts; @) identifying new geographic areas where CSW activities occurred or 
expanding geographic target areas; and (c) recruiting additional outreach workers. However, 
even with a collaboration with the CSW association, the IA faced substantial constraints in re￾mapping the situation after the closing of the commerci~l sex establishment resulting from 
the integration of sex workers into the community with false identities. 
. As numerous economic/financial consequences and sociopolitical ramifications of the crisis 
overwhelmed people in general, attracting their attention to issues related to HIV/AIDS and 
encouraging them to place these issues as a priority in their life while their most basic needs 
(e.g. food supply, sense of safety and security) were threatened entailed creative and 
persistent efforts of the HAPP staff and their partners. 
. The volatile economic and sociopolitical climate also affected management of the Project, 
especially in terms of the timing and scheduling of activities and the Project workplan. 
Consequently, the HAP Project had to accommodate changes in deadlines and reorient the 
workplan. Inflation and the fluctuation of exchange rates led to the readjustment of the 
approved budget of IAs and difficulties in the budget preparation of most PVOs and NGOs 
and ultimately, delays in proposal development. The crisis necessitated the 50% increment of 
all IA budgets, mentoring USAID FSN salary levels and the provision of COLA by FHI-HQ. 
Lessons Learned : 
The inflexibility of the PBC posed challenges to integrate social safety net concerns into the 
HAP Project. Such flexible funding mechanisms as RRF evidently allowed urgent response to 
the pressing and evolving needs of the community at large during the crisis. 
The economic crisis created demands for HN/AIDS prevention intervention because of 
anticipated growing numbers of people with high-risk behavior on account of a severe 
decline in their purchasing power and a decline in the Government's ability to address 
people's needs due to the depletion of public sector revenue. 
The economic crisis presented challenges to all aspects of the HAP Project: the reformulation 
of subprojects to integrate the social safety net responses, management and implementation 
(e.g. budgeting, scheduling and the on-time implementation of activities, reaching target 
audiences). Additional workloads to the normal workloads fell on the staff's shoulders in an 
attempt to respond to the varied challenges. 
Flexibility and creativity in project management and implementation on the part of the HAPP 
staff and their partners to overcome ongoing problems, endurance of their staff to deal with 
the challenges and good teamwork became the key to success. 
HAPP - Internal EvaIuation Report 
I Evaluation Report 1 
I. HAPP Project 
Context 
II. HAPPProject 
Design 
111. HAPP Internal 
Management 
IV. HAPP intra-office 
management - 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary , 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
11. HAP Project Design 
HAP Proiect Desim 
The project design has evolved over the life of the Project. Audiences 
and sites have been selected in response to assumptions, political 
agendas and increasingly, in response to better data and understanding 
of the stage of the epidemic in Indonesia. Unlinked sites and, to a certain 
extent, diverse, unlinked audiences made delivery of an integrated 
program capable of producing a 'critical mass' of impact difficult. 
Issues due to the specific contracting mechanisms are also the source of 
important lessons which are discussed in section .. X .. The end of global 
AIDSCAP funding and changes in project contracting meant that 
significant levels of time and resources went into learning new 
contracting requirements, start-up and close-out activities during the life 
of the Project. The inflexibility of the HAPP Performance-Based Contract 
made responding to evolving needs difficult. This was particularly 
evident in the case of new needs and priorities resulting from the 
economic crisis. 
However, experience gained, capacity built, working partnerships 
developed and a clearer understanding of HIV and STDs in Indonesia 
through HAPP has contributed greatly to the national ability to design 
and implement more effective programs. 
/ 
Lessons Learned : 
Widespread and diverse geographical sites and audiences increase 
the number of management issues and demands, which has an impact on HIV/AIDS prevention 
efforts through the diffusion of resources and effort. 
. Learning based on biological and behavioral data gathered through formal data collection 
activities and informally through documented experience needs to be combined with the learning 
about the realities of implementation when planning any new projects. 
. Being able to focus on linked and interacting audiences and sites has the potential to increase the 
potential for the impact of resources expended. 
HAPP - Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111 HAPPlnternal 
Management 
IV. HAPP intra-office 
management . 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
I VII. Working with I 
i Partners i VIII. Working With 
NGOs 
! IX. Building Support: 
Networking and 
I Coordination 
X. Beneficiary 
Involvenlent ! 
, XI. Technical Issues: I 
, XII. Surveillance 
1 XIII. Policy Support I 
III. HAPP Internal Management 
A. HAPP Internal Management 
Staff~g and Workhad 
Throughout the life of HAPP, the number of staff increased, particularly 
after the staffing review conducted as follow-up to the Internal Review. 
Among deficiencies in terms of staffinkwere HAPP being understaffed, 
incongruities in staffing and demands for supervision and technical 
input, as well as inadequate understanding of one's own and colleagues' 
job descriptions. The review resulted in the finalization of position 
descriptions (PD) for new staff, the revision of the position descriptions 
of all staff, a redesign of the HAPP organizational chart and a revision of 
the HAPP management plan. Since the Internal FHI Review, staffing 
improved and turnover was reduced. 
It should also be recognized that finding qualified staff with 
management skills and/or technical skills and experience in HN/AIDS 
prevention took a considerable time. Demands on the office at different 
points in time differed and required the establishment of a databank of 
consultants to fill gaps when required. 
Capacity Building of HAPPStaff 
/ Staff at the HAPP office received different types of capacity building 
1 activities, including regular supervision, training on technical matters 
and English language and orientation both in Indonesia and the FHI 
regional office. The results of the English training could be seen in the improved quality of the 
quarterly reports. Technical knowledge, management capacity and teamwork made it possible for 
Indonesian staff to complete planning and budgeting, e.g. the annual workplan and regular reports. 
Management Style 
Both staff and NGOs valued the HAPP COP/FHI CO's participatory style and innovative ways of 
combining staff for problem-solving. The advantages of this approach were that it improved 
understanding of HAPP overall and staff performance, it provided an opportunity for people to try 
new roles and to look at new problems. The approach was also reflected in the involvement of all 
HAPP partners in consultations in conjunction with various forms of evaluation (e.g. the internal and 
external assessnlent) during the course of the Project. Based on the recommendations, follow-up 
actions were conducted. The overall process led to improvement in different aspects of HAPP 
management, especially in the Jakarta office. 
Lessons Learned : 
. The quality and timely implementation of the Project requires openness and a willingness to 
reflect on the implementation process of HAPP and the commitment of HAPP/FHI to rectify 
deficiencies related to non-technical and technical aspects of the Project. 
HAPP- Internal Evaluation Report 
Teambuilding and Team work 
The effective team approach was reflected in the successful completion of complex tasks that required 
contributions from many staff, such as the development of the SAR, Annual Data Report and 
workplan. Office-wide retreats/training/problem-solving meetings were praised by staff members as 
useful in terms of building good understanding of the overall HAPP mission, strengthening 
communication, teambuilding, sharing experiences among staff, effective planning and advancing ' personal professional growth. It was put forward, however, that the meetings should have clear 
professional/organizational objectives and participants should be actively involved in appropriate 
substantive discussion. 
Lessons Learned : 
. Contributing factors to the effective application of a team approach are the careful selection of 
staff, a clear sense of mission and shared team values. 
Finance Matters 
Prior to the Internal Review in 1998, a transfer of funds from HQ required up to 3 weeks to arrive in 
Indonesia. There was also a lack of two-way communication and constructive feedbacks on the IAs' 
financial reports. Upon the recommendation of the Internal Review, a comprehensive analysis of the 
bottlenecks in processing financial reports (e.g. a review of procedures, filing systems, tracking 
systems etc.) was conducted. Based on the findings of the review, remedial actions were carried out in 
almost every aspect of the procedure of the financial process. The decentralization from HQ and ARO 
has allowed for payments to be authorized from BKK rather than Arlington. A tracking system, a 
time schedule for different types of payment and the computerized accounting system were put in 
place. As a result, the Finance department was able to expedite the transfer of funds to IA within 2 
weeks of the submission of complete documentation. In addition, the speedy process was also 
facilitated by the delegation of the authority to process disbursements to CD in the absence of the 
COP and assistance provided by provincial finance and Administration officers in checking all of the 
Recipient Monthly Financial Reports and supporting documents. The latter significantly contributed 
to the improvement of the monthly financial reports of HAPP IAs in terms of time and required 
attachments. 
Various forms were developed by the Finance department to facilitate the processing of financial 
matters, such as: 
a. guidelines for expenditures and reimbursements; 
b. budget proposal and financial report forms for WADS and RRF proposals to track 
expenditures versus budget; and 
c. Budget Analysis forms and the Recipient Monthly Financial Report tracking system for 
project financial reports and records. 
By and large, the concerted efforts to deal with delays in the processing of financial matters were 
more effective than previous ad hoc initiatives (e.g. the provision of cash advances based on IAs' 
estimates of expenses for a certain period of time). 
Other good finance-related practices have been carried out such as the conduct of an internal audit for 
all HAPP IAs as part of FHI HQ routine financial supervision and spending within a planned budget. 
However, the accuracy of funding projections (expenditures vs planned budget) remains to be 
strengthened. 
HAPP - Internal Evaluation Report 
Lessons Learned : 
. When bottlenecks in processing of the financial matters occur, a comprehensive review of 
procedure of processing of financial report, followed by concerted efforts to rectify the 
deficiencies, are required to result in efficient processing of the report and disbursement of 
funds to the field. 
. Technical assistance to IAs' financial report provided by knowledgeable staff to ensure 
compliance with the existing administrative requirements can contribute to faster prc~essing 
of the financial matters. 
Required Submisson of ProjerRelated Documents in English 
All project-related documents submitted to FHI and USAID have to be written in English. Many of 
the IAs, however, possess limited writing skills, even in Indonesian, and inadequate proficiency in 
English. Inevitably, analytical writing skills among IAs were restricted as reported by the assessment 
carried out by the Atma Jaya University. Translation of IAs' reports into English by a translator with 
insufficient knowledge of technical matters led to misunderstanding due to unfamiliarity with 
concepts and terminology. Likewise, the reports written in poor English could cause confusion. To 
deal with this situation, HAPP accepted IAs' reports written in Indonesian and encouraged the 
project manager to write the executive summary in English as part of a capacity building exercise. The 
translation and compilation of these into HAPP quarterly reports was assigned to a bilingual native 
English speaker or an Indonesian with knowledge of technical terms and concepts. 
Lessons Learned : 
. The translator of IAs' reports needs to be not only bilingual but also possess to knowledge of 
technical matters. 
. Capacity building efforts aimed at improving the report writing skills of the staff of Ws and 
their English proficiency remain priorities. 
Use of Erne and Cost-Sa ving Measures 
Various time and cost-saving measures have been applied to enhance the implementation of HAPP 
such as upgrading the computer software system and the employment of a computer technician. 
Training in MS Outlook and MS Office for HAPP staff resulted in the production of better planning 
documents and a better electronic filing system. Establishment of an email system and the 
appropriation of cell phones in the Jakarta office and the provincial offices facilitated communications 
between HAPP offices and with the HAPP partners. The use of email was found to be particularly 
cost-effective with the size of the telephone bills remaining the same in spite of the presence of 
additional staff. Communication with the Regional Office improved through the writing of a brief 
monthly report with a follow-up telcon between senior staff. RII's computerized accounting system 
(MTX) was established and the Finance Unit has continued to improve the MTX reports to fulfill FHI 
HQ standards and, in turn, a higher quality of financial reporting to USAID. 
In~plementation of the technology was not always smooth. MIS was delayed from December 1997 to 
the last quarter of 1998 due to ongoing problems with the technical design. Similarly, the FIS has 
improved reporting considerably but it took time to ensure that FHI formats were consistent with the 
formats required by USAID. 
-- 
HAPP- Internal Evafuahon Report 
Others 
The need for a bigger office space in Jakarta was reiterated in the Internal Review and led to 
renovations of the Jakarta office. The renovations were completed with minimum interruption by 
building a temporary office. 
Program PJaming 
Since 1997, FHI/HAPP has demonstrated improvement in the process of developing and finalizing its 
annual workplan. In the first year, preparation of the workplan was delayed due to failure to provide 
the required format at the beginning and several revisions and reformattings of the document based 
on Mission requests. The effectiveness/efficiency in implementation of activities based on what has 
been planned in the annual workplan needs further attention. 
Recognizihg CriticaJ Problem Areas And Opportunities 
Key HAPP staff, at all levels, still need to be more involved in holding group discussions to explore 
and clarify the nature of the problems, which can lead to good problem identification and solving. In 
addition, group discussions will enable FHI/HAPP to accept and offer staff ideas individually and 
privately about the nature of the problems. 
FHI will continue to recognize critical problem areas and opportunities in FHI's non-technical and 
technical abilities to comply with the current contract obligations which were found by FHI Internal 
Assessment Team in August 1998. We recognize that some of the specific recomn~endations have 
been implemented, which has added value to the Project. 
8. HAPP Internal Manapement Bullet Points 
Jntroduction 
The design of the HAP Project conlprised of partnerships with GOI, USAID, FHI (ARO-HQ), local 
NGOs or IAs and PVOs. Inevitably, staff of HAPP had to address requirements, regulations and 
expectations of all parties involved in the Project (GOI/USAID/FHI (ARO-HQ)/IAs/PVOs) as well 
as of colleagues (provincial staff and staff in the Jakarta office). They were also responsible for 
delivering on-time and quality work in consultation with all parties concerned. The collaboration was 
further translated into a lengthy bureaucracy and consultative process to meet the expectations, 
requirements and regulations of a11 parties. This became the major constraint to progress in the 
implementation of the Project, including the Project development and approval, which will be taken 
up again in the section on funding mechanisms. 
Staffing 
. For the first seven months of the HAP Project, the position of the Chief of Party was not filled 
and was represented by several staff of FHI-ARO and FHI who rotated on a bi-weekly basis. 
The acting COPS largely concentrated on negotiation for initiation of the HAP Project, rather 
than on providing technical orientation to staff in preparation for impIenientation. The lack of 
orientation, in addition to the lack of consistent leadership, inadequate planning and the 
different management styles of the Acting COP, adversely affected HAPP staff's ability to 
implement the Project with the required knowledge and leadership. 
. The HAP Project was fully staffed through recruitment of personnel to fill 13 available posts 
between January 1996 and January 1997. Since then, changes in staffing took place annually 
NAPP - Internal Evaluation Report 
(see Table 1). The most distinctive chanaes occurred after the staffing - review conducted as 
follow-up to the Internal Review. 
Table 1 Evolution of staffing in the HAPP Jakarta Office 
Number of fixed posts 
Number of additional 
posts 
Finding qualified staff with management skills and/or technical skills and experience in 
HIV/AIDS prevention took considerable time. Recruitment of the HAPP staff was often 
based on a limited number of qualified candidates. 
1996/1997 
13 
posts 
In the early phase of the Project, apart from being understaffed, staffmg of HAPP was 
characterized by: (a) incongruities in staffing and demands for supervision and technical 
input; (b) inadequate clarification of individuals' and colleagues' job descriptions, as well as a 
link between them for coordination purposes; and (c) limited skills and understanding of 
supportive supervision among HAPP staff. Consequently, provision of program supervision 
and technical support to IAs and provincial offices and managerial duties were not optimally 
discharged. 
Number of abolished 1 
A number of events were held to resolve some of the staffing issues mentioned above (such as 
retreats, meetings). Examples were the recruitment of an Evaluation Officer to coordinate the 
quarterly reports and the assignment of the Program Officer to focus on contract and grant 
management. However, the same issues related to staffing were identified again during the 
Internal Review. The Internal Review, staffing review and technical support provided 
resulted in the finalization of position descriptions (PD) for new staff, the revision of the 
position descriptions of all staff, a redesign of the HAPP organizational chart, a revision of the 
HAPP management plan, training for staff and recruitment of new staff. Since then, HAPP 
staffing improved. FHI-ARO and HQ technical support played a critical role in the Internal 
Review and its subsequent follow-up actions. 
1997 
13 
3 
1 3 
Both the partners and staff of the HAP Project conceded that the lack of understanding about 
each HAPP team member's roles and responsibilities remained a crucial issue. The staff 
found their job descriptions to be too broad with no clear boundaries of authoriv among 
those who shared responsibilities. 
1 
Total number 16 20 
Overall, staff turnover of HAPP during the life of the Project was not particularly high. 
However, senior staff functioned as contact persons for aU HAPP parlners and were key focal 
points in the decision-making of programmatic matters. Any changes in senior staff were 
accompanied by changes in the policy or approach to programmatic aspects of the Project. In 
turn, continuity of the programming process was disrupted and previous agreements or 
promises were nullified or contradicted. Finally, the programming process became lengthy, 
while relationships with partners and the leadership of HAPP were compromised. 
1998 
@re￾internal 
review) 
15 
1 
15 20 
1998/1999 
(post￾internal 
review) 
16 
7 
2000 
20 
1 
HAPP - Internal Evaluation Report 
. Demands on the office at different points in time varied. Consultants with matching technical 
backgrounds were hired intermittently to meet the demands. 
Lessons Learned : 
. A staffing review to anticipate the numbers, technical and managerial requirements and job 
descriptions of the staff for effective implementation of a project should ideally be conducted 
prior to the implementation of the Project. Factors to be taken into account are: (a) the scope 
of work of the overall project (e.g. the goals/objectives); (b) geographic coverage; (c) the 
number of parties involved (GOI, USAID, FHI-ARO/HQ, IAs) and the consequential 
demands for technical and managerial support; (d) the available funds; and (e) the capacity of 
the human resources available. 
. Acknowledgement from the FHI Vice President that HAPP was understaffed (after the HAPP 
task review) led to authorization for more staff recruitment and position creation. 
. As far as possible, the team leader and senior staff should be appointed prior to the 
implementation of the Project and given sufficient time to orient themselves and the staff 
with the Project in preparation for implementation. Staff orientation programs should include 
the project goal and strategy, technical issues and administrative matters. 
. Projects such as HAPP reasonably involve overlapping areas among staff's job descriptions. 
In this case, clear boundaries of authority anlong those who share responsibilities are more 
important than ever. 
. With respect to staff turnover, particularly the turnover of senior staff, sufficient overlapping 
periods of en~ployment behveen new and old staff is essential to orient the new staff. A 
smooth transition between them in terms of the continuity of the Project and honoring 
previous promises/agreement should take first priority. 
. The HAP Project was dynamic and fast-moving and, consequently, demands for staffing 
changed over time in response to changing demands on the office. Hence, it is crucial to carry 
out staffing reviews to identify the needs for both short-term and more permanent staff at 
different points in time during the life of the Project. 
Recommendations : 
. A database of consultants should be established to fill in gaps when required 
Capacity Buildng for HAPPStaff 
Issues : 
. As mentioned earlier, collaboration between USAID/GOI/FHI meant obligations in meeting 
the requirements and regulations of the concerned parties. Insufficient understanding of 
these factors on the part of the HAPP staff prolonged the completion of such tasks and led to 
frustration among the staff. One instance was a long delay in the preparation and approval of 
the workplan in the first year due to failure to receive information about the required format 
from the beginning and the multiple revisions and reformattings of the document based on 
Mission requests. 
. The staff of the HAP Project relied largely on trial and error or learning-by-doing/on-the-job￾training to become proficient in the regulations/requirements of GOI, FHI (HQ, ARO) and 
HAPP - Internal Evaluation Report 
USAID, given the staff's lack of experience of working with them and their inadequate 
orientation/briefing/training. An orientation on the FHI regulations was provided to staff of 
HAPP in line with the recommendations of the Internal Review but there was not enough 
refresher training. It was recognized that learning-by-doing or on-the-jobtrainiig was more 
effective to strengthen some skills that required a lot of practice to improve them. 
Nonetheless, a systematic orientation of the existing regulations/procedures was one area 
that was identified by the HAPP staff to be the most crucial to foster uniform 
understanding/interpretation. Hence, it would minimize the chance of providing 
contradictory information to 1.4s or PVOs. A systematic orientation should also include 
alternative solutions to handle special cases in addition to routine practice as part of an effort 
to expedite the processing of such cases. In connection with this, a clear definition of the 
delegations of authority must be considered. 
Many tasks in the HAP Project comprised technical, programmatic and financial components 
such as proposal development or amendments. At times, the lack of understanding about 
each other's areas of expertise among the technical staff, the fiance staff and the program 
staff, as well as the linkages between their roles in the completion of tasks, led to delays or 
deadlocks in processing the tasks at hand. One illustration was an inclination to deal only 
with the component of the proposal that was relevant to one's responsibility. Hence, the 
fiance staff only read the part on fiance and the program staff were concerned only with 
the program side of the proposal. The review resulted in a disjointed proposal (an activity 
without a budget) or a long delay caused by a deadlock as staff failed to agree on a solution 
that accon~modated all requirements from each of their areas of responsibility. 
All staff, independent of their positions and expertise, had to deal with 
administrative/management aspects of their duties. As expected, management training such 
as the Institutional Development Framework (IDF) was perceived to be an extremely useful 
tool, particularly with respect to developing and enhancing leadership qualities and 
improving interpersonal comnlunication skills. 
HAPP staff received different types of capacity building activities, including regular 
supervision, mentoring. training on technical matters and on English language and 
orientation on management issues both in Indonesia and in the FHI regional office. Training 
and mentoring were found to be effective capacity building methods because they enabled 
staff to concentrate on learning specific subject matters at a particular time with limited 
interruption. The high level of the workload of the supervisor or subordinate and the limited 
understanding of supportk-e supervision contributed to ineffective capacity building through 
supervision. Inadequate supervision automatically hampered the attainment of skis or 
understanding through trial and error or learning by doing. 
Lessons Learned : 
. Prior to project implementation, key stakeholders (USAID/GOI/FHI (HQ-ARO)) need to 
provide systematic orientation to staff in their critical regulations and requirements to ensure 
consistent interpretation among the staff. The orientation can focus on basic and key 
information, as well as available references for various regulations and requirements. 
Refresher orientation is also essential to ensure that both the existing staff and the new staff 
are able to achieve the standard performance. 
. Staff can develop appropriate coping/stress management strategies by having prior 
knowledge of potential challenges resulting from the nature of the Project (multi-partner, 
multi-layer, dynamic). 
HAPP - Internal Evaluation Report 
. Orientation for all staff on areas of the job description and the technical aspects of each unit 
(Finance, Administrative, Technical and Program Units), albeit at an introductory level or in 
terms of general outlines, the linkages among the unit., potential conflicts and alternative 
solutions, can improve the overall performance of the staff. This can include discussion of 
their experiences. 
. Successful capacity building methods are influenced by adequate opportunity for staff and 
technical assistants/supervisors/trainers to undergo the process without interruption. 
Subsequently, such capacity building activities as training and mentoring are more beneficial 
for staff than supervision. Learning by doing/trial and error will only be an effective learning 
method if accompanied by adequate and supportive supervision. 
. Irrespective of their position and expertise, all staff need both administrative/managerial and 
technical skills to discharge their duties, as these duties conlmonly encompass both 
administrative/managerial and technical responsibilities. Accordingly, capacity building 
activities, particularly in technical and managerial areas, are important (e.g. supportive 
supervision, stress management, communication skills, negotiation skills, conflict 
management, time management). 
Management of HAPP 
. The collaboration between USAID/GOI/FHI (ARO-HQ) demanded flexibility and 
responsiveness on the part of the HAPP management to accommodate the inputs of these 
parties. In addition to the lengthy process of decision-making, HAPP frequently had to make 
repeated changes to previous decisions. Further, it was sometimes difficult for them to take a 
firm stand and be consistent on some technical and non-technical matters. Therefore, HAPF's 
partners often questioned its credibility and leadership. 
. Both staff and IAs valued the HAPP COP/FHI CO's participatory style and innovative ways 
of involving staff for problem-solving. The advantages of this approach were that it improved 
the understanding of HAPP overall and staff performance, and provided an opportunity for 
people to try new roles and look at new problems. The approach also resulted in the 
involvement of all HAPP partners for consultation in conjunction with various forms of 
review during the course of the Project. The limitations included a lengthy process, no 
feedback to participants on the decision and disappointment experienced by some people. 
. The HAP Project underwent numerous on-going formal and informal reviews, which were 
conducted by both HAPP/FHI staff themselves and external reviewers such as USAID or 
independent teams. Such reviews encompassed Internal Review meetings (the Daichi and 
Acacia meetings), the Internal Management Review, the External Review, retreats and 
USAID's monitoring field visits. These reviews typically discussed issues and provided 
recommendations. FoUow-up actions were usually carried out by the HAPP Jakarta Office 
whenever they were feasible. The overall process led to improvements in different aspects of 
the HAPP management, especially in the Jakarta Office, as outlined in a number of sections in 
this report. 
. The Internal Review had a more profound impact than other reviews on the improvement of 
the HAP Project management because its remedial measures were recommended based on an 
in-depth understanding of the root of persistent complex issues. The understanding was 
gained through the conduct of appropriate tools, such as the staffing review, the 
comprehensive review of the Finance Unit and the meeting with PVOs. The conduct of the 
review by FHI-HQ and ARO facilitated the implementation of the required significant 
changes. 
HAPP - Internal Evaluation Report 
. The senior HAPP staff of the Jakarta Office perceived that they were given sufficient 
authority and delegation for independent decision-making in routine matters. This was, by 
and large, also true for the administrative management aspect at the field level. However, the 
extent to which it was actuallv utilized at the field level was a function of the different 
personalities and cultural practice influenced by the cultural historical backg~ound of the 
area. (Manado, for example, has historically a high degree of European influence). Issues 
pertinent to decentralization will be outliied in-the section on the intra-management of 
HAPP. 
Lessons Learned : 
The quality and timely implementation of a project requires the openness and willingness of 
the management to reflect on the implementation process of the Project and the commitment 
of the HAPP staff and FHI (ARO-HQ) to rectify deficiencies related to the non-technical and 
technical aspects of the Project. 
In light of the demands conferred on the HAP Project from numerous stakeholders and the 
intensity of the Project, a participatory style of management can provide an opportunity to 
see an issue from different points of view and can facilitate creative problem-solving of the 
issue. At the same time, it requires high levels of facilitating skills on the part of 
supewisors/managers to maintain objectivity, to manage expectations, to deal with potential 
conflicts and to ensure effectiveness given the available timeframe. 
Complex issues can be better addressed by carrying out corrective measures on the basis of 
full understanding of the root of the problem. The conduct of the review by those with the 
authority to approve substantial changes is also equally important to implement changes. 
Meeting the expectations, requirements and regulations of all stakeholders 
(GOI/USAID/FHI ARO-HQ), demands a long consultative process and very flexible 
management in the sense of making repeated changes, as well as fluidity in the process of 
accommodating inputs from all stakeholders. Inevitably, this is done sometimes at the 
expense of the credibility of the HAPP Office. Comn~unication of the costs to aU stakeholders 
can be attempted to manage their expectations. 
TeambuiJding and Team work 
. An effective team approach was reflected in the successful completion of complex tasks that 
required contributions from many staff members, such as the development of the SAR, the 
Annual Data Report and the Work-Plans. Office-wide retreats/training/problem-solving 
meetings were praised by staff members as useful in terms of building a good understanding 
of the overall HAPP n~ission, strengthening communication, teambuilding, sharing 
experience among staff, effective planning and advancing personal professional growth. A 
common philosophy was developed in the first reheat, which had implications on the 
development of the second stage of HAPP. 
. The annual retreats were found to be a particularly effective means of strengthening 
teamwork rather than resolving complex management issues, which usually needed time to 
properly examine and find appropriate solutions. 
. The level of teamwork among the HAPP staff varied over time. Critical influencing factors 
were the ability of staff to cooperate with each other, the ability to see each other's role and 
HAPP - Internal Evaluation Report 
contribution, levels of stress, the equal sharing of important information, and transparency 
and uniformity in the application of rules/regulations/privileges to all staff. 
Lessons Learned : 
. Teamwork is most essential in this kind of large, fast-moving, multi-partnered and multi￾layered project. Contributing factors to the effective implementation of a teamwork approach 
are the careful selection of staff in terms of ability to work in a team, a clear sense of mission 
and shared team values and cultural homogeneity (numerous local staff at management 
level). In addition, rules/regulations/privileges should be transparent and applied equally to 
all staff, which leads to all staff respecting the rules/systems. Likewise, sharing key 
information to all staff also prevents a feeling of being alienated among some staff. 
. Teamwork needs to be nurtured through office-wide retreats/training/problem-solving 
meetings. 
. Self, supervisory and peer assessment are essential elements of the staff evaluation system 
and a means of fostering teamwork as they enable assessment in its entirety of the 
professional and personal contribution of each member of the staff to the organization. 
. Monthly staff meetings, which must include all staff members, are strongly believed to lead 
to greater transparency in the system as a whole and thereby to a higher level of efficiency 
and teamwork. 
. To deal with office politicking and staff problems, the creation of a labor union could be 
advocated. This could also deal with other staffing conflicts and legal matters. Electing a labor 
union leader should be based on commitment and willingness to take risks. 
Finance 
. The Finance Unit financially reported regularly to FHI-HQ and to USAID and carried out sets 
of regular activities on a daily basis, namely: payments to suppliers and employers; petty 
cash payments; purchases of equipment; and reimbursements to IAs. The Unit usually met 
ARO deadlines in their reporting procedures. Other good finance-related practices were 
carried out, such as the conduct of an internal audit for all HAPP IAs as part of FHI-HQ's 
routine financial supervision and spending within a planned budget. 
. Prior to the Internal Review in 1998, the transfer of funds from FHI-HQ took up to three 
weeks. This was attributed to a long bureaucratic process involving FHI-ARO and HQ, the 
limited capacity of NGOs to write financial reports, insufficient two-way communication 
between the Finance Unit and IAs and IittIe constructive feedback on the IAs' financial 
reports. Understaffing of the HAPP Jakarta Office partly contributed to the lengthy process. 
The provision of cash advances based on IAs' estimates on previous expenses rather than on 
expenses of upcoming activities, also created gaps in IAs' cash flows. 
. Upon the recommendation of the Internal Review, a comprehensive investigation of the 
bottlenecks in processing financial reports (e.g. reviews of procedures, the filing system, the 
tracking system, etc.) was conducted. Based on the findings of the review, remedial actions 
were carried out in almost every aspect of the procedure of the financial process. As a result, 
the Finance department was able to expedite the transfer of funds to IAs within two weeks 
after the submission of the complete documentation. The provision of cash advances based on 
IAs' estimates of expenses of upcoming activities for a certain period of time smoothed IAs' 
cash flows. 
HAPP- Internal Evaluation Report 
To prevent gaps in funding to the IAs and to maintain their monthly replenishment of funds, 
the HAPP Finance Unit developed the Flow of Disbursement system in February 1999. In July 
1999, a Budget Analysis format and the Recipient Monthly Financial Report (RMFR) Tracking 
System were established to monitor the status of the 1.4s' budgets. Various forms and 
guidelines were developed by the Finance deparlment to simplify processing of financial 
matters, such as: guidelines for expenditures and reimbursements and budget proposal and 
financial report forms for KFAD and RRF proposals to track expenditures versus budget. The 
MTX computer system was established for generating regular financial reporting. The finance 
staff of the HAPP Jakarta Office received training in FHI-ARO on finance matters (e.g. 
regulations). A schedule was also established for different types of payments. These 
streamlining initiatives were also found to be very helpful and useful for amendments and 
Option Period budget purposes. 
The speed of the process was also facilitated by: (a) the delegation of the authorization of 
payments from HQ to FHI-ARO; (b) the delegation of the authorization to process 
disbursements to DCOP in the absence of the COP; and (c) the authorization given to the 
provincial finance and Administration officers to ensure compliance of aU of the RMFR and 
supporting documents submitted by IAs with HAPP regulations. 
Significant progress was made in the finance area, yet the need for further streamlining of the 
Finance Unit was expressed. In the absence of the CDICOP, fewer signatories need to 
approve a voucher for payment than the present practice. Decision-making about the timing 
of certain payments should be delegated to those with the best ability to realiticauy estimate 
the time required to process the payment. At times, disbursement was still delayed because of 
failure to provide the Finance Unit with timely prior notice about upcoming expenditures. An 
amendment for extra payment to an IA and reimbursement to IAs remained a very lengthy 
procedure, as it required FHI-ARO approval and proof of payment. The MTX system was 
complex and not always functional, leading to reliance on time-consuming manual 
calculations because of the complexity of the system. 
The abolition of the pre-award assessment for new IAs, which was used to examine an IA's 
financial management capacity, created concerns about the ability of the Finance Unit to 
anticipate the technical support necessary given that they had no prior knowledge of the IA's 
capacity (e.g. financial monitoring, understanding of FHI financial policy). 
Lessons Learned : 
The timely disbursement of funds or payments needs proficiency in the existing 
regulations/requirements by both the staff of the Finance Unit and the other parties 
concerned (e.g. IAs), the capacity to write financial reports, good cooperation and 
coordination between them (e.g. the on-time submission of prior notice) and a streamlined 
processing procedure. 
. When bottlenecks in the processing of the financial matters occur, a comprehensive review of 
the process, followed by concerted efforts to rectify the deficiencies, are required to achieve 
efficient processing of the report and disbursement of funds to the field. Aspects of the 
Finance Unit to be assessed, include bureaucratic procedures, the delegation of authority, 
administrative systems (eg filing system; forms; tracking system), staff capacity (e.g. 
proficiency in the regulations/requirements), and utilization of time and cost-saving 
measures such as the computer system. 
HAPP - Internal Evaluation Report 
. Streamlining of the finance procedure is an evolutionary process and needs follow-up 
reviews on a regular basis to ensure its effectiveness and to address new issues. 
Project Documentation 
. All project-related documents submitted to FHI and USAID had to be written in English. Staff 
of the HAPP Jakarta Office were responsible for compiling the reports from all partners into 
the HAPP Quarterly Reports for USAID. Both IAs and the staff of HAPP recognized that the 
program reporting process was mutually beneficial for HAPP and IAs, particularly for 
program work. Many IAs, however, possesses limited writing and analytical thinking skills, 
even in Indonesian and an inadequate proficiency in English. 
. Translation of IAs' reports into English by a translator with insufficient knowledge of 
technical matters, led to misunderstanding due to unfamiliarity with concepts and 
terminology. Likewise, reports written in poor English caused confusion. To deal with this 
situation, HAPP accepted IAs' reports written in Indonesian and encouraged the Project 
Manager to write the executive summary in English as part of a capacity building exercise. 
The translation and compilation of the reports into the HAPP Quarterly Reports were 
assigned to a bilingual native English speaker or an Indonesian with knowledge of technical 
terms and concepts. 
Lessons Learned : 
. The translator of IAs' reports needs not only to be bilingual but also to possess knowledge of 
technical matters. 
. Capacity building efforts aiming at improving the report writing skills of staff of IAs, 
analytical thinking and English proficiency remain priorities. 
Facilihes and Infrastructure Development 
. Various time and cost-saving measures were applied to enhance the implementation of the 
HAP Project, such as upgrading the computer software system and recruiting a computer 
technician. The presence of a full-time computer technician in the office meant that most 
computer-related problems could be solved immediately. Training in MS Outlook and MS 
Office for HAPP staff resulted in the efficient production of quality documents and a better 
electronic filing system. Establishment of the email system and the appropriation of cell 
phones in the Jakarta Office and the provincial offices facilitated communications within the 
HAPP Offices and with HAPP partners. The use of email was found to be particularly cost￾effective with the size of the telephone bills remaining the same in spite of the presence of 
additional staff and an increase in the volume of communication. Communication with the 
Regional Office improved through the writing of a brief monthly report with a follow-up 
teleconference between senior staff. FHI's computerized accounting system (MTX) was 
established to improve financial reports to conform to the standards of FHI HQ and USAID. 
. ImpIementation of the technology was not always smooth. As mentioned in the Finance 
Section, the MTX system could sometimes be troublesome. MIS was delayed from December 
1997 to the last quarter of 1998 due to on-going problems with the technical design. Similarly, 
the FIS improved reporting considerably, but it took time to ensure that FHI formats were 
consistent with the formats required by USAID. 
. The need for more office space in Jakarta was reiterated in the Internal Review and led to the 
renovation of the Jakarta Office. The first phase of the renovation of the office at the 
HAPP- Internal Evaluation Report 
beginning of the Project and the second phase of the renovation (the office in its present form) 
took about three months for each. The latter was completed with minimum interruption by 
building a temporary office beforehand. Although the renovated and temporary offices 
provided more condortable space to work in, they still did not meet demands for space from 
time to time when additional short-term consultants were hired to work in the office or when 
several meetings needed to be held at the same time. 
Lessons Learned : 
. The use of appropriate technology to facilitate the completion of timely and quality work, as 
well as communication/coordination such as by computer and ceU phones, is necessary. In 
terms of computer usage, building the capacity of staff to use the sofhvare effectively and 
making a full-time technician available at the office leads to time and cost-saving measures. 
. The utilization of technology can be beneficial as weU as detrimental. The latter especially 
applies to a combination of complex technology and the unavailability of an expert to 
immediately troubleshoot problems. In relation to this, the availability of a back-up technician 
to solve daily problems in the absence of the permanent staff is necessary. 
. Attention should be given to the provision of office space that enables staff to work 
effectively, prior to the implementation of the Project. As the number of staff grows in 
response to the amount of the workload, expansion of the space may need to be reconsidered. 
Limited office space also means the need for better planning of meetings. 
C. Administrative Management 
Histoni-a1 Overview 
Setting up the HIV/AIDS Prevention Project Office in 1996 was the first experience of such a nature 
for the administrative staff. This included not just arranging permits through the Ministry of Health 
(MOH), Government of Indonesia (GOI), but physically renovating the office space provided by the 
GO1 and developing and formulating a personnel policy in keeping with FHI's regulations as weU as 
the Indonesian labor laws. The first phase of the renovation of the office took about three months. The 
second phase of the renovation (the office in its present form) took another three months. The 
formulation of the draft personnel policy took between three and five months. The first draft was 
approved by the FHI Area Regional Office (ARO) Director in June 1996. After the Chief of Party 
(COP) joined the Project, a revision was made and the final personnel policy in its present form, was 
approved in January 1997. 
The project started with six local staff in May 1996. Before the present Country Director began her 
assignment with the Project in July 1996, a number of people from ARO (Bangkok) and from FHI 
Arlington came for short periods as acting Chiefs of Party. 
Recruitment was first initiated for the Jakarta sites and was then done for Surabaya and Manado. 
However, during the initial period between 1997 and 1999, there were a number of changes in the 
staff during the process of selecting ther'righf' person for a given job. 
A probation period of three months, was part of the recruitment process. This provided an 
opportunity for both the relevant senior staff at HAPP and for the person to assess whether the fit was 
matching, so to speak, before the candidate was taken on as a member of the HAPP staff. Over time, 
project staff improved their skills in selecting appropriate staff to fit with the existing team. 
HAPP- Internal Evaluation Report 
An annual staff assessment for the entire HAPP staff was done. This consists of a self-assessment 
followed by a supervisory evaluation. However, there was no formal process of peer evaluation built 
into the system, although some senior staff members did consider peer assessment of staff members 
during the annual staff evaluation on an informal basis. It was strongly felt, however, that self, 
supervisory and peer assessment must be a formal part of the whole staff evaluation system, to be 
able to assess in its entirety the professional and personal contribution of each member of staff to the 
organization. 
CoIIaboration with NGOs 
Because of the fact that the entire planning and implementation of this Project was a learning 
experience for everyone, initial collaboration with parher NGOs was difficult because of differences 
in the culture and the way of functioning between the NGOs and HAPP. Problems were encountered 
and overcome in the areas of: 
. Equipment procurement processes 
. Travel 
. Finance 
It was an evolutionary process where, initially, obstacles were encountered and, later, solutions were 
developed for each particular challenge. But this resulted in HAPP staff gaining a large knowledge 
and information base about NGOs and their functional processes. 
Aufhorify and Defegatjon 
The senior HAPP management staff perceived that they were given sufficient authority and 
delegation for independent decision-making. This was, by and large also true of the administrative 
management aspect at the field level. However, the extent to which it was actually utilized at the field 
level was a function of the different personalities and the cultural historical background of the area. 
Manado, for example, historically has a high degree of European influence. 
Management Systems 
It was felt that the management systems could be further improved with a greater level of 
decentralization and a regular procedure of monthly staff meetings (which must necessarily include 
all staff members). These processes, it was strongly felt, would lead to greater transparency in the 
system as a whole and thereby to a higher level of efficiency. 
Communication 
Another aspect which was somewhat of a concern and which, it was felt, required more attention, 
was that of communication between the different units and departments within the HAPP Jakarta 
Office. Proper communication was, at times, felt to be lacking, resulting in miscommunication and 
sometimes in misunderstanding. 
The Institutional Development Framework (IDF) was perceived to be an extremely useful tool in this 
regard, particularly with respect to developing and enhancing leadership qualities and better 
interpersonal communication skills. A strong suggestion from within the Administrative 
Management Department was that an IDF refresher workshop be held for all staff at least once a year. 
Another success in this context was the yearly retreat organized for and by the staff. This was 
considered to be a successful procedure that resulted in intensive and extensive evaluation, 
WP - Internal Evaluation Report 
assessment and introspection by all the participants with regard to the activities, successes and 
challenges they faced over the previous year. The retreat was valuable also as a means of dispersing 
misunderstandings which may have arisen over time and as a means of enhancing communication. 
Other Points of Concern 
. Inter-office communication between the GO1 staff within the HAPP Jakarta Office and the 
other staff members. This concern stemmed from a feeling. that perhaps there was some sense 
of isolation between the Government officers and the others and there was a strong felt need 
to address this issue. 
. Dealing with staff (and equipment) humanely and in a mutually agreeable way before the 
completion of the Project in September 2000. 
Conclusion 
It is notable that the Administrative Management Unit of HAPP was the result of an evolutionary 
process which "started from scratch" and over a period of time - which was a learning process for all 
involved - and that it evolved and developed into a full fledged department which finally dealt 
successfully with all aspects of management, including interaction with the GOI, international 
agencies, the private sector and non-government organizations. A very good management system 
evolved which included a working personnel policy, a procurement process, a travel policy and 
successful interlinks with other departments. 
D. Financial Management 
Background 
As was the case with the Administrative Management Unit, the Finance Unit also started a number of 
its systems from "scratch" when HAPP began in 1996. There were, however, some vital programs that 
were provided or stipulated by Headquarters. These included the MTX computer system for 
generating regular financial reports, which was tailored specifically to HAPP and which reported on 
the seven Contract Performance Objectives (00s) by converting them into CLIN (Contract Line Item 
Numbers) with a specific weightage attributed to each. 
There was a system of regular n~onthly financial reporting to Headquarters the quarterly reporting of 
cash flow per activity planned and, for this fiscal year, a report to USAID. The activities of the Finance 
Unit also included the preparation of budgets for all IAs after the IAs prepared the budgets required. 
The prepared budgets were sent to the Asia Regional Office (ARO), in Bangkok, which reviewed 
them and sent them to Headquarters for approval. At times, changes occurred during the 
inlplementation of activities, which required amendments to be made to the budget and supporting 
funds to be provided. 
Apart from the different types of regular financial reporting, the Finance Unit carried out sets of 
regular activities on a daily basis. These included payments to suppliers, employers (salaries, medical 
allowances, etc.), petty cash payments (including to Provincial Offices and the KPADs), the purchase 
of equipment and reintbursements to IAs. 
Strengths 
A number of strengths supporting the Finance Unit led to it functioning in a relatively easy and 
efficient manner and, thereby, to related aspects of the Project. 
HAPP - Internal Evaluation Report 
The Unit expressed its pride in the fact that they always met ARO deadlines in their reporting 
procedures, in spite of the difficulties they sometimes faced because of the MTX system not 
working at times. 
Good coordination and cooperation existed between the Finance Unit and the IAs, which was 
extremely important as it led to IAs' financial reports being sent on time, which resulted in 
payments to IAs also being made on time. 
The Finance Unit also took pride in having developed a system of making daily payments to 
suppliers, IAs and employees, in such a way as to be, most of the time, efficient as well as 
satisfactory for all the parties. 
Another point of pride for the unit was their having developed and implemented an insurance 
policy for hospitalization and accidents for HAPP staff. Given the high costs of medicine, 
hospitalization and treatment in Indonesia, this development was greatly appreciated by and 
added to the overall efficiency and satisfaction of the staff. 
Weaknesses 
Some of the challenges faced by the Finance Unit during the life of the Project included the following: 
- The MTX system was found to be somewhat complex and at times did not work and often 
required manual calculation, which, because of the complexity of the system, was very time￾consuming. - Reimbursement to IAs was also found to be a very lengthy procedure, as it required ARO 
approval and proof of payment, which often took time. - If an amendment was to be made for extra payment to an IA, it too was found to be time￾consuming, taking away from the optimum level of efficiency of the given activity. 
- It was felt that the staff of the Finance Unit should have been encouraged to attend finance and 
related topic seminars etc. organized either in-house or by other organizations on a regular basis 
as this would lead to greater motivation, learning and exchange of ideas. - There was a felt need by the staff of the Finance Unit for provision of English Language courses 
for them. - It was also felt that the staff of the Finance Unit should be given the opportunity to escort study 
tours, as it would be a learning opportunity for the staff and be of great value to the participants 
of the study tour in better understanding the financial procedures involved during the tour. 
- A need was also expressed for a greater level of authority and delegation in carrying out certain 
routine activities which would, it was felt, lead to improved efficiency in the system and a saving 
of time. Examples of this included: 
. Decisions about when certain payments should be made, being with the Finance Unit and not 
with others, as the Finance Unit felt they were the best judges of reaIistically promising how 
long a particular payment would take; 
If the Country Director was not available, it was felt that a voucher for payment need only be 
approved by the Head of the Finance Unit along with the second signatory, rather than by a 
relatively large number of people who had to approve it under the circumstances. 
- There were times when either the WAD or some RRF NGOs did not give the Finance Unit prior 
notice about upcoming expenditures and then suddenly required the payments to be made. 
- It was also felt and expressed that communication and teamwork between the different units of 
the Project needed further improvement as it sometimes led to miscommunication and 
misunderstanding. 
Ffnance Management Assessment 
Strengths 
1. In July 1999, the HAPP Finance Unit developed a Budget Analysis format and the MFR Tracking 
System for each individual implementing agency. This was intended to make it easier for both 
HAPP - Internal Evaluation Report 
HAPP and the 1.45 to control and monitor each budget item category of the IAs' budget as well as 
their expenditures. It would also detect the status of the IAs' remaining budget quickly. These 
tools made it easier to work more closely with partner organizatio~. In addition, they 
strengthened the relationships, communication and coordination between HAPP and paher 
organizations. They were very helpful and useful especially for amendments and Option Period 
budget purposes. The documents are available in hardware and hardcopy. 
2. The HAPP Finance Unit also developed a standard format of budget for both the KPADs and the 
RRF NGOs, divided into the budget proposals, financial rePo& and tracking systems of financial 
reports. This format was very helpful for reviewing expenditures. This tool made it easier to work 
more closely with both the KPADs and the RRF NGO staff. 
3. In responding to-and fulfilling the USAID/external review recommendations, the Fiance Unit 
carried out the following activities: 
To prevent gaps in funding and to replenish funds for IAs, since February 1999, the HAPP 
Finance Unit developed the Flow of Disbursement system. This system intended to monitor 
and detect any gap in-funding from the IAs submitting their RMFR (Recipient Monthly 
Financial Report) until the time HAPP reimbursed their expenditures/requests and to 
maintain monthly replenishment of funds. This also ensured that they had sufficient funds to 
keep activities ongoing. In case of any delay in the MFRs submitted by IAs, HAPP 
automatically sent a reminder letter to them. 
In order to delegate authority, responsibility and accountability to the HAPP Provincial 
Offices, the checking and reviewing of RMFRs submitted by IAs, including the supporting 
documents and making sure that their reports complied with HAPP regulations, was done by 
them. 
4. Staff Benefits: Payments of medical as part of employee benefits were consistently implemented. 
All staff were also entitled to medical and accident insurance. 
Payment System: Payment service were set 3 times a week, that is each Monday, Wednesday 
and Friday. The supplier and employee may get the reimbursement on the arrangement date 
above. Medical allowance will be reimbursed every hvo weeks at end of the week 2 and 4. 
Pay Day of Salary: Cash or transfer could pay Salary directly to the employee accounts. Every 
en~ployee has salary history form and signs afterward. 
IA Reports: Finance has issued tracking sheet in relation to financialstatus. This is also 
including for KPAD and RRF subcontract. The other tracking sheet also is providing 
information in lateness or regularly on time of the monthly MFR reimbursement 
Cash Flow: is prepared quarterly and send to HQ C.C. to ARO. The cash flow is pre-requisite 
to get reimbursement upon expenditure submitted on the MTX every month. The HQ will 
never arrange wire transfer for such amount and kept in HAPP accounts. The quarterly cash 
flow is crucial and must be ready before the end of the quarter. 
E. The HAPP Staff Develovment Strategy 
High priority was given by FHI to the provision of a quality staff development program. The 
program is motivated by recognition of the contribution of the HAPP staff and encourages staff to 
improve professional skills relevant to their current position in the organization and in support of 
HAPP - Internal EvaIuation Report 
career advancement. The program has been possible because of strong FHI /ARO support of HAPPs 
allocation of budget for local staff development. 
The HAPP staff selected English language training as a priority focus for personal and professional 
development. A variety of strategies for providing learning opportunities were evaluated. As a result, 
design of the staff development program has been based on an adult learning model that takes into 
consideration matching learning to current capacity, actual needs and situations. 
Points to be considered included : 
. HAPP staff are fully committed and they frequently work well over 40 hours per week. 
Work-related travel time is already considerable for most staff members; ease of access was 
an important.issue. 
Each staff person has specific learning needs in order to increase skills related to their 
position responsibilities. 
. Staff have varying levels of English language competency in speaking, listening 
comprehension and writing. 
. Staff requested assistance with developing presentation skills. 
In Jakarta, an initial 20-hour program was piloted by the British Council. Based on feedback and on 
staff needs assessment, a 12-month follow-on program was designed. 
Adjustments included providing the instruction in the HAPP office to ensure that staff were able to 
take advantage of the tutorial time. Office time was allocated in recognition of the potential benefit to 
the Project itself and in appreciation of the'beyond the job description' contribution of all HAPP staff. 
Highlights of the program include: 
I. Spaced learning - that is, group course work is interspersed with time spent on individual 
programs and tasks. 
2. Staff are encouraged to monitor their progress through weekly learner diaries maintained in 
English 
3. Additional monitoring is provided through : 
. Samples of written English at beginning and end of the course 
Maintenance of learner portfolios containing work completed over the course. This will 
provide an overall record of achievement. 
. The one-to-one tutorials provide optimal monitoring and feedback. 
4. Each group (by level) has selected a representative whose responsibility it is to monitor logistics 
issues and to communicate any problems to the contact at British Council. 
Special events provide another motivation and opportunity for relevant learning. For example, 
presentation skills were sharpened through an in-house event. Each unit was responsible for 
preparing and delivering a group presentation on a topic of their choice. Every person in the unit had 
an opportunity to deliver a part and their 'successes' were celebrated after with a staff party. This was 
a challenging new experience for most of the staff. A similar program is being implemented in both 
the other HAPP sites. 
Staff were invited to identify their priorities and to identify local resources to provide learning 
support. Objectives identified in the sites included: 
. To develop capacity in English conversation to handle daily duties related to position 
responsibility as a HAPP staff; 
. To prepare reports; 
HAPP- Internal Evaluation Report 
. To facilitate HAPP visitors and to provide them with information; . To improve listening comprehension and the ability to express ideas in order to participate 
more actively in meetings in which English was the main language. 
Each staff member set his or her own personal learning agenda and they designed their learning 
around activities such as formal language study, bringing native English speakers to office for 
practice sessions, report and document writing practice with feedback, business letter writing. 
While each of the learning programs has a formal evaluation component, the impact of these activities 
can also readily be observed through improved report and document preparation, the ability to host 
and explain the HAPP program to foreign visitors and the increased ability to express themselves in 
English during meetings. 
F. A Full-Time Computer Specialist 
"Ensuring A Well-Oiled Engine" 
The importance of retaining a full-time computer specialist within the organization began to be 
recognized and greatly appreciated by the entire staff after the specialist joined in October 1996 first 
as a computer consultant and then as a member of the HAPP staff in June 1999. 
Among his many other tasks, the specialist was responsible for designing and standardizing the office 
software and ensuring that it ran efficiently and for connecting the MTX system in the Finance Unit to 
the main network. At the three HAPP sites, he standardized the software, developed an inventory for 
the hardware and solved any problems as they arose. This meant that he was always on call, both at 
the Counhy Office where he is based and at the sites. This was a fact that was well appreciated by 
everyone, particularly as everyone also realized the fact that the specialist was often under pressure to 
conlplete jobs and to houble-shoot, but was, nevertheless, always ready to help when required. 
His contributions to the organization were many and varied within the field of computers. However, 
the specialist felt that he too, had learned a great deal from HAPP. Apart from leaming on-the-job 
with respect to his field - he often came across complex problenls that required time-consuming and 
labor-intensive solutions - he learned: how to manage his time better and more effectively; how to 
deal with different sorts of people, in particular, those who did not have a background in computers; 
and how to work with government officials, since part of his job involved dealing with KPAD 
members. 
The computer specialist is still in the process of learning and studying, which he does in the form of 
both informal reading on his subject and formally studying towards qualifying as a Nehvork 
Engineer. But he has a dream - that "someday all employees (of organizations like HAPP) will have 
access to and know at least the basics of computers", as he rightly feels that that is the world of the 
future. 
G. Fuel for the En~ne 
In the many very important project tasks which, of course, have to be accomplished, it is that much 
more important to remember that, in order to achieve them, there are many, many smaller tasks 
which must be completed and which require certain types of people to do them. Often in the 
satisfaction and pride of having successfully achieved the greater goals, these 'smaller' tasks and 
people are forgotten. With a view to acknowledging their invaluable conhibution, it is warranted to 
mention that the support staff of the HAPP Jakarta Office consisted of hvo drivers and hvo office 
assistants. 
HAPP - Internal Evaluation Report 
Both the drivers were always on call and often worked long hours helping the professional and 
administrative staff to meet deadlines and to achieve objectives. 
The office assistants, both of whom were senior high school educated and had received training and 
certificates in the basics of computer functioning, were intelligent, honest and willing workers, with 
scope for further development. They too, were always there, working alongside the rest of the HAPP 
staff, whether under pressure of deadlines or just giving the routine support of photocopying or 
providing food and drink when required, as the daily work went on. 
The support staff participated willingly and wholeheartedly, joining in the staff retreats and other 
non-work activities as well. 
Their importance cannot be exaggerated for its constant, consistent, if silent, presence and for the fact 
that none of the bigger goals and objectives of the Project could possibly have been achieved were it 
not for their ever-ready contribution and support. 
HAPP- Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
Ill NAPPIntra-Office 
Management 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
vIn. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
N. HAPP Inba-Office Management 
A. Communication within HAPP 
Initially, the many complex layers and cultures which characterized 
HAPP caused difficulties in keeping open limes of communication 
and preventing miscommunication. This sometimes created delays 
and misunderstandings. Pattern of communication within the HAPP 
office also showed a lack of sharing information and insufficient 
understanding of lines of reporting. Examples were delays or failure 
of the HAPP Jakarta Office to respond to the provincial offices' 
requests/needs, 1.4s receiving different advice from the HAPP 
Jakarta Office and provinces and the Jakarta office directly contacting 
IAs without informing the provincial office. The latter led to 
confusion for both IAs and HAPP in the field. 
A retreat was organized for HAPP, USAID, MOH, KPA and KPADs 
to overcome the above challenges. The retreat helped to build 
relationships and establish lines of communication within the HAPP 
office and behveen each of these organizations. A common 
philosophy was developed which had implications on the 
development of the second stage of HAPP. In addition, there was 
greater clarity in understanding individual job descriptions and their 
interrelationship between the HAPP Jakarta Office and the Provincial 
HAPP offices. However, noncompliance within HAPP with agreed 
lines of communication and office procedures by FOs, IAs and new 
staff was found during the Internal Review. Informal discussion 
between new and senior staff led to creative problem-solving and 
more shared responsibilities. 
B. HAPP Provincial Offices 
Staff of HAPP provincial offices were GO1 nominees or selected from a pool of candidates of those 
qualified with experience in HIV/AIDS prevention in the provinces. They were recruited with 
recognition of their needs for ongoing training to reorient their skills and to understand the HAP 
Project. The role of HAPP provincial managers was centralized on their being gatekeepers with Local 
Government. They also had great scope in their work with coordination of HAPP IAs and the 
provincial AIDS Commission and overall monitoring. Decentralization of the authorization of HAPP 
proposals and financial management to provincial program managers was not feasible in view of 
FHI's rigorous proposal development process, the complexity of FHI/USAID's financial regulations 
and the presence of many levels of authorization for HAPP proposals. 
In the earlier phase of the Project, a number of consistent observations were made by various parties, 
including USAID, FHI, external evaluators and NGOs. They included: (a) the weak roles of the 
manager and the deputy manager; (b) the limited delegation of authority, responsibility and 
accountability to provincial level HAPP offices; (c) insufficient provincial resources to support the 
implementation of IAs; (d) the inadequate management of provincial offices, including a lack of 
monitoring; and (e) insufficient technical capacity. A lack of delegation from FHI HQ to ARO, from 
ARO to the COP, and from the COPto staff in Jakarta, Surabaya and Manado led to a bottleneck in 
implen~enting activities and distributing tasks, such as in the disbursement of funds to IAs. 
HAPP - Internal Evaluahon Report 
Remedial actions were subsequently undertaken. Examples were: the provision of training for the two 
Deputy Program Managers at the ARO in program management; participation in various technical 
training; access to e-mail for provincial program managers; and the delegation of many tasks to the 
Administration and Finance Assistants at the provincial level. The supervision arrangement was also 
adapted. Some management-related initiatives were carried out by provincial staff through holding 
monthly monitoring meetings and assisting NGOs and KPADs in preparing technical proposals and 
financial reports to ensure compliance with the administrative requirements of FHI/USAID. 
In spite of the many opportunities for training, support and regular supervision, the management of 
the Manado and Surabaya sites remained short of the operating level required. Moreover, HAPP￾funded NGOs repeatedly reported the lack of field monitoring conducted by HAPP provincial staff 
over the course of the Project. Other issues also emerged, including the persistent self-identification of 
PPMs as being staff.of GO1 rather than HAPP, a lack of leadership and over-reliance on DPPM. 
Throughout the life of the Project, IAs continued to see the role of HAPP provincial office largely as a 
post office for the HAPP Jakarta Office and IAs. 
Lessons Learned : 
. A provincial office with limited authority and capacity reflects a negative image among 
partners. 
In view of the complexity of the Project, the number of IAs involved and the amount of the 
workload, the delegation of responsibilities and authority is necessary to ensure the quality 
and on-time implementation of the Project. 
. Capacity building to improve the technical and managerial skills of staff may not be 
successful because of staff's entrenched attitudes, values and practices and a lack of learning 
capacity to orient oneself with new tasks and roles. 
. A successful delegation of authority is influenced by the technical and managerial capacity of 
staff, the level of complexity of responsibilities or bureaucracy system and HAPF's ability to 
develop capacity among staff. 
Recommendations : 
. Prior to the establishment of a provincial office, a clear objective of the establishment of the 
office, including its roles and responsibilities, must be in place. This should guide a careful 
selection of suitable staff with the required capacity to discharge the responsibilities and to 
assume the roles. 
. Decentralization inherently entails strong coordination to attain a common goal. 
Consequently, any forms of decentralization must be accompanied by well-planned 
coordination linkages, compliance with the agreed coordination scheme and a regular review 
of coordination. 
C. Subproiect Management Issues 
Project Start-up 
Major constraints to a seamless transition between Phase I and I1 of HAPP were the time needed to 
resolve HAPP technical strategy/directions with USAID and to amend and modify the FHI technical 
and financial proposals as requested by USAID. In addition, despite support from COTR and the 
Contracting Officer, considerable time was taken by HAPP staff to reorient to the different contractual 
formats and processes during the transition between the first phase of HXPP and the commencement 
of the Performance-Based Contract. 
mPP- Internal Evaluation Report 
Process of ProposaI Development and Approval 
A 'Jumpstart' Proposal Development Workshop was held in August 1996 for 26 NGOs selected to 
work with HAPP. Participants of the workshop received conflicting information, which caused 
confusion owing to inadequate preparation time for the event and a lack of orientation for facilitators 
about the task and guidance from FHI-HQ and the Asia Regional Office regarding procedures. 
For Phase Il of HAPP, NGOs interested in working with HAPP in the three sites were required to 
attend a one-day orientation to HAPP. It covered HAPP's priorities and requirements, the USAID 
Strategic Objectives, writing and submitting a concept paper and the requirement of the submission 
of all subagreements in English. HAPP staff reviewed the concept papers and determined the 
disposition for each based on these categories: (a) subagreements; @) capacity building and 
networking; (c) Rapid Response Funding; or (d) as not fitting within HAPP. After selection, 
comments were made on the concept paper and returned to the submitting agencies. A three-day 
subagreement development workshop was then held in each site and facilitated by ARO and HAPP 
staff. The workshops focused on development of a logframe, a workplan and a budget. 
In light of the number of parties involved, the vaiying skill levels and the high standard required in 
the proposal development, the HAPP staff finally resorted to jointly designing the subagreement with 
the IAs but wrote the documents directly. An English writer was also employed to ensure a high 
standard of language. 
HAPP provincial staff worked with the KPADs on the budget and program components of their 
proposals, which was beneficial to both parties. This was evident in the improved quality of the final 
proposals and a better understanding of the FHI requirements and the KPAD approach as well as 
areas where support was most relevant. 
The whole process of proposal development continued to be time-consuming, including the process 
of subproject approval. Reaching a consensus on the Project's conceptual design, the technical 
strategies to be used with each IA and a clarification of the roles of key officers took a lot of time. This 
included delays in getting concurrence from USAID. Moreover, the proposals needed improvements 
both in the content of the intervention activities and the format. The HAPP team in Manado found it 
more difficult to work on the proposals than did those in Surabaya because of inadequate explanation 
of the content of the IAs' proposals and a failure to integrate lessons learned. They did not mention 
their previous institutional activities, particularly during the last ten months of the HAPP 
implementation project and the implication of these activities for the current proposal. Political 
instability also delayed the whole process (e.g. the evacuation of the senior foreign staff of HAPP and 
US-based PVOs). Similarly, considerable time was spent in the preparation of budgets and narratives 
for the Option Period. The approval process was also lengthy due to the multilevel management 
Delays in processing the approval of subawards could have a negative impact on some activities. One 
example was the delay in finalizing the YBHK subagreement, which hindered referral activities by 
other NGOs. 
To shorten the whole process, concerted efforts were carried out to streamline the processing for 
reviewing subagreements, letters of agreement and RRF grants. They involved changing the 
organogram, recruiting additional staff and developing checklists for the review of proposals 
(financial, technical, management and administrative reviews). 
Lessons Learned : 
. A serious gap occurred between the planning/contracting phase and the start-up of field￾based activities due to the complexity of the technical design, the multiple agency structure 
NAPP- Internal Evaluation Report 
and the time taken to respond to USAID/Indonesia's comments and to reach consensus 
about the annual workplan. 
. USAID requirements, the new type of contract, the complexity of HAPP and the volatile 
political climate made this more difficult than usual. 
Rapid Response Fund 
From the beginning, it was recognized that the role of RRF was to enable wider participation by and 
the trials of small and innovative activities by new IAs or more experienced IAs turning their 
attention to HIV/AIDS issues. At the same time, it was also realized that it would be time-consuming 
to plan and manage it. As mentioned earlier, procedures for the improved and speedier processing of 
RRF proposals were finalized and implemented. 
Supervision and Technical Support 
Both IAs and the staff of HAPP recognized that program reporting processes were mutually 
beneficial for HAPP and IAs, particularly for program work. At the beginning of the Project, 
insufficient feedback on progress reports from the staff of HAPP was expressed by IAs. IAs and 
provincial staff of HAPP valued face to face interaction in the context of program supervision and 
technical support. Inadequate program supervision and technical support took place in 3 sites, 
particularly North Jakarta. The major reason was the limited skill and understanding about 
supportive supervision among the HAPP staff in the Jakarta offices. In response to this, the 
Evaluation Officer was recruited to coordinate the quarterly reports, while the Program Officer 
focused on contract and grant management. 
Cooperative Work Environment with Counterparts 
Good collaboration with KFADs, CDC and MOH was maintained by HAPP from the start of the 
Project. The working relationship with the USA-based PVOs (PACT and PCI) was problematic. 
During the negotiation of the second phase of the HAP Project, the partnership between HAPP and 
PVO was discussed and a letter of commitment between FHI and each PVO was signed. It was agreed 
that the role of the PVOs was to provide technical assistance to the NGO projects and to provide input 
to their design through collaborative planning and strategizing meetings between HAPP, NGOs and 
PVOs. In reality, PVOs experienced a lengthy negotiation (repeated changes of the content/focus) or 
no initiation of negotiation. They were also not involved with NGO subproject development until it 
was too late and with very few follow-up meetings of PVOs and HAPP staff. 
Overall communication with HAPP staff was also problematic, including: (a) a lack of information 
about HAPP activities; (b) slow or no response to requests for meetings and feedback; (c) HAPP staff 
gave different responses to the same questions; (d) the repeated submission of reports and other 
deliverables as the HAPP office lost copies of them; and (e) a difficult communication process, which 
was characterized by a top-down approach, unclear directives and requests and a lack of partnership 
in making technical and managerial decisions. The delay in realizing the contracts created a financial 
burden on PVOs who had maintained staff in anticipation of signing the contract. 
Since then, the relationship between HAPP and PVOs has been strengthened. A series of Technical 
Advisory Group (TAG) meetings was conducted to keep all PVO partners well informed on the 
progress of HAPP, as well as to update PVOs on technical issues that are relevant to STD/HIV/AIDS 
programs. One of the TAG results was a consensus on conducting organizational capacity 
assessments for all partners. This was useful to assess the coordination of the provision of technical 
assistance. 
FHI/HAPP satisfactorily communicated with S05-R03 during regular meetings between FHI/Chief 
of Party, MOH HAP Project Manager and S05-RP 2 staff. These meetings are held to discuss the 
HAPP - Internal Evaluation Report 
progress of implementation activities, problems encountered and to seek USAID's input on problem￾solving. FHI has been cooperative and responsive to USAID's additional urgent requests, such as 
preparing documents, participating in the Mission HIV/AIDS External Assessment in February 1999 
and making a presentation to the HIV/AIDS Design Team in November 1999. 
Working relationships with the three Provincial AIDS Commission (KPADs), USbased PVO partners 
and local NGOs have improved through the capacity building effort across many audiences, technical 
areas and different types of organizations. 
Capacity Building of WP-Funded NGOs 
The majority of NGOs involved in the first and second phase of HAPP were newly-established and 
had little experience in working with an international donor agency. Their limited program and 
financial management skills evident in the first phase of HAPP substantiauy improved in the second 
phase of the Project, particularly in the areas of planning and financial management, according to the 
IDF. IAs also demonstrated a much higher level of capacity and understanding of the FHI regulations. 
This has been achieved through supervisory and technical visits and training. New RRF NGOs also 
required technical assistance on monitoring and the regulations of FHI. 
In terms of monitoring and evaluation, diversity was observed among the IAs in skiU and 
understanding of them. IAs possess considerable skills in data collection, reporting and narrative 
descriptions of activities. Their weaknesses were in data analysis, the identification of relevant data, 
the under-utilization of target groups in monitoring and the evaluation and use of data to foUow up 
and improve activities due to a lack of sMls and appropriate human resources. The provision of 
technical assistance was hindered by NGOs' over-commitments, NGOs' loss of interest in the Project 
due to failure to get the extension, the low capacity of NGOs and high staff turnover among a few 
NGOs. One of the consequences was a constant change of participants at workshops during each 
phase of the Project, which made it difficult to maintain continuity. 
IAs'Staff Turnover 
Some IAs experienced significant levels of staff turnover since the first phase of HAPP. Thii was a 
persistent issue for a few NGOs despite technical support provided by the staff of HAPP. Staff 
turnover resulted in the delay and postponement of IAs' activities. The high turnover of counselor 
volunteers required a mass of trained volunteers, ongoing supervision and refresher courses. 
Other Const-aints 
Public and religious holidays and religious observances (e.g. the fasting month) delayed 
subagreement finalization and the implementation of the Project. Likewise, the economic and political 
HAPP - Internal Evaluation Report 
climate affected the timing and scheduling of activities and workplan. Consequently, HAPP had to 
accommodate changes in deadlines and to reorient the workplan. 
D. HAPP Intra-Office Management Bullet Points 
. The FHI-HQ and ARO have been effective in backstopping the FHI Country Office with 
respect to both technical and managerial support. Yet the need for decentralization of 
authority to the FHI Country Office was an overarchi& issue with the most substantial 
impact on the development and approval of proposal. This will be addressed in the section on 
Funding Mechanisms. 
. The provincial offices were designed with Limited inputs of Lessons Learned from similar 
initiatives, as such initiatives were very few. PPMs were selected based on GO1 nominees and 
DPPMs were recruited from a limited pool of candidates with professional background of 
working for NGOs, due to the few existing qualified people with experience in HIV/AIDS 
prevention in provinces. They were recruited in recognition of their need for ongoing training 
to reorient their skills and to understand the HAP Project. The role of the HAPP provincial 
managers was centralized on their being gatekeepers with Local Government. They also had 
great scope in their work with coordination of HAPP IAs and the Provincial AIDS 
Commissions (KPAD) and overall monitoring. Provincial offices were envisaged to 
coordinate and support implementation of the HAPP activities in their respective sites 
through provision of managerial/administrative and technical assistance to NGOs and 
KPADs. 
In the earlier phase of the Project, a number of consistent observations were made by various 
parties. They included: (a) the weak roles of the manager and the deputy manager; @) the 
limited delegation of authority, responsibility and accountability to provincial level HAPP 
offices; (c) insufficient provincial resources to support implementation of IAs; (d) the 
inadequate management of provincial offices, including the lack of monitoring, although 
monthly meetings were held; and (e) insufficient technical capacity. A lack of delegation from 
the COP to staff in Jakarta and provincial offices led to a bottleneck in implementing activities 
and distributing tasks. Staff of provincial offices found that supportive supervision and 
monitoring were inadequate. 
. Actions were undertaken to rectify the deficiencies. Examples were the provision of training 
for the two Deputy Program Managers at the ARO in program management, participation in 
various technical training, access to e-mail for provincial program managers and the 
delegation of many tasks to the Administration and Finance Assistants at the provincial level. 
Supervision arrangements for senior provincial staff was also adapted. 
. At the beginning, substantial decentralization of the authorization of HAPP proposals and 
financial management to provincial program managers was seen as not viable in view of 
FHSs rigorous proposal development process, the complexity of FHI/USAID's financial 
regulations and the presence of many levels of authorization of HAPP proposals. Later on, 
the staff of the provincial offices managed to assist IAs and WADS in preparing technical 
proposals and financial reports to ensure compliance with the administrative requirements of 
FHI/USAID. 
. Throughout the life of the Project, IAs continued to see the role of HAPP provincial offices 
largely as a 'post office' for the HAPP Jakarta Office and for the IAs. Moreover, IAs reported a 
lack of field monitoring by HAPP provincial staff over the course of the Project. Other 
recurrent issues also emerged, including failure to establish self-identification as staff of 
HAP! - Internal Evaluation Report 
HAPP, the lack of leadership and an imbalanced division of labor between staff. The 
provincial staff ended up spending more time in administration and management than on 
technical issues. In lie with the staff's professional background, the Manado Office was more 
BCC-oriented and the Surabaya Office more STD-oriented. A combination of a lack of 
clarification of their job descriptions, inadequate initiative of the staff, little supportive 
feedback or supervision and lack of authority, constrained the ability of the provincial staff to 
effectively screen technical and management-related material by ensuring their quality and 
adherence to the existing regulations, before being submitted to the HAPP Jakarta Office. 
. Communication within the HAPP provincial offices was problematic, partly as a result of the 
different sectoral backgrounds of the staff, which prevented reaching consensus on critical 
issues and the necessary follow-up actions stemming from different problem-solving 
approaches. This was also compounded by inadequate and unequal shares of information 
between the HAPP Jakarta Office and provincial offices as well within provincial offices. This 
hindered effective decision-making and caused friction. Similarly, communication behveen 
the HAPP Jakarta Office and the provincial offices was adversely affected by noncompliance 
with the agreed lines of communication by all parties involved. One example was a direct 
communication between the Jakarta Office and the IAs, by-passing the provincial office. This 
resulted in difficulties faced by the staff of the provincial office when they were requested to 
follow up without adequate background information about what had taken place. It also 
reflected an unfavorable image of the HAPP office to its partners. 
. Overall, the roles, responsibilities and authority of HQ, the ARO, the local Jakarta Office and 
the provincial offices needed to be clearly defined and appropriate authority given to each to 
ensure both timely responses and high quality work. 
Lessons Learned : 
A successful delegation of authority is influenced by the technical and managerial capacity of 
the staff of sub-offices, the level of complexity of regulations, the responsibilities and 
bureaucratic systems of the parties involved and the Country Office's ability to develop the 
capacity of the sub-office staff. 
A provincial office with limited authority and capacity reflects a negative image among 
partners and can be harnukl to the progress of the Project. 
In view of the complexity of the Project (multiple stakeholders), the number of IAs involved 
and the geographic coverage, delegation of responsibilities and authority to subordinates and 
sub-offices is necessary to deal effectively with the amount of the workload. Nonetheless, 
striking a balance between consistent quality of control and the delegation of authority 
remains a challenging issue. 
Capacity building to improve the technical and managerial skills of staff is a function of 
staff's entrenched attitudes, values and practices and their motivation and learning capacity 
to orient themselves with new tasks and roles. 
Decentralization inherently entails strong coordination and good communication to attain a 
common goal. Consequently, any form of decentralization must be accompanied by well￾planned coordination linkages (including lines of communication), compliance with agreed 
coordiiation/communication arrangements and a regular review of the coordination 
/communication including good dissemination of Jakarta staff's field trip reports and 
recommendations to the provincial offices. 
NAPP - Internal Evaluation Report 
Rec ommendations : 
. It was suggested that, under the new project, the provincial offices have minimal staff and 
also have a 'regional role'. This appears to be contradictory. Many persons interviewed 
suggested the need for increased decentralization in the provinces. However, under the new 
project there could be fewer personnel in the provinces with an increased role across a region. 
It seems unlikely that 'decentralization in the provincial offices' will be able to take place. 
Prior to the establishment of a provincial office, a clear objective of the establishment of the 
office, including its roles and responsibilities must be in place. This should guide a careful 
selection of suitable staff with required capacity to discharge the responsibilities and to 
assume the roles. Project coordinators at the provincial office should be selected based on 
their technical and advocacy skills and have the ability to work as a team and be highly 
committed. 
When only a limited selection of suitable candidates is available, other alternative solutions 
should be explored, including scaling down the roles and responsibilities and/or the 
selection of less suitable candidates with a view to improving their capacity in the spirit of 
national capacity building. The first option renders rethinking of the cost-effectiveness of 
maintaining an office with minimal capacity to support the overall implementation. With 
respect to the second option, it is imperative to strike a balance between national capacity 
building/sustainability and the assurance of the quality and cost-effective implementation of 
the Project. Moreover, potential benefits of capacity building of local staff in the 
decentralization process within the GO1 should also be given due consideration. In this case, 
white it is not feasible to strengthen capacity in many areas at once orientation and training 
essential for the smooth start-up of the office should be carried out prior to project 
in~plen~entation. 
Another option could be to make the sub-offices completely administrative and management￾oriented and hire local consultants for technical assistance. 
ARO could back up technical issues and update and FHI-HQ could do cross-border updates. 
This could save time and effort. 
Recruiting NGO staff from the staff of IAs posed problems in the past. In the future, it should 
be done in consultation with the Forum and that person should sign a statement, certifying 
his/her termination of involvement with the respective NGO during his/her employment. 
Intra-Office Communication Bullet Points 
Initially, the many complex layers of bureaucracy and the diverse sectoral backgrounds of the 
HAPP staff (public, private and non-governmental organization sectors), which characterized 
HAPP, caused difficulties in keeping open lines of communication and preventing 
miscommunication. This sometimes created delays and misunderstandings. Patterns of 
communication between the different units and departments within the HAPP Jakarta Office 
also showed a lack of sharing information and insufficient understanding of lines of 
reporting. 
The yearly retreat organized for and by the staff was considered to be a successful procedure 
that resulted in intensive and extensive evaluation, assessment and introspection by all the 
participants with regard to the activities, successes and challenges they faced over a period of 
the previous year. The retreat was valuable also as a means of dispersing misunderstandings 
HAPP - Internal Evaluation Report 
which may have arisen over time and of enhancing communication. The retreat helped to 
build relationships and to establish lines of communication within the HAFT Office. 
However, noncompliance within HAPP with agreed lines of communication and office 
procedures continued to be one of the recurrent challenges, which was partly triggered by 
urgent needs. Recipients' refusal to deal with matters that did not undergo the agreed lines of 
communication or office procedures was found to be helpful in reinforcing compliance on the 
part of communication partners. On the whole, both partners and staff of the HAP Project 
noted the need to strengthen communication/coordinatioOn among staff about routine HAFF 
activities. 
Lessons Learned : 
. Coordinati0n.k a must among HAPP staff and the establishment of lines of communication 
as a means of coordination at the early stages, can curtail confusion and misunderstanding 
within the office and with partners. Given the complexity of such a project and the 
involvement of many parties, adherence to agreed lines of communication is sometimes 
difficult to maintain over a long period of time for various reasons and for different reasons at 
different .points of time. This means that a regular review of the lines of communication to 
amend them on the basis of understanding of the root of the problem should be carried out 
from time to time. Meanwhile, the importance of enforcing the agreed lines of communication 
or office procedures should be recognized by all involved. 
. Retreats are more effective for strengthening teamwork and relationships than for solving 
such a complex management issue as the establishment of lies of communication or office 
procedures, which may take more time to carefully examine underlying causes and to come 
up with viable solutions. 
HAPP - Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management 
V. Fundkg 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Nehvorking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
V. Funding Mechanism 
A. Fundin~ Mechanism Bullet Points 
Introduction 
There were three broad types of contracts: (a) subagreements (over 
US$100,000 or a project longer than 12 months); (b) letters of 
agreement/LOAs (up to US$ 100,000 or up to 12 months) and (c) 
Rapid Response Funding or RRF. The RRF was fully developed and 
approved by the Country Office. All other proposals were first 
developed and reviewed by the CountTy Office, then reviewed and 
approved by FHI-ARO and finally concurred by USAID. It also 
involved the execution of agreements by FHI-HQ before advance 
payments to IAs were authorized. Theoretically, the whole process of 
proposal review and approval by FHI (ARO and HQ) and USAID 
should have taken up to 2 months before authorization of advance 
payment. 
bnly 10 months. The focuswas on getting started quickly and getting 
activities going in the sites. During the first two months, twenty-three 
subagreements were developed and approved. Since then, a great deal 
of learning has taken place: (a) new projects with many 'institutional capacity development' needs; @) 
new expectations; (c) new regulations and procedures; and (d) new technical needs for the HAPP staff 
in Jakarta, Surabaya and Manado, as well as the staff of partner agencies and IAs. 
During the course of the HAP Project, subagreement proposals were 
developed through two different procedural models. The first model 
was a 'Jump-StarY Proposal Development Workshop, which was held 
in August 1996 for 26 NGOs selected to work with HAPP. During the 
two-week workshop, all NGOs developed their proposals with the 
assistance of facilitators. At the end of the workshop, all the NGO 
proposals were completed and approved by FHI-HQ and USAID. This 
process was necessitated by the short initial implementation period of 
The second model was adopted for Phase I1 of the HAP Project. NGOs interested in working with 
HAPP in the three sites were required to attend a one-day orientation of the HAP Project. It covered 
the HAP Project's priorities and requirements, the USAID Strategic Objectives, writing and 
submitting a concept paper and the requirement of the submission of all subagreements in English. 
HAPP staff reviewed the concept papers and determined the disposition for each, based on these 
categories: (a) subagreements; (b) capacity building and networking; (c) Rapid Response Funding or 
(d) irrelevant to HAPP. After selection, comments were made on the concept paper and returned to 
the submitting agencies. A three-day subagreement development workhop was then held in each 
site and facilitated by the ARO and HAPP staff. The workshops focused on the development of a 
logframe, a workplan and a budget. At this point, continuity of funding was considered essential at 
the same time as it was considered essential to spend more time developing solid projects and 
proposals. 
Issues : 
. The process of the 'Jump-start Workshop' was stressful and less effective as participants 
received conflicting information, which caused confusion. This was attributed to inadequate 
HAPP- Infernal Evaluation Reporf 
preparation time for the event and the lack of orientation for facilitators about the task and 
the lack of guidance from FHI-HQ and the ARO regarding the overall procedures. The only 
benefit of the workshop was to offer the opportunity for NGOs to network and to share 
experiences and ideas. 
. During the Phase I1 of HAPP, the processing of all types of contracts, including changes in 
contracts, which required budget amendments, were time-consuming and labor-intensive. 
There were delays on the part of all parties concerned. Among the major reasons were the 
requirement for high levels of specificity in the Project contract and workplans. The contracts 
under which HAPP and the IAs worked were not easily able to accommodate being 
responsive to evolving needs and project learning. As mentioned in the discussion about 
finance, budget amendments meant undergoing the whole process of proposal review and 
approval by.FHI-ARO. The complexity of the financial form in the budget section of a 
proposal demanded considerable time to complete. 
. Delays also occurred in getting concurrence from USAID and in going through the multilevel 
management. The limited authority of the Country Office to release money for proposal 
approval was also a major challenge. 
. In light of the number of parties involved, the varying skill levels of Us and the high 
standards required in the proposal development, the HAPP staff finally resorted to jointly 
designing the subagreement with the IAs, but wrote the documents directly. A native English 
speaker writer was employed to ensure a high standard of language. The whole process of 
joint proposal development remained time-consuming and labor-intensive for all parties 
involved. The proposals completed by IAs needed improvement, both in the content of the 
intervention activities and in the format. The IAs did not mention their previous institutional 
activities, particularly during the first Phase of the HAP Project and the implication of the 
Lessons Learned for the Phase I1 proposal. Insufficient analytical thinking ability and the 
limited writing skills of NGOs entailed the provision of intensive technical support. The 
support aimed at assisting NGOs to systematically and analytically formulate a subproject 
based on lessons from their previous activities, as well as in undergoing the creative process 
of finding innovative and viable strategies. Reaching consensus on the Project's conceptual 
design, the technical strategies to be used with each IA and the clarification of the roles of key 
officers was also a lengthy process. 
. The lack of communication/coordination/consensus among the HAPP staff involved, 
including the turnover of senior staff as the focal point without proper handover (see section 
on Staffing), partly contributed to the 'impromptu redesigning' of the proposals, even after 
their approval, and incongruities between proposed budgets and activities, among others. See 
section on Capacity Building. 
. External factors also played significant roles in the delay of subproject development. The 
Project was implemented in the context of a dynamic sociopolitical environment. 
Accordingly, the content/focus of the proposal was also changed over time to better address 
the emerging needs of the target groups, such as the needs for mitigating the impact of the 
economic crisis. Political instability also delayed the whole process (e.g. the nationwide 
temporary closing of offices for security reasons and the evacuation of senior foreign staff of 
HAPP and USbased PVOs). 
. The involvement of the provincial offices in the process of proposal development turned out 
to be double-edged sword. On the one hand, the staff of provincial offices lessened the 
burden of the Jakarta staff through investing considerable time in jointly developing the 
proposals with NGOs. Joint preparation by the HAPP provincial staff and IAs, for instance 
HAPP - Internal Evaluahon Report 
with KPAD, on budget and program components of KPAD proposals, was mutually 
beneficial to both parties. This was evident in the improved quality of the final proposals and 
a better understanding of the FHI requirements and the KPAD approach and of areas where 
support was most relevant. On the other hand, it created an extra bureaucratic layer and the 
additional time necessary to reconcile disagreements between the provincial offices and the 
HAPP Jakarta Office. By and large, a subproject could take months to be approved and for 
funds to flow, even when the actual project time was only a few months. 
. Similarly, in the early phase, the process of RRF development and approval became time￾consuming and labor-intensive and created a lot of confusion among both the HAPP staff and 
NGOs. This was attributed to the involvement of too many stakeholders (FHI-ARO, the 
HAPP Jakarta Office, HAPP Provincial Offices) with different philosophies (e.g. quality 
control versus local ownership) and little coordination/consensus among them. Political 
instability contributed to difficulty in coordination and communication among staff of HAPP 
and between HAPP and partners. At one stage, the involvement of a foreign consultant with 
little knowledge of Indonesian created miscommunication between HAPP and IAs through 
inexact translation. These factors were complicated by several turnovers of focal points with 
inadequate orientation with previous agreements and consensuses and the necessity to honor 
them and to take the progress further. 
. Delays in processing/approving sub-awards had a negative impact on the progress of the 
Project in general and specifically on interrelated NGO activities. An example was a delay in 
finalizing the YBHK subagreement, which encompassed the establishment of a referral clinic. 
This hindered referral activities by other NGOs. 
Time-saving measures were carried out to streamline the process for reviewing 
subagreements, letters of agreement and RRF grants. These involved a change in the 
organogram, the recruitment of additional staff and the appointment of one staff member to 
be the focal point for RRF and KPADs, the use of standard costs, training staff in proposal 
development, holding HAPP group meetings for reaching consensus about technical content 
and developing checklists for the review of proposals (financial, technical, management and 
administrative reviews). HAPP introduced to the IAs a force-field analysis to study their 
current activities and to identify future activities. IAs utilized the results of this analysis as a 
basis for developing their amendments. It was also helpful to change the order of the 
bureaucracy procedure of RRF proposal approvals from IAs-Provincial Offices-KPAD-HAPP 
Jakarta to IAs-Provincial Offices-HAPP Jakarta-WAD. The latter arrangement permitted a 
more efficient processing of amendments, since, otherwise, proposals would have needed to 
undergo a long processing point at KPAD several times. Even though the above activities 
facilitated proposal development and approval, considerable time still had to be spent 
preparing the budget and narrative of the proposal for the Option Period. 
. RRF enable& (a) wider participation of IAs and/or target audiences; (b) the trial of small and 
innovative activities or the adaptation of effective interventions from other countries or 
provinces; and (c) implementation by new IAs or more experienced IAs including HN/AIDS 
issues within their on-going activities. At the same time, RRF project development based on 
local needs and issues (bottom-up planning) and the management of a large number of RRF 
projects was time-consuming and labor-intensive. Considerable assistance in terms of 
managerial and technical matters also required inputs, especially for newly-established IAs or 
IAs with little experience in working in the area of HN/AIDS. In view of the short-term 
implementation period (3-5 months) of RRF intervention, they could not reasonably be 
demanded to show the long-term impact of their intervention. 
HAW - Internal Evaluation Report 
Lessons Learned : 
A serious delay occurred between the planning/contracting phase and the start-up of field￾based activities, due to the complexity of the technical design, the multiple agency structure 
and the time taken to respond to USAID/Indonesia's comments and reach consensus about 
the annual workplan. USAID requirements, the new type of contract, the complexity of HAPP 
and the volatile political climate made this more difficult than usual. 
A 'jump-start' approach will not prove to be quicker for a new project. A carefully phased 
approach with a priority of provinces based upon epidemiological criteria and provincial 
capacity in relation to responding to HIV/AIDS would be the most appropriate strategy. 
Each party involved in the development and approval of a subproject @As, HAPP, FHI￾ARO/HQ, USAID, GOI) can contribute to delay in the overall process due to understaffing 
and/or the turnover of focal points, inadequate capacity (knowledge of technical and 
administrative requirements), a lack of clarification of authority among reviewers, poor 
communication/coordination/consensus. In addition to the human factors, other factors, 
such as the system (bureaucracy procedure), time-saving measures (guidelines, use of 
standard costs, user-friendly checklists for the review of proposals, user-friendly/simple 
format of proposal), coordination/consensus facilitating means (HAPP group meetings for 
reaching consensus), are also important. 
Prior to the process for subproject development, the new project should hold an introductory 
workshop that wiU orient everyone to USAID's strategy; the Goals, Objectives, Design/Focus 
and the Structure/Management of the new project; FHI and their subagreement process. It 
should include opportunity for the partners to organize themselves and to develop a plan of 
how they will collaborate under the new project for reaching joint objectives. This helps 
clarification of the definition of partnership and the operationalization of the partnership 
approach. Allow the partners to meet to discuss their respective strengths, to define their 
roles in relation to the structure of the Project and to identify strategies for 
cooperation/collaboration prior to subagreement development. TA should be provided for 
subagreement developn~ent with cooperative planning between organizations in mind. 
The decentralization of proposal development can be both time-efficient and time-consuming, 
depending on the severity of the gap behveen the capacity of the Country office and the 
provincial office, the ability of each party to reconcile their differences in an efficient manner 
or the presence of an effective mechanism to deal with disagreements between them. 
The development of subprojects means striking a balance between taking sufficient time to 
crystallize ideas, including addressing target audiences' evolving needs and taking a shorter 
possible time to prevent losing invaluable resources invested to target audiences. A long 
time-lapse between phases of the Project could mean loss of human resources, loss of time to 
restart the Project and to reestablish rapport with respective target groups. Therefore, 
proposal development and approval in the shortest possible time through maintaining 
continuity of consensus, objectivity and uniformity of treatment to all submitted proposals 
are imperative. In this context, consideration should be given to the establishment of a 
standardized review procedure for subprojects, including the development of user-friendly 
monitoring forms for all types of subprojects (e.g. subagreements, LOAs, RRFs) to ensure 
adherence by aU parties involved. The standardized procedure should include the following 
requirements: (a) aU units of HAPP should read all relevant sections of a proposal; @) the 
relevant units of HAPP should hold group meetings for reaching consensus about technical 
content; and (c) the documentation of participants, time and main results (consensus or 
disagreement) oi meetings with partners or colleagues in conjunction with proposal 
HAPP - Internal Evaluation Report 
development and approval. The latter can be incorporated into the monitoring form, hence 
the form can also function as a tracking device to monitor progress and bofflenecks and to 
help new focal points to obtain complete background information on each subproject. 
Furthermore, deadlines for every step of the procedure should also be enforced to expedite 
the whole process (e.g. deadlines for submission, deadlines for review, deadlines for 
approval). There should be a stipulation that a proposal that cannot be approved within a 
certain period of time has to be taken to a special meeting to decide its merits to continue the 
process. 
. Preparation in all aspects of the Project is essential to facilitate a seamless transition between 
projects: (a) project management (project design, technical assistants and consultants, 
methods/guidelines of proposal writing, clear project indicators); (b) working partners 
~~~~~~~~~~~~~~and dissemination of information about the Project, selection of NGOs based on 
experience, accountability, audit, etc.); (c) NGOs' concept papers; proposals/program design 
(systematic, focused, assisted by a consultant); and (d) budget planning, approval process 
and signing of contract. Therefore, the Project contractor, the executing agency and partner or 
implementing agencies, need to be well prepared. 
. Given the rigidity of other types of contract and the implementation of the Project in the 
context of the dynamic sociopolitical environment, RRF, as a flexible funding mechanism, is 
an essential element of the HAP Project. 
Recommendations : 
. Assistance should start in priority areas before the Project starts. Situational assessment and 
needs assessment should also be done when necessary. 
. The start-up of the new project can be assisted by a regional exchange between provinces of 
the old and new projects. 
. The whole process needs to be shorter and more authority should be given to COP. 
Decentralization can be done at the designing so that only the Project has authority (not 
FHI/GOI/USAID). 
. HAPP could use the transition for building the capacity of IAs' staff. Because there are no 
overheads for NGOs, a certain amount of funds should be given to NGOs to sustain their 
existence. 
. The new project should be organized through an incremental approach - this includes a 
phased approach to engaging partners and a phased approach to geographical coverage of 
the provinces. The project start-up should be seen as a phased approach with some provinces 
considered 'priority' under phase one. Priority provinces could be chosen based upon 
epidemiological significance and the level of the current development of a local response. 
Within provinces, obviously, certain districts would also be considered priority. In relation to 
program development, there could also be some technical areas that could be given initial 
priority under phase I (i.e. intenrentions with sex workers and their clients). There was no 
detailed discussion about all the necessary components of phase I. 
. The Provincial/District level GO1 partners need to be involved early on in planning. 
Subsequently, the annual workplan for the new project will need to reflect the need for 
provincial/district level assessments, planning and approvals. 
NAPP - Infernal Evalualion Report 
. Foreign consultants involved in subproject development at the provincial level should be 
selected based on knowledge of Indonesian or accompanied by a good banslator who knows 
the Project goals and sbategies well. This should be in conjunction with good coordination 
with the HAPP staff (Country office and Provincial office). 
B. Fundinp Mechanisms 
(working notes) 
c---. - 
HAPP - Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management' 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
vIn. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
InvoIvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
Vl;. Working with KPADs 
A. The KPAD and the Program Advisor 
In each of the three HAP Project areas, the Provincial AIDS 
Commissions (KPAD) were found to be well established, with a 
relatively greater involvement in HIVIAIDS prevention, education and 
control activities. HAPP strengthened the role and functioning of the 
KPAD by assisting the establishment of a working secretariat, hiring 
full-time staff and enabling coordination between the KPAD and the 
HAPP implementing agencies and partners. 
Capaciv Building and Support 
Capacity building activities that members of the KPAD were directly 
involved in included the IDF leadership and institutional development 
training, the 100 percent condom promotion workshop funded by 
HAPP, HAPP sponsored study tours, both national and international, 
and the establishment of the KPAD Program Advisor (PA) and 
secretariat. 
The important points that arose from each of the three sites regarding 
capacity building were: 
1. Manado 
. The PA was perceived by the WAD as an advisor on the 
-) programs, activities and selection of NGOs and as someone who 
coordinated with the KPAD members and supported KPAD activities and facilitated 
communication with the executive staff of the government. He was seen to be staff of HAPP 
and, therefore, perceived to receive more attention and respect from government officials. 
However, the implementation of activities was not seen to be his strength as the HAPP 
Jakarta Office was felt to be the final decision-maker. 
. The PA himself saw his role as that of an advisor and coordinator, to make suggestions about 
issues such as which groups required what capacity building activities, to help in planning 
and strategizing and giving ideas for alternatives when required. He also kept track of reports 
and research data to provide as inputs and feedback to the KPAD. 
. The PA felt that one of his major contributions was that his inputs made the KPAD begin 
thinking about sustainability and self-sufficiency and the need for other sources of funding. 
This led to districts as well as different departments of the KPAD now allocating some funds 
for their programs and activities. The above mentioned capacity building activities were seen 
by the KPAD to be very useful because officials from different sectors attended them and this 
helped to integrate HIV/AIDS activities better at the implementing stages. 
. The study tours, in particular, were found to be very useful as they helped to broaden the 
participants' view, they had a domino effect, especially in sectors other than health and to the 
executive level of the government and they led to HIV/AIDS being included in other 
programs, to better collaboration between the KPAD and NGOs and to a significant increase 
in the HIVIAIDS budget allocation. 
HAPP- Internal Evaluation Reporl 
The capacity building was also found to be useful in that community leaders and senior 
government officials had to speak personally at certain forums and, therefore, had to ensure 
that they were well-informed and knowledgeable about the relevant issues. 
NGOs also got a chance to present their views and opinions at these forums. 
The PA and secretary were largely seen to regard themselves as HAPP staff, following mainly 
HAPP orders. Therefore, they were perceived to be less helpful than required. 
The PA himself also used to consider himself a HAPP employee only earlier and this made 
working within the government difficult. But he currently considered himself more in the 
role of a coordinator between the KPAD and the government, the NGOs and H.PP and so 
found it easier to function. 
All other capacity building activities were perceived by the KPAD to be very useful as they 
were all new, they were conducted differently and had different purposes. The IDF training 
was found to be particularly useful as it taught them to develop budgets for greater allocation 
of required funds. The study tours, specifically those outside Indonesia (e.g. to Thailand and 
the knowledge gained about the success of the 100% condom use strategy) were found also to 
be of great value. 
The desire was also expressed by the KPAD PA for further baining for the KPAD members in 
English, particularly with the objective of writing acceptable scientific articles and papers. 
Jakarta 
The KPAD PA and Secretary in Jakarta were also perceived by KPAD to regard themselves as 
HAPP staff, following the HAPP Jakarta Office instructions. 
The PA himself felt that his role was that of a coordinator and supporter for the KPAD 
working groups and the different sectors and this helped them to collaborate on and integrate 
their activities and programs better rather than each sector doing unlinked and individual 
activities which, he felt, often led to duplication and inefficiency. 
The PA expressed his view that the ultimate goal of the KPAD and the NGOs is the same. But 
the KPAD should act in an advisory, coordinating and facilitating capacit): while the 
operational and implementation functions should be left to NGCk. 
The WAD members felt that the leadership training for government officials is not of 
significant relevance because leadership training is part of their culture as government 
officials. 
They felt that study tours would only be useful for them if they are held out of Indonesia as, 
within Indonesia, they felt Jakarta was more advanced in most areas than other regions in the 
country. 
The WAD members expressed appreciation for the opporhmiv HAPP provided for 
developing their proposal development skills - an ongoing training, so to speak, as HAPP 
required a proposal to be submitted for each activity undertaken by KPAD (including 
procuring office equipment and operational funds). 
HAPP - Internal Evaluation Report 
The KPAD members in Jakarta expressed a need for further capacity building in financial 
management, public relations and the training of trainers for information in HN, drugs and 
hepatitis. 
Lessons Learned : 
. The WAD Program Advisor was perceived by all the KPAD in the three sites to be a staff of 
HAPP and, therefore, did not integrate completely with the KPAD itself. This took on a 
positive hue in Manado where he was better accepted and respected by government officials. 
But it had a somewhat negative connotation in the other two sites. 
. All capacity building activities were greatly appreciated and added value directly to the 
KPAD and also through a domino effect to sectors other than health. 
Recommendations : 
. To integrate the roles of the KPAD PA and secretary better with the WAD itself, the 
recruihent process could be in greater consultation with the KPAD and the two could be 
either a part of KPAD or taken as KPAD consultants. 
. Capacity building activities must be ongoing and continue to include officials from different 
sectors to help integrate HIV/AIDS activities better in the future. 
KPA D's Role 
. The KPAD perceived their role and involvement in HIV activities as well-coordinated and 
particularly important and deep with respect to HAPP because of their perceived sense of 
responsibility. 
. They realized that, in certain situations and circumstances (for example, when dealiig with 
high-risk behavior groups), NGOs are more flexible and have to play a bigger role and, 
therefore, the KPAD's role was more that of a facilitator. 
. Because of the many sectors involved in the KPAD, their role too was perceived by them to be 
multi-sectoral. 
. They felt that the activities of the health sector had, over time, increased in frequency as well 
as in quality, particularly in terms of becoming more integrated. 
. They felt that HAPP coordinated very weU the activities between KPAD and the NGOs (who 
did what activities, the joined activities, the shared role of the different government sectors 
and the NGOs, etc.). 
The head of the NGO Forum was a member of the WAD Core Team and this went a long 
way in improving communication and trust. 
-. 7 Surabaya 
. The KPAD in Surabaya perceived their role as a support and coordination body for HAPP 
activities between the government and NGOs at all levels. 
HAPP- Internal Evaluation Report 
. HAPP involvement led to the governor and the other sectors becoming more aware and more 
involved in HIV prevention activities. This led to an increase in the budget from the governor 
and, therefore, to an increase in activities with participation even from the district levels. 
Before the involvement of HAPP, they felt that "everyone was sleeping and now they have 
woken up". 
. They felt that they had developed a good relationship with NGOs, but improvement in 
activity reporting by NGOs could lead to better coordination on the part of the WAD. 
3. Jakarta 
. Although this KPAD saw their tole more as a facilitator for NGOs to implement their 
activities, they felt that in time, they would lie to work towards strengthening their role as a 
coordinating body between themselves, NGOs and donors. 
Lessons Learned and Recommendations : 
. The KPAD played an extremely important role as a facilitator and coordinator betiveen the 
different sectors and levels of the government, the NGOs, HAPP and other donors to 
successfully integrate and implement SlV/HN/AIDS activities. This role needs to be 
consistently strengthened to make the WAD more effective, involved and sustainable in the 
future. Increasing the frequency and quality of relevant capacity building activities is a means 
of achieving this. What requires to be looked at is the financial feasibility of different capacity 
building activities. 
. A slight restructuring of the KPAD (as suggested by KPAD Jakarta, as a result of a capacity 
building activity supported by the World Bank) could be another means to greater 
operational involvement by them. This would involve KPAD having a steering committee of 
an advisory capacity and then an operational level made up of members from different 
government sectors and the NGOs, for actual involven~ent at the implementation level. 
Lessons Learned : 
. The WAD "Think Tank" or the Core Group (in which the NGOs are represented) was found 
to be very useful and the need was expressed to continue it for more efficient functioning of 
the KPAD. 
. The KPAD Core Team also expressed the need to continue support for the PA and secretariat, 
if need be, through other sources of funding. 
. To sustain the efficiency and effectiveness of the KPAD, it would be important to provide 
advocacy to members of the provincial parliaments and officials from the different sectors to 
ensure their support for budget allocation for HN prevention activities, both through the 
KPAD and through the different sectors. A multi-sectoral approach is very important and 
should be continued. 
. NGOs and KPADs at the district and lower levels need to be empowered and strengthened 
through capacity building, some of which should be self-planned and self-sustained. 
HAPP- Internal Evaluation Report 
. Good monitoring and reporting, particularly the STD survey data, contributed greatly to 
making activities and decisions more transparent, helped KPAD to encourage other 
departments to get involved and to plan better for the future. 
. KPADs at the district level need to be more involved in STD/HIV/AIDS prevention activities 
and, to achieve this, there needs to be greater levels of interaction between them and the 
NGOs at the district and sub-district levels. 
. KPAD HIV prevention programs at the provincial levels should be linked with those at the 
district levels to help make them more comprehensive and effective. 
. Members of the Core Team of the KPAD also felt that a very important factor contributing to 
sustainability was the creation of a network of comprehensive health services, ranging from 
HIV testing, counseling onwards to care of HIV-positive people at all levels and, at the same 
time, the provision of programs for empowering HIV-positive people. 
. The representation of the NGO sector (the head of the NGO Forum) in the WAD Core Team 
was felt to be a very positive move, leading to sustainability and increased communication 
and trust. However, it was felt to be important for the KPAD to consistently involve the NGO 
representative in all relevant meeting and decisions. Other factors, as expressed by the 
members of the KPAD, contributing to sustainability were their continued commitment, the 
political will of policy makers, in particular the governor and of course, continued funding 
through different sources. 
. The point was also underscored that KPADs, as in the case of any other organization, would 
require diverse and ongoing funding, including funds raised by them directly, to keep it 
sustainable and effective. 
3. Jakarta 
. One of the factors which was felt to be of significant importance for the sustainability of the 
functioning of the WAD was consistent improvement in the relationship between the KPAD 
and the NGOs. 
It was also felt to be of importance that members of the KPAD and the different sectors at aU 
levels were aware of the importance of the coordination and facilitation of related activities. 
. Regular and good monitoring and reporting led to better networking between the NGOs and 
the different government sectors and thus in better coordination between them. 
Lessons Learned : 
. To avoid confusion, changes and delays with respect to the implementation of activities and 
programs, it is important to ensure consultations and, at times when required, joint decisions 
between the KPAD and the donor (HAPP, in this case) on activities which will be 
implemented jointly. 
. In the light of the ongoing plans for decentralization in Indonesia, capacity building and 
empowerment need to be provided to the WADS at the district and lower levels to increase 
NAP! - Internal Evaluation Report 
their capability to involve themselves in and to implement programs and activities at these 
levels. As mentioned earlier, however, this would need to be done within the financial 
feasibility levels. 
Surabaya 
Along with increased capacity building for KPADs at different levels, the same would be 
required for NGOs at different levels so as to encourage and enable them to work and 
coordinate directly with KPADs at their levels. 
Jakarta 
The autonomy of the KPAD was seen to increase over a period of time and this led to their 
planning and funding a number of their own programs and activities. The role of the PA, as 
perceived by him, was also seen to have become more autonomous. This led also to allocation 
of bigger budgets and the freedom to decide how to use these. 
The role of the NGOs must increase over time because, as pointed out by the PA, "the arm of 
the government is not long enough to reach all sections of the community" and the NGOs 
must start becoming independent and capable of planning and implementing programs. 
Partnership behveen the government and the NGOs, however, must be consistently 
strengthened. 
KPADs at the district levels have be strengthened and empowered in order to conduct 
activities and programs with NGOs in their areas. 
As can be seen from the above analysis, one of the main points that arose through the various 
discussions and interviews with members of the KPADs and the Program Advisors was that in order 
to assure sustainability and to move towards decentralization, it would be necessary to increase the 
autonomy and authority of both the KPADs and the NGOs at the district and subdistrict levels. And 
to do this, various types of quality capacity building activities would be required. There is no doubt 
that this is true. However, given the high levels of expenditure needed to do this - the actual cost of 
KPA and provincial level KPAD activities between October 1998 and September 1999 was close to 
US$201,000 - it would be reasonable to assume that the costs would be many fold greater if district 
level KPADs were also to be taken up. It would, therefore, follow that more than one agency would 
need to take on this responsibility. And other sources of funding, maybe even generated by the 
KPADs themselves, would need to be looked into. 
Issues : 
. The role and function of the KPAD must be understood in the light of: its role as the 
coordinator of HIV/AIDS prevention in the provinces; the advocacy capacity of KPAD 
members to higher level decision-makers (the Government, ministries) as well as advocacy by 
them within the multi-sectoral agency to create an enabling/supportive environment for 
HIV/AIDS prevention, encompassing the allocation of a sufficient budget; and the 
commitment of the members of the KPADs. 
KPAD members' understanding of the comparative advantages of various partners' (NGOs, 
private sector) abilities and to capitalize on their strengths in HN/AIDS prevention 
initiatives were important influencing factors. 
HAPP- Internal Evaluation Report 
Study tours were seen as the most popular capacity building activity among the KPAD 
members and the impact was different across sites. For example, because certain members of 
the KPAD in Manado were decision-makers, after their participation in the study tours, they 
had the capacity to and were very enthusiastic about their increased participation and 
involvement in HIV/AIDS prevention activities (World AIDS Day). 
HAPP attempted to promote the adoption of a multi-sectoral approach within the KPADs, by 
developing various schemes designed to ensure the participation of representatives from the 
different sectors of the Government and from the NGOs into KPAD activities. Adoption of a 
multi-sectoral approach within the KPADs in the three sites varied, with some KPADs 
showing the involvement of sectors other than the health sector as the focal point of 
HIV/AIDS prevention activities and sharing information about each sector's activities to 
avoid duplication. 
Repeated restructuring and political uncertainty resulting from successive changes in the 
Government during the life of HAPP following the economic crisis, led to people feeling 
somewhat demotivated and to a public budget constraint and this ultimately adversely 
affected the performance of the KPADs. 
The technical and managerial capacities of the NGOs and the KPADs at the district and lower 
levels were reported to be inadequate in anticipation of the decentralization of GOI. These 
inadequacies included effectively implementing programs, collecting data, reporting and 
disseminating STD survey data, which are supposed to be used for decision-making and 
planning for programs based on local needs. 
The representation of the NGO sector (the head of the NGO Forum) in the KPAD Core Team, 
was a very positive move, leading to sustainability and increased communication and trust. 
?ssons Learned : 
The effectiveness of a study tour is influenced by the selection of the right person with respect 
to their commitment to HIV/AIDS prevention, their advocacy ability and also by the 
participation of multi-sectoral representatives. 
Raising awareness of the importance of =/AIDS prevention and the first-hand obsewation 
of success stories of HIV/AIDS prevention programs/activities, are needed to strengthen the 
commitment of the KPADs. In addition, it is equally necessary to strengthen their advocacy 
capacity through the provision of training about advocacy tools/strategies. 
An opportunity for dialogue among potential partners in HIV/AIDS prevention (NGOs, 
private sectors and GOI) is critical for promoting a concerted and sustained effort in this 
endeavor. 
It is important for the KPADs to consistently involve the NGO representative in all relevant 
meetings and decisions. Other factors, as expressed by the members of the KPADs, 
contributing to sustainability, were their continued commitment, the political will of policy 
makers, in particular of the governor, and, of course, continued funding through different 
sources. As in the case of any other organization, the KPADs would also require diverse and 
ongoing funding, including funds raised by them directly, to keep them sustainable and 
effective. 
NAPP - Internal Evaluation Reporf 
Recommendations : 
. To sustain the efficiency and effectiveness of the KPADs, it is important to provide advocacy 
to members of the provincial parliaments and to officials from the different sectors to ensure 
their support for budget allocation for HIV prevention activities, both through the KPADs 
and through the different sectors. A multi-sectoral approach is very important and must be 
continued. 
C. Program Advisor 
Issues : 
. All PAS and Secretaries hired by HAPP to assist the KPADs identified themselves as staff of 
HAPP. In some sites, the staff capitalized on their position as HAPP staff to facilitate WAD 
activities, but in other sites, the staff translated this identification into merely taking orders 
from the Country Office. 
. Major contributions of the PAS of the KPADs, to further improve the functioning of the 
KPADs, varied. In some sites, the PAS managed to effectively give guidance to and vitalize 
the WAD, while other PAS were less effective in doing so. 
. Learning from its own experience, HAPP recognized that regular policy dialogue was 
essential within the KPADs to strengthen the role of the KPADs as coordinators of HIV/AIDS 
prevention. This led to the assignment of Advisors to the KPADs, which was an innovative 
approach. However, this move was not accompanied by clear boundaries of and linkages 
between, roles and responsibilities between the KPAD-PA and the HAPP PPM, which further 
contributed to ineffective management of the overall HAPP activities in the provinces. 
Confusion was also caused by unclear job descriptions of the senior staff, lines of 
reporting/authority, lines of communication within intra-HAPP offices and between HAPP 
and the WADS. The IAs and KPADs were also uncertain about which office had the 
legitimate authority. 
. The decentralization of authority to the PAS with respect to the implementation of activities, 
was brought up as an issue, but this was caused more by confusion about their role - whether 
as advisors only or also as program managers. 
Lessons Learned : 
. The identification of PAS and Secretaries as staff of HAPP was to assist the KPADs, but this 
could be beneficial or detrimental from the point of view of the KPAD members, depending 
on how the identification was used to the advantage of the WADS. 
. The main contributions of the PAS of the KPADs to the KPADs depends on, 
a. their commitment to HIV/AIDS prevention; 
b. their acceptance by the KPADs; 
c. their capacity and ability to adapt to their advisory role for the KPADs; 
d. their experience in working with the Government and their history of a good relationship 
with the GOI; 
e. the awareness of the staff of the WADS about the multi-sectoral nature of HIV/AIDS 
prevention that leads to openness in accepting contributions/support from different 
sectors, including from the PAS. 
N4PP - Internal Evaluation Report 
. The management and process of HIV/AIDS activities in provinces should be clarified at the 
beginning of the Project and properly socialized among aU stakeholders. 
. Whether decentralization of authority to the PAS in terms of implementation of activities is an 
issue or not depends on whether the role of the PAS is that of an advisor only or as a program 
coordinator/manager as well. 
HAPP - Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management ' 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working with 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
VII. Working with Partners 
Workine with Partners 
Partnerships in this section refers primarily to the relationships of 
HAPP with the International PVOs identified as 'partners' on the 
original project document. There were many other organizations which 
would certainly be considered 'partners' but without the expectations 
of shared authority. These included the Indonesian organizations 
which provided valuable research and capacity building support (see 
section.. capacity building and surveillance) Other bodies and 
organizations with which HAPP coordinated or collaborated are 
discussed under section x Coordination and networking. 
The HAP Project followed the worldwide AIDSCAP model: one 'prime' 
agency contracting with USAID, working with a number of sub￾contractor 'pariners.' In some cases, secondment of staff from a sub￾contractor to the prime agency enabled greater input into ongoing 
management and decision-making. Otherwise, the 'partner ' agencies 
were subcontracted to fulfill a certain scope of work with reporting 
requirements to the prime agency. The prime agency, FHI/HAPP is 
ultimately responsible to USAID for the quality of all program 
components and outcomes. 
Without going into detail, it is interesting to observe how similar many 
of the issues and problenls identified are from both sides: HAPP staff 
and PVOs. These included : 
a) The inexperience of the HAPP staff - and PVO partners - in 
contractual 'partnership' arrangements. 
b) A specific, operational definition of 'partnership' was not explicitly established at the 
beginning of the Project. This led to a lack of shared expectations particularly about levels of 
funding, actual roles and responsibilities and assumptions about authority/autonomy versus 
the high accountability requirements of the contract. 
c) HAPP was a new project with new staff attempting to meet many requirements and 
expectations while they learned on-the-job. This contributed to inadequate establishment of 
mechanisms for more closely collaborative sharing of the workload mandated by project 
design and contract. 
d) Frequent travel required for HAPP and PVO staff often meant that key people were not easily 
available. This sometimes contributed to inaccurate communication and delays on all sides. 
e) Changes in the management of several PVO partners led to some change of organizational 
priorities. This change, in part, contributed to changes in focus to activities differing from 
those previously planned and to delays in project development 
t) It was agreed that the role of PVOs was to provide technical assistance to the NGO projects 
and to input to their design through collaborative planning and strategy meetings behveen 
HAPP, the NGOs and PVOs. However cumbersome contracting mechanisms made it 
extremely difficult for HAPP to find ways to pay PVOs for their contributions to the work on 
HAPP - Internal Evaluation Report 
project planning and development. This contributed to the PVOs not being involved as 
expected with NGO subproject development. 
g) Planning and coordination meetings were inadequate. Better systems needed to be 
established to ensure smoother coordination and communication among partners, HAPP and 
HAPP provincial staff and NGOs. 
h) PVO partners experienced the same difficulties due to the complex and lengthy 
subagreement process as did NGOs. The delay in the realization of the contract created a 
financial burden on PVOs who had maintained staff with anticipation of signing of the 
contract. 
i) Both HAPP and PVO staff perceived a lack of communication and responsiveness from each 
other. Since then the relationship between HAPP and PVOs has been strengthened. The 
reestablishment of Technical Advisory Groups helped improve relationships. A series of 
Technical Advisory Group (TAG) meetings were conducted to keep all PVO partners well 
informed on the progress of HAPP, as well as to update PVOs on technical issues that are 
relevant to STD/HN/AIDS programs. One of the TAG results was consensus on conducting 
organizational capacity assessments for all partners. This was useful to assess coordination of 
the provision of technical assistance. 
Lessons Learned : 
Partnership relationships need to be carefully managed. Clearly defme the roles, expectations, 
funding levels, responsibilities, authorities, accountability and limitations of all 'partners' within 
any new project. Clearly articulate the process for subagreement development and approval. As 
much as possible, develop capacity for this process early and communicate the progress of 
subagreement approvals to agencies concerned. Establishment and providing adequate staff for a 
regular process for regular and frequent communication should be a priority. 
The concept of collaborative ongoing management is cumbersome and not practical. However, a 
priority need is establishing a practical mechanism for being able to reimburse local partner 
agencies for their contributions during planning phases, before subagreements are established. 
Collaborative planning meetings need to be held at the beginning of a project; Establish systems 
and operational plans for ensuring regular communication as well as ways to meet needs for 
exceptional or additional coordination. Plans should include agreements on how to deal with 
differences of opinion, grievances and other interagency problems. For 'partners' outside of 
Indonesia, consultation will need to be held through phone or email contact. The plan needs to 
explain roles and responsibilities and to have budgets that are attached to outcomes and 
accountability. FHI procedures need to be explained early in the process. 
Acknowledge that planning can never be perfect; establish a system for dealing with the 
inevitable changes that will occur and with unexpected events and needs. Since the time 
requirements for communication, planning meetings and follow-up are significant, one staff 
person may need to be assigned to manage this part of the program. 
I-IAPP - Internal Evaluation Report 
Evaluation Report 1 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VNI. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
VIE Working with NGOs 
Workina with NGOs 
NGOs are the front-line forces of HIV/AIDS prevention among those 
at highest risk. The majority of NGOs involved in the first and second 
phase of the HAP Project were newly-established and had limited 
experience in meeting USAID requirements and in behavior change 
interventions. This was just as true of the HAPP staff. Given the state 
of the HIV epidemic in Indonesia, this was not surprising. The 
willingness of NGO staff to take on the challenging task of learning 
how to reach and build trusting relationships with stigmatized and 
'at-risk' populations contributed greatly to the progress of HAPP. In 
turn, support, technical guidance and the high level of resources for 
capacity building provided by HAPP contributed to the increased 
quality of NGO efforts. In the HAP Project, NGOs reached brothel￾based and street female sex workers, male 'client' groups, waria, high￾risk youth and street youth. Lower risk groups reached included 
factory workers and 'women at risk.' Additional audiences included 
pimps and managers. NGOs also carried out advocacy and 
informational activities among many important stakeholder groups, 
including local government, business managers, authorities and 
~thers. 
Issues identified by both HAPP staff and NGOs during the evaluation: 
I. Delays in subagreement devefopnent and funding 
I 
Contributing factors included 
- The complexity of contracting mechanisms. The standards for 
subagreements and the complex, multi-layered requirements for approval contributed to a long 
start-to-finish period for subagreement development and approval - Capacity building (and experience) for HAPP staff and NGOs in designing appropriate projects 
and developing proposals and subagreements that meet requirements was needed in order to 
speed up the design of appropriate projects and proposals - The need for greater authority and appropriate capacity in HAPP field offices 
2. The type and level of need for capacity building in many technical and management aspects 
varied among NGOs 
Note: Issues relating to overall capacity needs are discussed in section x.. Working with Partners, part 
... x and Technical capacity, see sections on BCC and STDs. 
The program and financial management skills shown in the first phase of the HAP Project 
substantially improved in the second phase of the Project, particularly in the areas of planning and 
financial management according to Institutional Development Framework assessments. This self￾evaluation was highly valued by those who received this assistance. IAs also demonstrated a much 
higher level of capacity and understanding of the FHI regulations. There was high staff turnover 
among a few NGOs. One of the consequences was a constant change of participants to workshops at 
each phase of the Project, which made it difficult to maintain continuity : 
- New RRF NGOs also required technical assistance on monitoring and regulations of FHI 
HAPP - Internal Evaluation Report 
- NGOs providing capacity building to other NGOs contributed to the development of both 
agencies involved. 
- The development of self-managing NGO f0~mS increased overall NGO capacity and decreased 
dependence on donors. 
- High levels of staff turnover decreased the impact of capacity building and resulted in delays to 
and the postponement of some IAs' activities. It was anticipated that some of the organizational 
capacity building would help alleviate this problem. 
3. Ongoing technical assistance 
Technical assistance has been centralized at the HAPP Jakarta Office to ensure standardization of the 
quality of technical assistance. Nonetheless, the number of technical advisors was not commensurate 
with the workloads of backstopping and supervising a considerable number of IAs, including those 
received RRFs in the three project sites. NGOs reported that this system posed challenges because of 
the length of time needed to approve IAs' works or to trouble-shoot IAs' problems and ultimately to 
the on-time implementation of IAs' activities. For additional discussion, see Sections 
4. Monitoringas a management tool 
- IAs had diverse levels of skill and understanding of monitoring and evaluation. Initially, only 
quantitative monitoring was required. Monitoring indicators were externally determined based 
on the subagreement logframes. Improving management and intervention quality through 
strengthening the ability of NGOs to use monitoring as a management tool has been a focus of the 
second phase of HAPP. 
- As much as possible, individualized assistance was provided within the overall program 
provided by the HAPP partner providing assistance on monitoring and evaluation. HAPP was 
fortunate in having a partner that provided continuous assistance in this area from the beginning. 
- Some NGOs report successes in using information to identify program needs and to develop new 
activities. Others requested more assistance in learning how to make better use of information 
collected through monitoring. One factor appears to be difficulty in analyzing and interpreting 
what is learned. Anther of the factors contributing to this difficulty may be the lack of knowledge 
about different options or approaches which might be adopted in their project. Peer to peer 
sharing may prove to be helpful in broadening the 'repertoire' of technical and management 
solutions. IAs also recommended involvement of field and other program staff in regular 
reflection and problem-solving. 
5 Des@ingprojecfs, developingproposals 
Several different approaches have been used to work with NGOs on project design and subagreement 
development. RRF development used other approaches. More detail is presented in Section x on 
funding mechanisms. 
One of the most important challenges was that HAPP was a new project and there was little formative 
data available and limited ability to use what was available. As in any new project, experience in 
designing projects and proposal writing was extremely limited, which hampered HAPP's ability to 
provide quality assistance in this area. Much of what was designed was based on 'best guess.' In 
addition, since there is no simple, 'cookie-cutter' recipe for interventions, every design and proposal 
writing activity was also an on-the-job capacity development activity. 
Another challenge for NGOs was the changes in beneficiary group focus of HAPP. Based on 
increasing data and contract changes, some beneficiary groups were added to HAPP and others were 
dropped. The shift has been towards focussing resources on those at highest risk. These are the more 
difficult audiences to reach effectively. 
HAPP- Internal Evaluation Report 
Even one PVO found this difficult, resulting in the lack of high risk 'male clients' centered in N. 
Jakarta. However, for the most part, NGOs have demonstrated comrnibnent and creativity in 
identifying needs and creating solutions, providing valuable insights in the process. 
6. Increasing NGO andgovernmen t collaborahon: see section on networking 
Lessons Learned : 
As much as possible, HAPP and partner staff who will be involved in project design and 
proposal development need to be provided with early orientation and training, keeping in 
mind that most learning wiU inevitably happen on-the-job. Training should provide 
simulations which will help build decision-making and problem-solving skills needed for 
project design. The kaining also needs to provide actual experience in writing. Program, 
technical and finance staff need to participate together in such training. 
New projects should be started in phases, after all stakeholders have been consulted and 
informed of their role in the overall plan. However, this will mean that funding will not be 
immediately available for NGO project start-up or continuity 
Capacity building needs could only be identified after the start of the Project. For most needs 
identified, approaches needed to be conceptualized and processes and resources to meet 
needs had to be created. 
Ongoing training for HAPP financial, program and technical staff in assisting others to design 
projects and develop proposals and budgets is essential in order to meet evolving needs. 
Working through NGOs requires a balance of providing capacity building to enable 
increasing organizational independence and ensuring that activities adhere to 
epidemiologically sound priorities. 
RRFs and the development of NGO forums can contribute to increasing NGO capacity and 
can expand involvement and resources for HIV/%D prevention and support. 
Regular and responsive communication with NGOs is essential for maintaining good 
relationships. It can also enable early identification of problems and opportunities for 
providing timely support. Adequate systems and staffing are needed to do this. 
HAPP- Internal Evaluation Report 
Evaluation Report 
I. HAPP Project 
I 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management ' 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX Buildingsupport: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
tX. Building Support: Networking and Coordination 
A. Networking and Coordination Bullet Points 
Partnerships in this section refer primarily to the relationships of NGO 
Forum and KPAD and religious organization. Other bodies and 
organizations, identified as 'partners' on the original project document 
(International PVOs) are discussed under section vii Working with 
Partners. 
NGO Forum 
The NGO Forum was a medium that brought together NGOs 
working in the field of HIVIAIDS issues. HAPP was instrumental in 
the establishment of NGO Forums, especially in Manado and Surabaya 
and in strengthening their capacity through involving them in various 
activities (study tours, ongoing technical assistance in financial 
accountability, management training, etc.). The achievements of the 
NGO Forums included the realization of joint activities with the KPAD 
(e.g. the commemoration of World AIDS Day) and the promotion of 
partnerships among NGOs through monthly meetings, which was a 
means of learning from each other about technical and management 
issues and it also promoted greater transparency about each other's 
programs and roles. The technical and management capacity of most 
members of the NGO Forum and the ability to support each other, 
were also critical issues. 
. Perceptions of the NGO Forum about their roles in the three 
HAPP sites differed. One perception was that the NGO Forum was a I I 
forum to share information and sources of funding for those NGOs focusing on HN/AIDS 
issues. The other view was that, in addition to the above, the NGO Forum should be 
consulted during the selection process of NGOs to be supported by donors, in view of the 
members' knowledge of NGOs working in this field. 
. One of the roles of the NGO Forum - the head being a member of the KPAD (at provincial 
level) - is to provide the KPAD with inputs about the needs and situation in the field, as the 
NGO Forum is more knowledgeable about them. Representation of the NGO Forum in the 
core team of the KPAD as well as in the Working Groups was expected to promote the 
effectiveness of the KPAD in discharging its coordination role within the province. In reality, 
the KPAD was perceived in three sites of the HAP Project as not fully involving NGOs, which 
resulted in low participation of the NGOs in the decision-making process of the KPAD and in 
a lack of transparency of the KPAD about its plans and programs, as well as about each 
other's role. In the context of improving the capacity of the NGO Forum to advocate 
pertinent, policy-related issues (such as 100% condom use in the red light areas) and to 
influence decision-making at the provincial level, the NGO Forum in Jakarta was in the 
process of investigating the possibility of restructuring the organization of the Jakarta KPAD 
to increase the number of NGO representatives at the decision-making level of the WAD. 
. The NGO Forum and the KPADs successfulty implemented large-scale joint activities such as 
the conimenioration of important AIDS events (World AIDS Day and Malam Renungan 
AIDS). With respect to these activities, articulation of clear terms of reference for the overall 
scope of work and the role of each party at an early stage or during the preparation phase 
HAPP - Internal Evaluation Report 
contributed to an increase in each party's sense of ownership as well as their sense of 
partnership. 
. The sustainability of the NGO Forum was questioned. One of the key issues was the ability of 
members of the NGO Forum to strike a balance between reaching the common goal of 
HIV/AIDS prevention through prioritizing the interest of each individual NGO and working 
as a team within the framework of the NGO Forum. This was compounded by the absence of 
recognition of it as a legal entity and its inadequate establishment as a proper and 
accountable organization. An articulated vision and mission, long-term plan and transparent 
and accountable financial and management systems were not in place. Similarly, there were 
insufficient funds, a lack of committed and capable human resources serving on their Boards 
and no full-time executive secretary. Moreover, office equipment was also seen to be limited. 
. In the context of the nation-wide implementation of the decentralization of the Government, 
there was a significant gap between the capacity of the KPADs and the NGOs at the 
provincial level and those at the district level. 
Lessons Learned : 
The foundation for collaboration and partnership between the NGO Forums and the KPADs 
has been built through membership of the NGOs in the Core Group and Working Groups of 
the KPADs and joint activities between the two parties. However, their ability to work as a 
team needs further strengthening by providing an opportunity for regular dialogue, among 
other ways of regular communication. An appropriate mechanism to promote equal 
partnership between them such as an equal representation of NGOs in the KPADs at the 
decision-making level of the KPADs must also be considered. 
There is a need for broadening the scope of NGOs participating in HAPP capacity building 
activities, to include non-HAPP-funded NGOs and for involving local consultants, if possible, 
as time and cost-saving measures, as well as a means of building local capacity. 
In anticipation of the decentralization of the Government, capacity building activities for 
NGOs and the KPADs should include strengthening their mentoring skills to provide 
technical assistance to NGOs at district levels in implementation of HIVIAIDS programs and 
in finding diverse sources of funding. 
An effective collaboration between NGOs and the KPADs is feasible if they are clear about 
the terms of reference of the overall activity and about each others' roles. These should be 
reached through a joint decision-making process at the preparation stage of a program or 
project on the basis of the comparative advantages and disadvantages of each party. 
There is a need for building the capacity of the KPADs and the NGOs at provincial, district 
and sub-district levels, to anticipate decentralization of the Government. 
The NGO Forum needs further institutional strengthening in terms of the following factors: 
(a) members' ability to work as a team; @) legalization of its role; (c) its function and structure 
as a proper organization with a clear vision and mission as well as accountability and 
transparent management and financial systems; and (d) technical and managerial skills, 
including the ability to generate income. Taking all these factors into consideration, 
institutional strengthening of the NGO Forum is both labor-intensive and time-consuming as 
it is an on-going activity. Hence, these factors should be addressed based on priority, which 
should be developed on the basis of the severity or urgency of a particular factor. When it is 
viable, capacity building activities should include a systematic framework to measure and 
evaluate capacity development. 
HAPP - Internal Evaluation Reporf 
Efforts aiming to improve the capacity of NGOs working in the area of HN/AIDS should consider 
collaboration with the members of the NGO Forum to avoid duplication. 
B. Manado NGO Forum 
The NGO Forum in Manado consisted of 17 members, includkg the HIV/AIDS Rapid Response 
Fund (RRF) NGOs. Of these members, 5 were office bearers consisting of three Presidents, 1 finance 
person and 1 secretary. One of these office bearers was also a member of the KPAD Core Group. 
Issues : 
Perception of their Role 
. Better RRF management systems were required because it took too much time for approval of 
proposals or changes took place even after the proposal was approved and the proposal went 
back and forth from the NGO to HAPP Manado Office to the HAPP Jakarta Office. In one 
case, it took 8 months for a proposal to be approved - for a project that had a life of 4 months. 
The RRF system was not standardized for proposal writing. Related to this was the fact that 
there was at times either a communication gap or discrepancy in agreement between HAPP 
Manado Office (or the Senior Staff there) and HAPP Jakarta Office, resulting in delays and 
changes even after a proposal was agreed upon or approved. 
. The members of the NGO Forum also expressed dissatisfaction on account of the fact that 
although the representative of the NGO Forum in the KPAD Core Group had been elected in 
May 1999, so far (in the last year) he was invited by them to only three meetings - for any 
information from the KPAD, he had to go to them personally and inquire. 
. There was a good personal relationship between the previous head of the NGO Forum and 
the KPAD. But he did not always share all his information with the NGO Forum. Also, 
because of this situation, it was he who always went for training programs, etc. However, he 
was subsequently not re-elected. 
. Another issue of concern for the NGO Forum was the fact that selection of the NGOs for 
participation in certain activities, particularly study tours, was not always standardized or 
transparent. In fact, the Forum representation in the KPAD Core Group was supposed to be 
involved in the selection process, but this did not happen. 
. The fo&m felt that knowing the impact and success of the "100 percent condom use" strategy 
(some members having seen it personally during a study tour to Thailand), the KPAD had 
not pushed it enough for implementation. 
. Another point of great concern for the NGO Forum was that they felt that the NGOs who 
worked well with the KPAD (and with donors,-etc. also) were those who "marketed" 
themselves well. An example of this, the Forum said, was a HAPP-funded NGO whose 
previous founder was a HAPP employee. Because of this, the Forum felt that HAPP was 
biased in favor of this NGO and the NGO got prior information on activities and plans of the 
KPAD and donors, which gave this NGO an unfair edge over other NGOs. This NGO 
presented itself as being able to work well with different target groups, as compared to other 
NGOs who worked only with one particular target group. 
fL4PP - Infernal Evaluation Report 
Capacity Bugding 
AU the NGOs represented by the NGO Forum had taken part in one or more of the fouowing capacity 
building activities sponsored by HAPP: 
Financial Sustainability Training - PC1 
. IDF (Leadership) Training - Business Dynamics 
IEC Development Training - TFGI 
IEC Dissemination Training - DKT 
Study Tours 
All this training was given over the last one year so the NGOs have not yet had an opportunity to put 
all of it to use. However, they aU unanimously expressed their appreciation of the fact that they were 
given this training and felt that it was all very useful as it was new for then and they had gained a lot 
of knowledge. Some specific points mentioned by the NGOs were: 
. One NGO working with sailors and dock workers designed a box of condoms with use 
instructions, specifically for sailors. This was kept on the ships for use by them and was 
found to be very useful. 
. Some NGOs used the Leadership Training material they received during the training for their 
training of peer educators. 
. Almost all the NGOs developed some IEC material on their own, using the inputs they 
received from the IEC Material Development Training. 
The NGO Forum agreed that, as long as an NGO provided what the target audiences required, the 
NGO would continue to be sustainable. They did, however, express the fact that it was necessary for 
NGOs to have more than a single source of funding and other income generating sources. For 
example, NU has health clinics and 20 percent of the income generated by these clinics goes to NU 
NGOs for their other activities. But, the NGOs stated that, as long as target groups needed them, they 
would continue their activities as best they could. 
Decentralization 
Given the rapid decentralization process taking place, the members of the NGO Forum strongly 
expressed the fouowing: 
. NGOs at the district and subdistrict levels have to be strengthened with respect to focussing 
their efforts, their STD/HIV/AIDS activities, writing proposals and coordinating and 
networking among themselves and with donors and the government. 
. The KPADs at the district and subdistrict levels also would have to be strengthened with 
respect to collaboration with NGOs and initiating and implementing their own activities. 
The NGO Forum submitted a proposal in January 2000 to HAPP to train NGOs at the district and 
subdistrict levels with workshops and seminars to provide to HIV/AIDS and STD information and 
for collaboration activities between NGOs and the KPAD. However, as of the end of April, they had 
not had a response. 
HAPP- Internal Evaluation Report 
Recommendations : 
Keeping in mind all the above issues and concerns, the following recommendations would be worth 
consideration. Some of these were suggested by the NGOs themselves. 
. Income generation training and programs could be organized for target groups, along with 
their ongoing HIV/AIDS and STD prevention programs. This would go a long way towards 
the sustainability of NGOs. 
. A required capacity building activity for the NGO Forum could include English language 
courses and training. This would be specially helpful for proposal and report development 
and writing activities. 
. The NGO Forum suggested having their members trained for the relevant medical and 
counseling aspects to start up HIV testing centers, which would be run and managed by the 
Forum. 
. On-the-job internship programs for NGOs would go a long way towards more efficient 
functioning and sustenance of NGOs. 
. The KPAD could encourage the 100 percent condom use policy in red light areas through the 
Ministry of Tourism and related Ministries. 
. Further capacity building for NGOs could be given for better collaboration with and 
presentation to the government (KPAD) and donor organizations. 
. NGOs could begin to look at increasing the target groups they work with and thus diversify 
further. 
. Given the level of their concern for this issue, another recommendation put forth by the 
NGOs was that, in future, if a donor recruits a staff member from an NGO, that person must 
sign a statement ensuring that he/she would not in the future be in any way involved with 
that NGO. This must be adhered to in order to do away with any bias. Also in the event of 
such a situation arising, the NGO Forum must be consulted and involved. 
. Subcontract procedures must be more standardized and professional. 
C. Islamic Reli~ious Leaders' Response To HIV/AIDS 
"The challenge of HNIAIDS is not just a health problem, rather it has major political, economic, 
social, ethical, religious and legal consequences which sooner or later will touch all aspects of national 
life" -states the National AIDS Strategy of Indonesia as formulated in June 1994. The Strategy is very 
much in-keeping with the Islamic religious teachings of the country. Islam is the predominant religion 
of Indonesia and a majority of the population is Islamic. The Islamic religious leaders of Indonesia are 
not only well respected in their communities but are also deeply involved in several community 
development activities. The infrastructure they control (for example, all the mosques in the country) is 
more widespread than the government infrastructure. They are in a key position to mobilize 
communities and implement activities. Their role is reflected by the organizational structure of the 
National Commission for Prevention and Control of HIV/XIDS, in which the Minister of Religious 
Affairs is one of the four vice chairpersons. 
It is, therefore, not surprising that they were sought out and involved relatively early in the fight 
against AIDS in Indonesia. It was realized that their potential role in making the HIV/AIDS campaign 
ELAPP - Internal Evaluation Report 
in the country successful depended to a large degree on involving and empowering them with 
appropriate information, tools and methods to, in turn, empower the community. 
But, as has been the case with all other conununities in the early days, misconceptions about 
HIV/AIDS and related issues, abounded even among the religious leaders and at the very be,-g 
they were reluctant to get involved. A number of informed and enlightened members of the 
community, including from the MOH, GOI, decided to correct the situation by taking early steps. This 
was manifested in the establishment of a specific committee on AIDS at the National Institute of 
Health Research and Development as early 1986, when there still had been no AIDS cases reported 
from Indonesia (the first case was reported in 1987 in Bali). It was however not easy to change 
perceptions and everything that was said had to be backed by accurate scientific information. Doctors 
from the MOH lobbied endlessly in the persuasion of religious leaders to start taking an active 
interest in the issue. Towards that end, a number of debates and talks were held with them and this 
eventually culminated in the first workshop for religious leaders being held in Bandung in November 
1995. 
The National Ulama Seminar on HIV/AlDS, Bandung26 - %November 1995 
The organization of this fist seminar was in itself the acknowledgment of the fact that a problem of 
epidemic proportions existed and showed the willingness of the Islamic religious community to 
actively begin to do something about it. With this as a given, the aim of the seminar was to generate 
responsible atiitudes, guided by Islamic teachings, towards preventing the spread of HIV/AIDS, 
through communication, information, education and motivation. After much discussion, the fiveday 
seminar brought forth the following recon~mendations: 
. That the Indonesian Council of Ulamas (ICU) establish a Task Force to deal primarily with 
preventing the spread of HIV/AIDS and methods of handling infected people. 
. That the Fatwa Commission of the ICU convene discussions on the establishment of Fahva on 
methods to prevent the spread of the infection, with special emphasis on issues related to 
euthanasia, quarantine and the sterilization of people with HIV and AIDS. 
. That the ICU, in collaboration with the Government, produce guidelines on nursing people, 
with a view to preventing the spread of the infection. 
. That the National Commission on AIDS Prevention insure the involvement of ICU at the 
Provincial and District levels in the Provincial and District Commissions. 
. That the Government, in its development efforts, take into account the "religious values" and 
"supreme national culture". 
. That infected people be guided by religious rites in order to be "granted forgiveness". 
The Plan of Action brought out by the seminar proposed future programs, which included: 
. The ICU setting up a Task Force to assist the Indonesian Ulama in educating people about the 
risks and prevention of HIV/AIDS. This Task Force was expected to come up with a detailed 
plan on preventive measures against the infection and to maintain continuous collaboration 
with the National Commission on AIDS Prevention. 
. Studying, firsthand, areas and groups vulnerable to the infection with the objective of 
understanding in-depth the sociodynamic environment conducive to its spread. A 
comparative study with other Islamic states battling with HN/AIDS was also proposed. 
mPP- Internal Evaluation Reporf 
. Doing research and carrying out studies on sexual behavior in relation to HN/AIDS, within 
the Islamic thinking. 
. Developing a special curriculum for lecturers, especially in religious schools, enhancing 
knowledge about HIV/AIDS. 
Developing guidelines and leaflets giving information about HIV/AIDS. 
Strengthening collaboration between the Ulamas, the Government, other Islamic 
organizations and NGOs, international organizations and universities, on research, planning, 
implementation and evaluation of the HIV/AIDS Prevention Program. 
. Convening seminars etc., to enhance this collaboration, including: 
.:. Forming Friendship FoN~S among various ICU levels, others Moslem religious leaders 
and scholars, other Islamic organizations and NGOs. 
Introducing relevant information at all educational levels, both formal and non-formal. .:. Including government bodies and institutions at all levels. 
6 Including modem and traditional mass media. 
This was a major first step taken by a predominant religious community in the country and was done 
relatively early in the evolution of the infection in Indonesia. 
Study Tours of Islamic Leaders to Malaysia and Thailand 
The purpose of these visits was to gather firsthand knowledge about the host countries' policies on 
W/AIDS programming, background, design, involvement of related sectors, implementation, 
constraints and achievements; and also to provide an opportunity to exchange views on the different 
approaches to HIV/AIDS, particularly with relevance to the Indonesian Master Plan 1995/1996 - 1999/2000. The visit was coordinated by PATH/Indonesia in conjunction with PATH/Thailand and 
PACT/Malaysia. 
The main objectives of the study tours were: 
. To study the Thailand and Malaysia experiences with planning and implementing their 
HN/AIDS programs. 
To consider the relevant implications of Thailand and Malaysia for the Indonesian National 
AIDS Program. 
. To gather important views and inputs to help improve ability in reformulating and 
redirecting the Indonesian National AIDS Program within the religious framework. 
In particular, the tour to Malaysia was to learn firsthand how programming in an Islamic society can 
be effectively carried out. In Bangkok, the objective was to learn firsthand how a "full-blown 
epidemic" can affect a society and a nation. 
In each country, several meetings and discussions with host organizations were held. Some of the 
more outstanding outcomes of the exchange were: 
. The presenter of the presentation on the HN/AIDS Program for Recovering Ex-Drug Users 
at Rumah Pengasih, acknowledging his appreciation to the Indonesian Islamic Leaders for 
visiting the Rumah Pengasih, was grateful to have been given the opportunity to host such a 
HAPP - Internal Evaluation Report 
group for the first time. He hoped this would lead to the Malaysian Islamic leaders doing the 
same. 
. During the presentation on Islamic Approaches by the Govenunent in Relation to HIV/AIDS, 
made by Jabatan Kemajuan Islam Malaysia, the need for dialogue between the two countries 
on Islam and HIV/AIDS and to exchange ideas and learn from each other was expressed. 
. After a visit to the Suan Santitham AIDS Hospice in Patumthani, Thailand, the Indonesian 
group understood better the need for hospices for AIDS 
However, the main results of the tours were: 
. The expression of a need for Islamic leaders of MEAN to meet together and discuss issues, 
concerns and preventive measures to regionally combat the problem of HIV/AIDS. 
. Indonesian Islamic leaders realized the need to act immediately - it was made clear to them 
that Indonesia could not afford to delay the implementation of HN/AIDS prevention 
programs any longer. 
. They realized also that the involvement of Islamic leaders in the overall national effort in the 
battle against the spread of HNIAIDS is of the utmost importance. 
Workshop of the Indonesian Islamic Leaders on HIV/AIDS, Jakarta, 22-25 June 199% 
One of the most important follow-up actions of the Study Tours was the second workshop of Islamic 
leaders, held in Jakarta in June 1997. This was fiianciauy and technically supported by 
USAID/HAPP, with particular emphasis on its first main program component, that is, policy 
support. 
Various aspects of HN/AIDS, with particular regard to religion, were discussed during the course of 
the workshop. These included issues relating to HIV-infected pregnant women, the babies of these 
women, circumcisions of HIV-infected children, HN-positive CSWs, the home care of people living 
with HN and AIDS and the handling of people who die of AIDS. All these issues were seen in the 
light of both medical ethics and the relevant religious teachings and took into account the entire life 
span of the human beingl'from the womb to the tomb". 
One of the main results of this workshop was the development of a set of guidelines for 
implementing HN/AIDS prevention programs and activities, within the framework of Islamic 
teachings and importantly, with a strong emphasis on the fact that the dignity of human beings must 
be respected with no exceptions, including that of HIV/AIDS infected people. A controversy on 
euthanasia for HN/AIDS infected people ensued during the workshop and discussion on the topic 
raged. However, given the importance of human dignity and the fact that, as per Islamic teachings, 
only the Almighty has the right to give and take life, the Indonesian Council of Ulamas rejected this 
idea. This then led to a decision where euthanasia, discrimination and isolation of people with 
HIV/AIDS would not be "accepted, approved or tolerated". 
The First HW/AZDS ASEA NRegonal Workhop of Islamic Rehgious Leaders, jakarta, November30 
- December3,1998. 
This was attended by delegates from Bmnei Darussalam, Indonesia, Malaysia, the Philippines, 
Singapore and Thailand. The Deputy Country Director of HAPP chaired the workshop. The opening 
ceremony of the workshop was linked to World AIDS Day (December 1,1998) and the participants 
were addressed by the Minister of Religious Affairs and by the then President Significant papers 
were presented by deIegates from all the represented countries and each presentation was followed 
HAPP - Internal Evaluation Report 
by lively discussion. Each country representative spoke about the health situation in general and then 
about the HIV/AIDS problem in particular in his country, prevention and control measures being 
undertaken, problems faced and the role and involvement of the religious community within the 
social and cultural framework in their country. Other than this, several Indonesian speakers spoke on 
a number of different aspects and issues related to HIV/AIDS. However, the major results of this 
workshop were the development and adoption of a Plan of Action and the Jakarta Declaration. 
The Plan of Action 
It is commonly accepted that Moslem Leaders in Islamic communities are not only highly respected 
within these communities, but are also deeply involved in community development activities. They 
can, therefore, play an important role in the HIVIAIDS campaigns in their communities and 
countries. However, an obstacle being faced by them at the time of the formation of the Plan of Action 
was that the IEC component of the National AIDS Program was reaching only a few of them in the 
urban areas. This was a major drawback. It was felt during the workshop that "feasible methods and 
appropriate instruments for empowering the majority of Moslem religious leaders in (the) ASEAN 
member countries (were required) to enable them (to play an) important role in (the) HIV/AIDS 
campaign in their respective Moslem communities". 
Therefore, the objectives of the Plan of Action were: 
. To implement HIV/AIDS campaigns within the given religious framework. 
. To strengthen cooperation among Moslem leaders in the ASEAN region in combating 
HIV/ AIDS. . To set up information exchange networks among ASEAN Moslem leaders. 
A number of activities were proposed towards successfully achieving the above objectives. These 
included: 4 
. Holding seminars, workshops etc., on HIV/AIDS for Moslem leaders throughout the region 
with the objective of getting rid of misconceptions about the issue; 
. Establishing expert committees on sex education within the Islamic cultural framework, to 
develop guidelines for teaching modules for children, youth and married couples; 
. Developing technical working groups (within the Islamic cultural framework) on risk 
reduction programs; 
. Implementing programs for the prevention of the spread of HIV from mother to child; 
. Strengthening cooperation between the Government and Moslem leaders for solving the 
problem of prostitution by implementing religious and public health programs. 
TheJakarta Declaration 
As stated earlier, the other major result of the First HIV/AIDS ASEAN Regional Workshop of Islamic 
Religious Leaders was the formation and adoption of The Jakarta Declaration. This basically summed 
up the conclusions and findings of the workshop and in addition, provided the following guidelines: 
. HIVIAIDS is acknowledged as a global threat. 
. Euthanasia is unacceptable. 
. Knowingly transmitting the infection is against Islamic Law. 
. Marriage between people living with HIV/AIDS is permitted. 
. Marriage between an HIV-positive and HIV negative person is not prohibited but is 
preferable to avoid. 
. Divorce due to the infection is lawful. 
. A married couple must use preventive measures (condoms) to prevent transmission of the 
infection if one of them is HIV-positive. 
HAPP - Internal EvaIuation Report 
. Abortion is prohibited even if the pregnant woman is Winfected. 
. A pregnant HIV-positive woman having got the infection through IDU must be treated 
humanely but must be "made aware of her sin and guided to ask for forgiveness". 
. PLWHA should stay with their families and families should take care of such people. 
. The delivery of a baby of an HIV-positive woman should be handled by well trained 
professionals. 
. The circumcision of HIV-positive children must be done and handled by well trained 
profe&ionals. 
. HIV/AIDS positive accident victims should be taken care of and all precautions should be 
taken to avoid HIV transmission. 
. Deceased AIDS patients should be given a proper Islamic burial. 
From fien To Now 
So, briefly, the following have been the stages of involvement of the Moslem religious organizations 
and leaders in the recognition and active participation of HIV/AIDS preventive activities, nationally. 
. The fist stage, as has been the case with all communities globally, was that of denial, of not 
recognizing the gravity of the problem and, therefore, not wanting to get involved. Active 
lobbying, talks and debates with religious leaders by Government officials, doctors and 
others, led to a slow but positive shift in their attitudes towards recognizing and 
acknowledging HIV/AIDS as a global problem of which Indonesia is a part. 
. This resulted in the setting up of the special committee on AIDS in 1986. But, misconceptions 
and inaccurate information still lingered in the minds of most Islamic leaders. Further debates 
and talks were held to disperse these misconceptions. 
. The National Ulama Seminar on HIV/AIDS Prevention Measures was then held in Bandung 
in November 1995. This resulted in the deeper involvement of Islamic religious leaders in the 
issue of HIV/AIDS through a number of recommendations involving the ICU and plans for 
the future. 
. Two study tours to Thailand and hlalaysia 
. A workshop of the Indonesian Islamic Leaders on HIV/AIDS was then held in Jakarta in June 
1997. Here, for the first time, a guideline was developed to implement HN/AIDS programs 
within communities and within the framework of Islamic religious laws and teachings, with 
special emphasis on respecting the dignity of human beings, with respect to issues such as 
euthanasia and discrimination. 
. The First HIV/AIDS ASEAN Regional Workshop of Islamic Religious Leaders took place in 
Jakarta in December 1998. This was attended by representatives of six ASEAN countries with 
high Moslem populations. The most important results of this workshop were the formation of 
a regional Plan of Action and The Jakarta Declaration. 
Lessons Learned : 
Religious Islamic Organizations in Indonesia are amongst the biggest, strongest and most influential 
the world. Almost all the people belon,$ng to the Moslem religion in Indonesia (and it is the 
predominant religion of the country) are members of one or the other major Islamic religious group. 
These religious organizations are old and have been, from the beginning, intensively and extensively 
involved in community development activities. Their activities cover a gamut of issues and concerns 
including health, education, religious teachings, counseling, family life enhancement, etc. And 
HAPP- hternal EvaIuation Report 
activities implemented by them geographically cover, by and large, the entire country. The major 
Lessons Learned, therefore, have been as follows: 
HAPP recognized that, as the Moslem population is dominant in the country, the 
involvement of the religious leaders was a must and HAPP did so at an early stage of the 
Project. 
Moslem leaders are respected decision-makers in their communities and all program 
decisions endorsed and encouraged by them have very high rates of success. 
Having once introduced them to and got them into the fold of the issues and problems 
related to HN/AIDS, introducing new programs into their ongoing ones will prove to be 
easier. 
The problem of HN/AIDS was recognized and accepted by the Moslem religious community 
early and the challenge now is to ensure that the momentum in their involvement and 
cooperation is kept at a peak so as to thereby ensure the involvement of the majority of the 
country's population and, thus, the ongoing success of the Indonesian National AIDS 
Strategy. 
Relipious-Based Oreanizations 
Introduction 
"The challenge of HN/AIDS is not just a health problem, rather it has major political, econon~ic, 
social, ethical, religious and legal consequences which sooner or later will touch all aspects of national 
life" - states the National AIDS Strategy of Indonesia as formulated in June 1994. Islam is the 
predominant religion of Indonesia. The Islamic religious leaders are respected and involved in 
development activities in their communities. Their key position to mobilize communities and 
implement activities is reflected by the organizational structure of the National Commission for 
Prevention and Control of HIV/AIDS, in which the Minister of Religious Affairs is one of the four 
vice chairpersons. 
Issues : 
. In the early days, the religious leaders had misconceptions about HIV/AIDS and related 
issues and they were reluctant to get involved. Doctors from the MOH lobbied endlessly in 
the persuasion of religious leaders to start taking an active interest in the issue. Towards that 
end, a number of debates and talks were held with them and this eventually culminated in 
the first workshop for religious leaders being held in Bandung in November 1995. 
. The study tours to Malaysia and Thailand resulted in. the realization of the need to act 
immediately to prevent the spread of HN/AIDS; the realization of a need for Islamic leaders 
of ASEAN to meet together and discuss issues, concerns and preventive measures to 
regionally combat the problem of HIV/AIDS; and, religious leaders understanding better the 
need for hospices for AIDS patients. 
. One of the most important follow-up actions of the study tours was the second workshop of 
IsIamic leaders, held in Jakarta in June 1997. Various aspects of HN/AIDS were discussed in 
the light of both medical ethics and the relevant religious teachings and took into account the 
entire life span of the human being "from the womb to the tomb". The main results were: 
HAPP- Internal Evaluation Report 
a. the development of a set of guidelines for implementing HIV/AIDS prevention 
programs and activities within the framework of Islamic teachings and respect for 
the dignity of human beings, including that of HIV/AIDS infected people; 
b. a decision where euthanasia, discrimination and isolation of people with 
HN/AIDS would not be "accepted, approved or tolerated". 
c. it was also specified that condom use for couples where one of the hro is HIV￾positive, is compulsory. This was a major step. 
. The First HN/AIDS ASEAN Regional Workshop of Islamic Religious Leaders took place in 
Jakarta in December 1998. This was attended by representatives of six MEAN countries with 
high Moslem.populations. The most important results of this workshop were the formation of 
a regional Plan of Action and the Jakarta Declaration. 
Lessons Learned : 
Involvement of the religious leaders is a must and should be done at an early stage of an 
HIV/AIDS prevention project. 
Having once introduced religious leaders to and got them into the fold of the issues and 
problems related to HIV/AIDS, introducing new programs into their ongoing ones will prove 
to be easier. 
Ongoing and intensive advocacy activities play a significant role in the enlistment of support 
from the religious leaders in HN/AIDS prevention programs. 
The problem of HIV/AIDS was recognized and accepted by the Moslem religious community 
early and the challenge now is to ensure that the momentum in their involvement and 
cooperation is kept at a peak so as to thereby ensure the involvement of the majority of the 
country's population and, thus, the ongoing success of the Indonesian National AIDS 
Strategy. 
Participation In ICAAP-5m, 
Kuala Lumpur, Malaysia 
HAPP participated in the Fifth International Congress on AIDS in Asia and the Pacific (ICAAP-Sh), in 
Kuala Lumpur, October 20-27,1999. 
An attempt was made to maximize NGO and GO1 participation in the ICAAP-5h. This included the 
capacity building activities, which enabled direct project representation at the Conference. 
Indonesians working under the HAPP umbrella were facilitated in developing capacity in order to 
bring their experiences to Kuala Lumpur to share in a forum of their peers. Out of 30 HAPP 
sponsored participants, 14 persons representing HAPP partners gave presentations at the Conference. 
The other HAPP participants attended the Conference and participated in region-wide nehvorking 
workshops, managed the FHIIHAPP exhibit and provided technical support to abstract presenters. 
In addition, HAPP staff were involved with the planning and implementation of heightened public 
relations activities specific to HAPP's participation in the conference. 
HAPP - Internal Evaluation Report 
Capacity building outputs from this activity included the following: 
. Twenty-three professional abstracts were written in English by HAPP partners and submitted 
to an international HIV/AIDS conference. 
. Two in-country panel discussions in English were organized and held, which provided a 
forum for abstract presentations and dialogue with peers around professional issues. 
. Fourteen individuals successfully gave professional presentations in English at an 
international AIDS conference. 
. Thirty individuals representing HAPP partners attended an international HIV/AIDS 
conference and participated in activities such as skill-building workshops, regional 
networking, management of the FHI/HAPP exhibit and other professional exchanges. 
. HAPP staff gained skills in planning and designing an exhibit area, the design and 
management.of HAPP PR activities and printed materials. 
The overall goal of this activity was to facilitate projectwide participation in an international 
HIV/AIDS conference, thus heightening Indonesian experiences and concerns as an integral part of 
the Asia Pacific Region. Specific objectives were to build the capacity of HAPP partners in the area of 
abstract writing and oral and poster presentations. In addition, there was also the objective of 
promoting HAPP as an FHI/USAID/MOH project in Indonesia through an exhibit area and 
involving HAPP staff and members of HAPP IAs in PR activities. 
As much as possible, the actual author of the abstract was coached and facilitated in writing the 
abstract. Many abstracts were first conceptualized and written in Indonesian and later translated into 
English. HAPP provincial staff played an important role in providing direct assistance to individuals 
submitting abstracts, many of whom were writing abstracts for the first time. 
The HAPP consultant organized the HAPP Jakarta Office staff in assisting in the final production of 
abstracts to be submitted to the ICAAP organizing committee. Abstracts needed to be typed and fit 
into a standard format on an original form. The authors' personal data was also collected and 
recorded. HAPP selected 23 abstracts for submission to the Conference committee, making sure that 
these arrived prior to the deadline. This process alone was very labor-intensive and time-consuming. 
Oral presentations were written in English based on the author's technical paper. Presenters were 
given one-on-one assistance in both writing and presenting in English. Technical Assistance in the 
development of colorful visuals was provided by graphic artists.. 
Two separate seminar activities (one each in Surabaya and Jakarta) were planned and held in advent 
of the Kuala Lumpur Conference. These activities consisted of a panel discussion featuring the 
selected abstract presenters. The activity was designed as a rehearsal for the presenters to gain 
confidence in presenting and fielding questions in English. It also allowed other HAPP partners 
(KPAD and NGO) to be aware of how their province would be represented at the Conference. 
Poster Presentations were assisted with both technical and artistic input. Poster content was reviewed 
for accuracy, adherence to the suggested poster guidelines and attractiveness. Graphic artists were 
engaged to help create the final poster from a document of printed data, photos, tables and graphs. In 
one case, the poster content needed to receive clearance from CDC Atlanta as this office held 
intellectual rights to some of the material being presented. HAPP staff worked closely with the artists 
and reviewed aU poster "mock-ups" prior to printing. Professional, single-printed posters were 
produced with HAPP, USAID, FHI and MOH signature logos. Posters were transported to Kuala 
Lumpur and placed in the specific viewing area by the poster presenter. After the Conference, the 
posters became the property of the individual presenter/organization for use in their respective 
agencies. 
UAPP- IntemalEvaluahbn Report 
Upon arrival in Kuala Lumpur, all HAPP participants were briefed on the conference activities. Oral 
and poster presenters were twinned with helpers (other HAPP participants) and given written 
explanations of the place and time of their presentations. 
Two exhibition booths were used for FHI/HAPP. The HAPP exhibition booth had the goal of 
promoting HW/AIDS projects in Indonesia under HAPP umbrella. 
A post conference assessment was made with HAPP sponsored participants. Participants were asked 
to complete a brief survey which was done by 56 percent of the 
Lessons Learned : 
The following are some of the "lessons learned" gleamed from this experience: 
. Developing capacity for writing and presenting professional papers is useful for both NGO 
and GO1 organizations. The ski1 level from person to person and organization to 
organization varied significantly. A great deal of technical assistance was required in this area 
sometimes influencing the actual "intellectual ownership" of the product Due to most of the 
TA being given on a one-on-one basis, a series of workshops that would facilitate writing and 
presenting abstract ideas over a period of time is recommended. This method may prove to 
be more time and cost-effective for all parties. 
. The FHI/HAPP exhibition booth promoted a sense of national pride and facilitated 
teamwork among participants. This was especially apparent behveen the GO1 and NGO 
participants who worked side-by-side as booth attendants, answering queries about 
HW/AIDS work in Indonesia. 
. This was the first experience for HAPP staff to plan for a professional exhibition. The use of 
professional graphic artists was invaluable to the entire process. More lead time in art 
production is recommended, especially if multiple requests are made of the same agency. If 
possible, abstract authors should have more participation in making recommendations to the 
artists to reflect more of their personal style and emphasis. 
. In order to ensure adequate staffmg at the FHI/HAPP booth, it is recommended that when 
FHI sponsorship is given to an individual to attend an international conference, there is 
requirement that the individual participates in booth attendant duties. 
. For future FHI participation in international conferences, it is recommended that more 
interactive ways be used to tell others about work in Indonesia other than through an 
exhibition booth. This might include music or drama performances, hosting informal 
networking meetings (similar to the sex worker nehvorking meetings), etc. 
. In July 1999, FHI/HAPP, through the Futures Group, held a journalism contest for 
Indonesian journalists to encourage coverage HW/AIDS issues in Indonesia. The winner of 
the contest was awarded free participation, travel and lodging to attend the 5''' ICAAI' in 
Kuala Lumpur. Throughout the conference, this journalist had an opportunity to become 
better acquainted with the Indonesian delegates and, more importantly, to learn about 
specific interventions and to hear concerns. The winner, a writer for Femina Woman's 
Magazine, produced several articles which were published based on learnings from the 
conference. Needless to say, the strategy of facilitating the capacity development of a local 
journalist was successful. 
HAPP - Internal Evaluation Report 
F. Draft: capaciw buildina; evaluation framework 
(Working notes) 
a. Capacity In- a&witZev we' On-guhq thvw the 2&3 of 
tha Proj'ect dMd attempt t@ reapd deveZopw neab w L&,VUX.~ 
bypavture*-6.dMdbyHAw. 
HAP'- Internal Evaluation Report 
HAPP- Internal Evaluation Report 
G. The HAPP Capacity Buildina Model 
Introduction 
Building capacity among host country nationals and institutions must be a goal that underwrites the 
activities of any development project. HAPP has allocated significant attention and resources to this 
end. 
Capacity building refers to the enhancement of abilities that are at least one step removed from the 
beneficiary, i.e. the person who will benefit from the HIVIAIDS prevention activities provided 
through HAPP. Capacity building includes a range of activities that contribute to the development of 
technical and management ability among those involved in HAP Project implementation. 
Those benefiting from HAPP capacity building activities include HAPP staff, KPAD, local 
government, IA staff, staff in government clinical and lab facilities and members of the private sector. 
Capacity building activities are on-going throughout the life of the Project and attempt to respond 
developing needs as identified by partners and by HAPP. Capacity building may be multi-layered, 
with HAPP providing support to an agency which in turn provides capacity building support to its 
own audience. 
As much as possible, capacity building activities are done through partnerships: 'with' and not 'to' 
participants so as to strengthen 'fiV and the potential for uptake by participants. Partnerships include 
collaborative needs assessment and 'learn by doing' activities as described below. 
Capacity building activities are implemented through a variety of mechanisms. For example: 
. Directly, through HAPP technical and management/fiancial staff in all sites; 
. Through contracting with an external agency NGO, CBO, university, PVO , private sector; 
. In collaboration with other donors and GOI; 
. Through budgetary and TA support for capacity building provided by IA s and RRF grantees 
to staff and communities; 
. Through technical support to GOI; 
. Indirectly through resource mobilization (e.g. through Durex - which supported WAD 
activities; 
HAPP - htemal Evaluation Report 
. Through funding and arranging for international training, study tours, meetings and 
conferences (ARO, International AIDS conferences, Johns Hopkins); 
. Through funding and arranging internal/national study tours. 
Capacity building activities are implemented using a variety of approaches. These include: 
Formal workshops with specific outcome objectives; 
Support for peer to peer learning through internships, mentoring, consultation, exchange 
visits; 
Support for international and local study tours; 
Informal on-the-job capacity building implemented by HAPP staff and KAPP technical 
consultants during site visits (to HAPP staff and IA s, HAPP); 
Support to imperson information sharing and networking activities; 
Providing facilitation and support for self-analysis and reflection on technical and 
management capacity by HAPP staff, HAPP implementing agencies and WAD; 
Support for problem based 'learn by doing' activities (BCC ~~LcuhmdeveZopment, EC 
matwk& dadopmW)( what about the Ivaininft. and, Kqsm 
tfapitd?); 
Through invitations to GO1 representatives to capacity building activities; 
Through formal and informal presentations (e.g. BSS and other data). 
Capacity building activities are monitored through: 
. Qualitative/quantitative reporting 
. Technical review by HAPP staff 
. Debriefing with participants and implementing staff 
. Monitoring meetings 
. Pre-post tests 
. Workplans 
'Opinionaires' at end of activities (participant perceptions) 
The HAPP capacity building model and activities will be evaluated through a multi-tiered 
participatory process: among and by HAPP offices, implementing agencies, recipients of capacity 
building. Due to the breadth and scope of capacity building, the evaluation will focus on a carefully 
selected sample of activities and approaches. 
Management systems capacity development is being evaluated through application of an Institutional 
Development Framework process. The HAPP offices trialed the process which has now been 
facilitated with the 9 major HAPP IAs as well as with the KPADs. 
30 additional 'safety neV NGOs being supported though an agreement with an international PVO 
(PACT) are being assisted with a similar assessment, using a PACT-developed approach. 
H. Capacity Buildinz Introduction to IDF 
Key activities of the HN/AIDS Prevention Project were implemented through NGOs. The 
organizational orientation of many NGOs is often more outwardly focussed, on field or technical 
activities rather than inward on management and the development of a professional organization. 
The HAP Project identified hvo essential components for the effective implementation of Behavior 
Change and clinical HIVIAIDS prevention intewentions: sound technical strategy and skills and the 
institutional and management capacity to implement the strategy. 
HAPP- Internal Evaluation Report 
In an effort to promote quality work and to introduce greater accountability and sustainability into 
NGO activities, FHI adopted two synergistic capacity building approaches, both related to 
management capacity: 
1. Increasing the capacity for planning, carrying out and using the results of outcome-linked 
monitoring and evaluation to strengthen program management and 
2. Ensuring NGOs mature into strong, effective institutions 
A first step forward in institutional development was to provide each IA with the opportunity to 
carry out an organizational situation assessment. FHI identified and reviewed several tools that 
provided an operational framework for institutional or organizational assessment. The Institutional 
Development Framework was selected as it addressed core organizational components identified by 
HAPP staff during their reviews and the ID Framework had been successfully translated, adapted 
and used in Indonesia. While an external assessment might be more objective, the participatory, 
reflective process of self-assessment using the IDF process is intrinsically capacity building. Finally, 
skilled facilitators to assist with the participatory process were already available. 
I. Institutional Develovment Framework 
The IDF was used to assess institutional development within the HIV/AIDS Prevention Project 
(HAPP). The Institutional Development Framework was selected as it addressed core organizational 
components identified by HAPP staff during their reviews and the IDF Framework had been 
successfully translated, adapted and used in Indonesia. While an external assessment might be more 
objective, the participatory, reflective process of self-assessment using the IDF process is intrinsically 
capacity building. Finally, skilled facilitators to assist with the participatory process were already 
available. 
The IDF is essentially a matrix con~posed of commonly held resource characteristics seen as crucial to 
institutional effectiveness in almost any organizational unit. These characteristics are further 
subdivided into discrete "key component" areas against a four-stage institutional development 
continuum for each of the key components cited. The continuum describes textually the "life cycle" of 
each of the key components from the typical "start-up" phase to a "development" stage on to an 
"expansion/consolidation" phase to a phase which could be considered as "sustainable." 
Start-up Development Expansion/ Development 
Consolidation 
In its current form, the IDF is purposefully inclusive - there are enough resource characteristics and 
key components identified (with accon~panying four-stage development continuum criteria) to satisfy 
almost any type of organizational unit. The common resource characteristics used with HAPP 
partners included: organizational mission and vision, leadership style, planning, monitoring and 
evaluation, staff skills, management practices, financial management, communications, public 
relations, community participation, autonomy and advocacy. Through group process, the 
organization is tasked with reviewing the characteristics and key components to determine which 
ones apply to their own organization. 
The organization then self-assesses their "location" or progress on the four-stage continuum giving 
themselves a 14 "score" (using decimal points as desired) for each of the key components they find 
relevant to their organization. 
I-IAPP - Internal Evaluation Report 
This simple self-assessment scoring mechanism enables the organization to immediately generate a 
graphic picture as to their institutional "progress" over time. (The scores are depicted in a simple bar 
graph.). More importantly, the dialogue process among participants during the self-assessment helps 
forge a common understanding of the organization's purpose, vision, identity and sense of progress 
towards "sustainability" as defined by the criteria stated for each of the key components. 
By obtaining an average of the key component scores, the organization is able to report a single 
number, which, in an admittedly very general way, represents the organization's overall progress 
towards sustainability.). 
For internal management purposes, the organization can assign a priority to each of the key 
components (in the form of 1-4 weighting, with 4 being high priority and 1 being low). Doing so 
allows the organization to plot out the location of each of the key components in terms of "urgency of 
attention" (on an "x-y" axis) with one axis depicting the 1-4 continuum progress "score," and the 
other axis depicting the assigned weight for each of the key components. The key components will 
fall into one of four categories: 
Q low priority/high continuum score 
-2 low priority/low continuum score 
'3 high priority/high continuum score 
-3 high priority/low continuum score 
The key components that fall into this last category, the "high priority/low continuum score," qualify 
for "urgency of attention". These are key component areas in which the organization places a great 
deal of importance, but has yet to make much progress towards sustainability, but which command 
more immediate attention/review by the organization and the development of action plans. 
The IDF assessment involved a process of in-depth discussions and was administered with each 
organization over a period of 3-4 days. Each IDF workshop involved groups of 15-30 persons 
requiring 1-2 group facilitators/professional trainers. An external party with group training 
experience was sought out as facilitator. Not only was neutrality a preference, but also there was a 
realization of the labor-intensive nature of thirteen separate workshops and the impact on current 
HAPP staffing. Therefore, Business Dynamics was contracted to implement the IDF. In addition, 
HAPP identified a dedicated staff member to coordinate and manage the entire process. 
Business Dynamics is a Jakarta-based firm with experience in conducting corporate education and 
training programs for business and non-profit organizations in Indonesia. Business Dynamics refined 
the framework and adapted it to the characteristics of the HAPP organization. Business Dynamics 
provided three professional bilingual trainers who either worked individually or as a team with 13 
HAPP organizations. All nine HAPP implementing agencies, three HAPP-supported Provincial AIDS 
Commissions (KPAD) in Jakarta, Surabaya and Manado and the HAPP Jakarta Office voluntuily 
participated in this assessment process. Business Dynamics was the primary agency administering the 
IDF, howwer HAPP provided close monitoring, 'ouble shooting and logistical support The 
following steps were taken in administering the IDF. 
Weekly phone monitoring was done with the Business Dynamics facilitators. Feedback from 
experiences with some of the first organizations assessed resulted in modifications to the workshop 
design and schedule. The same facilitator was designated to work with each of the Provincial AIDS 
Commissions (KPAD), as the organizational structure and style differed greatly between KPAD and 
NGOs. HAPP Jakarta and Provincial staff provided assistance as needed for logistical support. 
c-n 
HAPP- Infernal Evaluation Reporf 
Follow-up Action Plans 
As the cumulating activity of the IDF assessment, Business Dynamics facilitated each organization in 
making action plans based on the IDF findings. The plans were written up in specific tasks and 
members of the organization took responsibility for its completion. Business Dynamics also made a 
site visit to each of the organizations in order to verify information that was provided during the 
assessment. 
Outcomes of IDF 
Some of the main incentives to utilize the IDF were the value it has as an accountability tool and as a 
means for HAPP organizations to promote themselves, thus increasing chances of sustainability. 
Anecdobl Summary by Business Dpamii-s 
Business Dynamics documented their observations of each organization during the IDF Assessment 
process. Based on their observations, Business Dynamics made recommendations for the HAPP 
Jakarta Office in their supervisory role of each organization. 
Lessons Learned : 
This was HAPP's first experience in conh.acting with a private agency for an intervention with NGOs 
and government organizations. The initial concern expressed was whether or not an agency from the 
business sector could relate to the culture of NGOs and government. Business Dynamics were aware 
of this sentiment from the onset and made efforts to use language and processes that were familiar to 
NGOs and government organizations. In addition, they spent considerable time studying each 
organization and visiting their workplaces. 
By using Business Dynamics as the facilitator of the IDF assessments with all thirteen (13) 
organizations, the result was consistency, neutrality and professionalism. HAPP partners were 
motivated to improve their management in line with their private sector counterparts. 
An important linkage was established between HAPP organizations and this agency which could 
develop into future non-profit and private sector collaborations. 
Lesson Learned: 
Utilizing the private sector in organizational assessments (as HAPP did with the contracting of 
Business Dynamics) contributed to the professional completion of 13 rounds of 4-day IDF assessments 
within a time period of five months. The results of the assessments included quantitative and 
qualitative documentation as well as action plans for each organization. 
Findinrz: 
A considerable concern coming from the IDF results was the varying degrees of professional 
leadership and management capabilities of directors and managers in HAPP partner organizations. 
Organizations with less participatory management practices showed slower development progress. 
The concern was: How to support those leaders who have developed effective leadership skills and to 
strengthen leaders which have been ineffective, thus limiting the development of their organization? 
Lesson Learned: 
HAPP - Internal Evaluation Report 
Because effective leadership/management is believed to be essential for institutional growth, a 
follow-up activity to the IDF assessments was planned. A 3-day leadership training for each HAPP 
organization to address the needs of ineffective leadership styles and ways to maintain healthy 
leadership behaviors was offered in each of the three HAPP sites. The leadership training was both 
skill and knowledge-based building on the findings of each organization from the IDF. The leadership 
training was made available for up to four participants of HAPP partner organizations who worked 
in positions as managers and supervisors. 
Finding 
HAPP partner organizations are very much aware of the ongoing need for institutional growth and 
development. Sustainability of an organization is paramount to the organization's survival. When 
organizations are dependent upon international grants or other funding sources, having an 
"ath.active'' organization with a documented track record is essential. The IDF assessment facilitated 
under HAPP provided two sets of scores which furnished a systematic way of describing the growth 
of an organization over time. 
A repeat IDF Assessment was completed for HAPP IAs in July 2000. 
Lesson Learned: 
Systematically administered IDF assessments can document the growth of an organization over time 
and strengthen an organization's bargaining power with donor agencies. 
J. Donor Coordination Bullet Points 
. In the last six years, regular quarterly coordination meetings chaired by WHO/UNAIDS 
enabled the sharing of information where UNAIDS facilitated small working meetings. HAPP 
demonstrated an effective coordination with AusAID's HIV/AIDS Prevention and Care 
Project at many levels, which allowed the hvo projects to benefit from Lessons Learned and 
ongoing strategic planning. There was very good collaboration behveen AusAID and CDC on 
STDs and with HAPP on the BSS. AusAID saw the hvo projects as complementary as RII's 
approach offered greater focus on high-risk groups and AusAID's approach was a greater 
focus on institutional strengthening. 
. Building partnerships among donors, based upon respective strengths and common issues 
and for a common goal has proved to be effective. The IDU Rapid Assessment was a good 
example of a collaborative effort WHO should be seen as having technical expertise with 
high level Government assistance and USAID/HAPP should be seen with technical expertise 
and with the ability to implement in the field. 
Lessons Learned : 
Overall, collaboration and coordination behveen HAPP and AusAID's HIV/AIDS prevention 
project improved with time and effort. 
. Collaboration is best operationalized around a common goal or activity. In the future, 
collaboration and coordination behveen AusAID and USAID projects are essential. The 
model of collaboration should be on issued-based training and coordinated geographic 
coverage. 
Recommendations : 
HAPP - Internal Evaluation Report 
. Coordination between two projects should be both in geographical placement of the Projects 
(e.g. Irian Jaya &Jakarta) and in issue-based collaboration in assessments, training and policy 
discussions. 
. In the future, AusAID sees that the majority of the work should be with a provincial focus 
with locally tailored strategies and responses. 
hXPP - Internal Evaluation Report 
( Evaluation Report 1 X. Beneficiary Involvement 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
ID. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
WI. Working With 
NGOs 
IX. Building Support 
Networking and 
Coordination 
X Beneficia'y 
Involvement 
XI. Technical Issues: 
XII. Surveillance 
XIII. Policy Support 
A. Beneficiaries 
' 
1 
1 
: 
( " 
c 
1 
\ 
C 
t 
r 
Issues : 
Apart from €he information received from outreach workers, 
all target audiences (CSWs, sailors, youths, waria/gay men, bck 
drivers and dockworkers) also gained information about 
STD/HIV/AIDS from mass media or extension work provided by the 
government agencies. Discussion and interpersonal communication 
with outreach workers, however, gave the opportunity to conduct two￾way communication to clarify some misconceptions and uncertainties, 
as well as to discuss personal and sensitive matters. 
Good relationships between outreach workers and 
beneficiaries, which was established through regular communication, 
:ed to openness to discuss concerns with regard to HIV/AIDS and 
;exual health on the part of the beneficiaries. An example was that the 
,pen communication allowed the provision of condom negotiation 
;kills, which, in turn, encouraged CSWs to discuss !+TI3 issues with 
:lien& and to use condoms with them. Some beneficiaries also reported 
ncreased awareness about their high-risk behaviors (CSWs and 
varias) and some behavior changes such as the willingness to buy 
:ondorm themselves if they run out of them. Other beneficiaries 
:]aimed increased empathy with people with HIV and felt assured that 
hey would still be accepted even if they were infected with HIV (gay 
nen). 
I I 
. Some groups of people with high-risk behaviors felt 
uncomfortable about using public health services, but reported satisfaction with health 
services provided by NGOs, such as the clinic for waria. Favorable features of this type of 
clinic reported by the beneficiaries were that they: (a) were friendly and non-judgmental; @) 
were affordable (free examination and treatment); (c) were accessible; and (d) had credible or 
good quality of services (including counseling services). Their good experience with the clinic 
encouraged them to bring their friends to have STD check-ups. 
. OveraU, all groups of beneficiaries stated that involvement with HAPP-funded NGO 
programs enhanced their self-esteem, pride and self-worth because they were treated with 
respect, were made to feel useful and fuUy part of the Project, which they did not receive 
from the community. One example was that NGOs gave them the opportunity to become 
peer educators and resource persons at seminars or training, which permitted them to build 
networks with people from diverse backgrounds. 
. Sex workers never used condom with their boyfriends as a symbol of their special 
relationship and trust. 
. The integration of some income generation activities or vocational training, which were 
perceived as beneficial by target audiences, contributed to the sustainable involvement of 
some beneficiaries in the Project. 
HAPP - Internal Evaluation Report 
. All beneficiaries reported that their friends were sources of information about reproductive 
and sexual health, including relevant health care, as well as sources of encouragement for 
their behavior changes. Their ability to communicate about these issues were facilitated by: 
(a) the real care and concern shown by their friends; and (b) the accessibility of their friends. 
. In some HAPP-funded NGOs, regular meetings for peer educators organized by NGOs 
became a starting point to organize themselves into a new organization focusing on 
HN/AIDS and STD-related issues. 
. Based on the experience of some NGOs, the provision of transportation money to some peer 
educators led some commercial sex workers to leave their profession, as they now perceived 
their peer education activity as a job. 
. Street CSWs were reluctant to admit their profession because they were worried their family's 
would find out about it. They could not be fully involved in various peer education activities 
such as attending meetings, serving as resource persons, carrying condoms and taking IEC 
materials home. 
. Some owners of commercial sex establishment explained that the involvement of CSWs in the 
HAP Project was beneficial for their business as they gained the reputation of providing 
'clean' services and their clients were willing to pay more. 
Lessons Learned : 
Outreach Workers needs to have good communication and simple counseling skills (see 
section on Outreach Activities - BCC). 
In view of the flexibility of NGOs, they are more effective at providing health care services for 
marginalized groups of the community, especially if they manage to deliver client-centered 
(friendly and non-judgmental), affordable, accessible and credible or good quality services. 
Once a group of the marginalized community is mobilized through establishment of their 
association, for instance, they can be effective at reaching out to more marginalized people 
(e.g. Bandungwangi). 
Boyfriends of coniniercial sex workers play a critical role in the adoption of safe sex; 
accordingly, future intervention should consider this factor. 
Income generation programs can contribute to the sustainability of HIV/AIDS prevention 
programs. 
Integrating financial incentives into peer education activities can lead to the perception that 
peer education is a 'real' job, hence an alternative job to commercial sex work. 
Associating 'clean services' with good business can he used as an advocacy message to solicit 
support from owners of comn~ercial sex establishment for STD/HIV/AIDS prevention 
programs. Concerns about the clienfs health through the provision of 'clean services' can be 
used to negotiate condom use with clients and to enforce the adoption of 100% condom use in 
a commercial sex establishment. 
Peer education is a potential means of influencing behavior change among all beneficiaries, 
particularly if peer educators can show real care and concern to their peers and be accessible 
to them. 
NAPP - Internal Evaluation Report 
B. Bandunmangi 
From Sex Work to Self Help 
Bandungwangi is an organized group of community members who have had legal identification as a 
full-fledged organization since June 1999. They were formed with the help of Yayasan Kusuma Buana 
(YKB), a non-profit organization providing family health care through community-based health 
services. Under HAPP, YKB focussed on outreach services to brothel-based commercial sex workers, 
pimps and brothel managers in Kramat Tunggak, North Jakarta. This successful empowering of 
commercial sex workers to form their own organization for H~~/AIDS prevention activities is the 
first one of its kind in Jakarta. 
YKB trained a number of the sex workers in peer education and designated them as peer educators 
(PE). Outreach workers (ORW) were then recruited from among the PEs and trained and paid a 
monthly salary. 
From among the many advantages that the members of the Bandungwangi gained through this 
experience, the most useful were: 
The opportunity to become ORWs, which further gave them the chance to become facilitators 
and resource persons for future training programs. 
. The opportunity to build networks with people at all levels in the community and outside. 
. The opportunity to network with commercial sex workers in other areas, through national 
and international study tours. 
. The chance to undergo condom negotiation training, which has since been of help. 
. Knowledge for preventing themselves from getting STD/HIV/AIDS and the ability to 
disseminate this information and skills to their peers, thereby creating a domino effect. 
. The increase in their self-esteem and self-image, particularly because of the respect they got 
from YKB throughout. 
And all the above leading gradually to their becoming part of the community from earlier 
being a marginalized and stigmatized group. 
Another notable achievement of the Bandungwangi was that, in 1996, even before they gained legal 
status, when they were still an informal group, YKB organized a photography course for them (this 
activity was sponsored by the Ford Foundation). The course led to them organizing two photo 
exhibitions and eventually to the production of a book entitled, "Menghadang Mentari Pun Tak 
Peduli" ("Even Facing the Sun I Don't Care") in 1997. This contained the best selection of about fifty 
black and white autobiographical photographs depicting their lives and feelings, along with poems 
written by them. The book was launched at a seminar organized by Bandungwangi where they 
invited a speaker from a similar organization in Germany. The main outcomes of this activity were 
that it gave the women the opportunity to recognize their talents, develop and express them and to 
have Bandungwangi as a group recognized, thus again increasing their self-image. 
As Bandungwangi evolved as a group and even earlier, during their interaction with YKB, the 
women felt that a number of their problems began to get solved. Some of these included: 
. An increase in their treatment seeking behavior (for example, they began to go for treatment 
for vaginal discharge, which earlier they had not). The referral system developed by YKB 
gave them a chance to use the same treatments and services as other, non-sex worker patients 
had access to. 
. The affordability of adequate STD treatment. 
. A change in their own behavior after becoming PEs towards using condoms with clients. 
. Getting the opportunity to become paid outreach workers and thereby being able to stop 
commercial sex activities. 
NAPP- IntemalEvaluahbn Report 
Notable behavior change over the period of their involvement with the Project, as stated by the 
women themselves, included important elements such as: giving up smoking and alcohol; not being 
commercial sex workers any longer; becoming emotionally and mentally more mature and secure; 
getting along better with people; self-awakening, leading to finding opportunities to help others; and 
very importantly, learning to budget and live within that budget as now their income was limited and 
regularly spaced and what they now referred to as being 'halal'. Another noteworthy change in 
behavior, as mentioned earlier, was the use of condoms with clients. 
Bandungwangi's future plans included the hope of becoming a completely independent organization, 
receiving independent capacity building with the goal of procuring funding to provide services and 
to disseminate HIV/AIDS/STD prevention information and skills to commercial sex workers. The 
Bandungwangi women felt that continuing their outreach and peer education programs was now all 
the more important since Kramat Tunggak as a localization was banned and the sex workers from 
there had now spread out into the community. And because it was now more difficult to reach them, 
STD prevalence among them had increased (BSS data). The women also felt that they would like, in 
the future, to build stronger networks and links with the government because they felt that this 
would greatly legitimize their activities and, at the same time, work towards minimizing the 
stigmatization of their group. 
As another sign of their high motivation and ambition, Bandungwangi expressed the need for 
capacity building and technical assistance for their organization in the use of computers 
organizational development, human resource development and, importantly, training in income 
generation programs as they realized the need very early for diversity in their sources of funds. 
The mission and vision of Bandungwangi as an organization was summed up in a statement made by 
one member - "Being a sex worker is not what I want to be." And Bandungwangi has given her and 
others like her, the opportunity, means and confidence to go ahead in life and be what she wants. 
C. Putri Mandiri 
An Organization and SelfHelp Success Story 
Putri Mandiri is a registered organization of commercial sex workers set up in June 1999 by Yayasan 
Investasi Kemanusiaan (YIK), a non-government organization working with communities in the 
coastal areas of North Jakarta, MK's activities include raising STD/HN/AIDS awareness among 
members of these communities which include commercial sex workers and brothel-based sex 
workers. Putri Mandiri was set up with help from YIK and the members were from among YIK's peer 
educators. Putri Mandiri was earlier the name of a newsletter developed by the sex worker peer 
educators, consisting of stories about themselves; the name was then adopted for the organization. 
Putri Mandiri, as a self-help community organization, is rather unique for a number of reasons. For 
one thing, after being legally formed, the members of Putri Mandiri took over certain YIK activities 
such as the dissemination of HIV/AIDS information. Motivation of their peers to practice safer sex, 
literacy classes and condom promotions also featured among some important activities carried out by 
Putri Mandiri. In fact, the features of their condom promotion were very interesting and innovative. 
Ever since its registration, Putri Mandiri began a direct reIationship and coIlaboration with the Task 
Forces of the different condom manufacturers. Putri Mandiri not only ensured the availability of 
condoms in this particular red light area, but also did so in a way that generated income for their 
members. They bought the condoms from the Task Forces at a price, sold then for a small profit and 
were permitted to keep 10 percent of the profit. 
Another income generating activity carried out by Putri Mandiri was a result of income generation 
training and seed money they received from YIK. They collected Rp.500 per day from each of their 
members over a period of two months. With the money thus collected, they bought sacks of peanuts 
HAPP - Internal Eva/uation Report 
and smaU containers. These containers were fiUed with peanuts and sold as snacks to clients. The 
profit collected was divided, one part going to the organization and the other to the sex workers 
themselves to do with what they liked. 
They also received vocational training which led a number of sex workers having the opportunity to 
join pubs and discotheques as dancers and singers. Apart from giving them additional income, it 
actually resulted in five of them leaving the sex work indusby completely to take on new professions 
as dancers and singers. 
The members of Putri Mandiri took pride in the fact that sex workers constantly wanted to join the 
organization. But there were conditions for doing so including: participating in aU the Putri Mandiri 
meetings for at least two months and always using condoms. There were a number of reasons for the 
organization's popularity and credibility. It was a good opportunity for meeting and interacting with 
others in similar situations and for sharing and solving problems. It offered opportunities for 
counseling, vocational training, literacy classes and participation in income generation activities such 
as the 'peanuf scheme. The members also felt that joining the activities of YIK and thereafter of Putri 
Mandiri led to a realization of their own levels of risk of STD/HIV/AIDS and of the need to do 
son~ethiig about it. 
They claimed that they now always wear use condoms with clients and because of this the SI'D rates 
among them are low. 
The further plans of the members of Putri Mandiri included the organization becoming independent 
of YIK and being in time, able to obtain support and funds to broader the scope of their present 
programs with sex workers to include not only STD/HN information dissemination but to focus 
more a vocational and income generation programs. This, they felt, would result is greater 
independence from their present profession and provide them the opportunity to enter other 
professions. 
D. Services for Suecia1 Communities 
Ikatan Ahli Kesehatan Masyarakat Indonesia (IAKMI) is a society for public health professionals. 
IAKMI is also registered as a non-profit organization providing public health activities. Under WP, 
IAKMI provides STD/HN/AIDS education, prevention and treatment services to more than 800 
waria (transvestites) in North Jakarta. Their innovative activities include a dropin center and STD 
clinic, outreach activities and education and training activities with waria. A point to be noted is that 
IAKMI is the only STD clinic for waria and gay men in Jakarta. 
IAKMI trained waria and gay men as outreach workers (ORW) for the dissemination of knowledge 
and information about STD/HIV/AIDS and condom negotiation. The waria, on the other hand, 
found this information very useful as, earlier they used to self-treat and did not use condoms since 
health was not their main concern -just the desire to earn money through sex. Condoms are now very 
popular among the waria, to the extent that waria are now willing to buy condoms from the clinic 
even though earlier they used them only if they got them free. They claim that they "usuauy" use 
condoms with clients now. 
Through the outreach carried out by the waria ORWs, the clinic has become very popular among 
their community. Over a period of time, the ORWs trained by IAKMI have become very confident 
and feel that, given the opportunity, they can even act as resource persons. 
Since IAKMI is the only clinic for waria and gay men in Jakarta, the waria expressed their concern, 
saying that, if the clinic were to close down or to stop servicing their community for any reason, they 
would perforce have to return to the practice of self-treatment. But they also said that, in this event, 
HAP' - Internal Evaluation Report 
they would, however, continue condom use for self-protection as, knowing the risks now, they would 
no longer want to put themselves at risk of STD or HN. 
They realize how fortunate they are in the facilities and opportunities available to them. But their 
longer term dream is that clinics Like IAKMI become available to the many others in their community 
- IAKMI works only in North Jakarta, whereas the largest waria population is actually in East Jakarta 
and in Bekasi. Given the success and impact of IAKMI among this group, it would be more then 
justified for others to take up the cause and make this dream come, true. 
t-tAPP- lnternalEva/rration Report 
XI. Technical Issues 
A. Technical Aspect 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP 1nh.a-Office 
Management . 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues 
XU. Surveillance 
XIII. Policy Support 
Technical assistance 
! 
solicit their support for the IAi' activities. Appointment of new 
government officials or the reorganization of geographic jurisdiction 
Technical assistance has been centralized at the HAPP Jakarta Office to 
ensure the standardization of the quality of technical assistance. 
Nonetheless, the number of technical advisors was not commensurate 
with the workloads of backtopping and supervising the considerable 
number of IAs, including those receiving RRF in the three project sites. 
Further, NGOs reported that this system posed challenges in the length 
of time needed to approve IAs' work or to trouble-shoot IAs' problems 
and ultimately in the on-time implementation of IAs' activities. In the 
absence of evidence on the adverse effects of the imbalance between the 
workloads and staffing on the quality of technical assistance, no 
discussion can proceed on this matter. 
required NGOs to hold orientation meetings again. On the one hand, the changes in the local 
government officials in the area of service delayed implementation of the IAs' activities as permits 
needed to be re-initiated and rapport re-built. On the other hand, the government officials could 
facilitate the IAs' activities. One IA reported that the government officials' interventions overcame 
skepticism of CSWs on the IA's program. 
Technical support provided by foreign technical advisors was reported 
by HAPP partners to be satisfactory. Yet the language barrier reduced its 
effectiveness. As planned, a BCC foreign advisor was replaced with 
Indonesian staff and an Indonesian STD advisor to assist the foreign 
staff. Between field visits by technical advisors, implementation needs 
support. No action has been undertaken to address this. 
Solkiting Support fiom Parties Concerned 
Some IAs held orientation meetings with local government officials to 
To formalize and ensure the smooth implementation of their activities, some IAs organized meetings 
with all partners involved and community leaders on a regular basis (e.g. local government officials 
such as the head of the neighborhood, professional organizations, neighborhood chief of the red light 
district). The benefits of gaining their parhers' support included conducting wide discussions on the 
threat of burning down brothels and reverted any problems by one IA in Surabaya with the assistance 
of the larger community. Routine meetings with CSW peer educators, the staff of the local health 
center and local village officials (local health and community officials) strengthened collaborative 
relations and allowed continued access to red light areas for outreach workers and to follow-up on 
referred CSWs. 
HAPP- Internal Evaluation Report 
Coordination 
Coordination with relevant parties could facilitate attainment of IAs' goals. YBHK networked with 
private medical doctors to get urine samples and trained health providers in requesting and taking 
samples. In 1998, the WAD and NGOs worked together from planning to implementation of 
activities surrounding World AIDS Day. Drop-in center in Bitung for STD screening and treatment for 
CSWs was equipped with effective coordination with GO1 (MOH-drugs and equipment, Depsos-bed, 
BKKBN-gynaecologic bed and services of a medical doctor). 
On the other hand, coordination also places a condition that a common goal can be reached if each 
partner delivers its commitment. FHI did not meet standard target for 1999 in terms of increasing the 
proportion of those at risk of STDs who are sewed at HAPP participating clinics according to 
standard STD diagnosis and treatment protocols. This achievement also depended on PIL 
implementation activities and drug availability, for which FHI was not directly responsible. 
Outreach Activities 
Capaciw Buildina for Outreach Workers 
High levels of burn-out among outreach workers was found but responses to outreach workers' 
needs for refreshment and upgrading both substantive and outreach skills, were limited. Outreach 
workers of some NGOs targeting CSWs had very limited condom negotiation skills and counseling 
skills. Outreach workers of some IAs (YPEKA, PKBI and YKB) did not possess adequate skills in 
persuading their target groups to seek medical care or to use the referral system. 
Male Responsibility 
The need to strengthen outreach activities to better focus on male clients was highlighted during the 
negotiation of HAPP in 1997 and continued to be an unsolved issue towards the end of the Project. 
Technical Terms 
Considerable time was needed to explain the term of repeated outreach contact. How tecc~ptLwe,& 
qLcaLity oftbintevd~~v~y? 
The Impact of the Economic Crisis and Political Instabilitv on Outreach Activities 
Pressure from some community factions to close down brothels and other entertainment places 
employing CSWs were more intensified during the reform era, which began following the downfall of 
the second President of Indonesia. The latter was partly attributed to the economic crisis that took a 
socioyolitical dimension. Together with the low number of clients, the actual or threatened closing of 
the commercial sex establishments (CSE) in Jakarta and Manado increased the mobility of CSWs to 
other red light districts or the street (YIK, YKB, YAA). Subsequently, the number of CSWs identified 
by outreach workers reduced and CSWs were reluctant to meet outreach workers. One IA redirected 
its activities to a different red light district. Other solutions adopted by other IAs were identifying 
new geographic areas of CSWs or expanding geographic target areas. Even with collaboration with 
HAP'- Internal Evaluation Report 
the CSW organizations, the IAs faced a substantial constraint on re-mapping the situation after the 
closing of the CSE resulted in the integration of sex workers into the community with false identities. 
Political instability and security problems owing to the economic crisis also caused a higher mobility 
of waria because of violence targeting their house. Recruitment of outreach workers to anticipate the 
increased mobility of this target group was carried out. In parallel, a large numbers of CSWs returned 
to their villages to participate in the election or to avoid security problems. In turn, this halted YKB's 
activities. The other compounding factor was difficulty experienced by outreach workers in entering 
activity areas that were hard hit by the May riots. 
The crisis also increased the number of migrant CSWs from other parts of Java coming to Jakarta and 
the number of transient sex workers in Manado due to the sociopolitical unrest in the eastern region 
of Indonesia. Anecdotal observation suggested that more factory workers turned to sex work. Based 
on their survey, the University of Indonesia indicated a rise in the number of younger and less 
experienced CSW after the crisis. 
Influence of Relidous Holidays and Observances' 
Two IAs reported that negative reaction from the community to commercial sex activities during the 
fasting month led to difficulties in monitoring and implementing outreach activities. 
Cavacitv and Attitude of Pimvs and/or Manaeers 
Some IAs found that pimps' ability to talk with CSWs and clients was low; some trained pimps did 
not support CSWs in negotiating condoms with their clients; and there was a lack of information 
about HIV/AIDS prevention among CSE managers and workers (bartenders, managers etc.). 
Reachine CSWs throueh Rewlar Meetings 
Monthly meetings among the pimps and CSWs using arisan practices and requiring each participant 
to invite another two CSWs to meet quotas was effective. 
Mwegayuatt.ended.mmddy wavhM~ (uvrerpecfedmLtwme) 
Reaching CSWs and Waria through their 'Professional' Organizations 
Collaboration with a sex worker association and a waria association, as well as the establishment of 
sex worker associations, facilitated the respective Us' outreach and peer education activities. 
Other Challenges 
- 
HAPP - Internal Evaluation Report 
Evidence of Capacity Building 
Peer Education 
PE Strategy 
A high level of competition among CSWs meant that they shared info with CSWs living in the same 
compound and with their guests. Information dissemination centered among PEs. Recruitment in the 
afternoon was difficult because most CSWs are outside the compound. 
Factorv Workers 
YPRCA: ~~forctw~hmm~leY&bfwovkev mo76LlQ &n*lothwvevy Zav. 
TvcLivred PEt we forctbvy wwkevy who he ve.i2d&~ kwb laid off. ~~ 
 contact^. 
Content of IEC: 
OWs messages include cost-saving benefits of condom use. 
Language: 
YEKA: CW of CSW we f..veipb aeaftww~ who- do Mot bpeak.Itdm&mb - NEED: 
IEC hdhw b+pa@& 
Development process: 
Distribution: 
HAPP - Internal Evaluation Report 
Role Model Story: waria can develop new stories but like the story with pictures - can be co￾facilitators in future workshops. 
Target group: 
IEC strategy: 
m: 
YAA: 
YIK: 
m: 
m: 
BMS: 
w73: 
m: 
Outcome: 
ww: 
low ley&ofed.ucaE~~~rofmCSWrw&~toeducati 
apprd of YAA; YAA: *eMt hela- of educatm kampwd 
wmofinfrw- 
nAPP - Internal Evaluation Report 
Planning of strategy: 
As a result of this detailed planning and regular consultations' with the GOI, the CSM activities 
contain the appropriate messages while using a style appropriate for the Indonesian community. 
Promotion of Condoms 
MESSAGES 
Condom Market 
CSM: 
The impact of the crisis caused consumer brand switching. - The highly -. subsidized ~ktra brand has 
increased its market share at the expense of commercial brands. 
Condom Promotion/Marketinz Activities 
The percentage of condoms used among female CSWs increased significantly, from 37% in 1998 to 
4856 in 1999, after three years of essentially no change. The increase of condom use among CSWs 
resulted from adequate outreach intervention activities among high risk groups, especially on 
HAPP - Internal Evaluation Report 
condom use negotiation/promotion intervention and condom distribution/social marketing. 
Currently, FHI emphasis has been on the CSWs as change agents to use condoms. In order to create 
more demand for condom use, FHI will focus the intervention more on the clients of CSWs (Annual 
performance evaluation - 24 February 2000) 
Condom Use 
YAA: free wndmw we veqw&&ecovromic wi6iY; the ~~~LWVV wgdy & 
ihdequate? 
YAA: emw,miowi6iY - mavy CSW2velucta~ttG-Wbtheiv ~tcr~condorm; 
w CSW2 s.. feaw ldq -;lew.I.9 of -- 
hwmpwea~of~iwuzyL: 
FGD mCSE mmwgwx thew&& veducedthe numbw of CSW clieuLta: CSW2 more 
COnzp&LC.e The, imp&- of'@ni&mt othevp (?)' mla the cugh,w 
that dto. htwget& 
Condom Distribution 
Referral Sysfern 
YAA: 
YAA: 
HAPP - hfemal Evajuahon Report 
YMM: 
YBHK: 
Ym: 
PmI: 
YMM: 
MANADO: 
KB: 
YAA: 
IAKMI: 
YPW: 
PKBI: 
MND: CLI m&udwstm Iretween, MND aMd BitLuq IAr covlcw- vefwvd 
w*w. 
P~omofion of the-s~~ Clinic 
WHK: hpvWmoff;eeSTD t&i ivL wuw%i with. fP sevvW veceived. 
mp~w~v~onae- appvopviate6twdid notmeeth- of 
patients-vnenopcubd wmpaVrLcipated￾CORRECTION: cledv ~ofhrw~~Pof~ 
opw-dVld-pdqe. 
Pd- vwed.1*, pavhwz Pm-stooathat* 
waLldbet&edby pap krnedv - NEED: nuwee&xdS6waMd 'to 
~~~a++eh~ofl.Oth.~CdZIp~PkivnuEt~~ 
~~~~duv~tv~ cumwu'dpap wneav if 
vqatiye STDP tet. 
NAPP - Internal Evalt~aiion Report 
STD Clinics 
Finding STD Clinics to Utxrade 
Qualihr of Service 
STD Clinic for Waria 
Facilities 
IAWI: STD clinioZacku*abylmckrupfrw%i&pectdSTD cmwa.vu%nodvq9 
fiK STD tv-. 
STD cliniovtotopthdy ut&p%(why, whidvone?), htheabmwoffiee . . -, only anl3.miceywepvovidedr 
Others: 
hXPP - Internal Evaluation Report 
Issues During Counseling 
PKBI: delay hpvweynentofpvov~ veqLLiVedfov tb~~01~,of~np~ 
fov STD gcv~. 
Policy Support 
B~whmTfAwst4-h-*and.coov-wtbpdiy 
dwitii with.tbKPftD. 
La&ofa/~e~fov~~~new state-of-theavttechnkd 
data/and.&&mHIV pv~ammim@ok.ey pdiy vnakev~dothw 
cLd'dm&qpavtwk 
po~iykuppovt: spov~.soyedtr~~~~~~~~of~s~a*~io~~ 
Leadwy mHIV/AIDS which re&.&& hthJ-Declavath. 
SBY: a/dvcLftwwkh+yplavl/foveach.oftbKPAD tecknical.wbvkh+ygvmp. 
MND: ht+wmm&ne&oftrWVLmGovwnor whsuaedtshe hIvGa.wJcya/m 
HIV/AIDS acriitii. 
KPAD SBY: fovwudded@Ld&hWfov th ~~~oflOO% covubmrw 
pdiy h+vido~mpPou veawmm. 
KPAD-JKT: W~-Sectbvd/~~~~~ith.KPAD hEaatJtWtc-and. 
v&ed NGOP Lnw& (WB, P&a: Bwaa@). 
MND: &&WOW ~OY~HIV/AIDS fvw, ~op~d nazr 
judgwwntal.~ - fd2ow~a/wieyofjaw~wbvkahoprcorrducted~ 
MOH thvmi&PIL &othwpuWowevLtyconductdby HAW IAk 
Research 
During training implementation of surveillance sentinel HN and AIDS, it was observed that skills on 
surveillance are very important and useful for Health Services staff (hospital and health center). Many 
medical and paramedical staff do not understand the procedure of Sero testing for HIV and the 
diagnosis of HIV. We believe that those training will improve the quality of HIV/AIDS data. 
Mass Media Strategy 
NAPP - Internal Evaluation Report 
CEDPA withdrew its interest in working with HAPP after 18 months' preparation, two technical 
visits and a completed technical design. This led to time lost to strengthen the KPAD, partners and 
staff in gender issues because of the withdrawal of CEDPA. 
Impact of the Economic Gisis 
Knowledge of Acceptable Ways to Prevent HIV Transmission Among CSWs 
B. Behavior Change Communication 
Findings : 
While many agencies involved in HIV/STI prevention have adopted the new terminology, most still 
view BCC as 'IEC materials' that promote awareness and provide basic information. Supporting this 
shift created a major challenge related to the limited local material and human resources for building 
technical skills. Identifying needs, designing locally appropriate approaches, developing training 
materials, identifying agencies able to assist, strengthening human resources and mobilizing other 
needed resources were all part of an ongoing process. 
Identifying skilled agencies able to assist with management capacity building proved easier than 
finding resources for building technical capacity. (See Capacity Building and Working With Partners) 
Various strategies have been proposed or implemented. At the beginning of HAPP U, it was 
suggested that each of the technically-oriented PVO partners should budget for a% of technical staff 
time to contribute to a review and response team. This was not taken up. One PVO which had 
provided important support to technical capacity building changed management and the focus of 
their proposed contribution to HAPP. Working together with other donor programs helped to 
increase resources and also to develop a more integrated understanding of behavior change. 
Quality control of BCC activities and materials was a continuous challenge. Capacity among many 
agencies proposing activities was low. While provincial staff were not hired or trained as technical 
experts, they attempted to provide as much support as possible. However, the BCC advisor was 
HAP'- Internal Evaluation Report 
accountable for technical quality. Attempts to review and respond to activities and materials 
proposed were constrained by the high level of need, the limited BCC staff and the size of the 
technical workload. One possible solution to the need to provide consistent, ongoing assistance had 
been proposed at the beginning of HAPP 11; it was that a technical review and response team be 
established comprising technical personnel from each of the technically-oriented partners. It was 
suggested that each could budget for a % of staff time for work as a part of the team. This was not 
taken up. 
Behavior Change Communication Skiffs 
Most activities and materials developed by IAs and during early stages of HAPP focussed on the 
awareness/knowledge stage of behavior. This was appropriate at the beginning of the Project, but 
there was a great need to help IAs and other agencies to refocus efforts on behavior change rather 
than awareness. 
A Behavior Change Communication training curriculum based on behavior change models did not 
exist when HAPP started. Most training was not skills based and generally consisted of 'expert 
Iectures.' Trainers skilled in adult learning methods and capable of assisting IAs to train 
communication workers (outreach workers and peer educators) in behavior change were rare. 
Trainers who could assist IAs to develop IEC materials with a specific audience and behavior change 
objective were extremely limited. In collaboration with one HAPP partner, a BCC curriculum was 
developed and field tested through a participatory process. Participants were trained in adult 
learning methods as part of the process. A continuation was planned, but the management of the 
partner agency which had facilitated the first set of activities changed. 
The curriculum is still being used by some individuals and agencies who were involved in the 
development and testing. Users expressed enthusiasm about the curriculum and were able to make 
concrete suggestions for revisions. A collaborative revisionlupdate is underway with AusAID. 
Another major challenge to the BCC program was the unavoidable early departure of the BCC 
advisor in early April 1998. There was a gap of 4 months before a replacement could be recruited, 
resulting in a large backlog of work. The new BCC Advisor had to assume full responsibility without 
the benefit of the orientation and handover that had originally been planned. 
IEC Materials 
HAPP provided funds within subagreements for materials development. Due to the workload of the 
BCC Advisor, the ability to provide direct TA as needed at field level was limited. To provide basic 
materials, HAPP focussed on materials for STD clinic attendees and women attending reproductive 
health services for the treatment of RTIs, female sex workers and doctors providing RTI/STD 
services. Additional support was provided for the development of materials for male transport 
drivers, female sex workers, waria, fishermen and others. 
Although HAPP required that the IA submit plans in order to provide TA and quality control, this 
was often not done. 
Training in IEC development was provided by HAPP through a consultant contract. Almost all IAs 
interviewed stated appreciation for the IEC materials, but emphasized such training needed to be 
provided earlier in a project. Inadequate distribution systems for IEC materials were an issue for 
HAPP and for NGOs. Training in collaboration with another donor project, with guidance from the 
BCC Advisor addressed these issues. Over the life of HAPP, many agencies have been able to 
advance their understanding and ability to meet audience needs. Adding staff to the HAPP BCC unit 
helped as did the training in dissemination provided by DKT. 
IIAPP- IntemaJEvaJuation Report 
Role model stories, a new communication approach based on behavior change theory and focussing 
on modeling actual behaviors rather than on facts. was introduced and tested among waria. The 
group was extremely enthusiastic. An international consultant who was involved in development and 
testing this process was contracted to work with HAPP BCC staff and others. The challenge was to 
adapt a sophisticated and proven model so that it would be feasible - and still retain quality in 
Indonesia. 
Outreach workers' complaints that, once IEC materials were distributed, they were unable to answer 
questions stimulated by the materials led to the development of an SID field handbook for outreach 
workers and Peer Educators (non-clinical field workers). The booklet, accompanying male and female 
'body maps' printed on durable fabric were contained in a 'belly bag' which had space for condoms 
and other materials. These proved very useful to these workers and contributed to their confidence 
and credibility with audiences. 
Mass Media 
To support requirements for mass media communication targeting general population, HAPP 
collaborated in the production of radio dramas, spots, TV dramas and TV spots and the training of 
media personnel during the first project period. Due to a shift in audience and the achievement of 
high levels of knowledge re. two ways to prevent HIV infection, less emphasis was given to mass 
media during the next project. Instead, the focus was on building a supportive environment for HIV 
prevention. An innovative approach was contracting with a public relations firm which was able to 
ensure positive publicity and widespread disseniination for all program activities. 
The CSM component of HAPP is covered in section X. The BCC Advisor also provided significant 
input into the development of the mass media component of the condom social marketing program 
implemented by one HAPP parber. 
Lessons Learned : 
. IEC materials are a support tool for use in an overall behavior change intervention. The 
differing geographic sites, characteristics of the different audiences and audience segments 
contributed to the high number of variables to be considered when producing behavior 
change messages, materials and approaches. Helping NGOs to move from a focus on 
distributing IEC materials to being able to plan and manage an intervention aimed at 
behavior change is an enormous challenge. The ability to conceptualize, plan and implement 
behavior change communication as a part of an ongoing set of activities with spdc 
behavior change objectives still needs high levels of ongoing capacity building. 
. A combination of directly providing basic IEC materials, building IA capacity building in 
materials development and on-going quality control was needed in order to ensure that 
quality materials were designed appropriately for each audience and audience segment. 
NGOs still need help to understand that IEC materials are not a substitute for interpersonal 
communication. 
. Generic approaches and materials were useful among some audiences, but additional 
training in behavior change cornmunication skills, adaptation of existing materials and the 
role of IEC materials as a support to behavior change are still needed by NGOs and others. 
. Early development of a capacity building plan based on previous experience of needs, 
resources and time needed should improve the ability to sequence and implement a balanced 
program of technical capacity building activities. Ideally, capacity building is provided in 
response to the current needs of each NGO or government program. However, the scope of 
HAP' - Internal Evaluation Report 
the needs and the limited resources require compromises. Additional resources for providing 
technical assistance, quality control and capacity building are still needed. 
. While a resource center with a full inventory of HAPP-produced and non-produced 
Indonesian IEC materials was suggested, many materials are not appropriate except among a 
limited audience and reprinting small quantities based on a localized need is not cost￾effective. Training IAs to produce or adapt materials for local use, training in using materials 
effectively and building capacity for other, non print-based, interpersonal communication 
approaches is likely to be more useful. 
. Developing new models to address behavior change beyond stages of awareness and 
knowledge appears to be possible. Introducing the approach to NGOs and others will require 
adequate human resources to ensure that quality is maintained while the role model story 
intervention is being adapted and trialed in Indonesia. 
Peer Education: 
A variety of Peer Education programs were established within HAPP - Peer education programs 
evolved differently depending on the management and the beneficiary group. During the evaluation 
interviews and in their regular reports, NGOs expressed many lessons they had learned about the 
recruitment of peer educators, what can reasonably be expected from PEs, the challenges of Peer 
Educators' work, the support needs of Peer Educators and how these can be met. 
A workshop/consultation has been held to facilitate gathering Lessons Learned through these 
experiences. The emphasis was on the management of peer education programs within a behavior 
change framework. The outcome of the workshop is being written up as a practical guide to peer 
education programs for Indonesia. 
Initially, the focus of peer education was limited to providing information to peers. Later, selected 
peers (based on condom negotiation skills) participated in a workshop to build training skills and to 
develop a condom negotiation training program. This program was then presented in each of the 
communities from which the participants came. The activity was perceived as very successful and 
contributed to increased self-esteem and confidence among the peer educators. 
The transformation of peer educator groups into self managing 'self-help' groups was another very 
encouraging finding. Several groups noted that becoming peer educators and having regular 
meetings and support provided a starting point for organizing themselves. The earliest of these is the 
Bandugwangi group of female sex workers developed by and with YKB. This group was able to 
continue work despite the closure of Kramat Tunggak brothel area which caused sex workers to 
scatter and greatly increased the ability to work with them. 
Several other groups formed: YIK assisted with development of Putri Mandiri, comprising female 
street sex workers and GemtahC - a group of youth. Peer educators working with YAA (calling their 
group 'Harapan Sejati) also suggested that they could form their own NGO. Mamis (female pimps) 
have formed a group called 'Sudi Asih.' Several of these groups are carrying out small income￾generating activities. 
Lessons Learned : 
. Improved management and understanding of peer education programs is essential to using 
this interpersonal approach effectively. Drawing on a combination of international and local 
experience and providing a framework to ensure effective 'operational' analysis provided a 
valuable and practical resource for Indonesia. 
NAPP - Internal Evaluation Report 
. Support to capacity building activities for peer educators can lead to 'empowermen? and 
decreased dependency on the facilitating NGO. While these groups are new in Indonesia, 
Peer Educator-led organizations have played a valuable role in other countries; providing a 
cost-effective approach expanded the reach to many of their peers. The increase in self￾confidence and skills has aUowed some to find other employment. It also seems likely to 
contribute to the ability to negotiate for safer sex. Independent self-help groups also 
contribute to expanded reach among beneficiary populations and have high potential for 
contributing to the sustainability of activities. 
Outreach 
All non-clmical HAPP-supported interventions included an outreach component. Outreach staff 
characteristics were highly variable. Many programs hired people similar to, but not from, beneficiary 
groups so that they might more easily communicate with the audience. Others hired young people, 
often university student age. Still others hired outreach workers from within the beneficiary groups. 
Hiring 'proven' peer educator volunteers as outreach workers was also recommended as this would 
ensure understanding of the work and audience and tended to increase the credibility of the 
organization among the beneficiaries. 
Benefits of outreach identified during the NGO and beneficiary interviews included: 
. Familiarity with audience, ability to reach marginalized audiences 
Helping to build trust and good relationships with beneficiary groups, more members of the 
audience actively sought help and information from them and their organization 
. Able to reach work with many members of marginalized groups 
. AUows for one-on-one communication and 'helps in behavior change' 
. Able to provide and help with referrals for needed services 
. Provide support for Peer Educators in the field 
. More able to monitor progress: 'condoms are becoming more popular with OR," " they are 
now willing to buy condoms.' 
For the outreach workers themselves it was noted that: 
. The work increases ORW experience and self confidence 
. Based on their experience, they were able to help develop projects for the future 
. They are utilized as speakers for other organizations, building credibility of their program 
and for the prevention messages they share 
. They were able to act as advocates with government and others on important issues such as 
100% condom use and against brothel closure. 
Challenges: 
. For ORWs from other populations and backgrounds, it may be difficult for them to communicate 
effectively about sexual issues. 
As yet, there are no Indonesian guidelines for outreach, although that was the intent of an early 
collaborative workshop implemented in Yogyakarta. As with peer education, inadequate capacity 
for the management of outreach may contribute to problems such as high levels of bum-out 
among outreach workers and the limited responses to outreach workers' needs. 
HAPP - Internal Evaluation Report 
For refreshment and upgrading both substantive and 'outreach skills' were limited. Outreach 
workers of some NGOs targeting SWs had very limited condom negotiation skills and counseling 
skills. Outreach workers of some IAs (Yl'EKA, PKBI and YKB) did not possess adequate skills to 
persuade their target groups to seek medical care or to use the referral system. 
. Economic crisis: Pressure from some community factions to dose down brothels and other 
entertainment places employing SWs were more intensified during the reform era, which began 
following the downfall of the second President of Indonesia.,The latter was partly attributed to 
the economic crisis that took a sociopolitical dimension. Together with the low number of clients, 
the actual or threatened closing of the commercial sex establishments (CSE) in Jakarta and 
Manado increased the mobility of SWs to other red light districts or the street (MK, YKB, YAA). 
Subsequently, the number of SWs identified by outreach workers reduced and SWs were 
reIuctant to meet.outreach workers. One IA redirected its activities to a different red light district. 
Other solutions adopted by other IAs were identifying new geographic areas of SWs or 
expanding geographic target areas. Even with collaboration with the CSW organizations, the IA 
faced a substantial constraint to re-mapping the situation after the closing of the CSE resulted in 
the integration of sex workers into the community with false identities. 
. Political instability and security problems owing to the economic crisis also caused a higher 
mobility of waria because of violence targeting their houses. Recruitment of outreach workers to 
anticipate the increased mobility of this target group was carried out. In parallel, a large number 
of CSWs returned to their villages to participate in the election or to avoid security problems. In 
turn, this halted YKB's activities. The other compounding factor was difficulty experienced by 
outreach workers in entering activity areas that were hard hit by the May riots. 
C. Condom Necotiation Training 
Getting condoms on clients has proven to be one of the greatest challenges of HIV/AIDS prevention 
programs for sex workers. The HIV/AIDS Prevention Project (HAPP) has recently designed and 
supported a training approach which draws on the expertise of the real experts: sex workers who 
know how to get the job done right. 
HAPP partner NGOs working with female and transvestite sex workers were asked to identify their 
'condom use stars' among their peer educators. These peer educator experts were asked to volunteer 
to help develop and to participate in a master trainers' course during a week-long program 
implemented by Lentera, a Yogyakarta-based NGO, through a contract with HAPP. Additional 
support was provided by the Condom Social Marketing Advisor from another HAPP partner, The 
Futures Group. 
The first day of the training focused on development of content; based on their real life experiences, 
the participants identified common problems and shared strategies on how to overcome client 
resistance without losing their business! Lots of creative ideas were shared through discussions and 
several enlightening demonstrations. The second half of the workshop focussed on the development 
of adult learning and active training skills, with plenty of time for discussion and practice. The high 
level of enthusiasm lasted throughout the week. The training ended with roleplays to demonstrate 
newly acquired skills. 
During the follow-up phase, the new master trainers will provide the condom negotiation training to 
sex workers in their own areas. Assistance will be provided by Lentera trainers to support and 
reinforce the new role. 
The greatest challenge during the training was learning the unfamiliar participatory training skills. 
But, by the end of the training, it was clear that more than just skills had been developed, as 
HAPP - Internal Evaluation Report 
participants expressed new self-confidence. Several of the new master trainers voiced interest in 
working professionally as HIVIAIDS prevention outreach workers in NGO projects. In the words of 
one participant "I am sure now I can do something beyond being a sex worker." 
Whatever their profession in the future, the 'graduates' of this program are certain to provide a 
valuable resource to their peers and to HIV/AIDS prevention efforts. 
Condom Social Marketing Component BuUet Points 
Prior to the economic crisis, most private condom companies spent a small proportion of their 
expenditure on marketing condoms in Indonesia for numerous reasons. At the same time, 
there was little demand for condoms in the general population and a large proportion of the 
market was occupied by highly-subsidized condom brands. During the economic crisis, these 
companies further cut down their spending on marketing in light of a sharp increase in 
production costs and a plunge in people's purchasing power. For the same reasons, donors 
that subsidized some condom brands increased their support to non-profit condom 
companies and the Government (through grants) with a view to meeting people's needs. This 
led to difficulties for local private condom companies to survive. HAPP aimed at assisting 
condom companies to create condom demand and sustainable marketing for condoms in the 
context of HIV/AIDS prevention. 
One of the marketing strategies adopted by HAPP was the establishment of Special Task 
Forces to bridge the gap between suppliers and retail outlets as well as to create non￾traditional outlets for condom. This Task Force actively distributed condoms at strategically 
convenient points to the target audience including gatekeepers of the commercial sex 
industry (e.g. owners or managers or staff of brothel, mama-sans and pimps) as well 
"disguised" brothels (bars, hotels, massage parlors, discos and other entertainment 
establishments). As a result, improvements in the availability of condoms in non-traditional 
outlets such as discos, bars, wismas and kiosks were observed. Stickers conveying symbols of 
condom brands and/or messages on ways of preventing STD/HN/AIDS were found at over 
50% of the non-traditional outlets. This implied acceptance of non-traditional outlets as places 
for selling condoms. 
Capacity building activities for the Special Task Force included educating them about 
STD/HIV/AIDS and condom-related issues to enable them to deal with myths and 
misconceptions about them. The activities also encompassed salesmanship training (self￾motivation, sales careers, teambuilding, the psychology of selling, team creativity, 
merchandising success, merchandising simulations, retail distribution and self-assessment), 
condom social marketing, NGO activities and the dynamics of the commercial sex industry. 
Finally, the Task Forces required ongoing supervision and technical support. 
The staff of the Special Task Force were employed by the respective condom companies, even 
though their salaries came from HAPP funding. This fostered staff loyalty to the companies 
and a sense of belonging of the Task Force on the part of the companies. All the companies 
recognized the benefit of having the Special Task Force; in fact, one company maintained the 
Task Force. 
Partnership between the Private Sector and NGOs 
. Within the context of the HAP Project, partnership between the private sector and NGOs was 
fostered through the establishment of the Task Force and the implementation of joint 
activities, which provided opportunities to benefit from each other (e.g. learning from each 
HAPP- Internal Evaluation Report 
other, filling in each others' gaps). The partnership was further reinforced by recognition of 
the effectiveness of the low budget promotional activities conducted by NGOs. 
. The creation of the Consortium of Concerned Condom Companies encouraged collaboration 
among private condom companies, both local and multinational. 
. In view of the benefit of the promotional activities for their products, the managers of the 
Consortium manufacturers have invested considerable time in HAPP by attending countless 
management meetings and going on field trips to the HAPP demonstration areas. These 
managers have been very "hands on" in managing program activities and in overseeing the 
selling, merchandising and promotional activity of their field Task Forces. They have also 
proactively collaborated with NGO partners. 
The intensive KondoMania promotion contributed to: (a) strengthening collaboration among 
the three Consortium members through regular meetings held in conjunction with 
promotion; (b) building relationship among the Task Forces of different companies; (c) 
building collaboration between NGOs and the condom companies; and (d) the adoption of a 
"generic" mass media campaign. Field experiences showed that it was easier to find Durex 
and Simplex Task Forces at a local NGO rather than at the respective distributor office. 
Marketing Strategies 
Marketing strategies adopted by HAPP comprised the following steps. They were: (a) the 
creation of brand image, targeting the general population through the implementation of 
various promotional activities, which used different forms of mass media; @) assurance of the 
availability and accessibility of condoms to the target audience; and (c) maintenance of the 
brand image through intermittent promotional activities. A good illustration of the first step 
was the implementation of the three-month KondoMania promotion. The KondoMania 
promotion evidently had a demonstrable impact on condom sales and condom use. As a 
result, consortium brand sales increased by over 30% during the three-month promotion 
period over the previous three-month period. Moreover, KondoMania and other CSM 
activities made a sigruficant contribution towards increasing condom use in the 
demonstration areas, as indicated in the 1999 BSS. 
. The image of condoms has been controversial and associated with illicit sexual activity. For 
instance, it was not permitted to present condom television advertising before 930 p.m., 
although having condoms promoted on TV at all is a landmark. Although, in the 1980s, the 
social marketing of the "Dua Lima" condom brand successfully positioned it as the family 
planning brand, research (IDHS) showed that less than 2% of family planning acceptors use 
condoms. Thus, the primary use of condoms is indeed for STD prevention and, inevitably, 
condom promotion has been associated with the promotion of illicit sex. The HAPP-funded 
Condom Campaign was designed based on: (a) consumer research findings (a series of focus 
group discussions), targeting people with high-risk behaviors and the general population; (b) 
consideration of sociocultural norms and Ministry of Health policies; and (c) consultations 
with the Ministry of Health, USAID and the Consortium of Concerned Condom 
Manufactures. When it was launched, it received very little negative reaction from the 
community at large. 
In light of the controversial nature of condom promotion, the need to focus on people with 
high-risk behavior and the limited financial resources, a "media tunnel" was selected as a 
marketing strategy. The purpose of the "media tunnel" was to bombard clients and 
commercial sex establishments with messages aiming at: (a) increasing their perceptions 
about their risks from STD/HIV/AIDS; and (b) mainstreaming understanding of the benefits 
WP - Internal Evaluation Report 
of condoms as a protective device. The "media tunnel" was carried out by blanketing the 
entire red light area with a variety of promotional and IEC materials including posters, 
stickers, hanging mobiles, banners and brochures. These materials were affixed and placed in 
bars, wisma, massage parlors and retaii outlets, including cigarette kiosks (find photos). 
. At the beginning, non-traditional retailers were reluctant to stock condom because of various 
factors such as religious taboos, young family members managing the outlets and fear of the 
police. One of the most effective strategies used to deal with this was offering consignment 
mechanism. Moreover, these outlets were serviced periodically mainly to ensure that the 
required inventory was maintained to meet the consumer demand and required stock of 
particular brand were prominently placed to catch the attention of the customer. 
Lessons Learned: 
Donors with different visions, missions and mandates and the absence of a common strategy 
to promote condom use in Indonesia can threaten the sustainability of condom demand and 
marketing for condoms. 
In the context of STD/HIV/AIDS prevention among people with high-risk behaviors, the 
formation of Special Task Forces to bridge the gap between suppliers and retail outlets as well 
as to create non-traditional outlets for condoms can be effective. It requires capacity building 
of the staff and supervision and is facilitated by collaboration with those with good 
relationships and experience of working with the target audience, such as NGOs. 
An innovative and new strategy to promote condoms should be integrated into the existing 
system of a private company to ensure the sense of ownership of the company. 
Partnership between private sector and NGOs as well as between private companies can be 
fostered through well-planned activities designed to achieve a common goal through a 
process that is mutually beneficial for all parties involved. 
Condom campaigns must address public health issues while remaining sensitive to religious 
and cultural issues. While emphasizing the use of condoms (through mixed media), care 
should be taken to target the myths and stigmas attached to them. One way is to pre-test 
condom promotion strategies and drafts of material not only with the target population but 
also with opinion leaders and policy makers. Perceptions of what may be offensive are not 
always accurate. At times, the public is more accepting of challenges to convention than 
condom promotion planners give them credit for. 
Sales of condoms can be increased by applying comprehensive and appropriate marketing 
strategies that include components of market creation and maintenance through promotional 
activities and supported by assurance of the accessibility of the product at the retail level. It 
will be time-consuming and labor-intensive. It should be realized that the market for condom 
brands produced by members of the Consortium has been created and further maintenance is 
required to safeguard its sustainability. 
A consignment mechanism can develop a positive attitude among retailers towards condoms, 
especially for non-traditional outlets (for the first trial). 
In a situation where condom promotion is controversial, people with high-risk behaviors 
should be targeted given the low prevalence of HIV/AIDS and the limited financial 
resources; constructing a 'media tunnel' around the red light areas can be considered a cost￾effective strategy. 
Collaboration with Public Relations in Promoting Condom Use 
Introduction 
In recognition of the importance of creating a supportive environment for promoting HIV/AIDS 
prevention activities, including the promotion of condom use as one of the prevention methods, 
HAPP collaborated with several public relations agencies to conduct a number of activities. These 
included the dissemination of information to journalists, the dissemination of press releases, a feature 
writing competition, the publication of newsletters and monitoring the coverage of issues pertinent to 
HN/AIDS in various forms of mass media (e.g. audio, audio-visual and printed media). 
Issues : 
Overall, collaboration with public relations agencies increased the number of articles printed 
in the media, although significant improvement was more noticeable around major AIDS 
related events such as World AIDS Day and The International Candlelight Memorial. 
. The presence of an agency responsible for monitoring the content of media coverage on 
HN/AIDS issues, especially with respect to printed articles, facilitated adoption of damage 
control measures. An illustration was that in dealing with the publication of an article that 
hampered promotion of condom use (e.g. incorrect information about condoms), the PR 
agency approached resource persons to respond to the article by providing the correct 
information. 
. Collaboration with PR agencies was initiated by educating them about the Project and about 
other pressing or emerging issues. In spite of these time-consuming and labor-intensive 
efforts, they needed ongoing dissemination of HIV/AIDS issues. The training resulted in the 
use of less judgmental language, the mention of the word 'condom', rather than a euphemism 
and the mention of using condoms as a means of HIV/AIDS prevention. Nevertheless, they 
did not fully manage to stimulate readers' concerns about HN/AIDS and condoms, as 
shown in the absence of any heated debate about it. In addition, adoption of the sensationalist 
approach still persisted. This could be attributed to the general policy of the media. 
C-: rn lm+ cmztahtik@ tevw LWA k$we and aftw tv- fh jawnabfa 
(pc.%a* correct and irzccwrect tww), m exa4qb of a/ hurt $'3nmtzd a.4-E& 
~HIV/AIQS vww~axtii. Or w~of~ional.~~ 
Lessons Learned : 
. PR agencies are useful in facilitating the publication of project-related news and in 
strengthening coordination/relationships with journalists and opinion leaders and helped the 
Project staff to deal with media appropriately. However, generating on-going coverage of 
HIV/AIDSrelated issues requires persons with writing and technical skills and who are 
deeply concerned about HIV/AIDS. This latter element is what is sometimes difficult to 
foster. 
. In view of the sometimes potentially harmful approaches of the coverage of HIV/AIDS 
concerns adopted by decision-makers in the mass media (e.g. chief editor), advocacy activities 
aiming at mass media organizations and the higher level decision-makers of such 
organizations (e.g. the media producers), are essential. 
HAPP - Internal Evaluation Report 
. Journalists can be educated and trained in technical issues concerning HIV/AIDS and can be 
sensitized to respect human rights related to HN/AIDS issues through training and regular 
dialogue. 
. Monitoring of articles published in the mass media is useful to conduct timely damage 
control measures. 
E. STD Services 
Ensuring that HAPP target populations had access to appropriate, acceptable, high quality STD 
services remained a challenge throughout the Project for various reasons. Various quality 
improvement activities were undertaken with a number of government and NGO facilities, focussing 
primarily on improving counseling and access for higher risk audiences. 
The HAPP STD Advisor's contract as a US Government (Centers for Disease Control) employee rather 
than as an FHI/HAPP staff member made it 'difficult to coordinate and collaborate within the 
framework of the HAP Project. 
Achievements included: 
The STD Advisor contributed to the adaptation of Syndromic Management Guidelines, which were 
then translated and printed using PIL funds. Support was provided for training based on the 
guidelines. TOT was provided to the CDC (DepKes) training team and lead trainers from Community 
Health (Puskesmas), Medical Care (hospitals) and Laboratory divisions. This group was then 
responsible for the training of staff under their programs. 
The STD advisor contributed greatly to the surveillance and other research activities supported by 
HAPP and in relation to requests from USAID. Particularly valuable was work done to clarify the 
picture of STD prevalence among the groups targeted by HAPP and the annual review quality of STD 
care provision based on WHO Performance Indicators. .. . 
The STD Advisor also introduced improved lab technology which allowed for accurate diagnosis of 
Chlamydia and thus, documentation of prevalence levels among selected groups of high risk and 
other populations. There are now GenProbe machines in NAMRU, the National Institute of Research 
in Health (NIRH) and the provincial laboratory in Surabaya. Training in necessary lab techniques was 
provided to public and private sector HAPP clinical partners through support from CDC (USA) 
While the GenProbe is generally too expensive to allow for screening and diagnosis, it does provide 
an important reference facility. 
Anti-microbial susceptibility studies were also supported, which enabled identification of GC drug 
resistance and recommendations of more effective treatment. 
Providing technical guidance in STD service provision was a priority focus for the CDC advisor. 
During the evaluation, it was generally agreed that, if STD service delivery was to be prioritized 
under the new project, then services that will be accessed by the female (and male) sex workers and 
their clients should be a priority. (See: External HIV/AIDS Assessment, Indonesia 11/99). However, 
the current project showed up a number of constraints to be considered. These are as follows: 
HAPP was mandated to 'upgrade' STD facilities so the STD Advisor spent considerable time 
attempting to locate appropriate clinics and hospitals to work with. However, few facilities salv more 
than a handful of STD patients/month, which limited the usefulness of this effort. 
h!APP- Infernal Evaluation Report 
In general, three different clinic models were trialed during HAPP. 
1. Well-established NGO clinics providing reproductive health services to family planning 
populations integrated ST1 care into their services using a combination of 'syndromic 
management' and simple laboratory tests. This was more appropriate during the first phase 
of HAPP which included " women at risk" as a target population. Selection of this population 
was based on the assumption that many women were at risk of HN and STIs through their 
husbands' sexual activities outside of marriage. 
During the first phase of HAPP, this was reasonably successful and one project developed 
innovative outreach activities working through traditional medicine vendors which resulted 
in increased referrals (and increased status of vendors). In another, the promotion of free STD 
testing in conjunction with FP services received a positive response. 
To varying degrees, these projects also attempted to function as referral resources for 
outreach projects working with female sex workers and high risk unmarried youth. A shift in 
the second phase resulted in dropping the 'women at risk audience' as levels of STIs did not 
suggest this was a priority over other, higher risk audiences. As in most countries, trying to 
integrate sex workers into general population clinics did not prove particularly successful for 
a variety of reasons. 
2. A study was implemented to assess the feasibility and costs of integrating STD/RTI 
management for women into Puskesmas (government clinics) The effort took a high level of 
effort and again, the clinics did not attract the higher risk populations. In addition, the failure 
of syndromic management as a method of identifying or treating ST1 in women limited the 
effectiveness of treatment based on this approach. 
Another government clinic receiving high levels of support, including staff, equipment and 
training, through HAPP and directly through CDC, was located next to a brothel area in 
Surabaya. Additional staff were hired to give counseling during the aftemoon when 
additional clinic hours were held. (4 @me&w about this... did a*y of this. & LY) 
ath.dq rvran3, we x-w wwkew?) Smaller number of clients than the morning clinics 
- the aftemoon clinic is found more attractive as there is a shorter waiting period and 
individualized attention. However, findings re: sense of ownership of the GO1 vs. the 
HAPP/CDC-supported clinics suggest that, without ongoing fmancial support, most new 
services and 'technology' wilt not be continued in the HAPP/CDC side of the clinic program. 
NGO outreach staff made referrals. Referrals were problematic for many reasons: Reasons 
could be grouped under: 
a) Perceived financial costs of visit and medications 
b) Perceptions of inferior quality of services 
C) Perceived social or personal costs (embarrassment, judgment) 
d) Distance to the clinic and inability to attend during clinic hours 
e) Some referrals were inappropriate, with population members being sent to 
reproductive health services for other problems. 
The economic crisis also had a negative impact on the ability of government clinics to provide 
treatment due to the decrease in the availability of appropriate STD drugs in health centers. 
3. The most successful clinics reaching high risk groups were those established specifically for 
beneficiary groups: one for waria and one for truck-drivers and others in the Surabaya port 
setting. Because of CDC support, the waria clinic was able to provide free drugs for 
treatment. The drugs were provided through. Clients were recruited by peer outreach 
MPP - Internal Evaluation Report 
workers who also provided educational sessions at the clinic. Transport costs were also paid. 
The clinic has become quite popular and is also now attracting some male sex workers and 
gay men. It has also allowed for STD prevalence studies among this population, which 
provided valuable new information. 
The truckdrivers' /port workers' clinic was able to negotiate with local government (I;anWil) 
for medicines. Negotiation was facilitated by the HAPP Program Manager. The clinic was 
located in a busy port area. During the second phase of HAPP, the Project expanded to serve 
all the populations of the area, including female sex workers and waria. Consideration was 
being given to charging a small fee. 
Unfortunately, the syndromic management approach worldwide has proven to be limited for 
accurate case identification and management, due to the frequent lack of STD symptoms 
among women and the lack of syndrome specificity. Since lab back-up is not usually 
available, use of syndromic management among women cannot be expected to have a 
significant impact on STD prevalence. 
Lessons Learned : 
Changing care-seeking behavior is a slow process, as is any behavior change. The majority of 
CSWs still resort to self-treatment with antibiotics even though there was increased self￾awareness of signs and symptoms of STIs rather than following the suggestions of Outreach 
Workers (i.e. routine condom use, examination by a health professional). 
The clinics most able to attract high risk populations were those specifically established to 
meet their needs. The problem of not being able to buy medicines for clinics severely limits 
the ability to establish appropriate clinics. Obtaining government drugs depends on local 
decisions and supplies. 
Developing referral networks was also tested by one NGO family planning clinic; the low 
involvement of other clinics meant the system was not effective. The level of involvement 
may relate to the low number of STD patients actually seen. 
Syndromic management remains a useful approach for the diagnosis and treatment of Sns in 
men; however, regional and international data has shown that it has a very low usefulness in 
screening or diagnosis and treatment in women. Most clinics do not have laboratories and 
labs do not have the facilities to allow for the diagnosis of Chlamydia, one of the most 
common 'important' =Is. There is still no simple solution to the management of SlDs even 
among women with a high prevalence of infection. 
A more effective approach may be to focus on promoting the care seeking and correct 
treatment of men, with as much parher referral and treatment as is feasible. 
Systems for making referrals to high quality HW testing services are also needed. Pre and 
post-test counseling would need to be provided by the referring agency in order to ensure 
quality and follow-up on any positive cases. 
Outreach workers, peer educators and clinic service providers need to work together closely 
to facilitate giving appropriate referrals and uptake of those referrals. Providing 'tours' and 
IEC on what clients can expect at the STD clinic may help. 
Providing a counselor at clinics allowed more time for the visiting doctor to examine and 
treat patients. Trained counselors are more likely to be able to discuss issues of sexuality and 
HAPP - Intemal Evaluaiion Report 
specifics of safe and unsafe sexual practices, condom use and negotiation and partner 
notification when appropriate. 
. Adherence to universal precautions needs to be carefuuy monitored. 
Anti-microbial susceptibility studies provide important information and should be 
periodically repeated. 
. The challenge of providing adequate STD services to sufficient numbers of populations at 
highest risk remains. 
STD Service Provision 
In the area of a future STD strategy, the CDC advisor suggested that primary prevention of STD/HIV 
through condom promotion should be the most important strategy for the new project in the area of 
STD services. 
F. Supportinz Field Outreach 
"Wllen I give people the brochures on HJV and STDs, they ask questions.. and sometimes 1 don't 
know the answers? This statement is representative of the comments of many HIV/AIDS AeId 
workers, who, with simpJe training, must coniident(y be able to raise issues and to help people 
j?ersonalize'informahon if HJVprevention is to move beyond simple awareness raising. 
Field outreach is a tough job - hours are irregular and venues are frequently difficult. Maintaining the 
morale of workers is an important concern of any street outreach program. Feeling inadequate 
doesn't help anyone's morale - nor does the lack of confidence in workers increase the program 
credibility. 
In response to needs voiced by outreach workers and based on experience with providing field 
worker support, the HIV/AIDS Prevention Project, has produced a 'field workers' kit - now a 'hot' 
item not only among the field workers but also with many others invoIved in HN/AIDS prevention 
work. 
The kit was tailored for 'street' needs. The original included: 
A tough 'hands-free' belly-bag with HAPP logo, to carry materials needed during outreach - big enough for IEC materials, condoms and other supplies. 
An outreach workers' handbook on HIV and STDs. Written in a simple question and answer 
format, the book was based on actual questions provided by outreach workers. The questions 
were referred to both HAPP STD Advisor and Indonesian specialists to ensure clinical and 
linguistic accuracy of information. 
. A flashlight, in recognition that much of the work is done at night. 
. A set of 'aprons' - fuU-size images of the male and female reproductive systems, screen 
printed on fabric for durability. These were developed to supplement the small pictures in the 
bookIet. 
On a recent visit to Yayasan Mitra Masyarakat, a Manado-based NGO working with youth at risk, 
USAID was able to see the materials in action. Young people, trained by the YMM field workers, 
HAPP - Internal Evaluation Report 
demonstrated use of the 'aprons' in their peer education activities. Young men and women were able 
to lead discussions on reproductive health, illustrating their presentations by pointing to appropriate 
areas on aprons being worn by other peer educators. The visiting team was impressed with the ease 
with which the peer educators were able to use the materials to open discussion of these usually 
'taboo' subjects. 
The bags have become a 'professional' status symbol for outreach workers. They are being requested 
by many other HN/AIDS projects, including those of universities and Government and have been 
used as 'awards' for peer educators successfully completing training. The kits even accompanied 
Government and NGO representatives on a recent study tour to Thailand. 
These popular HIV/AIDS prevention tools are about to go into a second edition, including a new bag 
design and an update of the handbook to include more advanced information. 
MPP - Internal Evaluation Report 
Evaluation Report I 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
111. HAPP Internal 
Management 
IV. HAPP Intra-Office 
Management. 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues 
XJL SurveifJance 
XIII. Policy Support 
XII. Surveillance 
Surveillance 
HAPP established the Behavioral Surveillance System in the three 
project sites and produced reports on (3?) rounds of surveillance. 
Other research activities to monitor progress against the Project CPOs 
included the studies carried out by JEN which covered STD prevalence 
among sex workers (avyw &?) and the progress of STD care 
providers toward providing quality care using the WHO Prevention 
Indicators. 
Several major challenges were identified: 
1. Building adequate capacity to design and implement the 
studies and to ensure the quality of analysis and interpretation. The 
STD advisor and ARO staff provided high levels of guidance and 
review. 
2. Writing up results and dissemination also remain challenges, 
as does ensuring that potential users have the skills for interpretation 
and the programmatic application of findings. 
3. The cost of the surveillance versus the scope of need and 
resources available 
Sero/ogica/ Surveiffance 
Although the majority of respondents in this consultation felt the sero￾surveillance system was generally simplified, several key issues still need to be addressed. A recently 
conducted review of the national response to HN/AIDS including the sero-surveillance system 
revealed several issues relating to strengthening the surveillance system in Indonesia. SpecificaUy, the 
surveillance guidelines nationally need to be strengthened to ensure that implementation is clear 
concerning: 
. the type of target group to be sampled, 
. the appropriate sampling procedures and 
. lines of reporting from district to province to the national level. 
It is critical that enough funds are available to ensure basic surveillance with a minimum coverage of 
sex workers and blood donors. However, surveillance within additional high-risk groups is required, 
especially with transvestites and injecting drug users. In addition, the implementation of linked 
serological surveillance with behavioral surveillance should be considered. 
STD Surveillance Based on Service Clinic Popula fions 
Issues identified included: 
. problems in the quality of STD data collection and analysis. 
. inability to diagnose specific STDs through the syndromic approach used. 
H4PP - Internal EvaluahbnReport 
. need for the improvement of SIT data management and reporting. 
Behavioral Surveillance 
Several informants, including the GOI, within this review suggested that the BSS results may be the 
largest contribution of the HAP Project. However, the utiiization of the BSS data at local level has 
been minimal and activities to build capacity among decision makers to use data was limited. It was 
generally believed that either a short version of the BSS or a different tool for collecting and analyzing 
local data is required under the new project. 
Discussions have also taken place as to how the BSS can be decentralized. Although there is 
the potential to simplify the BSS, several issues must be taken into account while planning for 
'local' implementation and use of data and still maintain quality and cross reliability. 
Overall, the following questions need to be answered prior to simplifying and implementing the BSS 
with local parlners: 
1. What data is required at a local level? 
2. Which questions are critical in the BSS in relation to local needs? 
3. What is the minimum criteria/experience of organizations to implement the BSS or a similar 
tool? 
4. What are the capacity development needs? 
5. How will the process and findings be linked to both policy development and the end users? 
In conclusion, it was not clear whether the BSS is the most appropriate tool for provincial and district 
level authorities for decision-making in association with programs primarily due to the complexity of 
implementing it. Other tools, including qualitative research, need to be a part of locally appropriate 
evaluation methods. 
Dissemination and Use of Findings for Decision-Making 
Under HAPP, the dissemination of findings generally took the form of official reporh and other 
documents designed for use by central government and donors. 
Experience suggests that ensuring the 'usability' of findings by provincial and district level 
government and by NGOs will require that reports be developed with each audience's needs and 
capacity in mind. There is a need for links between the sero and behavioral surveillance implemented 
at a provincial level with governmental and non-governmental involvement. 
Lessons Learned : 
There is an ongoing need to improve data management, including data collection, diagnosis, 
analysis and reporting 
In general, for sero-surveillance, it was felt that the procedure has been fairly simplified. The 
overall requirement is to decentralize and how to accomplish decentralization. In many places, 
the local capacity to implement is already there; however, in some places, equipment and overall 
funding is needed 
The new project may want to consider the model of implementation used under the AusAID 
project for the BSS in their provinces, where there was a combination of quantitative and 
qualitative data collected and local organizations were involved in the collection of this data. 
HAPP- InternaIEvaluation Report 
If the collection of data with a tool like the BSS is to be done by local organizations then several key 
issues need to be addressed including: 
Increasing the number of partners will increase the complexity and increase the requirements 
associated with quality control. Therefore, the role, responsibility and financial/technical 
authority of every organization must be very clear. 
A plan for the Capacity Building of local institutions must be articulated, provided with sufficient 
time, provided with appropriate follow-up and properly funded. As in all other program 
activities, compromises will need to be established as resources are not available to ensure as high 
a standard of quality as was possible within HAPP. 
HAFP - Internal Evaluation Report 
I Evaluation Report I 
I. HAPP Project 
Context 
11. HAPP Project 
Design 
III. HAPP Internal 
Management 
N. HAPP Intra-Office 
Management 
V. Funding 
Mechanism 
VI. Working with 
KPADs 
VII. Working with 
Partners 
VIII. Working With 
NGOs 
IX. Building Support: 
Networking and 
Coordination 
X. Beneficiary 
Involvement 
XI. Technical Issues 
XII. Surveillance 
Xlll. PolicySupport 
XU. Policy Support 
Policv SUDPOI~ 
The HAPP Delivery Order provided no support for the Provincial 
AIDS Commissions in the 3 sites although there was substantial 
support for NGOs. The KPAD in each province had been formally 
set up in Keppres 36 but not yet operationalized in the 3 HAPP￾supported provinces. Family Health International designed an 
"umbrella proposal" to support the KPAD and to mobilize the 
Secretariat in an innovative, flexible and sustainable manner. This 
consisted of funding for a range of activities that would result in the 
Secretariat becoming a functional unit, consisting of quarterly 
coordination meetings, study tours and policy meetings. This 
approach created a big administrative load for HAPP staff and was 
not easy for the KPAD. It was clear after 3 years of implementation 
that flexible direct funding is the best way to support local 
govemment. While gradual transfer to KPAD funding support was 
planned initially, it proved impossible because of the impact of 
'krismon' (the monitory crisis). 
As a result of the Institutional Development Framework, HAPP 
provided leadership training and management training through 
Business Dynamics. The KPAD found this very valuable. 
uI. 
Lessons Learned : 
Donor support to provincial AIDS Commissions should be 
long-term because of the time needed for sustainable organizational change. 
Simultaneous donor support to both KPAD and the NGO Forum is the most appropriate, 
effective way to build partnership and dialogue between the provincial players. 
Direct funding is a very user-friendly mechanism to support KPAD. 
Leadership and management training would have been more beneficial earlier in the process. 
Study Tours and Conferences 
These were a requirement as a part of FHI's Delivery Order and the Performance-Based Contract. FHI 
implemented study tours through contracts with PATH as well as directly with ARO support. Direct 
implementation was preferred, despite the heavy workload, as the Project benefited directly from the 
nehvork established. 
While study tours were motivating for the provinces involved, the selection of participants could be 
contentious. It was also impossible to meet all participants' needs despite attempts to build in options. 
One obstacle in this process was the shucture of the KPAD, which led to a turnover of staff and a lack 
of continuity. A second problem was that the KPAD neither maintained a database of trained staff nor 
systematically used or evaluated the expertise the participants gained. 
NAPP - Internal Evaluation Report 
Lessons Learned : 
. The Project has benefited greatly from the stronger regional linkages and better use of 
resources and Lessons Learned from FHI's other regional projects. 
. It was very productive to use both local and international tours. 
. A systematic database of participants (KPA or KPAD) is needed. 
. Any follow-up activities should target key decision makers in strategic locations. 
. Better cross donor coordination is needed, for example AusAID's IASTP is training people in 
North Sulawesi, where HAPP is providing support, while many other provinces have no 
donor support. 
The Project Manager led the team; this is a full-time position. She regularly advised and consulted 
and that enabled FHI to have an immediate understanding of GO1 needs. FHI could then react fast 
when necessary, for example, the procurement of HIV test kits. 
Lessons Learned : 
. This model is very effective for FHI. 
. It maximizes GO1 ownership. 
GO1 gets critical needs met. 
HAPP - Internal Evaluation Report 
D. Fourth Stage - Disseminating the Data 
I. . LessonLeamed 
11. Resport Structure 
m. Second Framework 
IV. Final Framework 
Attachment : 
The Time Line and Key Event 
Glossary 
mPP- IntemalEvaluation Report 
D. Fourth Stage - Disseminating the Data 
I. Lessons Learned 
While writing the individual reports contained in Part 3, the writers became aware that there 'was too 
much information for practical dissemination. For this reason, a new framework was drawn up and 
the individual texts were combined and edited. This inevitably led to some information Wing cut but 
it was to disseminate "Lessons Learned" to a wide readership of those who would not 
necessarily have access to or want to read long reports. The findings were distilled into practical 
lessons learned that could be used by many organizations or individuals implementing HW/AIDS 
related activities. The style is deliberately positive and concise rather than attempting to convey the 
complexity of the many viewpoints that are contained in this document. 
11. Report Structure 
Besides the lessons learned report, other versions of the report stressing particular elements of HAPP, 
were produced for different audiences. Key chains for beneficiaries and booklets were produced. 
There was a plan to produce leaflets specifically for the KF'ADs but it was decided that this was 
unnecessary as they would be receiving the "Lessons Learned" report. Working closely with the 
design company, which provided valuable input regarding the graphics and layout, HAPP produced 
the following items, all of which were meticulously field tested prior to going to the publishers: 5 
versions of key chain products with the essence of HAPP Lesson's Learned for sex workers, waria, 
and MEN populations; booklet for sex workers with key BSS results. 
111. Second Framework 
The following framework was the second attempt at organizing the lesson's learned book. It is as 
follows: 
Cover Page 
Table of Contents 
Foreword 
Glossary 
Summary 
Working with Partners 
A. Introduction to Working with Partners - "Managing Expectationsn 
The Partners 
B. Planning and Coordination 
C. Capacity Building 
D. Policy Support and Funding 
Project Overview 
A. Introduction to Project Overview 
Background 
Mission and Goals 
The Project's Phases 
Main Activities 
HAPP - Internal Evaluation Report 
Map of Project Sites 
Evaluation Methods 
I B. Behavior Change Communication 
Outreach 
Peer Education 
IEC Materials 
Collaboration with Public Relations Agencies and the Mass Media 
C. Condom Social Marketing 
D. Sexually Transmitted Disease Treatment 
Trial of Three Basic Clinic Models 
I E. Suweillance 
HAPP Management and Staff 
Conclusion 
Annex 
IV. Final Framework 
The final framework for the Lesson Learned Book is as follows: 
able of Contents 
xeword 
cronyms 
ilmmary 
Project Ovewiew 
A. Introduction to Project Overview 
The Start-up of HAPP 
Mission and Goals 
Social and Political Changes 
Main Activities 
Map of Project Sites 
Evaluation Methods 
B. Behavior Change Communication 
Outreach 
Peer Education 
BCC Materials 
Technical Assessments 
C. Condom Social Marketing and Mass Media 
Collaboration with Public Relations Agencies and the Mass Media 
Condom Social Marketing 
D. Sexually Transmitted Disease Treatment and Surveillance 
HAPP- hfemdEvaluation Report 
Implementation of Three Clinic ,Models 
STD Surveillance 
E. Behavioral Surveillance System 
Working with Partners 
A. Introduction to Working with Partners - 'Managing Expectations' 
The Partners 
B. Planning and Coordination 
C. Program Management 
D. Capacity Building 
E. Policy Support 
HAPP Management and Staff 
Conclusion 
;lossary 
mnex I - Participants in the HAPP End of Project Evaluation 
Acknowledgement 
Appreciation is expressed to the following persons 
for their meticulous assistance in the production of this report. 
Doreen BieHe 
HAPPprofessionaland support sraff 
Hardiyanti Gunadi 
Jane Wilson 
Maria RMafi 
Rashmi Rajan 
Susan Alexander 
Tl~c Time Line and Key Events from 1993 - 2000 
Family Health International 
I HAPP Strategy drafted USAID-GOI agrccrnent $ignsd ( 4 USAI;;;;;;;~ ~ith I4 for brlntsr=l March'95 - M~rch.00 - - lrnplcrnent.tion Phn designed I I I 
December 93 Much '95 Septernbcr 95 Oetobcr/Novernbsr '95 
First HAPP rhff hired IiAPP Officc opcnr at Dcp.Ker HAPP Chiefof Pnrly hired 
Jmanry96 May '96 July '96 
Performmcc Bated PI-I1 submits propotd to FIHl Ncptiztion opened; rkff hired USAID / lndoncria with PVO pwtncrt 
Dcccmber'96 Much/Aprilt97 May "97 
extended to July 31; Cbv out Tmrition fram Delivery 
Order to PBC - NGO fundine I -4 AIDSCAP-HAP@ find rcportr per end of AIDSCAP and 1 I I DO#14 rtopt -I I completed 
Junc/July ' 97 July '97 - 
July - Auyrt'97 u 
August '97 
AMWVI WOI~~IYII i~dm~itld to 
USAID (due 1 Dsc) with 7 NGOI psndingqqxovd of workPlm 
Scptembcr'97 October 97 October . Dcccrnbsr'97 Octobcr'97 . Janunry'98 
The Time Line and Key Events from 1993 - 2000 
Family Health International 
I Submirrionof BSS-2dntn for Re-submission of workplm, ctc 
mission's R4 reporting 1 toUSA1D 
Re-ncPtiatingof PBC with 
CSM meetingr with USAlD ad FHI; TF FHI nnd USAlD on CSM 
88240Technical Asrirtnnt for rc-design ~tivitic~ 
I 
Jmuuy- Februnq'98 
HAPP Monitoring nod 
Full implcmcntntion of7 NGO Re-design & re-negotiating of Uncrpected dcpnture of BCC Evduntion Workshop with 
projects with USAID waiver CSM component with TFG Implcmcnting Agcncicr 
Febmaiy '98 Februq/Mnrch'98 Mzch'98 April '98 
USAlD approval . . of HAPP - Recommendntion to improve Pull initiation ofCSM projcct work with I'VOr 
July/Aupst '98 
President Suhzrto resigns jurted nll NGO contmct by 50% for inflation 
Tcchnicd Advisoiy Gmup wi DKTdesign proporal for FHI; 
PC1 md Armnjnya begin Field CEDPA design completed Fomd notific&on from 
CEDPA of its resignation 
August '98 September '98 November '98 Dccembcr '98 
One-week intcrnntioxxnl study 
5 new positions were filled in PC1 & DKTpropord find tour for North Sulzwcri 2nd 
HAPP Jnhrtn EnrtJnw KPAD 
Jmunq-Mnrch '99 Februnq '99 Februq '99 Februq'99 
General HAPP skff meeting, 8 RRF NGOr in Manado and Completion of rubrnirtion of Innugumtion of the renovated 
W~rkrhop on prapord revicw 
process for HAPP rtnff 
March-April'99 April '99 May '99 June '99 I I I 8 
The Time Line and Key Events from 1993 - 2000 
Family Health International 
Jnsugurstion of thc North Jake* KPAD Pro- ruff 
Sulawcri KPAD Secrctnriar surr Approval of DCOP nr n Subrnirrion of PHI proposal 1 The 5th intcmntionnl Congress 
signntory on finnncid mnttcrr USAID for 5-month sxtentio on AIDS in Aria Pacific 1999 
Second qwrtcr 
Approval of lhc HAPP 
Novcrnbcr '99 Decemhrr999 lulv'rm 
MPP - Internal Evaluation Report 
GLOSSARY OF ACRONYMS 
AIDS 
AIDSCAP 
Atma Jaya 
AusAID 
BCC 
BSS 
CD 
CDC 
CHR-UI 
CLINS 
COTR 
CPO 
CSE 
CSM 
DKI 
FA 
FCO 
FHI 
FSW 
FUTURES 
GO1 
HAPP 
HCP 
HCW 
HN 
I A 
IDF 
IDU 
IEC 
IP 
IPOOS 
Jabotabek 
JEN 
Kanwil 
KPA 
KPAD 
Litbangkes 
LO1 
Iokalisasi 
MFR 
Acquired Immuno Deficiency Syndrome 
AIDS Control and Prevention Project 
Atma Jaya University 
Australian Agency for International Development 
Behavior Change Communication 
Behavior Surveillance Survey 
Country Director 
Center for Disease Control (Atlanta) 
Center for Health Research - University of Indonesia 
contract Line Item Numbers 
Contract Office Technical Representative (Country 
Director) 
Contract Performance Objective 
Commercial Sex Establishment 
Condom Social Marketing 
Daerah Khusus Ibu Kota (Capital City Administrative 
District of Jakarta) 
Finance Assistant 
Final Cost Objective 
Family Health International 
Female Sex Worker 
The Futures Group International (PVO) 
Government of Indonesia 
HIVIAIDS Prevention Project 
Health Care Provider 
Health Care Worker 
Human Immunodeficiency Virus 
Implementing Agency 
Institutional Development Framework 
Injecting Drug User 
Information, Education and Communication 
Infection Prevention 
Ikatan Persaudaraan Orang-Orang Sehati, A Jakarta Gay 
Organization 
Greater Jakarta (Jakarta, Bogor, Tangerang, Bekasi) 
Jaringan Epidemiology Nasional (Indonesian Epidemiology 
Network) 
Provincial Office of a Government Ministry 
Kornisi Penanggulangan AIDS (National AIDS 
Commission) 
Kornisi Penanggulangan AIDS Daerah (Provincial AIDS 
Commission) 
Health Research and Development Office 
Letter of Intent 
brothel area 
Monthly Financial Report 
HAPP - Intend Evaluation Report 
MOH 
MSI 
MSM 
MTX 
NAMRU 
NAZA 
NGO 
OC A 
OR 
ow 
P2M & PLP 
PASA 
PE 
PI 
PIL 
PKM 
Puskesmas 
PVO 
R AR 
RI 
ROC 
RRF 
RSCM 
STD 
ST1 
s W 
T A 
TKBM 
TOR 
TOT 
UP 
US AID 
waria 
WHO 
Ministry of Health 
Management Systems International 
Men who have sex with men 
Accounting software for office use 
Navy Medical Research Unit (US) 
Narcotics, Alcohol and Addictive Substances 
Non-Government Organization 
Organizational Capacity Assessment 
Outreach 
Outreach Worker 
Pemberantasan Penyakit Menular (Communicable Disease 
Control and Environmental Health, MOH RI) 
Participating Agencies Service Agreement 
Peer Educator 
Performance Indicator 
Project Implementation Plan 
Pusat Kesehatan Masyarakat (Center for Public Health 
Education) 
Primary health center 
Private Volunteer Organization 
Rapid Assessment and Response 
Republic of Indonesia 
Repeated Outreach Contacts 
Rapid Response Funds 
Cipto Mangunkusumo Hospital 
Sexually Transmitted Disease 
Sexually Transmitted Infection 
Sex Worker 
Technical Assistance 
Tenaga Ke ja Bongkar Muat (harbor labor) 
Terms of Reference 
Training of Trainers 
Universal Precautions 
United States Agency for International Development 
Wanita Pria (transvestite or transsexual) 
World Health Organization 
Glossary - 2