REPORT Abstinence and Risk Avoidance for Youth Project Report (ARK) END OF PROJECT EVALUATION FOR HAITI, KENYA AND TANZANIA SUBMITTED TO: World Vision United States Program Director: Happy Kumah August, 2010 BY: MASAZI DEVELOPMENT ASSOCIATES, cc ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 2 TABLE OF CONTENTS TABLE OF CONTENTS...............................................................................................2 ACKNOWLEDGEMENTS...........................................................................................3 ACRONYMS.................................................................................................................4 EXECUTIVE SUMMARY ...........................................................................................5 I. BACKGROUND .................................................................................................14 II. OBJECTIVES......................................................................................................15 III. EVALUATION METHODOLOGY ...............................................................15 IV. FINDINGS.......................................................................................................20 a. Operations and Accomplishments of ARK Project .........................................20 1. Overall Structure and Management of the ARK Project .............................20 2. Key Interventions and Accomplishments of ARK Project..........................23 3. Monitoring and Evaluation ..........................................................................39 b. Outcomes and Impact ......................................................................................41 1. Knowledge and Attitude: AIDS Stereotyping .............................................43 2. Behaviour.....................................................................................................46 3. Abstinence and “Be Faithful” ......................................................................50 4. Sources of information.................................................................................53 5. Life skills Youth ..........................................................................................56 V. STRENGTHS, WEAKNESSES, CONSTRAINTS & OPPORTUNITIES ........61 a. Strengths ..........................................................................................................61 b. Weaknesses......................................................................................................63 c. Constraints .......................................................................................................64 d. Opportunities....................................................................................................65 VI. DISCUSSION..................................................................................................66 a. Lessons Learnt .................................................................................................68 b. Project Sustainability and the future................................................................69 c. Recommendations............................................................................................71 VII. APPENDICES .................................................................................................76 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 3 ACKNOWLEDGEMENTS This report describes in detail the final evaluation of the ARK Project in Haiti, Kenya, and Tanzania conducted between March and June 2010. The report was prepared by MASAZI Development Associates of South Africa, contracted by World Vision US to conduct the end of project evaluation. The evaluation of ARK project in Kenya and Tanzania was facilitated by Dr Lewis Ndhlovu and Mrs Kgetheng Dlamini and in Haiti by Dr Chiweni Chimbwete in collaboration with in-country evaluation teams. World Vision US would like to express sincere thanks to all parties involved in this evaluation: World Vision ARK Project Program Managers and their dedicated staff, USAID who funded the project and the many stakeholders across the countries who took the time to participate in the evaluation. They are all making a great contribution to helping young people adopt abstinence and reduction of risky behaviour to fulfil a dream of an HIV-free generation. Funding for this evaluation was provided by the United States Agency for International Development under Cooperative Agreement GPO-A-00-05-00004-00. Points of view or opinions in this evaluation report do not necessarily reflect the views or policies of the United States Agency for International Development, nor does the mention of trade names, commercial products, or organizations imply endorsement by the United States Government. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 4 ACRONYMS AB Abstinence and Be faithful ADP Area Development Programs AIDS Acquired Immune Deficiency Syndrome ARK Abstinence and Risk Avoidance CBO Community Based Organisation CCP Center for Communication Programs F/CG Faith and Community Groups FBO Faith-Based Organization FGD Focus Group Discussion FGM Female Genital Mutilation HIV Human Immune Virus IEC Information, Education and Communication IGA Income Generation Activities JHUCCP Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs KAPB Knowledge, Attitude and Behaviour Practices KCSF Kenya Christian Students Fellowship KII Key Informant Interview MOE Ministry of Education MOH Ministry of Health MOU Memorandum of Understanding NAC National AIDS Coordinating NGO Non-governmental Organisation NS Not Significant PAG Parents Action Group PEPFAR Presidential Emergency Plan for AIDS Relief SACCOs Savings and Credit Co-Operative Societies SU Scripture Union SUK Scripture Union of Kenya TOT Training of Trainer USAID United States Agency for International Development VCT Volunteer Counseling and Testing WV World Vision WVUS World Vision United States YAG Youth Action Group ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 5 EXECUTIVE SUMMARY This is an end of project evaluation report for Abstinence and Risk Avoidance for Youth Project (ARK) conducted in May 2010 in Haiti, Kenya, and Tanzania where the project had been implemented. The report reviewed the achievements over the past five years and gave recommendations for implementation in future. BACKGROUND AND RATIONALE World Vision United States (WVUS) in partnership with Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs (JHUCCP) has been implementing ARK (Abstinence and Risk Avoidance) project for five years in three countries namely; Haiti, Kenya, and Tanzania. The ARK Project was designed to be implemented through World Vision Area Development Programs (ADP) in partnership with government sectors, faith-based organization (FBO) partners, national, district, and village-level civil society. The goal of ARK was to expand and strengthen HIV / AIDS prevention efforts for young people through behaviour change that will decrease risk of becoming infected – primarily through abstinence and being faithful (A&B) and mutual monogamy. The project offers young people a safe, enabling environment where small groups of youth, small groups of parents and Faith and Community Groups (F/CGs) can explore, foster, support and promote positive traditional norms of behavior. OBJECTIVES OF THE ARK PROJECT 1. Strengthen the capacity of youth, adults and sub-grantees for A&B programs; 2. Increase family and community support at district, ward and village level for A&B; and 3. Create an enabling environment for A&B. The objectives of this review are to: a. Assess the achievements of the programme goal against the original log frame; b. Determine the impact and results achieved over the project’s five-year life span; and c. Identify the strengths, weaknesses, opportunities and constraints of the project implementation, and d. Document lessons learned and provide recommendations for scale up of ARK and related projects. EVALUATION METHODS Two sites from each of the three countries of Haiti, Kenya, and Tanzania were selected for the study based on site selections done during the mid- term review. A number of approaches were used for gathering data, namely: i) a desk review; ii) structured and semi structured interviews; and iii) focus group discussions. A quantitative survey instrument was administered to about 2030 young people aged 10- 14 and 15-24 years to assess their knowledge, attitude, behaviour and practice (KABP) related to the ARK project interventions. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 6 FINDINGS 1. Operations and Achievements a. Operations Management  Senior staff members from ARK and World Vision indicated that there were no difficulties in the relationship between the project and other WV structures or other projects.  Where the ARK Project was housed within the ADP offices it benefited from better infrastructure and administrative support. In Haiti, all ARK staff were designated as HIV and AIDS coordinators.  ARK worked directly with community stakeholders. In Kenya, ARK worked with sub-grantees but they were later used as training contractors. Consequently, there was no proper follow up of trainees and sustainability was not well established. b. Monitoring and Evaluation  Programme indicators were designed to report on PEPFAR’s output and outcome indicators. Some of the selected output indicators include numbers of youth and parents trained and reached in value-based life skills and AB messages using interpersonal approach.  The routine monitoring summaries (monthly, quarterly and annual) were available and well kept. However, there was no in-depth documentation of the quantitative project outcomes and their quality.  Outcomes of the midterm evaluation are compared to those of the end of project evaluation. c. Achievements The key ARK interventions were primarily Information, Education and Communication (IEC) of which the main components were training, radio, dialogue meetings, common ground melting pot meetings, drama, poems, handouts and other ARK commodities (t-shirts, bangles, bags, and ARK passports). Training  A cascading training program was carried out: master trainers from the ARK core team in Kenya and Tanzania and a JHUCCP consultant trainer trained trainers who then trained facilitators. These trained facilitators conducted training for youth, parents, teachers, NGOs, CBOs and representatives from some of the government departments (e.g. MOE and MOH).  Training covered topics like human development, values and behaviour, sexual health, relationships and personal skills. Materials were developed and equipment was purchased through the project.  From qualitative data it was perceived that ARK had made great achievement in reducing number of teenage pregnancies, HIV infection school drop outs, enhanced teacher-learner relationships, and improving academic performance and school attendance. Support for A&B  There was overwhelming passion and support for the ARK project in the three countries in both the younger and older age groups, parents, religious leaders, ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 7 traditional leaders and government officers who found the AB messages appealing to them. This provided a big opportunity for the ARK project to take root in the community and expand. Enabling environment  ARK project complemented government policies on child rights and HIV prevention. In each of the countries, there was an opportunity for integration of ARK into various HIV prevention projects. Kenya  Youth indicated the life skills training had helped them to make informed decisions on important sexual and life skills issues facing young people. Parents reported better communication with their children and added benefits to their family life and improved relationships.  Different kinds of media reinforced each other and to reach new audiences. Parents also liked the radio and suggested that they be involved more in the radio shows; dialogues improved communication between children and adults; children enjoyed the spirit in which sports events were held; products such as T-shirts, bangles, bags, and ARK passports and murals were highly valued by beneficiaries in the project areas. However, they were reported to be always in short supply. There was overall appreciation that the project closely collaborated with government departments, NGOs and other organisations. Youth in school recognized the benefit of their teachers’ involvement in ARK.  People in the communities also benefited from the linkages that ARK had made with government officials and participants felt they were relating better to them. Tanzania  ARK was commended for changing young people’s negative behaviours such as walking around at night, watching pornographic movies, having many boyfriends and girlfriends, avoiding bad company, smoking, avoiding esoto (night dances where youth have sex) and taking money from strangers.  The participants in the FGDs and in depth interviews reported that ARK approach was unique, participatory and sensitive to their needs; this was appreciated by Maasai communities. However, participants reported that they would have liked interventions among parents and teachers.  Young people liked participating in the radio talk show, but they reported that some teachers imposed restrictions at school because they felt that no time was specifically allocated to the project.  The use of video shows, a drama and radio program helped make ARK known even outside the project areas.  Using dialogue meetings, concerned community members challenged people who showed pornography videos in the community to stop. Drama and dialogues also played an important role in sensitizing people about VCT. Through participation in sports young people said their minds were diverted from thoughts about sexual matters in school. T-shirts were popular and it was felt they should be given to all students not just ARK members. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 8  ARK mobilized community stakeholders like CBO, FBOs and local government leader’s parents, teachers, youth who realized they have one common goal. For example Christians and Muslims worked together in Monduli. Hard-to-reach traditional healers were also influenced. Haiti  Parents, community leaders, religious leaders and key informants unanimously mentioned the graduation ceremony after training as the most memorable event with ARK. It was more than the awarding of certificates but a sensitisation to show that they had valuable resources in their community. There were testimonies of individuals who had reconciled with others.  However, community stakeholders complained about the limited reach of the ARK project in terms of the number of people trained and the geographical coverage within the ADP.  The animators1 were responsible for ARK and Hope Child programming. For sustainability the animators will continue training peer educators on ARK activities beyond the original ARK intervention regions. This can be cited as a best practice on integration of ARK into existing projects of WV and sustainability of its activities.  Peer educators underwent drama training that included writing and the final act of staging a play on sensitisation of HIV/AIDS. The process and the final output were noted as best practice examples.  ARK promotional products were visible in the community. However, community members complained there was a short supply of training and promotional material: In Haiti, ARK resolved to directly work with community-level partners such as schools, churches and development groups. The AIDS response faces added challenges due to the 12 January 2010 earthquake with World Vision stepping up of relief programmes. Ministry of Health and World Vision work together at community level without necessarily having a formal relationship.  There was a concern that supporting social networks created by ARK disappear once young people move to city to pursue education and career opportunities. One possible solution was to link them with other HIV prevention programmes at World Vision.  Young people shared that ARK has helped them reflect on their lives and to have goals. Parents testify to how they were now empowered to communicate and to reach out to their children. Within the school environment, pupils, teachers and parents indicated that with ARK there were less early pregnancies, less teacher-pupil relationships, improved school attendance and better academic performance. Thus an enabling environment was created at household and community level. 2. Outcomes and impact Quantitative data on knowledge, attitude and behaviour provides some interesting insights on the ARK programme. 1 Animators (animateurs) are facilitators and employees of World Vision Haiti. They report to ADP Coordinators and oversee peer educators. Each animator has a kit that contains ARK facilitation guides, ARK Passport and the OUCH quality improvement checklist to assess the quality of delivery of peer education training sessions. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 9  Knowledge of the key elements of the ARK Project, that is, abstinence and ‘be faithful’ were well known among the groups in qualitative interviews. The manual indicates abstinence as “staying away from sex, drugs, alcohol and risky behaviour”. Staying away from sex was the most well known definition of abstinence among youth of the two age groups.  When the young people were individually asked in the survey about some of the ways that they could use to avoid HIV infection, at least 61 % of youth (10-14 years) ranked abstinence highest. Among those aged 15-24 years, they ranked being faithful first as a strategy of choice (range 69 to 74 %). These findings should be viewed in context that the youth had undergone training of the ARK programme; therefore their knowledge levels would have been expected to be higher for the key elements.  For questions on stereotypes associated with people living with HIV, it was noted while there was compassion for those infected; there was also belief that AIDS was punishment for bad behaviour and general shame associated with having a member of the family infected. That there was variability within the countries and between the countries suggests that the stereotypes have not been adequately addressed throughout the ARK programme.  The older age group (15-24 years) appears to be more tolerant and accommodating of people infected; they would not feel as much ashamed if their family member was infected with HIV, and they would show compassion and love to those infected.  Sources of information for messages on “being faithful” vary between the countries. In Haiti, the health centres ranked highest followed by the traditional birth attendants and Anti AIDS clubs. In Tanzania, the radio, the health centres and the schools played the key roles. Kenyan youth identified the radio, television and church as key sources of information about “faithfulness”.  The key sources of information (radio, schools, clinics, churches) for reproductive health, A and B messages are consistent with ARK’s intervention strategy of using the community.  The younger age group (10-14 years) appeared to know more about the two methods of contraception (pills and the injectable) compared to the older ones (15-24 years). The younger age group was also relatively well versed on signs and symptoms of sexually transmitted infections. They were more likely to mention that avoiding multiple concurrent partners would be a future strategy for them to avoid contracting HIV infection (compared to their older counterparts). 3. Strengths, Weaknesses, Opportunities and Constraints a. Strengths  Availability of resources for effective communication  Participatory engagement of key groups (parents, teachers and youth): Life skills have improved the communication between youth and their parents.  Good partnerships with stakeholders, including traditional leaders and healers: The relationships with the groups assisted in making the key stakeholders accountable to the people they serve. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 10  ARK promotional products such as food and drinks given during training and seminars, graduation ceremonies were acknowledged as important ingredients for motivating participants.  Stakeholders support and ownership of ARK Project:} The success lay in the passion, commitment and self motivation of many volunteers in the communities who worked as TOT, peer educators and in other different capacities b. Weaknesses  Inadequate material resources: During some training sessions, the quantity of training and development materials was not adequate and this was cited as reducing training efficiency and effectiveness in meeting program goals.  Due to limited resources, stringent criteria were developed for distribution of materials to the target population. However, TOTs and some volunteers pointed out that there was lack of transparency in the distribution of some items. o Community expectations were not met at certain sites. It was felt that programme personnel made promises that were not fulfilled (for example, bangles, footballs) o Coverage of the programme areas was limited and there was unmet need. c. Constraints  In general, there was satisfaction with the ARK project by different groups of participants. Therefore, constraints presented were few. The main one was the need to increase geographical and radio coverage. d. Opportunities  Supportive Policy and community environment- evaluation found support and cooperation from the government and the community  Endorsement by traditional leadership  Existence of Trained Cadre and materials for training of trainers, peer educators and drama groups are now available to continue with ARK activities.  Shared experiences across projects and location- in different cultures, and project situations provides an opportunity for sharing lessons drawn from each of the countries under different circumstances. 4. Lessons learnt  The use of in-country WV staff is critical for project support and success.  It is important to recognize that local small sub-grantees require capacity building to help them handle large funds and learn donor guidelines. This was a constraint for some of the local sub-grantees to manage large funds in the project.  The ARK project was multi level and multi country. Opportunities exist for learning and drawing lessons in administrative and technical processes. Coordinating activities and funds from different country locations slowed the ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 11 decision making process due to institutional bureaucracies. Therefore, bringing decisions as close as possible to the ground where projects are implemented is a better strategy. 5. Project sustainability In all the in-depth interviews and group discussions, the message was clear and consistent that the project was beneficial for the communities and some way should be found to continue with the activities. Some ways for ARK project sustainability include:  Better Integrate ARK project with World Vision program at ADP level.  Put ARK equipment and materials at the disposal of the community through community non governmental organisations.  Expand the ARK approach beyond its current borders. For example In Haiti, animators from outside the ARK Project, ADPs have been trained on ARK from its inception.  Introduce the different groups of the ARK project to stakeholders who may be interested in funding the project further. For example, in some districts in Kenya and Tanzania, faith based organisations have expressed interest in adopting some of the relevant messages for their faith teachings.  Strengthen the ARK Life Skills approach in schools  Integrate some key element of the ARK programme into government programmes, such as schools. This could be done through curricular and extracurricular programmes. 6. Recommendations These recommendations are discussed in the context that the Programme is coming to an end and that the lessons learnt will be applied to similar programmes or be used for sustaining the ARK Project i. ARK: A youth and community-friendly project  Project authorities should seek funds to continue to build on the completed phase of the ARK Project. This will allow time to adequately document the impact of behavior change in reducing risk for HIV infection among youth in the three countries.  Scale Up coverage of ARK Project in the target countries.  Utilize ARK platform to implement comprehensive combination HIV prevention intervention for youth and other at risk groups in the community. ii. Human Resource management  The project is designed around the concept of volunteers. In some places this led to inadequate trainings for youth and the community in some geographical areas.  Improve the use of ARK staff members as well as the volunteers in terms of their capacity and skills to handle the work assigned. iii. Improve incentives for Volunteers  The project staff should provide and clarify the incentive issues for volunteers and make them transparent and consistent at the beginning of the project in the target areas. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 12 iv. Linking ARK Project with ADPs in World Vision The implementation of the ARK project was viewed as superior where ARK was located within the World Vision ADP offices.  Strengthen the integration of ARK within the ADP framework in order to efficiently use resources. For example ADPs should support more ARK activities in satellite offices. v. IEC, Training and Promotional Materials in ARK Project There were not always enough T-shirts, bangles, bags and training facilitation guides. While key informants noted that there was distribution criteria, participants felt it was inconsistent and not transparent.  Ensure that as part of planning that there is always enough training and other project materials and ARK commodities for distribution to the beneficiaries.  IEC material developed should be integrated within youth and child health policies at the national level. vi. Create opportunities that will sustain behaviour change  Support people’s behavior change by creating opportunities including income generating activities that will sustain their new chosen behaviour.  Compile a resource book of activities that youth and others on alternatives to risky life style for those who decide to change their behavior can be involved in. vii. Increase the accessibility of VCT & access to a combination of comprehensive HIV prevention packages, with linkages to households and families as necessary.  Participants said people who were willing to go for VCT after ARK sensitized them could not test in some cases because there were not enough testing kits at the time of counseling. Recommendations  Improve VCT uptake in ARK Project by working closely with local health authorities to provide both mobile VCT and home-based HIV testing for individuals and household within ARK project. There is need to balance demand creation with supplies of such services.  Explore opportunities of giving more information on comprehensive HIV prevention services including condoms. viii. Operations Research Studies A number of operations research studies could be conducted to assess the effect of different combinations of interventions under different programme settings.  The use of control sites was not implemented due to budgetary constraints and lack of support from the funder and yet this could have offered a very powerful comparison to investigate quantitatively the effect of interventions. As the programme expands, clear indicators before and after interventions should be measured.  World Vision should seek to collaborate with agencies conducting social, health and demographic surveys. Few questions may be added to gather data on key impact indicators. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 13  World Vision should continue to take advantage of its regional presence to share data gathering tools, processes and lessons learnt from different site and country settings.  At some point in future, World Vision needs to evaluate the different strategies of interventions and their combinations; for example, TOTs and voluntarism, the understanding and effect of materials on the different audience. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 14 I. BACKGROUND World Vision United States (WVUS) in partnership with Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs (JHUCCP) has been implementing the ARK (Abstinence and Risk Avoidance) project for five years in three countries namely; Haiti, Kenya, and Tanzania. The ARK Project is funded by PEPFAR (Presidential Emergency Plan for AIDS Relief) through United States Agency for International Development (USAID) Grant Number USAID APS-M/OP￾04-812. In Kenya, the project was implemented in five districts: Teso, Bungoma, Suba, Migori and Nakuru. Activities in Haiti were targeted at the population in two regions Central Plateau and the Island of La Gonâve. In Central Plateau, the ARK Project was implemented in 3 communes Hinche, Thomonde and Thomassique; and 2 communes - Anse-à-Galets and Pointe-à-Raquettes in La Gonave. In Tanzania, ARK was implemented in four regions; Arusha (Monduli District), Tanga (Handeni District), Kilimanjaro (Hai District) and Kagera (Bukoba rural now known as Misenyi) and Karagwe Districts. The ARK Project was implemented through the World Vision Area Development Programs (ADP) in partnership with government sectors, faith-based organization (FBO) partners, national, district, and village-level civil society. It combined the reach and power of the faith community with the weight of proven public health communication methodologies to strengthen HIV prevention efforts for young people. The project offered young people a safe, enabling environment where small groups of youth, small groups of parents and faith and community groups could explore, foster, support and promote positive traditional norms of behavior. This helped youth traverse the risks they face in their daily lives, and better understand how to challenge harmful norms that present barriers to positive health practices, including risk reduction for HIV and AIDS. The ARK project came to an end in June 2010. A mid-term review was conducted in July and August 2007 and a report was written on the project’s accomplishment at that time. At the end of the project, there was a need to assess achievements in the past five years and to draw lessons that could be scaled up and replicated in other projects and communities. This is a report of the evaluation conducted in May 2010 in the three countries where the project had been implemented. Objectives are presented, followed by the study methodology. The findings are presented to cover the structure and management of ARK, operations of the project, linkages with stakeholders, outcomes and impact as well as strengths and weaknesses of the project. The report concludes with recommendations and thoughts on the future and ways towards sustainability. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 15 II.OBJECTIVES The Goal of ARK was to expand and strengthen HIV / AIDS prevention efforts through behaviour change that would decrease risk among youths from becoming infected – primarily through abstinence and being faithful (A&B) and mutual monogamy. Risk reduction Objectives of the ARK project were to: 1. Strengthen the capacity of youth, adults and sub-grantees for A&B programs; 2. Increase family and community support at district, ward and village level for A&B; and 3. Create an enabling environment for A&B. This is an end-of-project evaluation of the performance of ARK over its five-year life span (June 2005 – June 2010). The objectives of this review are to: 1. Assess the achievements of the programme goal against the original log frame; 2. Determine the impact and results achieved over the project’s five-year life span; and 3. Identify the strengths, weaknesses, opportunities and constraints of the project implementation, and 4. Document lessons learned and provide recommendations for scale up of ARK and related projects. III. EVALUATION METHODOLOGY The evaluation focused on assessing the performance of the ARK project during its five-year life span, starting in June 2005 and ending in June 2010. The methodology took into account the ARK project’s Transformational Development Approach2 , to achieve its main goal and objectives. 2 Transformational Development Approach:  Children affected by AIDS, youth, families and communities move toward wholeness of life with dignity and justice; and  Designed to manage context instead of content providing people with necessary skills and knowledge to change contextual factors. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 16 The data collection, analysis and recommendations measured the extent to which ARK has achieved its objectives, that is:  to what extent ARK provided an enabling environment where youth, parents and teachers can explore, foster, support and provide positive traditional norms and behaviours as well as to help youth avoid daily risks and understand how to challenge harmful norms that are barriers to positive health practices;  assess the outcomes (and impact) of the project on the beneficiary populations – assessing whether the outcomes of the project have been in line with the original log frame, and  Evaluate the linkages between the international implementers, national implementing partners and the relationships beneficiary populations of parents, teachers and groups of young people. Study Design and Site Selection Two sites from each of the countries were selected for evaluation. The rationale for the site selection was that these areas were covered in both the Phase I and II Mid￾Term Reviews to allow for more accurate comparison and assessment for the end of project evaluation. The selected sites were as follows:  KENYA: Teso and Suba Districts  HAITI: Central Plateau and the Isle de la Gonave regions  TANZANIA: Monduli and Kagera (Bukoba Rural, now known as Misenyi and Karagwe) districts Data Collection Techniques The data collection techniques adopted for the ARK end of project evaluation include: a guided desk review; structured interviews, key informant interviews (KII), semi￾structured interviews and focus-group discussions (FGD) (see Appendix). The guided desk review of all relevant available information about the ARK project, both in general and for each country was conducted. This included the information listed below:  ARK’s Programme Design / Proposal  ARK’s log frame  Quarterly Reports  Annual Reports  The mid-term review of the ARK Project The Key Informant Interviews were conducted on policy issues to explore the overall structures and initiatives for ARK. Based on a semi-structured interview guide (which included open-ended and close-ended questions), key informant interviews were conducted with key managers, officials and international implementing partners both at national and local office. For instance, interviews were conducted with representatives from the John’s Hopkins Bloomberg School of Public Health, Centre ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 17 for Communications Programs (CCP), PEPFAR / USAID and ministries of health, education and agriculture in Haiti, Kenya, and Tanzania Quantitative Structured interviews: For the end of project assessment, two quantitative instruments were included to gather data from the two youth groups 10-14 years and 15- 24 years. These were administered to learners in schools. The quantitative instrument gathered data on background characteristics of the learners, their knowledge of reproductive health, their attitude towards various HIV issues, sexual practices, life skills and livelihood. These were topics covered during ARK interventions for the young people. Focus group discussions (FGDs) were conducted to assess normative behaviour in order to assess the impact of the ARK project on the beneficiary population groups within each site. Specifically, FGDs were conducted with youth and parents at each of the selected sites in the three countries. This was done to get the normative views of the different beneficiary community members of ARK. Training and Field Work Two-day training for the evaluation team and data collectors preceded the fieldwork. Data collection was conducted between the 15th and 31st May, 2010. In Kenya, training was conducted for 28 data collectors at the two training sites (Suba and Teso districts) while in Tanzania, 17 data collectors were all brought together to Arusha for training. In Haiti, a field team of eight was trained at the WV national office in Port au Prince. At each of the training sites, the data collectors were trained on the general principles of data collection for qualitative and quantitative data collection methods as well as the actual content of the data collection instruments. Topics covered included knowledge, attitude, practice and behaviour for reproductive health and HIV prevention. Final sampled/selected figures on which the report is based are indicated below in Table 1. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 18 Table 1: Distribution of final sampled/selected units for the evaluation by country and age groups Target Group Sample Size per country Quantitative Kenya Tanzania Haiti 10-14 years 401 406 208 15-24 years 417 397 203 Focus Group Discussions 10-14 years 5 5 4 15-24 years 5 5 4 Parents - 2 2 Indepth Interviews PEPFAR - 1 - Johns Hopkins - 2 2 KII CBO 5 2(meetings) 2 KII FBO 2(1 interview+1 meeting) 2(meetings) 2 KII 9 21 12 Other discussions Parent/Youth Action Groups (PAG, YAG) 3(1 YAG+2 PAGs meetings) 3 (PAGs meetings) 2 (Peer educators) Training Of Trainers (TOTs) - 2 (meetings) KII=Key informant interview Data processing and analysis The teams collected qualitative and quantitative data using FGDs, in-depth interviews as well as structured interviews. For qualitative data, data was analysed using themes formulated by the researchers. Key thematic areas were KAPB, organisational issues for the partners, linkages of the project with stakeholders, recommendations for the future and how the project could be sustained. The software, Nvivo was used in analysing the qualitative data. Quantitative data for the two youth groups were gathered from schools. The software Epi Info was used for creating a database as well as generating frequency distribution tables. Further analysis was conducted using the statistical software Stata. Data was entered at the three different sites in Kenya, Tanzania and Haiti. Limitations of the Evaluation One of the limitations of the evaluation was that the long term impact of ARK could not be more precisely ascertained. There was no baseline that could be used nor was there a comparison site. Going to a comparison site would have required more resources which the project did not have. However, there were case studies from non ARK sites. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 19 Quantitative data was captured differently for site in Kenya compared to the sites in Tanzania and Haiti. There were challenges in standardizing the EPI INFO data entry template. Multiple responses were captured as mutually exclusive responses and this affected the comparative analysis between the countries, that is, the Kenyan data could not be easily compared to that of Tanzania and Haiti for the youth group 15 to 24 years old. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 20 IV. FINDINGS The findings are presented based on the Terms of Reference. The first section covers the overall structure and management of ARK. The management of the project provided the context within which the project was implemented and this was not consistent throughout the sites of the project. There were differences in operations, linkages with stakeholders. These issues are important to help assess the sustainability of ARK. Secondly, the outcomes and impact are discussed and this section is based on quantitative data. An assessment of the strengths, weaknesses, opportunities and constraints is made. The report is concluded with a discussion section followed by recommendations. a. Operations and Accomplishments of ARK Project 1. Overall Structure and Management of the ARK Project The implementation of the project was designed to be within the structure of World Vision’s Area Development Programmes (ADP). During the first three years, a part￾time Project Director, originally based at WV headquarter in DC, together with a Regional Project Manager stationed in Nairobi, was in charge of project implementation in Kenya and Tanzania. Oversight was from WVUS and JHUCCP headquarters. Programme Managers or National Coordinators were stationed in each of the three countries in Haiti, Kenya, and Tanzania. For Haiti, the Project Manager directly reported to World Vision in the USA. The Regional Project Director in Kenya left his position and the post was not replaced. At this point, the overall management was filled by a Director stationed in the World Vision headquarters in Washington DC. At the same time, the National Coordinators were the first level of managers of the project in country. Program-wise, in each country, they reported directly to the Project Director in the USA, a position which also had high turnover. The ARK project was directly implemented by WV in all the three countries. However, in Kenya, two subgrantees were identified and they were used in the first two years of the project. The subgrantees had a specific role which was to train peer educators. The two subgrantees were the Scripture Union and the Kenya Students Christian Fellowship. However, recognizing the limited capacity of the subgrantees, in the third and fourth years of the project, the ARK Project modified their role and they became contractors for specific services. World Vision was the prime contractor to USAID and the organisation was responsible for the overall coordination of the programme; on the other hand the Johns Hopkins University Center for Communication Program was a sub contractor with a specific mandate to provide technical support in communications. WV supervised ARK coordinators, programme implementation and design, community programming activities and contract management. Annual and semi annual reports which were for monitoring the project were submitted to the ARK director in the USA for further distribution with the Donor-USAID and other relevant parties. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 21 With this set up, the end of project evaluation sought to identify strengths and bottlenecks in general management, coordination, communication and supervision in the implementation process. Responses were synthesized from interviews with sub￾grantees, donors and senior programme staff members as well from government departments. a. The ARK Project and Johns Hopkins University Center for Communication Programs (JHUCCP) Coordinating a project that is located in three countries with WV as the prime and JHUCCP in Baltimore, USA as Subgrantee, would pose challenges under any circumstances; even under situations where roles and responsibilities are clarified. This arrangement was important given the fact that the project was centrally funded by PEPFAR in the USA but the JHUCCP did not have a big presence in the three countries. In the later life of the project, project funds were administered through the broader JHUCCP account in the Dar es Salaam office and yet approvals had to be obtained from the USA. This arrangement slowed down the implementation process because financial approval had to be obtained from Baltimore. The fact that the manager was located in the US in the later part of the project posed further challenges. b. The Ark Project and World Vision Senior staff members from ARK and World Vision indicated that there were no difficulties in the relationship between the project and other WV structures or other projects. The ARK project was designed so that it would operate within the framework of the Area Development Programme (ADP) of World Vision. The ADP Coordinator and ARK district coordinator were at the same level and this initially caused some confusion in terms of hierarchical relations. The two had to report to the World Vision Zonal Manager who also supervised the two officers. In Haiti, the HIV/AIDS Coordinator at national and regional levels oversaw the ARK Project. At the ADP level the HIV/AIDS “animators” were supervised by the ADP Coordinator. However, for any technical matters they reported to the HIV/AIDS Coordinator. At local or district level, the relationship between the ARK project and the WV ADP presented two situations: the first scenario was of a situation where the ARK was located within the broader WV office. For instance, this was the case with Suba in Kenya, Misenyi in Tanzania and the sites visited in Haiti. The second situation was of a case where an ARK office was opened solely to support the project, for example in Monduli, Tanzania and Teso in Kenya. Outcomes in terms of functionality were observed to be different in the case where there was an ADP office and where it did not exist. Where the Ark Project was housed within the ADP offices, there project benefited from a better infrastructure and administrative support. Where there was no ADP office, the Project District Coordinator had to report to an office outside the area and transport and communication posed a challenge. Even during the end of project evaluation, transport posed a problem for the data collectors. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 22 ARK started first in Kenya and followed soon after in Tanzania. Haiti followed a year later. A staggered implementation process had advantages since the late starters were able to learn from the early starters. In Haiti, key informants testified that a later start in that country made it easier to get started with the ARK Project within the well coordinated structure of the AIDS and Health Programmes nested within the ADP in Haiti. La Gonave and Plateau Central were selected for ARK on account of this strength and the project was built on a well established structure. This ensured that there was better integration at all levels. All cadre of staff up to animator level were designated as HIV and AIDS officers and not just specific to ARK. At national level ARK was managed by the HIV and AIDS Programme Coordinator within the Health Section of World Vision. Key informants at national and regional level articulated how well ARK was integrated into existing HIV and AIDS programmes. In the field, at ADP level however, this integrated approach was not clear to some of the coordinators and animators. The staff viewed themselves as being responsible for ARK and sometimes would mention “another project”, specifically Child Hope. This perception may be either because the evaluation was focussing on ARK or that the integrated approach is not well communicated to the field staff. Integration of HIV prevention programmes for field staff would ensure sustainability of ARK activities after the end of current funding. c. ARK and sub-grantees In Kenya, ARK had been working with two subgrantees; the Kenya Christian Students Fellowship (KCSF) and the Scripture Union of Kenya (SUK). The KCSF trained teachers and youth on HIV/AIDS prevention and life skills using the Tambua Life Manual in the five districts of Suba, Naivasha, Teso, Migori and Bungoma. The Scripture Union (SU) also trained Peer Educators and Peer Influencers (Teachers) in the same five districts. Senior staffs from the Kenya Christian Students Fellowship (KCSF) were of the opinion that their relationship with ARK was functional but there were areas that could have been improved. In particular, there was a desire for greater involvement of the KCSF in decision making and implementation. The perspective of the sub-grantee was that their organizational structure and institutional expertise was not optimally utilised. The partners expressed the view that the ARK project did not integrate well into the structures of the KCSF and SUK. Further, they would have liked greater grass root level involvement in mobilization and follow up in the districts covered by KCSF and SUK. It was however, explained by ARK to the evaluation team that the structural changes were due to the partners’ limited capacity to manage huge resources. They were then used as training contractors. The sub-grantee organisations felt they were considered as just “any other training consultancy firm”. This was the case in the later years when the ARK Project contracted sub-grantees for specific training activities. Consequently, there was no proper follow up of trainees and therefore, sustainability was not well established. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 23 2. Key Interventions and Accomplishments of ARK Project The key interventions of ARK were primarily dissemination of Information, Education and Communication (IEC) materials and conducting training on HIV and AIDS for the reduction of risky behaviour among youth, parents and community members. The main components of the interventions were carried out through training of youth, parents and community leaders, radio messages, dialogue meetings, common ground melting pot meetings, drama, poems, and handouts, and distribution of ARK commodities (T-shirts, bangles, bags, and ARK passports). The discussion in this section is based on findings from focus group discussions and in-depth interviews. The basic interventions of the ARK Project were as follows;  Training on ARK  Radio Messaging  Video Shows  Sport Activities  Community Dialogue meetings and drama  Promotional activities and materials  Linkages and collaboration with stakeholders In the three countries, all the interventions were implemented but the scale varied. For instance, in Haiti, sports and video shows were not strong activities in comparison to the other two countries. Training Training was the main intervention for ARK. Training was conducted for youth, parents, teachers, NGOs, CBOs and representatives from health, education and agriculture ministries. Trained facilitators were to become TOTs, peer educators youth mentors and coaches whose function was to train others. Facilitation guides, namely “Tambua Life: A Value Based Life Planning/Skills Guide for Youth (10-14 year olds)”, “Tambua Life: A Value Based Life Planning/Skills Guide for Youth (15-24 year olds)” were used. The guides were designed to promote abstinence as one of the key strategies to avoid HIV infection risk among young people. As stated in the introduction of the guides, the training is aimed to “encourage young people to learn more about risks to their health and make conscious and informed choices to change their attitudes and behaviours”. For the two age groups (10 to 14 years and 15 to 24 years), training was the same in content and process but with varying degrees of depth for some of the topics. The materials were intended to be used by youth, mentors and coaches for training key intervention groups. Training covered the following units and topics:  human development- the human body, body image and puberty  values and behaviour- values, gender, religion and values, and the media and values  sexual health- sexuality throughout life, rape and sexual abuse, and HIV and AIDS,  relationships – my family, friendship, love marriage and parenting, and ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 24  personal skills- self-esteem, understanding risk, decision-making, communication and looking for help The main differences between the guides for the two age groups were the additional unit and topics for the older age group as follows:  Putting it all together- this was aimed at consolidating all the main lessons on HIV prevention. The topics include; journey of hope and looking for help  Relationships- marriage Supplementary materials were also later added to training to cover topics that were either not covered at all or covered in much detail in the original guides. They were targeted for the age group 15-24 as it was felt that some of the content was ‘too mature’ for the younger age group. The topics included:  multiple concurrent partners  cross generational sex  transactional sex  condom use  male circumcision  gender based violence, and  alcohol and drug abuse. It was recommended that these materials be used as additional sessions; to be added to specific topics in the main guides, mainly sexual health; or as standalone discussion topics during group sessions. The evaluation team’s impression of the ARK training materials is that they are indeed ‘value-based’ as stated in the title of the guides. The values are grounded on religion and how it affects different areas of people’s lives including health, relationships, behaviour, including sexual behaviour. However, it was felt that in the supplementary material the ‘value-based’ emphasis was not as evident. Linkages with Stakeholders The successful implementation of the ARK Project interventions depended on links with organisations that interacted with the communities where the project operated. ARK, by design, was meant to work with existing community structures to maximize the impact of its intervention. Links with other organisations and projects had a bearing on issues of coverage and sustainability of the project. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 25 The linkages of the ARK project in the three countries are highlighted as part of the operations. For all the three countries, the views presented on linkages between ARK and stakeholders were drawn mainly from key informants and group meetings. A more detailed description of the interventions and linkages by country are presented below. Kenya a. Training From teachers and parents in Kenya, discussions indicated that the training was adequate and of high standard. Participants recognized that the staff that trained them knew their content clearly and that they were friendly. Consequently, the training in Kenya was perceived to be having good outcomes in terms of imparting knowledge and the ability to train other people. As illustrated by key informant in Kenya ‘ARK has trained me and I can train others.’ (Teso, KII) Youth from FGDs indicated the value of the life skills training that they had received and how it helped them to make informed decisions on important sexual and life skills issues facing young people; improving on self image, associating with right friends and resisting peer pressure. Some illustrations follow; “It has helped us with self esteem and to accept who we are and what we have” (Suba, 15-24 FGD) One girl said “I used to have girls who are drop outs as my friends and after ARK training I know they can influence me negatively and now I have good friends who encourage me with my school work”. (Suba, 15-24 FGD) One boy said “I am now able to see peer pressure and deal with it. I have managed to change a brother of mine who was a drug addict. Now he has changed” (Suba, 15-24 in school FGD). Parents acknowledged that the life skills have empowered their children. This was exemplified by the quote: “Life skills education has scored highly improving the bargaining power against sexual abuse among the youth” (Suba, KII). Parents reported additional benefits to their family life and improved relationships. “Most families now talk about sex openly and the couples practice complete faithfulness” (Suba, parents, FGD). ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 26 b. Radio broadcasts The use of radio had worked very well in the two districts in Kenya. ARK used local radio stations to air messages on A & B and sexual risk reductions your youth. The content covered in the radio messages was consistent with that in training with minimal variations in the format from place to place depending on the cultural context. The intention was for the different kinds of media to reinforce each other and to reach new audiences in cases where ARK was unable to reach. In all the groups and interviews conducted many participants mentioned that radio made a huge impact in terms of behaviour change in the community such as reduced early pregnancy and school drop outs. For example, “I liked the radio presentation programs; some of our people in the society have experienced drastic change in behaviour through listening to radio programs” (Teso, 15-24, FGD). Parents also liked the radio and suggested that they be involved more in the radio shows “ARK should considerably involve parents in their radio programme as facilitators or to give their life experiences” (Suba, Parents Group Discussion). Youth from the FGDs liked the fact that the radio reached the wider audience than it would normally do and that it enabled people who are illiterate to get information. They also felt that it was another useful instrument in encouraging communication with parents. c. Video shows Videos shows were part of interventions over the five years and a number of videos were available for use3 . Different age groups recognized the importance of video shows and lauded that as a good approach. An FGD participant gave her response; “‘Bush Fire’ that trained people to focus on their goals and avoid any risky behaviours was good” (Suba, FGD 15-24). 3 Examples of videos shown in Tanzania and Kenya include: STIs; Mother to Child; Yellow Card; A Fighting spirit; Center (series); Neria ;Dead Mums Don't Cry; Think (Sex or Love); Joto la Roho (Swahili); Why Me? (Kwa nini Mimi? Ho Ea Rona; Fikiri pamoja; A red ribbon around my house; The faces of AIDS; Be Afraid (Ogopa Ukimwi); Ngoma ya Fatuma; Gharika la ndoa; Johari; Sikitiko langu ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 27 d. Dialogue, Meetings, Drama The dialogue meetings conducted for youth to youth, parents to parents and parents to youth. These were very popular in all the places visited. According to both adults and youth who participated in FGDs and interviews, the dialogues improved communication between children and adults. For parents, the dialogues enabled them to confidently talk about sexuality issues with their children. They also enhanced the parents understanding of youth. “We have been made aware of the challenges facing young people through dialogues, drama and other outreaches. We have been made aware of our role to support the growth and development of our children” (Suba, Meeting, parents). According to one youth: “ARK program encouraged youth on creative arts like drama performance, poem and composition writing, public speaking, drawing and arts” (Suba, FGD 15-24). The youth felt that they had learnt additional skills that were useful and important in their lives as a result of their involvement with ARK. ARK created a space for them to explore their strengths and to express themselves. e. Sports and other competitions Sports as outreach activities were used to promote exchange of ideas amongst peers. In Kenya, children in the age group 10-14 years consistently mentioned sports ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 28 activities and what those meant to them. For instance, a youth who participated in an FGD, remarked, “ARK sports and tournaments provided a learning ground on life skills and promoted youth interaction.”(Suba, FGD 10-14). Even in older age group of 15-24 years, there was a call to do more in this regard. In addition to organising sports, ARK was involved in promoting an exchange of ideas among youth through competitions of poems, drama and debates. This seemed to work well as children clearly enjoyed the spirit in which these competitions were held. They liked going to other places, meeting other children and said that the prizes they won in the form of mathematical instruments and ARK commodities motivated them. f. Other ARK Promotional Materials Artefacts such as T-shirts, bangles, bags, and ARK passports and murals were highly valued by beneficiaries in the project areas. According to information from the FGDs and KIIs “Provision of ARK t-shirts, caps, bangles has strengthened ARK messages and many youths have been involved in ARK activities as a result” (Suba FGD 10-14). In Suba, ARK murals were visible in schools and at bus stops and this reminded people about the message that ARK sought to pass to communities. They were viewed as valuable in sustaining ARK messages. There were some challenges associated with availability of various promotional materials. Inability to provide enough ARK T shirts, bangles, caps and mathematical sets for all pupils who were involved in the project was consistently mentioned as a weakness in the discussions and interviews. A participant concluded “Unfair distribution of ARK materials such as bags; t-shirts and bicycles sometimes caused controversies” (Suba, PAGS, and Meeting). All participants interviewed in the ARK project reported that handouts were important, useful and reliable source of information that should be given to participants whenever possible. They were also viewed as key to sustaining the messages of ARK. Many in the groups however mentioned that they were always in short supply and that whenever they attended some course teaching there was a shortage of materials. They found this disappointing. g. Linkages Key Informant Interviews provided insight of how ARK approached and worked with key stakeholders and thoughts on the successes and the limitations of the approach. Informants from relevant government departments, FBOs, CBOs, NGOs and other influential entities in the communities were identified and approached by ARK to be interviewed by the evaluation team. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 29  Views from government The relationship between the ARK Project and government departments at operational level varied from project site to project site but in general, there was overall appreciation that the project closely collaborated with government departments. Typically, the departments that had established a relationship with ARK were those of education, local government, youth, culture, agriculture and health. At both sites, key informants from government acknowledged that ARK strove to involve all stakeholders in the community. A religious leader acknowledged the participatory approach, “ARK program has been effective in partnering with other organizations e.g. Ministry of Youth Affairs and Sports” (Teso, Demco Religious leader). A government officer in Suba surmised “ARK was able to use the right structures through the provincial administration” (Suba, DO Central, KII). Involving other participants helped the ARK messages to reach further, that is, “involvement of other development partners within the division helped in dissemination of their message of A&B and risk avoidance” (Suba, DO Central, KII). “ARK has promoted free communication on HIV/AIDS through reaching out to different leaders e.g. chiefs, assistant chiefs, beach leaders, CBOs, youth groups, FBOs” (Suba, DO Central, KII). The same officer recognised a positive outcome in using the wider approach “Through reaching out to the beach management union (BMU) leaders ‘fish for sex’ practices have been tremendously reduced e.g. Women/girls are able to resist fish for sex (Suba, DO Central, KII).  NGO and other organisations Key informants from NGOs and acknowledged that they were involved in ARK training and that ARK interacted with them, “The ARK officials were good people and were very much ready to co-ordinate with other NGOs staff” (Teso, KII). An appealing element of the ARK programme is the message of “abstinence” to young people. It was well accepted by the parents especially the religious leaders. “ARK worked with the whole community, starting from children to adults, church leaders, government leaders and other community leaders” (Suba FBO, KII). However a number of concerns were raised. Firstly, involvement of local leaders could lead to their interference, “The first constraint was interference from local leaders, that is, the leaders were forcing them to employ people from their communities (against the rule)” (Teso, KII). At the same site, there were also complaints about lack of clear communication about issues. This could be as a result of the inadequate infrastructure of ARK; at this site they did not have a fully operational ADP (World Vision) office. Through FGDs, youth in school recognized the benefit of their teachers’ involvement in ARK. They sighted this as one good reason why ARK worked well in their schools because they could participate freely and make time for ARK activities. In cases where ARK organized activities out of school, pupils spoke at length about how these links gave them positive exposure, promoted positive interaction and exchange of ideas amongst them as youth. They enjoyed going to other places on trips and ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 30 enjoyed participating in various competitions and activities. This is what one peer educator said: “ARK has brought exposure we can now go anywhere and exchange ideas with others freely, the tours and trips they offered to us were helpful in that we were able to meet people from other districts and share core messages” (Teso, 15-24 out of school, FGD)  The Community Participants in the FGDs reported that people in the communities also benefited from the linkages that ARK had made with the government ministries in that they were able to have government officials to participate at meetings. They mentioned that they had established a relationship with them and government officials participated in community activities. Bringing together different stakeholders in the communities had spinoffs for community members. Participants mentioned that they felt they were relating better to government officials. They said “The government officials had been involved in most ARK seminars, hence we get to know each other and relationships have been made strong in such a way” (Teso PAGs). “I relate well with hospital nurses, they are no longer harsh to us” (Teso, out of school 15-24, FGD). In addition, government officials were perceived as responding better to invitations as one participant said “The government is showing positive response whenever they are called upon and the government contributes positively towards the program” (Teso, PAGs). Tanzania In Tanzania the evaluation was conducted in Monduli and Misenyi districts. Monduli is dominated by Maasai people who strongly believe in cultural practices surrounding sexual behaviour that may increase the risk of HIV infection (expounded under training below). It was reported that there had been very few HIV/AIDS interventions before ARK started its operations in Monduli. Misenyi, on the other hand, is an area in Bukoba on the border of Uganda and Rwanda and it is a border town with all the typical challenges of a border and transitional town. There is a heavy inflow of people from neighbouring countries, heavy trucking business and consequently, transactional sex seems to be a major challenge. The community responses to the interventions in the two areas are discussed in this section. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 31 a. Training In Tanzania participants in the FGDs and in depth interviews reported that ARK training increased their awareness about HIV/AIDS. The participants commended the training model which was based on the ‘training of trainers’ approach. A participant from the Monduli site remarked “ARK has been able to produce peer mentors who have attended different seminars and they came back to teach the others and hence the messages spread further” (Monduli, Meeting). Youth in all the groups reported that they received training on how to avoid risky behaviour, and they mentioned in the FGDs that training in life skills has helped them change their ways to good behaviour. Youth in all the groups mentioned that they were taught about what risky behaviour was and how they could avoid it. “The education program has been able to change behaviour mostly of youth towards the prevention of HIV and AIDS” (Misenyi FGD, 10-14 out of school). Some examples of behaviour change included: walking around at night, watching pornographic movies, having many boyfriends and girlfriends, avoiding bad company, smoking, avoiding esoto (night dances where youth have sex) and taking money from strangers. Many pockets of areas of the Monduli society seemed not to have been reached by HIV/AIDS messages before ARK; “Before the ARK project the community was not aware of HIV/AIDS but now they know” (Monduli, FGD 10-14). The participants in the FGDs and in depth interviews reported that ARK approach was unique and participatory. They also liked that ARK, especially in the Maasai areas, used members of their community to pass its messages. This approach endeared ARK even more to Maasai communities as it showed that they respect them and were sensitive to their needs as a community. The participants indicated that they were respected by the facilitators and this was something that motivated them to learn. They also said “We had training guides which made the work easier” (Monduli, FGD 10-14 in school). Teachers, parents and youth in the meetings, FGDs and in depth interviews talked at length about the harmful cultural practices, namely esoto, female genital mutilation (FGM), polygamy and wife inheritance that were likely to spread HIV infection. In one group discussion parents mentioned that: “Some parents are no longer mutilating girls as they are aware of health consequences” “It has taught the society and helped them realize the effects of polygamy and how it leads to AIDS infections” (Monduli, Parents, FGD). In the youth FGD, participants said ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 32 “I have also stopped to attend ‘esoto’ and I now just use abstinence” “ARK has really saved me from contracting HIV/AIDS as I used to have sex with many girls but now I have decided to change and stay with one and be faithful to each other” (Monduli, Meeting of YAGS, PAGs, peer educators). In Misenyi, the participants reported that the interventions among parents and teachers were not adequate, as they were trained only once and that had been a long time ago. This was confirmed by the participants in the youth FGDs when they said “They [ARK] have not reached the whole community mostly the parents”. In a youth FGD one of the participants remarked, “ Some of our parents are not taught about sex and other things like that and do not talk about them to the extent that we cannot face or talk with them as they will punish us” (Misenyi, FGDs). What the children said about parents was confirmed by researchers’ observations who found that their understanding of questions about the programme was very limited. b. Radio broadcasts In Misenyi children participated in the radio show through listener groups discussions. Some of them called in to the radio station to ask or answer questions during the programmes. Young people liked participating in the radio talk show and this was mentioned on several occasions during FGDs. The outcome of the radio intervention is summarized by a youth from Misenyi; “The radio outreach was successful in reaching more people and youth involvement on the programs was huge and motivating” (Misenyi, FGD 15-24). In Monduli children did not mention radio much but the outcomes of the radio were summarized by a teacher in an in-depth interview when he said; “Radio programmes and seminars conducted by ARK brought great awareness among the stake holders and community as a whole” (Monduli, KII Teacher). There were some challenges however as children reported that “Teachers restrict the listening of radio programmes” (Misenyi, FGD 10-14). The teachers imposed restrictions at school because they felt that no time was specifically allocated to the project. This may also be associated with schools where teachers were not adequately sensitized. c. Video shows A wide range of videos were available for show by ARK staff to different audiences. The selection was both in English and local languages, that is KiSwahili in Kenya and Tanzania. The topics were varied and covered reproductive health, life skills and ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 33 social life issues. These were great avenues for creating a conducive environment for abstinence and be faithful messages. Participants in Monduli FGDs and meetings thought video shows were powerful because people who could not read could watch, see and understand. The perceived impact was demonstrated by the quote from one of the participants; “The video show was the best way to reach people especially youth, we saw in reality how HIV is. It showed HIV as it is, practically. It showed how you get it and how we can avoid the risky behaviours that can lead us to HIV. We know why we have to be faithful and we know why it is the safest thing to abstain” (Monduli, Group Meeting, 15-24 years) Another participant commented that; “The use of video shows, a drama and radio program has made ARK known even outside the project area” (Monduli, 15-24 in school). d. Dialogue, Meetings, Drama In Misenyi video shows as training method were less frequently mentioned and when that occurred it was in the context of misuse by other members of the community who showed pornography. Pornography videos were shown in the afternoons at certain points in the city. It was in this context that the power of the dialogue meetings was harnessed when concerned community members used the dialogues to challenge people who showed such videos. To deal with this problem dialogues were facilitated and different stakeholders like the police, the parents, religious leaders were called and this problem tackled. Each stakeholder had to say how they are tackling the problem. That was handled successfully and pornography videos are no longer shown publicly. Participants in all adult and youth FGDs loved drama. One participant commented about the entertainment value of drama that it entertains while at the same time educating the community. FGDs and in depth interviews with parents and KIIs indicated that drama and dialogues played an important role in sensitizing people about VCT. As one participant put it “Seminars and dialogues have motivated people to check their HIV status and consider the treatment once they know they are infected.” Misenyi, KII ADP Youth in Misenyi did not mention the dramas as much as the adults did. In Monduli the youth spoke highly of dramas and dialogue meetings. This difference could be as a result of the differences in cultural context. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 34 e. Sports activities and sports equipment The distribution of sports materials and organisation of sporting events by ARK was recognised by community groups in both project sites in Tanzania as one of the strong points of the ARK project. The project staff distributed sports kits (footballs and basket balls) that were received from World Vision US Gift in Kind products. In some cases, sports uniforms were bought for participating teams. A key informant from the district council in Misenyi observed that the provision of sports facilities promoted sports competition among schools in the Misenyi ward; the result is that youth are now busy with sports instead and their minds are diverted from thoughts about sexual matters in school. f. Other ARK promotional materials Participants cited ARK materials such as t-shirts, bags and murals. T-shirts were so popular that one youth participant remarked that “T-shirts should be provided to all students and not to ARK members only” (Misenyi, FGD 15-24 in school). However, not all ARK promotional materials were recognised by groups in Tanzania. According to the researchers in Monduli, youth in the 15-24 year age group in the school FGD were not aware of ARK passports, bangles and other IEC materials. These materials had not been distributed to in those areas. Linkages As in Kenya, interviewees in the KIIs revealed that ARK worked closely with relevant government departments and that this relationship led to good cooperation between ARK and the government departments. Pupils in FGDs in Monduli confirmed that it was easy for ARK facilitators to access teachers and pupils for the ARK training and seminars as they were able to attend easily. However, as noted earlier, some pupils mentioned that time to attend radio programmes were restricted by some educators. In the case of a local government one district official said “There was a good relationship between District Council and ARK” (Misenyi, KII). NGOs and other organizations In all the interviews participants mentioned that ARK involved all the community stake holders in their trainings. The participants also talked about how ARK made sure through the meetings and dialogues with leaders that the community understood the objectives of the project. The general view from Tanzania was that ARK project coordinators made an effort to ensure that the community based organisations were fully conversant with the project objectives before the project commenced implementation. This, according to one participant interviewed, made it easy for parents to allow their children to participate in ARK activities and break the silence on talking about risky behaviours and sex in particular with their children (Misenyi, Mapec, KII). ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 35 ARK’s effective interaction and engagement with the faith based communities was praised in the communities. This is highlighted in this quote; “ARK program has been able to implement its activities effectively regardless of religious differences and biasness” (Misenyi, 10-14, in school). This was confirmed by one of the religious leaders who reported that “I have built a good network with others FBOs” (Misenyi, KII, Religious leader). A case study of a harmonious relationship is illustrated by the relationship that has been established between Christians and Muslims in Monduli [see Appendix 1]. The Community In Monduli participants said that they admired the way that the ARK project reached out to almost all groups of community at different levels compared to other projects they had had experience with. They stated; “ARK mobilized community stakeholders like CBO, FBOs and local government leader’s parents, teachers, youth and realized we have one common goal. It brought us together and encouraged us to work together” (Misenyi, 15-24 out of school). In other FGDs they mentioned that they liked the cooperation of ARK with village leaders as they said this made it possible for them to access the communities. As one participant reported; “ARK messages have reached even the traditional leaders and even the healers who are often hard to reach and change” (Monduli, PAGS, Meeting). It was clear from what the participants said that reaching traditional leaders made it easier for people to accept ARK activities and they were more likely to listen to ARK messages as a result of the engagement of the traditional leaders. C. Haiti i. Training The standard ARK training materials were translated into Creole, the local language of the target population. JHUCCP modified the material used in Kenya and Tanzania for the Haiti context. A Behaviour Change Coordinator from JHUCCP was based in World Vision Haitian and coached field animators, as trainer of trainers. They in turn trained volunteer peer educators, who then trained young people, parents, and community and religious leaders at community level on ARK messages on risk reduction and behaviour change. A peer educator facilitated discussions for a group of a maximum 25 people at a time. When asked what the strength of ARK was, parents, community leaders, religious leaders and key informants unanimously mentioned the graduation ceremony after ARK training. The high level key informants from World Vision and the Ministry of Public Health and Population testified to this. Graduation after training is public spectacle! This was described as more than the awarding of certificates but a ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 36 ‘Community sensitisation. It’s a way of telling the community that these people are like you, after the training, they are a valuable resource in your community.’ (Key informant). For example, parents who have graduated become mentors and peer educators to share their experiences with other parents. It is also a public declaration of the commitment that one has made to practice abstinence or to be faithful. The people invited to such graduating ceremony included community based organizations, church, civil leaders and community members. The positive acceptance and enthusiasm for ARK among community members where it was implemented created a big challenge for the project. Community stakeholders complained about the limited reach of the ARK project in terms of the number of people trained and the geographical coverage within the ADP. The evaluation team also noted during the field interviews at schools that there was a sense of envy and admiration by some pupils for those that were participating in ARK activities. The issue of coverage was clarified at national level interviews. ARK was operating within specific ADPs (10) within the two regions. However since the beginning of ARK project in Haiti, more animators than those within these ADPs had been invited for training to implement ARK within their regions nationally. It was further emphasised that within the structures of World Vision all AIDS projects were managed centrally. As mentioned earlier, while ARK was built on the strength of existing AIDS programme structures, ARK was also of benefit to the other programmes through its peer educator approach. All ARK staff were designated as HIV and AIDS coordinators. For example, World Vision Haiti Hope Child Project has a youth prevention project within a comprehensive AIDS response programme covering prevention, treatment, care and support. The animators are responsible for and ARK and Hope Child programming. For sustainability the animators will continue training peer educators on ARK activities beyond the original ARK intervention regions. This is can be cited as a best practice on integration of ARK into existing projects of WV and sustainability of its activities. ii. Radio broadcasts A lot of airtime was sponsored by the project for radio spots, jingles and soap operas in both Plateau Central and in La Gonave to reinforce messages covered in ARK, educate the wider public and stimulate public discussion. The Radio messaging was powerful in reaching wide population. However, it was mentioned that it is limited to those who owned radio sets. In Central Plateau key informants mentioned that after the current funding cycle the radio will continue to be used as a form of communication on ARK and other health related matters. In two focus groups with young people participants reported having learnt about abstinence and faithfulness on the radio. However, it was not clear whether this was ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 37 specific to an ARK-sponsored programme. It can be assumed it is since during focus group discussions young people said that it was ARK that enlightened them on practising abstinence for HIV prevention and reduction of risk of early pregnancies. iii. Dialogue, meetings and drama Parents and young people commended ARK for improving communication between parents and their children and that of the parents themselves and even mending broken relationships among neighbours. One memorable thing that key informants mentioned about the graduation ceremony was testimonies of individuals who had reconciled with others. Fora for dialogue mentioned include: community dialogue (dialogue communauté) and child-parent dialogue (manman papa pitit an pale; paran pitit an’n pale). A focus group of religious leaders in Central Plateau proposed that this format of dialogue needs to be extended community-wide to enable different parties resolve conflict and live together in peace. Impact and sustainability of community dialogue and drama is exemplified in the narrative below. The case study illustrates the impact of ARK and some routes towards sustainability. One experience that has been documented and narrated to the evaluation team was the impact of drama training on peer educators. In collaboration with National School of Arts, 50 peer educators underwent drama training that included writing and the final act of staging a play on sensitisation of HIV/AIDS. The process and the final output were noted as best practices examples. The peer educators revealed remarkable talent. The play entitled “AIDS is among us” was produced into a DVD and reviews of it indicated that it is professional. It was reported that The Ministry of Public Health and Population also found it so effective that they were going to make it available nationally to reach out to other young people in promoting HIV prevention. iv. Other ARK promotional material One of the remarkable first impressions that the evaluation team got in La Gonave was the visibility of ARK project in the community. At first one may have thought that there were many young people with ARK bangles at interview venues because they were prepared for the team. However, it was exciting to see a member of the community wearing an ARK t-shirt even in remote areas of the island. One of the weaknesses cited was the short supply of training and promotional materials. It was limited to the people who had participated in the training. In addition, as mentioned before the training was also limited to specific number of people as per project design. However, the participants in the focus groups wished it was open to more people. i. Linkages  Government views The end-evaluation in Haiti was conducted in the two regions where ARK has been implemented: La Gonave and Plateau Central. Haiti provided a unique case for the ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 38 ARK project in that it was the only non-African and Francophone country. Apart from these geographic and linguistic differences, Haiti AIDS response is not as collaborated as it is for countries in eastern and southern Africa. The Joint United Nations AIDS Programme (UNAIDS) recommends the “3 Ones” approach in the National Strategic Plan for HIV/AIDS. This means having One coordinating authority, One strategic Framework and One M&E framework for AIDS programmes at all levels in a country. Being in the epicentre of the AIDS epidemic and having had UNAIDS support for longer, the ARK project sites in Africa countries are more experienced in being part of a coordinated national AIDS response. In Haiti, the AIDS response faces added challenges due to the 12 January 2010 earthquake. This has not only affected government and key stakeholder infrastructure and human resource capacity but also poses new socioeconomic challenges and threats to the population. While ARK project is winding down, World Vision Haiti has stepped up its relief programmes in the light of the earthquake.  Government and NGO Partnership The regulation and collaboration of NGOs with government in Haiti is still developing. Although there are guidelines, the Government has limited resources to institute mechanisms to foster such relationships. Some efforts have been made to map NGOs working in different regions in the field of AIDS in Haiti. However, key informants both from government and World Vision confessed that NGOs choose where they want to implement their programs. The Ministry of Public Health and Population has formulated a strategic document for the formation of a National AIDS Coordinating (NAC) body. However, there are no resources yet to launch it. In the absence of such a coordinating body AIDS programmes will continue being implemented in a piecemeal fashion. World Vision has visible presence in all the areas that were targeted for evaluation by the team and across the countryside from Port au Prince to Plateau Central. This is one of the strengths of World Vision Haiti and the national government has entrusted World Vision to run health and development programmes in the country. World Vision has specifically signed a Memorandum of Understanding with the government for all its HIV and AIDS programmes at all levels of operation. A government key informant remarked; “Ministry (of Health) and World Vision recognise that there was need to work together at community level without necessarily having a formal relationship”. This working relationship was said to be stronger at regional (implementation) level than at national. It should be noted that in Haiti, right from the start, ARK resolved to directly work with community-level partners such as schools, churches and development groups. During this evaluation, the team was unable to interview any government stakeholders at regional level due to competing demands on their side.  Community level At community level, no formal partnerships with other implementing partners existed. In La Gonave, up to about a year before the project end, ARK had an informal ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 39 working relationship with Concern, an NGO working with young people in schools. In Plateau Central ARK also informally collaborated with Zamila Sante, a health promotion NGO to implement some of its work. Otherwise, ARK worked with teachers, religious and community leaders in implementing the project. The number of health care workers sensitised to promote and support A&B messages increased during the project life from a target of 18 in the first year to 108 in the final year. Very few (a maximum of 6) government officials were recorded to have ever been targeted to be sensitised on ARK activities in Haiti throughout the life of the project. In one focus group with young people, they noted that they got ARK messages from home, school and church. This increased the number of entry points to the young people to reinforce the messages learnt. Key informants also commented that the young people who had participated in ARK were well cushioned by a supportive social network for as long as they were in their community. However, there were concerns that these supporting social networking might be lacking should they move to Port au Prince to pursue education and career opportunities. For example, it was reported that of the 305 peer educators trained in 2008 about 28% have left the project to pursue for further studies, jobs or because they left the area for other regions. Given that ARK volunteers are a young age group about to start a career; resources were continually committed to train replacement peer educators. While some of the peer educators for example were known to have been active in training others where they went, many would have lost the supporting social network that would hold them accountable. Two key informants discussed this at length and proposed that there should be programmes that ensure that these young people are supported beyond ARK. One possible solution was to link with them with other HIV prevention programmes at World Vision. In summary, the qualitative analysis shows that young people, parents, community and religious leaders, and key stakeholders from different organisations agree that ARK has accomplished its main goal. The different groups shared their experiences on how the ARK project has encouraged positive behaviour change in their communities. Young people shared that ARK has helped them reflect on their lives and to have goals. Parents testify how they were now empowered to communicate and to reach out to their children. There were also testimonies of how ARK has facilitated reconciliation among couples and brought peace in the homes and the community at large. Within the school environment, pupils, teachers and parents indicated that with ARK there were now less early pregnancies, less teacher-pupil relationships, improved school attendance and better academic performance. Thus an enabling environment was created at household and community level. 3. Monitoring and Evaluation As the ARK Project was funded mainly through USAID PEPFAR funds, the monitoring of activities had to fall within the country’s operational plans. Programme indicators were designed to report on PEPFAR’s mandatory output and outcome indicators in accordance with USAID Mission Country Strategy Plans for 2004-2010. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 40 The evaluation team found that a consistent data collection and reporting system was established in each of the project sites of the three countries. Monitoring and Reporting The progress for the project was monitored through monthly reports. Output indicators which mainly recorded the number of items according to the three strategic objectives and direct results of operational activities were collected for the project. Some of the selected output indicators include the following;  the number of youth trained in value-based life skills  the number of youth 10 to 24 years old reached with AB messages using interpersonal approach  the number of Anti AIDS Club members trained as Peer Educators and  the number of parents trained in parenting skills disaggregated by location. On a monthly and quarterly basis, data were collected (and where necessary disaggregated according to sex) by ARK District Coordinators and reported to the ARK Program Director. In Haiti the reports were compiled semi-annually. This was submitted to the key stakeholders and the donor including USAID. Data collection tools were examined for completeness and consistency at the different offices in the field. The research team found that the data were well collected and filed in the offices. There were records that corresponded to outputs of meetings held in the project area. The routine monitoring summaries (monthly, quarterly and annual) were also available and well kept. Quality Assessment All the project sites achieved targets for the output set out at the beginning of the project. However, the main short coming of the monitoring and evaluation system of the Project was that there was no in-depth documentation of the quantitative project outcomes and their quality. This concern was summarized by one key stakeholder as: “How do you ensure quality interventions that reach beyond the numbers? What does it mean reaching so many people?” A number of reasons can be attributed to this apparent discrepancy. One of them could be that the “Tambua Facilitation Guide” covers a wide range of topics expected to be covered in a total period of 24 hours or 3 days. A person is defined as ‘reached’ ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 41 when exposed at least one hour of ARK activities4 . This can lead to multiple counting of individuals who have had repeat exposures. In addition, the short contact period for ‘reached’ and ‘trained’ might have affected the depth of understanding of some of the topics as shown by the quantitative analysis under the outcomes and impact section of this report. Evaluations In the original project document, a baseline assessment was included and this was to be followed by a midterm evaluation in Nov/Dec 2006, and a final evaluation at the end of the project. Unfortunately, the baseline evaluation was not conducted because at the early stage, USAID/PEPFAR indicated that PEPFAR would carry out external project evaluations to ensure objectivity. The midterm evaluation was detailed in the process and findings are consistent with those from the end of project evaluation. However, the omission of quantitative data to measure outcome indicators during the midterm review was a limitation of the project. The midterm evaluation and final evaluation were used to assess the progress toward achieving the targets, the use and effectiveness of the strategy and interventions, program quality, promising practices, constraints and strategies used to address them, as well as recommendations to improve program performance. Outcomes of the midterm evaluation are compared to those of the end of project evaluation. The final evaluation used the same methodologies and focused on the achievement of the program, the changes at the program and beneficiary levels, sustainability of program benefits, and lessons learned. The final evaluation was used to monitor the degree of community participation5 and quality of community relationships6 . The qualitative data analysis has shown that ARK has successfully achieved these outcomes. The midterm review serves as a baseline evaluation; therefore findings from the final and mid -term reviews will be compared. The location of the ARK Project in the different countries and sites provided ideal opportunity for conducting Operations Research studies to identify different strategies of programme implementation and interventions. This opportunity was not optimized in the project. This could have been a constraint imposed by terms of reference of the donor and other key stakeholders. b. Outcomes and Impact This section uses quantitative data from the survey of young people who participated in ARK project to explore outcomes related to their knowledge, attitude and behaviour and practice (KABP). The findings are presented for two age groups; 10 to 4 World Vision’s definition of “reached” includes all individuals who receive at least one hour of exposure to the ARK A and/or B messages in interactive small group settings. ARK definition of “trained” is those individuals who have completed the full ARK life skills curriculum and/or at least 24 hours/3 days of exposure to the ARK A and/or B program through interactive small group sessions. 5 Men, Women, adolescent boys and girls perceive they actively participate in ARK program planning, implementation M&E. 6 Defined around dimensions of use (sharing) of community resources, gender relations, protection and valuing of children, well-being of vulnerable persons, and conflict prevention/resolution. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 42 14 years and the older group, 15 to 24 years to highlight differences in overlapping questions. The results are presented to give an overall picture for the three countries (Haiti, Kenya and Tanzania), and where relevant the data are disaggregated country wise. Indicators are comparable for the age groups 10 to 14 years for the three countries. For the younger age group, the total number of respondents was 1015, while 1017 youth were interviewed for the older age group of 15-24 years. Statistically significant levels are indicated as a column for all the relevant tables, that is, to indicate if the findings are of statistical significance. Statistically significant data have to be interpreted carefully within context, taking into account that statistically significant results may not necessarily translate into significant cultural or behavioural findings. For the 15 to 24 year old age group, not all indicators are comparable country wise. For some indicators, data from Kenya is directly comparable to that from Haiti and Tanzania. In Kenya multiple response questions were entered as single response questions that are mutually exclusive. During the analysis, comparisons are pointed out and clarified. The socio-demographic characteristics of youth groups are indicated in Table 2. The total number of participants in the younger age group (10-14 years) was 1015, of which 55 percent were females and 45 percent were male. Nearly two thirds (62 percent) lived with their fathers and mothers, with 23 percent living with mothers only, 5 percent with fathers only and 10 percent with relatives. As expected with this age group, 99 percent were attending school at the time of survey. For the older age group (15-24 years), 1017 completed the questionnaire; over three quarters (76 percent) were less than 21 years in age. Over half (56 percent) were living with parents, 24 percent with the mother, 8 percent with the father and the rest with relatives (12 percent). Only a small proportion was married (6 percent) while the remainder were single. Less than one in five of the respondents said they had a boyfriend or a girlfriend. The majority were currently in school (81percent). Table 2: Socio-demographic characteristics of young respondents for the two age groups (10-14years &15 to 24 years) 10-14 years 15-24 years Age Kenya (n=401) Tanzania (n=406) Haiti (n=208) Kenya (n=417) Tanzania (n=397) Haiti (n=203) 10-11 11 13 24 12-13 59 44 40 14 31 43 36 15-16 26 18 32 17-18 26 30 37 19-20 19 29 18 21-22 11 10 7 23-24 18 13 6 Who do you live ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 43 10-14 years 15-24 years Age Kenya (n=401) Tanzania (n=406) Haiti (n=208) Kenya (n=417) Tanzania (n=397) Haiti (n=203) with Mother 23 23 25 18 23 37 Father 7 4 4 9 8 5 Mother and father 59 63 63 62 57 44 Other relatives 11 10 8 11 12 14 Currently in school 99 100 100 96 63 84 Marital status Single 94 54 64 Have boy friend data not available 31 34 Married 4 12 1 Other 2 3 1 Total 100 100 100 100 100 100 Findings on knowledge, attitude and behaviour change with respect to reproductive health in general and specifically on HIV and AIDS is reported. Life skills issues are presented at the end of the section. For each of the thematic areas, the report has been presented in according to the two age groups 10-14 years, and then followed by the older age group 15 to 24 years. 1. Knowledge and Attitude: AIDS Stereotyping   Youth 10 to 14 years Issues of stereotyping people living with HIV were assessed through four main questions. From the four questions, the respondents were to indicate whether they agreed or disagreed with the statement. The level of agreement was divided into two categories; those strongly agreed or disagreed with the statement. In the analysis, responses were summed together to form two categories of those who agreed (agreed plus strongly agreed) or disagreed (disagreed or strongly disagreed). Figure 1 presents the attitude of young people towards HIV and AIDS. In general, the young people would show compassion to people with HIV and AIDS and not discriminate them because of their HIV status. From the same table, we see that over half of the young respondents believe AIDS is a punishment for bad behaviour and that they would feel ashamed if one of the family members got HIV infection. The differences between countries are significant and no clear explanation can be offered for the variation by country. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 44 Figure 1: Percent of youth agreeing (strongly agreeing & agreeing) to the statement on HIV and AIDS issues (10-14 years) The topic is further explored in Table 3, where we also note variation between the three countries with Kenya generally showing the highest percentage responses in expressed opinions. That there is wide variation in the responses for each of the statements between the countries indicates that the message is not consistently understood by this age group. One would have expected a very high percentage of a response from pupils to say “you can’t really tell” if one has HIV infection unless he or she is tested. Across the countries we note that there is general agreement, (that is, no statistical differences) that HIV infected persons will be “always sickly” and “that their mouth looks different”. Attitudes were examined in the context of the ARK Facilitation guides. There is an activity under HIV and AIDS topic, titled ‘Dealing with inaccurate information”. This involves challenging some stereotypes about HIV and AIDS. The evaluation team felt that the topic of challenges cannot have been adequately addressed in one session. For example in some of the FGDs in Tanzania, participants discussed one of the video tapes entitled ‘Faces of AIDS’ that they had watched during ARK activities. They referred to what the video showed with respect to people who have AIDS. One participant said that “scared him off” and made him to abstain. Clearly, addressing of inaccurate information on HIV and AIDS needs to be reinforced in HIV and AIDS interventions but clear transmission of information is important. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 45 Table 3: Percent of respondents reporting perceived looks of someone with HIV and AIDS (10-14 years) Attitude Overall (n=1015) Kenya (n=401) Tanzania (n=406) Haiti (n=208) P-value % % % % Thin 52 63 38 56  Their skin looks different 37 50 34 14  Their mouth looks different 39 39 37 43 NS Coughs 49 63 46 31  Always sickly 52 56 50 50  Hair looks different 37 38 27 55  You can’t really tell 19 18 27 3  Significance Level: p ≤ 0.001 NS=Not Significant * multiple responses possible Youth 15 to 24 years Issues of attitude and stereotyping people infected by HIV are explored with the older group of youth for the three countries the results are presented in Tables 4 and 5. As with the younger age group, responses showed general compassion for people infected with HIV; high percentages of this group would not discriminate or stigmatize people with HIV. At the same time the numbers are still high for those who consider AIDS to be a punishment for bad behaviour and those who would feel ashamed if a family member were to have HIV/AIDS. Table 4: Percent of youth agreeing (strongly agreeing & agreeing) to the statement on HIV and AIDS issues (15-24 years) Attitude Overall (n=981) Kenya (n=398) Tanzania (n=391) Haiti (n=192) P-value % % % % AIDS is a punishment for bad behaviour 41 34 47 43  I would feel ashamed if someone in my family got AIDS 33 27 36 28  I am happy to show compassion for people I love whether they have HIV and AIDS not 89 92 92 79  I do not discriminate or stigmatize people living with HIV and AIDS 86 88 95 63  Significance Level: p ≤ 0.001 NS=Not Significant For the older age group, direct comparison can be made between Tanzania and Haiti. For the two countries we observe that there is no statistical difference for the youth who thought that those perceived with HIV would be thin, that their skin or mouth would look different, and they would be always be sickly. In Kenya, the largest percentage of youth thought the infected people would always be sickly (36%), followed by those who thought it would not be possible to tell (24%). ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 46 Table 5: Percent of respondents reporting perceived look of someone with HIV and AIDS (15-24 years) Attitude Kenya (n=408) Tanzania* (n=397) Haiti* (n=203) p-value % % % Thin 23 47 54 NS Their skin looks different 5 34 45 NS Their mouth looks different 2 28 32 NS Coughs 4 51 28  Always sickly 36 52 50 NS Hair looks different 4 36 66  You can’t really tell 24 24 4  Fat 2 8 10 NS Total 100 N/A N/A Note: Significance Level: p ≤ 0.001 N S=Not Significant * multiple responses 2. Behaviour Youth 10 to 14 years The participants were asked what some of the ways were that young people can follow to avoid contracting HIV. Results are shown in Table 6 for the younger age group. Overall, the largest proportion (79 percent) of youth mentioned abstinence as one of the ways to avoid “getting HIV AIDS”. Results also show variation between countries. Responses from Kenya showed the highest response of 85 percent compared to Tanzania (78 percent) and Haiti (71 percent). Figures of young people who mentioned “being faithful” were far lower for the countries. Haiti scored significantly lower at 38 percent. The smaller percentages of those who mentioned “being faithful” could be a reflection of the age group of the respondents of which only 12 % reported that they had ever had sex. The use of condoms was scored by a sizable number of the young people with a range between 26 to 45 percent. Avoiding having sex with multiple partners was most frequently mentioned in Kenya (38 percent) and least cited in Haiti (5 percent). Table 6: Some of the ways young people can avoid getting HIV and AIDS (10-14 years) Attitude Overall (n=1015) Kenya (n=401) Tanzania (n=406) Haiti (n=208) p-value % % % % Avoid mosquito bites/sleeping under net 3 2 4 5 NS Sleeping with a minor 10 7 17 3  Abstinence 79 85 78 71  Being faithful 53 53 61 38  Using condoms 35 26 38 45  Not having sex with many people 27 38 28 5  Significance Level: p ≤ 0.001 NS=Not Significant ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 47 The young people were asked at a personal level what strategies they intended to use in future to prevent HIV infection. Overall, abstinence was ranked highest by the 10- 14 year old groups (Table 7). “Be faithful” and “using condoms” was ranked second and third. The variation between countries should be highlighted with the highest figures noted in Kenya (60 percent) and Tanzania (61 percent) and the lowest in Haiti (38 percent). The use of condoms as a future prevention strategy and avoiding multiple concurrent partners was mentioned by a sizable percentage of young people in each country (range 37 to 43 percent). It is noted that youth in Haiti ranked use of condom as second. For condoms, the variation between countries is not significant. Table 7: Strategies young people intended to use in future to prevent getting HIV infection (10-14 years) Attitude Overall (n=1015) Kenya (n=401) Tanzania (n=406) Haiti (n=208) P value % % % % Avoid mosquito bites/sleeping under net 9 4 3 30  Sleeping with a minor 8 5 14 3  Abstinence 75 76 77 68 NS Being faithful 56 60 61 38  Using condoms 37 37 35 43 NS Not having sex with many people 26 25 31 20  Significance Level p ≤ 0.001 NS=Not Significant The young people were asked question about their sexual behaviour. Figure 2 shows that overall 12 percent of this group had already initiated sexual intercourse. There was no difference between the countries. At the time of the evaluation, 39 percent of the respondents from Tanzania said they had a boyfriend, compared to 12 percent in Kenya and 6 percent in Haiti. As expected of this young group, knowledge of contraceptive methods is also low and there is no variation between countries. However specific knowledge of contraceptives varied by country with pills and injectables most commonly known in Haiti and Kenya and least known in Tanzania. There was significant variation in the knowledge of use of condoms as a contraceptive method ranging from 17 percent to 26 percent. Knowledge of sexually transmitted infections also varied with the highest percentage recorded in Kenya (66 percent) compared to Tanzania (58 percent) and Haiti (49 percent). ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 48 Figure 2: Knowledge of reproductive health for age 10-14 26 16 30 20 11 36 26 17 26 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Knows any contraceptive method (10-14 years) Pills Injectables Condom 46 29 26 36 23 14 48 10 15 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Knows any signs and symptoms of STI's (10-14 years) Discharge in private part Ulcersin private part Lower abdominal pains 66 58 49 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Knows any sexually transmitted infections (10-14 years) Youth 15-24 years For the three countries, the two highest ranked strategies that were cited for avoiding HIV infection were consistent with teachings of ARK, that is, abstinence and being faithful. The data for youth aged 15 to 24 years were directly comparable for Tanzania and Haiti. For the two countries, abstinence was mentioned by just less than three quarters of the young people (Table 8). Among the 408 young people in Kenya, being faithful was ranked first (61percent) as a method to protect against HIV infection. It is worth noting that having multiple partners did not feature high in any of the countries. A third of respondents in Haiti identified “having sex with many people” as a risk compared to lower percentages in Tanzania (11 percent) and Kenya (0 percent). These topics were covered as part of the supplementary content of the ARK facilitation guides. Selection of topics in the supplementary guide was left to the discretion of the trainers and it is possible that that the coverage was different in the various contexts. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 49 Table 8: Some of the ways young people can avoid getting HIV and AIDS (15-24 years) Attitude Kenya (n=408) Tanzania (n=397) Haiti (n=203) P value % % % Avoid mosquito bites/sleeping under net 1 1 23  Abstinence 61 74 73 NS Being faithful 22 52 50 NS Using condoms 14 29 47 Not having sex with many people 0 11 33  Getting married 0 2 14  Marrying a young girl 1 1 8  Avoid sharing sharp instruments 1 32 41 NS Not having sex with prostitutes 0 15 8 NS Total 100 N/A N/A Significance Level p ≤ 0.001 NS=Not Significant For the two countries for which data were available (Kenya and Tanzania) abstinence and “being faithful” were ranked highest as strategies that young people intended using in future (see Table 9). In Kenya, being faithful was ranked highest by 69 percent of the youth group and this was followed by abstinence. In Tanzania “being faithful” was mentioned by 74 percent and abstinence was ranked second by 50 percent of the youth group. The report on the future strategy to prevent HIV infection does not include Haiti because the quality of data from for that variable was of suspicious quality. Table 9: Percentage of young people reporting different ways they intended to use in future to prevent getting HIV and AIDS infection (15-24 years) Attitude Ways of prevention Kenya (n=408) Tanzania (n=397) % % Avoid mosquito bites/sleeping under net 1 1 Abstinence 24 50 Being faithful 69 74 Using condoms 4 32 Not having sex with many people 0 24 Getting married 1 7 Marrying a young girl 1 1 Avoid sharing sharp instruments 1 37 Not having sex with prostitutes 1 15 Total 100 N/A Table 10 shows reproductive health behaviour for the youth aged 15-24 years. For each country, between 30 and 37 percent of the respondents had initiated sex and between 31 percent and 46 percent had had a boyfriend or girlfriend. For those who ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 50 had ever had sex, ever use of contraceptives was highest in Kenya and Haiti, respectively 58 percent and 60 percent. In Tanzania, having ever used any contraceptive was lowest (38 percent). The use of condoms as a contraceptive method is noticeable for all the three countries but Kenya and Haiti ranked highest in the use of this technology. Further, knowledge of signs and symptoms of sexually transmitted infections was highest in Kenya. Table 10: Sexual Behaviour and practice (15-24 years) Kenya (n=408) Tanzania (n=397) Haiti (n=203) P value % % % Action Ever had sexual intercourse 37 30 36 NS Has a boyfriend/girlfriend 46 36 31  Use any contraceptive method (n=271) 58 38 60  Method Known Pills 6 21 3  Injectable 6 31 29  Condom 86 37 55  Signs & symptoms of STIs (Kenya n=144; Tanzania n=397; Haiti n=203) Discharge in pvt parts 53 15 15 NS Ulcers in pvt parts 17 17 7  Lower abdominal pains 23 10 6 NS Significance Level p ≤ 0.001 NS=Not Significant 3. Abstinence and “Be Faithful” Youth 10-14 years Results from the 10-14 year age group show that they were highly knowledgeable about abstinence and “being faithful”. The groups of young people gathered was drawn from the school where the ARK programme was running and they were expected to know the key messages of abstaining and “being faithful”. Nearly all mentioned that they had heard of the word “abstinence”, averaging 92 percent for the three countries. An equal percentage level of 92 percent was recorded for those who said they had heard of “being faithful”. The meaning of abstinence was sought with four options given. The options are indicated in Figure 3a. The largest proportion of respondents understood abstinence to mean staying away from sex (overall 72 percent). The percentage was highest in Kenya (85 percent) and lowest in Tanzania (58 percent). There were responses from the young people about staying away from drugs, alcohol and risky behaviour but the levels are low. Responses to staying away from drugs were highest in Kenya and lowest in Tanzania. The finding was similar for alcohol use. However, staying away from risky behaviour was highest in Tanzania and lowest in Haiti. The meaning of faithfulness was defined in the context of staying with one partner or staying away from sex. Overall, 70 percent defined faithfulness as staying with one partner. The percentage was highest in Kenya and lowest in Tanzania (55 percent). On the other hand, the option of staying away from sex was highest in Tanzania and lowest in Haiti. The figure further shows that nearly all respondents indicated that they had heard messages about abstinence and being faithful. The variations between the countries should be viewed in the context of what the programme in each country teaches and emphasizes. Figure 3: Percentage of young people aged 10-14 with knowledge of Sexual and Reproductive Messages 85 58 75 33 14 25 25 11 16 36 53 17 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Figure 3a: What does abstinence mean? (10-14 years) Staying away from sex Staying away from drugs Staying away from alcohol Staying away from risky behaviour 83 55 72 38 51 27 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Figure 3c: What does being faithful mean (10-14 years) Staying with one partner Staying away from sex 83 69 50 22 21 39 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Figure 3b: What abstinence messages heard(10-14 years) Abstinence is 100% safe Virgin power virgin pride 84 69 43 22 22 28 0 10 20 30 40 50 60 70 80 90 100 Kenya Tanzania Haiti Percentage Countries Figure 3d: What being faithful messages heard(10-14 years) Abstinence is 100% safe Virgin power virgin pride ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 51 Youth 15-24 years For all the three countries, the vast majority of youth in the age group 15 to 25 years had heard of the word abstinence and “being faithful” (Table 11). Abstinence was also associated with staying away from risky behaviour. The majority of the group had heard the risky behaviour message that “abstinence is 100 percent safe”. Less than a third of the respondents had heard of the message “virgin power virgin pride”. Over 70 percent of respondents in each country interpreted. “Being faithful” to mean staying with one partner. The most frequently heard message on “being faithful” was “being faithful is 100 percent safe”. This message was most popularly heard in Kenya by 94 percent of the youth compared to Tanzania (67 percent) and Haiti (50 percent). Table 11: Knowledge, behaviour and practice for abstinence and faithfulness among youth (15-24 years) Attribute Kenya (n=404) Tanzania (n=397) Haiti (n=203) P value % % % Ever heard word abstinence 99 94 96 NS What does ‘abstinence’ mean? Staying away from sex 83 84 80  Staying away from drugs 3 4 4  Staying away from alcohol 1 5 5  Staying away from risky behaviour 10 36 17  Staying away 3 0 0 NS Total 100 N/A N/A Ever heard any messaged about abstinence (Y) 96 89 90  What are the abstinence /avoiding risky behaviour messages you have heard Abstinence is 100percent safe 94 71 46  Virgin power virgin pride 6 30 24 NS Total 100 N/A N/A Ever heard the word ‘being faithful’? 98 91 95  What does the word being faithful mean Staying with one partner 70 74 82 NS Staying away from sex 28 21 11 NS Ever heard messages about being faithful (Yes) 94 84 90 NS Which ‘being faithful’ ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 53 Attribute Kenya (n=404) Tanzania (n=397) Haiti (n=203) P value messages have you heard Being faithful is 100percent safe 94 67 50  Virgin power virgin pride 6 21 0 NS Significance Level p ≤ 0.001 NS=Not Significant 4.   Sources of information Youth 10 -14 years Sources of information about reproductive health issues, abstinence and faithfulness for the youth group 10-14 years is shown in Tables 12 to 13. Table 12 shows the sources of HIV and AIDS and reproductive health information for the 10-14 year old group. In Kenya, the highest ranked sources of information were the school (69 percent), followed by radio/TV (61 percent) and ARK (59 percent). In Tanzania three top most cited sources of information were the clinic (60 percent), ARK (59 percent) and radio/TV (45 percent). In Haiti, ARK was ranked highest (58 percent) followed by the school (57 percent) and then the Anti-AIDS clubs (44 percent). The school as a source of information was likely to be related to the ARK programme in the country. Since ARK uses radio, schools and TV as vehicles for information, there is a chance of an overlap of responses for the sources of information. Table 12: Sources of information for HIV and AIDS and reproductive health for youth (10-14 years) Overall (n=1015) Kenya (n=401) Tanzania (n=406) Haiti (n=208) Significant Source % % % % Home 27 39 21 13  School 53 69 35 57  Clinic 50 48 60 34  Church 22 38 12 11  Radio /TV 46 61 45 21  Books /Magazine 35 42 39 15  Posters/Leaflets/Brochures 20 26 23 4  Anti-AIDS Club 36 43 25 44  Home Based Care 15 20 15 4  Counselling /VCT Centre 36 45 35 19  Youth friendly corner 14 15 13 13 NS ARK 55 59 58 46 NS Other community-based organisation 13 12 19 5  Note: Significance Level: p ≤ 0.001 NS=Not Significant,* =multiple responses ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 54 Table 13 shows sources of information for abstinence for young people. The rankings of three most common sources of information were radio (69 percent); school (64 percent) and the ARK programme (57 percent). Similarly, in Tanzania the same sources were ranked in the top three but the order is different; the ARK programme was ranked first (64 percent) followed by radio (41 percent) and school (39 percent). Haiti respondents ranked the school (51 percent) followed by the health centres (38 percent) and radio (32 percent). Table 13: Sources of information for abstinence and being faithful messages for young people (10-14 years) Sources of abstinence messages Sources of being faithful messages Kenya (n=401) Tanzania (n=406) Haiti (n=208) Kenya (n=401) Tanzania (n=406) Haiti (n=208) % % % % % % Source Radio 69 41 32* 72 43 32* Television 55 31 16* 55 31 17* Fliers /posters 33 21 2* 36 22 5* Church 40 10 12* 48 9 7* School 64 39 51* 59 37 49* Health centre 46 33 38* 49 34 39 Community health worker 26 21 4* 27 24 4* Traditional Birth attendant 15 7 11* 14 9 15 Anti-AIDS clubs 11 11 13 11 10 37* CBO 19 23 21 15 14 9 ARK Programme 57 64 5* 3 0 0 Home 13 9 0* Note: * Significance Level: p ≤ 0.001 Further, the table indicates the sources of information for “be faithful” messages. The three most common sources of messages were the radio, the school and television for Kenya. For Tanzania, the levels are lower but the rankings indicate the radio (43percent), the school (37 percent) and health centre (34 percent). In Haiti, the school ranked highest (49 percent) followed by the health centre (39 percent) and finally the Anti AIDS clubs (37 percent). As a source of “Be faithful messages”, the ARK programme is probably being indirectly referenced in schools and in radio programmes since the project tended to be the driving force behind the programme. Youth Group 15-24 years Sources of information for reproductive health and HIV and AIDS are shown in Tables 14 and 15 for the older age group 15-24 years. For this group, the youth in Tanzania identified ARK, radio / TV and schools as the three top most important sources of information. Anti AIDS clubs, the school and clinic were identified as ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 55 most frequent sources in Haiti. In Kenya, the school ranked highest, followed by ARK and then the clinic. Table 14: Sources of information for HIV and AIDS, and reproductive health for youth (15-24 years) Kenya (n=408) Tanzania (n=397) Haiti (n=203) Significant Source % % % Home 4 25 7  School 39 38 53  Clinic 9 32 50  Church 4 19 7  Radio /TV 6 45 33 NS Books /Magazine 1 37 13  Posters/Leaflets/Brochures 1 30 3  Anti-AIDS Club 1 29 60  Home Based Care 7 15 3  Counselling /VCT Centre 5 35 20  Youth friendly corner 0 25 14  ARK 21 62 47  Other community-based organisation 1 15 8 NS Internet 1 7 8 NS Total 100 N/A N/A Note: Significance Level: p ≤ 0.001 N S=Not Significant,* =multiple responses Table 15 shows the sources of information on abstinence for the 15 to 24 year old youth. Tanzania and Haiti both ranked radio, the schools and health centres as the top most sources for information about abstinence. In Kenya, the church together with the school and radio were ranked highest as sources for abstinence. Table 15: Sources of information for abstinence and being faithful messages for young people (15-24 years) Sources of abstinence messages Sources of “ be faithful” messages Kenya (n=391) Tanzania (n=397) Haiti (n=20 3) Kenya (n=391) Tanzania (n=397) Haiti (n=203) % % % % % % Source Radio 23 37 34 27 36 39 Television 10 20 19 25 18 19 Fliers /posters 6 24 4* 11 22 5* Church 22 5 12 12 14 9 School 19 39 30 9 24 38* Health centre 12 35 44 7 30 44* Community health worker 3 24 9* 5 25 6* ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 56 Sources of abstinence messages Sources of “ be faithful” messages Traditional Birth attendant 1 9 20* 0 11 43* Anti-AIDS clubs 2 22 12 4 19 42* CBO 2 29 16 0 19 13 ARK Programme 0 8 1* 0 0 0 Home 0 0 0 0 0 0 Total 100 N/A N/A 100 N/A N/A Note: * Significance Level: p ≤ 0.001 NS=Not Significant Sources of information for messages on “being faithful” vary between the countries. In Haiti, the health centres ranked highest followed by the traditional birth attendants and Anti AIDS clubs. In Tanzania, the radio, the health centres and the schools played the key roles. Kenyan youth identified the radio, television and church as key sources of information about “faithfulness”. 5. Life skills Youth 10-14 years In the three countries, between 70 and 83 percent of youth had attended life skills courses (Table 16). The highest reported figure of attendance was in Kenya while the lowest was in Haiti. Nearly all who had attended the life skills training said they would recommend the training to friends and other people. There is general variability in the skills learnt in different countries thus probably reflecting the emphasis of the courses in each country. In Kenya, the most popular skill learnt was self esteem compared to goal setting in Tanzania and decision making in Haiti. The participants indicated that the life skills had changed their lives. In Kenya and Tanzania, the largest percentage of respondents indicated that they had “improved their skills” as an intervention resulting from attending a life skills course. In Haiti, the largest percentage said the life skills programme had made them wiser. The young people indicated that they could teach others various elements of life skills. Table 16: Life skills for young people (10-14 years) Attribute Overall (n=749) Kenya (n=298) Tanzania (n=308) Haiti (n=143) Significant % % % % Attended life skills training 80 83 81 70  Would recommend the skills to others/friends 97 98 96 97  Life skills learnt Communication 25 32 23 22 NS Goal setting 39 42 46 24  Decision making 31 48 24 27  Avoiding peer pressure 28 48 20 26  ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 57 Attribute Overall (n=749) Kenya (n=298) Tanzania (n=308) Haiti (n=143) Significant Negotiation skills 19 24 20 12.5 NS Assertiveness 17 35 12 10  Self esteem 33 60 34 4  Role models 14 30 13 2  Critical thinking 13 25 12 2  Changes brought by life skills Helps to be independent 26 23 34 19  I have a lot of friends 14 16 10 18 NS I have improved my skills 39 48 41 19  Makes me think wiser 35 25 41 44  Helps me to think critically 25 31 30 6  I have learnt to respect people 36 36 33 40 NS No one can bully me 9 8 10 9 NS Helps me not to be lazy 16 15 16 16 NS Helps me to decide on right & wrong 22 22 25 16 NS Helps me to get what I want 6 6 9 3 NS Life skills elements that youth can teach Communication 21 35 16 19  Goal setting 37 37 40 29 NS Decision making 33 51 25 30  Avoiding peer pressure 33 50 24 34  Negotiation skills 20 20 20 17 NS Assertiveness 17 32 13 12  Self esteem 32 53 35 4  Role models 16 27 17 5  Critical thinking 17 32 16.5 2  Life skills elements that I will be using in next 12 months Communication 21 37 15 17  Goal setting 40 39 46 30  Decision making 30 47 27 19  Avoiding peer pressure 28 43 24 21  Negotiation skills 19 23 23 6  Assertiveness 18 36 13 11  Self esteem 31 54 34 2  ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 58 Attribute Overall (n=749) Kenya (n=298) Tanzania (n=308) Haiti (n=143) Significant Role models 19 34 18 5  Critical thinking 16 29 17 0  Note: Significance Level: p ≤ 0.001 NS=Not Significant A large percentage of young people had thought about their future 12 months before the survey (Table 17). The parents were the most favoured persons who the youth had talked to about their future goals. After knowing ARK, the majority (66 percent) of youth said they thought differently about issues compared to the time before. These findings are summarised in Table 17. Table 17: Young people (10-14 years) and their future goals Overall (n=956) Kenya (n=391) Tanzania (n=393) Haiti (n=172) Significant % % % % In the past one year thought about the future 84 95 70 94  In the past year talked about these plans with someone Person talked to Friend (close in age) 28 33 19 34  Parent 49 51 52 42  Brother/sister 14 10 17 16  Relative (close in age) 3 3 3 1  Relative (older) 1 0 0 3  Church leader 1 0 1 0  Programme staff 3 0 6 3  Other 0 0 1 0  Thought same or differently after knowing ARK Thought differently 68 74 61 69  Thought in similar way 32 26 39 31  Note: Significance Level: p ≤ 0.001 N S=Not Significant Life skills and Youth 15-24 years A significant percentage of the older youth group (15-24 years) had attended life skills training one year before the interview. The largest proportion of those trained was in Kenya (88 percent) followed by those in Tanzania (79 percent) and then Haiti (59 percent). A wide range of skills were learned and between 87 and 98 percent of the youth in this age group said they would recommend the skills training to friends or other people, thus suggesting the value they derived from the course. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 59 The highest percentage of youth mentioned communication, goal setting, decision making and avoiding peer pressure as skills that they had learnt. Life skills brought different changes to respondents. In Kenya, the three most ranked changes were independence, an increased number of friends and general improvement various skills. In Tanzania, the youth ranked independence, general skills improvement and ability to decide “on right and wrong” outcomes related to life skills intervention. In Haiti, the three most commonly given answers were; thinking wisely, having improved skills, and respecting people. The youth also thought they could teach some of the elements of the skills programme that they had participated in. Goal setting and decision making were ranked highest as elements of the skills courses that youth could teach. Table 18 shows that the youth had thought about their future 12 months before the survey. In Kenya and Tanzania, the highest percentage discussed their future goals with their parents, friends or brothers/sisters. The effect of the ARK intervention is clearly indicated by the high percentage of young people who said they “thought differently after they were exposed to ARK. These results are consistent with qualitative findings. Table 18: Life skills for young people (15-24 years) Kenya (n=407) Tanzania (n=390) Haiti (n=177) Significant % % % Attribute Attended life skills training 88 79 59  Would recommend the skills to others/friends 98 96 87  Life skills learnt Communication 24 19 13 NS Goal setting 32 53 24  Decision making 20 33 25 NS Avoiding peer pressure 12 27 18 NS Negotiation skills 2 17 8 NS Assertiveness 1 15 6 NS Self esteem 7 0 0  Role models 1 20 2▪  Critical thinking 1 11 0▪  Total 100 N/A N/A Changes brought by life skills Helps to be independent 13 37 12  I have a lot of friends 15 12 15 NS I have improved my skills 43 31 16  Makes me think wiser 12 23 29  Helps me to think critically 5 24 3▪  I have learnt to respect people 2 19 22 NS No one can bully me 1 13 6 NS ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 60 Kenya (n=407) Tanzania (n=390) Haiti (n=177) Significant Helps me not to be lazy 17 4▪ NS Helps me to decide on right & wrong 9 26 17 NS Helps me to get what I want 0 12 6 NS Total 100 N/A N/A Life skills elements that youth can teach Communication 17 17 15  Goal setting 30 65 28 NS Decision making 28 38 26  Avoiding peer pressure 11 31 22  Negotiation skills 4 19 14 NS Assertiveness 0 16 7  Self esteem 8 22 4▪  Role models 1 28 2▪  Critical thinking 1 16 0▪  Total 100 N/A N/A Life skills elements that I will be using in next 12 months Communication 18 17 12 NS Goal setting 25 61 28  Decision making 27 41 24  Avoiding peer pressure 13 29 18 NS Negotiation skills 3 22 12 NS Assertiveness 0 19 4  Self esteem 11 23 5  Role models 2 29 2  Critical thinking 1 17 0  Total 100 N/A N/A Note: * Significance Level: p ≤ 0.001 NS=Not Significant Table 19: Young people (15-24 years) and their future goals Kenya (n=353) Tanzania (n=392) Haiti (n=190) Significant % % % In the past one year thought about the future 92 95 88  In the past year talked about these plans with someone Person talked to Friend (close in age) 18 21 37  Parent 66 42 29  Brother/sister 9 14 19  ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 61 Kenya (n=353) Tanzania (n=392) Haiti (n=190) Significant Relative (close in age) 1 11 8  Relative (older) 1 10 7  Church leader 2 5 6  Teacher 2 1 1  Program staff 1 0 0 Total 100 N/A N/A Thought same or differently after knowing ARK Thought differently 72 59 72  Thought in similar way 28 41 28  Note: Significance Level: p ≤ 0.001 NS=Not Significant,* =multiple responses Quantitative data on knowledge, attitude and behaviour provides some interesting insights on the ARK programme. In terms of attitude, there is evidence of stereotyping people with HIV and AIDS. The variability between countries is noted. Generally, the younger age group (10-14 years) had more stereotype views about people infected with HIV compared to the older age groups (14 -24 years). The A and B messages are well known in the groups. It is an unexpected finding that the younger age group appeared to know more about the two methods of contraception (pills and the injectable) compared to the older ones. The younger age group was also relatively well versed on signs and symptoms of sexually transmitted infections. They were more likely to mention that avoiding multiple concurrent partners would be a future strategy for them to avoid contracting HIV infection. The key sources of information (radio, schools, clinics, churches) for reproductive health, A and B messages are consistent with ARK’s intervention strategy of using the community. Life skills which are of great concern for young people are a source of empowerment for them and it is important to note that this intervention was viewed by young people as life changing. V. STRENGTHS, WEAKNESSES, CONSTRAINTS & OPPORTUNITIES a. Strengths Respondents were also asked to identify some of the successes associated with the ARK Project. They mentioned that they thought communication had improved between different stakeholders that cultural practices that were viewed as harmful were, at that moment, being viewed critically. Success on these outcomes is discusses in this section: Availability of Resources for effective communication For all the three countries, the methods of teaching adopted by the project were lauded. The use of Training of Trainers, seminars, video shows and drama groups was well received by the communities in Kenya and Tanzania. The distribution of books, ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 62 T-shirts, passports, bangles and other communication materials reinforced the AB messages for youth. The communities further felt that the facilitators were well trained and skilled. Use of audio visual equipment provided great learning opportunities for the youths. In some areas, TOTs were provided with motor bikes and bicycles to ease transport problems. This was a significant input to the project thus contributing to wider and greater accessibility of the project. Participatory engagement of key groups (parents, teachers and youth) Bringing together the three key groups of parents, youth, teachers and parents contributed to effective participation in the project. As one key informant noted in Haiti, often the different groups are targeted separately. The strength of ARK is its ability to engage all the parties in a dialogue together to face sexuality issues. Different group discussions identified that life skills have improved the communication between youth and their parents. Some of the youth and parents noted improved communication in their relationships. They indicated that they are now able to talk more freely over sexual issues with their parents. Parents realized that they serve as role models for their children and that their behaviour had to be exemplary for the youth. Good partnerships with stakeholders The participation of government and local NGOs was viewed as important for programme sustainability and success. In Monduli, ARK had established a good network of partnerships that became a useful resource to key stakeholders. For instance, an example was given of SACCOS (Savings and Credit Co-Operative Societies) who were cited as having helped in providing loans to teachers because of the relationship created by ARK. A number of participants mentioned that their relationship with government officers had improved and they had easy access to them. Engagement with as many stakeholders as possible, including traditional leaders and healers was effective in improving participation by these groups and coverage for the ARK project. In particular the relationships with the groups assisted in making the key stakeholders accountable and responding to the needs of the people they serve. Providing Stimulating activities and incentives Some community members identified incentives offered by ARK as one of the strong aspects of the programme. The material incentives that were mentioned were T￾shirts, mathematical sets, ARK passports and bangles that were given to young people, especially in Kenya. In rural communities, food and drinks given during training and seminars were acknowledged as important ingredients for motivating participants. Sports activities were an effective tool for participation, motivation and behaviour change among young people in the ARK project. The field trips, out of school ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 63 activities were very engaging for young people. In Kenya and Tanzania the youth mentioned how important award ceremonies were to them as well as bringing great recognition of the project. In Haiti, certificates after competition of courses were mentioned as a great motivation aspect of the project. Stakeholders support and ownership of ARK Project The passion for the project in the three countries in both the younger and older age groups was identified as one of the key factors in what was perceived as a successful implementation of the ARK Project. The success lay in the passion, commitment and self motivation of the army of volunteers in the communities who worked as TOT, peer educators and in other different capacities. b. Weaknesses Inadequate material resources At all the sites visited in Haiti, Kenya and Tanzania, participants pointed out that there were some shortages of materials that were important to the programme. Groups pointed out that sometimes activities could not be executed for lack of items. For instance, in one site in Kenya, balls were not available for sports activities and these demotivated participants. The issue of materials shortages was raised during the midterm review. The report summarized “lack of training materials was mentioned across areas. Both staff and participants identified lack of adequate quality of training and development materials as reducing training efficiency and effectiveness in meeting program goals”. The issue of unfair distribution of ARK materials such as T-shirts, bicycles, bags and such items was raised. Participants, who raised the issue, felt that there was lack of transparency in the way the materials were given. TOTS and volunteers had inadequate handouts to provide to participants after training and dialogues. Seminars and workshops were some of the popular means of engaging communities and participation was high. However, some participants reported that they did not always receive transport allowances when they participated in such group meetings. Many times they were provided with food though they would have preferred cash. They also pointed out that refreshments were not consistently provided to groups especially during radio listeners groups. Commenting on this shortage, key informants reported that due to limited resources stringent criteria was developed for distribution of materials to the target population. In addition, in situations where training was conducted far from participants’ homes, then transport allowances were given in the form of fare reimbursements. Unequal distribution of resources TOTs and some volunteers pointed out that there was lack of transparency in the distribution of some items. This was particularly the case with the issue of bicycles and T-shirts. The 15-24 year olds in Monduli mentioned that they were “Not aware that there were training materials such as passports, bangles and other IEC materials”. At some sites, there were complaints that the volunteers did not know how the community participants such as youth volunteers were selected. The community thought that some of the volunteers were selected to receive materials. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 64 Community expectations Teams pointed out that programme personnel made promises that were not fulfilled. For instance, in a group meeting in Kenya, participants remembered that mosquito nets were promised to them but these were never delivered. In Tanzania, complaints were made about game and sports materials for schools that were not delivered. Some of the unfulfilled promises were related to the broader WV project and not to the ARK Project per se but complainants were unable to distinguish between WV ADP and ARK as a project. This affected some people’s confidence in the programme. Coverage of the programme areas The ARK project was defined to cover a defined geographical area, but key informants and parent’s focus groups felt that program coverage was limited. For example, KII interviewees in Teso District indicated that their district is large and the population is big and hence there were difficulties of coverage in implementation of project activities. In Monduli, KII felt that ARK did not cover the rural areas or villages where there was a greater need. The staff assigned to cover the required geographical area was not enough to do so. A teacher in Kenya commented; “Understaffing did not enable them [ARK staff] to have direct contact with staff [teachers]. There was lack of motivation for teachers to be efficient. There were very many teachers but only a few trained” (KII, Suba). Associated with the large geographical area was the lack of transportation. Bicycles were provided to TOTS and volunteers but still this was not adequate. It is some credit to the project that community members felt that there was an unmet need for the services of the ARK Project. In other words, the positive works of ARK created a huge need which fell outside its defined geographical boundaries. The Process of Training & follow up As stated above under ‘Monitoring and Evaluation’ section there were some sites where the depth of knowledge of the programme appeared to be superficial because the number of training sessions was conducted only once. In Kenya the teachers pointed out that the number of seminars held for their groups were few and they would have preferred further repetition. In a number of schools, only a few teachers participated in the programme. In Tanzania lack of deeper teacher-parent participation in the schools meant that the ARK training curriculum was not provided space in the school curriculum and this limited the effect of intervention. Some of the teachers were reluctant to allow the ARK volunteers to train the school children and to make use of the religious sessions for training. c. Constraints By and large there was satisfaction with the ARK project by different groups of participants. Therefore, constraints presented were few. The first constraint cited by participants had to do with the vastness of the terrain and need for more staff to cover the area. In the context of the resources given, this would not have been possible. In Tanzania, some participants indicated that the project did ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 65 not extend to areas where the language was primarily not KiSwahili. Radio coverage was well appreciated in all communities but coverage was not as wide as the community would have desired. Consequently, a suggestion was made that the network coverage should be extended. Where involvement of teachers was limited, not enough time was set aside to conduct ARK project activities such as radio listener groups at the schools. d. Opportunities Supportive Policy and community environment In all the countries, the evaluation found support and cooperation from the government and the community. The various government departments supported the programme; permission was obtained from the departments of education in the three countries. In each of the countries, the government policies on children’s rights were supportive and complementary to the work of ARK. In addition, in each of the countries, the governments potentially have access to AIDS funding (e.g. Global Funds, PEPFAR) which provides an opportunity for integration of ARK into the projects. Endorsement by traditional leadership The traditional healers in the communities tended to support the AB strategy that was the cornerstone of the ARK strategy of work. The support of village leaders and those of religious organisations found the AB message appealing to them and therefore supported the ARK programme. This provided a big opportunity for the ARK project to take root in the communities and to expand. Existence of Trained Cadre and other resources The existence of a trained cadre (TOTs, peer educators, drama groups) is a great resource that exists in the community. The cadre are members of the community and they have been trained through the project. As noted in other sections earlier, there was great community recognition of the high level of skills of the cadre. In developing the IEC interventions, a number of resources have been developed. The equipment for developing radios programmes and videos now exists. Materials for training of trainers, peer educators and drama groups are now available to continue with ARK activities. Shared experiences across projects and location The ARK Project has been implemented in three countries and within a country; there are a number of sites. This situation of implementing a project in different cultures, and project situations provides an opportunity for sharing lessons drawn from each of the countries under different circumstances. Case studies of success, lessons learnt can be drawn from the different study sites. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 66 VI. DISCUSSION 1. The process of implementation According to the key stakeholders, an appealing aspect of the design of the ARK Project was its engagement process with many stakeholders at different levels. The different activities or elements reinforced each other for better effect. For instance, the objectives of A&B for youth were supported and reinforced through parents, teachers, community elders and structures; thus leading to longer term sustainability. Engagement at different levels with stakeholders ensured that they were aware that the project was for their benefit and that of their communities. In its formulation, the ARK project sought World Vision to identify areas where there was a need; where the project could be supported by the ADP structure, partner organisations (such as NGOs) and government structures. At first, the integration of ARK with ADP management faced challenges. This was also acknowledged by the midterm review. However, by the end of the project in June, such problems between ARK and the ADP seemed to have been resolved. In any case, during the last year of the project, many sites were working out exit strategies which explored the possibility of integrating the ARK Project into the ADPs. The sites for evaluation in Kenya and Tanzania showed that the integration between the ARK Project, the government and NGOs was very strong but not to the same level and not in a similar way. In Haiti was an exception in the sense that WV Haiti had an MOU with the Government to implement Health and AIDS programmes in support of its programmes. In such a case integration may not be evident, but the programmes have the blessing of the Government. 2. Tools for fostering social transformation A wide range of tools for social transformation were adopted by the ARK project; and these were training programs, radio programs, adult training seminars, video and drama program, music programs, football clubs, debates and discussions, and peer mentoring. In all the three countries, there was general consensus from the different groups that that training was of high quality. Overall and at country level, there was consistency in the number of interventions implemented but variability was also noted that the tools were not implemented to the same depth and level in three countries. In Kenya and Tanzania, radio programmes and sports activities were highly praised. Video programmes in Misenyi were competing with videos shown by local community members who were promoting pornographic material. In the quantitative survey for youth, we noted that in Kenya and Tanzania, the school and radio were ranked in the top three as sources of information for reproductive health, “be faithful” and “abstinence” messages. This was also the case with Haiti though the clinic tended to feature high in that country compared to the African countries. The end term evaluation as was case with the midterm review noted that there was shortage of training and other ARK materials. From one perspective, shortage of ARK materials could be an indication of the high demand and popularity of such materials. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 67 On the other hand, for volunteers whose recognition came in the form of materials; this shortage constituted a great disincentive. 3. Changes in Knowledge, Attitude and Behaviour The key interventions tools for the ARK project were information, education and communication activities to youth, parents and the community. After five years of implementing interventions, the question is asked if they were effective in mediating change in the behaviour of youth and other groups. The qualitative data from the youth, parents and teachers support the view that there was a change brought about by the ARK Project interventions. Evidence from qualitative data suggested improved and accurate knowledge on reproductive health issues for all groups. Pupils also reported behaviour change at individual level; they avoided risky behaviour, associating with peers who support them in pursuing their goals. Parents reported improved ability to talk about issues of sexuality. In addition, an unintended benefit of the project was reported improved harmony between husbands and wives. Testimonies were born out clearly by reports from qualitative interviews in Kenya and Tanzania; there was a reduction in the number of teenage pregnancies, HIV infection and school drop outs. There were reports of improved teacher-learner relationships because teachers understood learners better. Improvement in academic performance and school attendance were also reported. Specifically, knowledge about abstinence and “be faithful” was nearly universal among both groups of youth, but practice was far less. Between countries, the meaning of each of the concepts was not consistently understood as indicated by the variability of responses. The manual indicates abstinence as “staying away from sex, drugs, alcohol and risky behaviour”. Only staying away from sex is the most well known definition of abstinence. Similarly, the message on abstinence most well known was “abstinence is 100% safe” compared to the “virgin power virgin pride”. This variability may suggest that these messages are not being communicated strongly to the youth or they are less appealing to them. Results of a change in the effectiveness of the ARK Project are corroborated by findings from the Mid Term Review. In that review, the authors concluded “these scores indicate that all stakeholders perceive that the ARK program is effective in helping youth make and persist in choices to reduce their risk behaviours for HIV/AIDS, …” During the MTR, the researchers also had control sites from which they collected comparative data. The analysis arrived at the same conclusion of effectiveness. 4. What was the state of knowledge and Attitude? The qualitative data presented a strong impression of high knowledge and modified sexual behaviour because of ARK intervention. These findings are contrasted with quantitative data gathered from the two age groups 10-14 years and 15 to 24 years. As expected, a higher percentage of the older youth (15-24 years) had ever had sexual experience compared to the younger age group of 10-14 years. For each of the countries, less than half of the younger youth had ever had sex compared to the older ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 68 age group. Contraceptive use among the older age groups was far higher in Kenya (58%) and Haiti (60%) compared to Tanzania (38%). It is interesting to note that knowledge of contraceptive methods and sexually transmitted infections for the younger group of youth was comparable to that of the older group, and at times even higher. Statements seeking to evaluate the perception of youth about people infected with HIV suggest lack of clarity among the respondents. Asked to indicate agreement, or disagreement with statement on people infected with HIV/AIDS, there is a indication of general variability showing lack of consensus among the youth (for both age groups). At the same time, the older age group (15-24 years) appears to be more tolerant and accommodating people infected, they would not feel as much ashamed their family member was infected with HIV, and they would show compassion and love to those infected. This may reflect an attitude modified more by life experiences rather than teaching from ARK. Case study 2 is a self selected sample of young people who have been in interaction with ARK. Asked if ARK had influenced them to go for VCT, or to practice abstinence, we note that the majority mentioned that they had done so because of ARK. V. LESSONS LEARNT 1. The use of in-country staff is critical for project support, especially where there are changes in the staff management in the project. 2. It is important to recognize that local small sub-grantees require capacity building to help them handle large funds and learn donor guidelines. This was a constraint for some of the local sub-grantees to manage large funds in the project. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 69 3. The ARK project was multi level and multi country. Opportunities exist for learning and to draw lessons in administrative process and technical processes. 4. Coordinating activities and funds from different geographical locations slowed the decision making process due to institutional bureaucracies; bringing decision as close as possible to the ground where projects are implemented is a better strategy. VI. PROJECT SUSTAINABILITY AND THE FUTURE The issue of sustainability and the future of the ARK Project became of critical importance as the funding for the project was coming to an end in June 2010. Programme managers have worked out different exit strategies for the project. In all the in-depth interviews and group discussions, the message was clear and consistent that the project was beneficial for the communities and some way should be found to continue with the activities. This section provides a number of scenarios that could be explored for sustaining the work of the ARK project. The different options are not mutually exclusive but can be adopted in different ways by the project implementers. This section is based on reports from the in-depth interviews with different key stakeholders and key informants from different organisations. Since the last quarter of 2009, districts where the ARK project was implemented convened meetings to explore sustainability mechanisms for the project. Some of the draft reports were also made available and the strategies suggested were compared with data gathered from field interviews. 1. Integrate ARK Project With World Vision The original design of the ARK Project was that its activities should be integrated into World Vision’s Area Development Programmes. In Haiti and Tanzania this was already achieved since officially the ARK project was integrated with HIV and AIDS programmes within the ADP structure. The ease of integration will depend on how closely the project has been collaborating with other components of the World Vision in the past five years. An illustrative example is drawn from Misenyi in Tanzania where the ADP HIV and AIDS Coordinator and the ARK Project Coordinator in the district worked together to avoid duplication in implementation (See Case Study 1 in the Appendix). The arrangement freed some funds for ADP to be used for other activities in the community like buying more VCT kits when they realized more people were coming for VCT services. After HIV testing, those who were positive were integrated into food programmes and support groups and referred for clinical examination at the health facilities to determine their needs for antiretroviral treatment or prophylaxis. However, the challenge in Kenya was that the ARK Project covered wider areas (a whole district) than the ADP. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 70 2. Register An Entity That Will Continue With Key Activities Of The ARK Project There is overwhelming positive response about the value of the different Information, Education and Communication elements of the ARK project and that these resources can be used by other community based organisations and government to build their social capital. Since the TOTs, peer educators and drama groups are an existing resource in the communities, it is important to help them to form an entity that will provide a framework within which they can continue to provide such services to government and NGOs. Some of these ideas are being tested in some districts where TOTs and peer educators are establishing NGOs from which services can be obtained (probably at cost recovery level). 3. Put ARK Equipment And Materials At The Disposal Of The Community In the past five years, investment has been made in purchasing audio equipment and developing IEC materials. The equipment can continue to be used by the community but it can be put under the management and supervision of the ADPs. Equipment and materials can be made available to former ARK structures to support the continuation of ARK TOTs, radio listener groups and drama activities. The existing materials can be reproduced and the trainers and participants can be utilized to help implement and expand activities. The materials can be shared with other projects. Schools can put the materials in their libraries for use and reference. For example in Kenya training guides were reprinted in bulk in 2009 and distributed to schools, MOE officials, churches and youth groups. 4. Move The ARK Approach Beyond Its Current Borders The concept of A&B is well accepted by parents, CBOs and government. At all the sites, there was overwhelming acceptance to the participatory approach of the ARK project. To sustain the messages, all the WV country offices should take advantage of the acceptance of the approach to introduce the different groups of the ARK project to stakeholders who may be interested in funding the project further. The ADPs should continue to encourage different groups to adopt the approach and messages from the ARK programme. In the exit strategy meetings, different stakeholders have expressed interest in mainstreaming some elements of the ARK programme. For example, in some districts in Kenya and Tanzania, faith based organisations have expressed interest in adopting some of the relevant messages for their faith teachings. In Haiti, animators from outside the ARK project ADPs have been trained on ARK from its inception. 5. Strengthen The ARK Life Skills Approach In Schools One key element of the ARK Project was the life skills programme in schools. Research teams reported meetings with brilliant young people who had been trained through the programme and had embraced the AB approach. For sustaining the programme, WV should engage more with ministries Government ministries of ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 71 education to explore the possibility of adopting the programme more formally into schools. This could be done through curricular and extracurricular programmes. 6. Integrate Some Key Element Of The ARK Programme Into Government Programmes The general involvement of government in the implementation of ARK activities at district and local levels provides an opportunity for relevant government departments to adopt some of the programme elements. They can facilitate the introduction and adoption of some of the programme elements. Government can play a crucial role in encouraging district leaders to continue to support ToTs, peer educators, drama groups and others. For example, a Ministry of Health Representative in Suba, Kenya, suggested that the Ministry of Health, through its HIV and AIDS directorate, could continue to promote ARK activities (for example, outreach HIV and AIDS activities). Therefore, district teams should be encouraged to integrate ARK interventions in their plans so that they can be allocated funds to support local TOTs and CBOs for implementation. VII. RECOMMENDATIONS The recommendations in this evaluation report are based on findings from the desk reviews of ARK project implementation documents, focus group discussions, site visits and observations and key informant interviews conducted in the three countries. These recommendations are by no means exhaustive but provide guidance on how to scale up and replicate best practices in a sustainable way as well as address weaknesses observed in the ARK project. These recommendations are discussed in the context that the Programme is coming to an end and that the lessons learnt will be applied to similar programmes or be used for sustaining the ARK Project ix. ARK: A youth and community-friendly project Different aspects of the ARK project suggest successful implementation and outcomes. The project has produced models of best practices in engaging youth, parents, communities, government departments for children and youth, CBOs, and FBOs and other stakeholders on risk reduction for change in high risk behavior among youth. Recommendation: i. Project authorities should seek funds to continue to build on the completed phase of the ARK Project. This will allow time to adequately document the impact of behavior change in reducing risk for HIV infection among youth in the three countries. ii. Scale Up coverage of ARK Project in the target countries. iii. ARK as a platform for comprehensive HIV prevention intervention at the community level. The ARK project has demonstrated best practices in ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 72 implementing targeted multi-country youth-friendly risk reduction programs at the community level. Recommendation:  Utilize ARK platform to implement comprehensive combination HIV prevention intervention for youth and other at risk groups in the community. 2. Human Resource management The project is designed around the concept of volunteers. In some places this led to inadequate trainings for youth and the community in some geographical areas. This led to inability to follow up on programs and greater reliance on community volunteers. For example, in Haiti, about a third of volunteers left the project mainly to pursue further education opportunities or start a career. Recommendations  To reconsider the use of ARK staff members as well as the volunteers in terms of their capacity and skills to handle the work assigned.  Increase the frequency of training and follow up activities at the community level and find ways of keeping in touch with trainers.  Provide regular refresher courses for the workers and volunteers implementing the program. 2. Improve incentives for Volunteers Compensation of volunteers continues to be raised as a thorny issue in the communities and was acknowledged by key informants as a challenge. World Vision does not provide monetary incentives to its volunteers; non-cash and other materials support are often used for compensation. Inconsistent incentives and lack of transparency in compensation was identified as demotivating to different field volunteers (teachers, peer educators, TOTS). Participants also frequently raised the issue of certificates as a way of recognition for attending ARK trainings. In Haiti, training in theatre for volunteer peer educators equipped them with skills that they could use in their future career. Recommendations  The project staff should clarify the compensation issues for volunteers and make them transparent and consistent at the beginning of the project in the target areas.  This and other project should consider awarding certificates and training opportunities for rewarding volunteer participants in the program.  Certificates of recognition to be offered to different levels of staff working on the programmes. (This is already happening at some sites). ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 73 4. Linking ARK Project with ADPs in World Vision The implementation of the ARK project was viewed as superior where ARK was located within the World Vision ADP offices. The infrastructural support was better and ARK tended to be more visible because it was associated with other ADP projects. The overlap between the ARK district and ADP coordinators was strength in that they could pull resources together and prioritize intervention implementation. This was the standard practice in Haiti. Recommendation  Strengthen the integration of ARK within to ADP framework in order to efficiently use resources. For example ADPs should support more ARK activities in satellite offices. This will also improve communication and utilization of other resources in the field. This will ensure sustainability of ARK activities in the absence of USAID funding. 5. IEC, Training and Promotional Materials in ARK Project Information, education, communication; training and promotional materials including T shirts, bangles, bags and training materials like facilitation guides, were essential component of the ARK project. At some sites, some of these materials were inadequate and inconsistent in supplies. Also the Radio messages were utilized to reach many more participants’ especially those who owned radios. However, participants especially those in the border towns suggested the use of billboards for beneficiaries who do not have access to radio. Recommendations iv. IEC Materials Several issues were raised with regard to the T-shirts, bangles, bags and training materials like facilitation guides. These were not always enough and while key informants noted that there was distribution criteria, participants felt it was inconsistent and not transparent. Recommendations  Ensure that as part of planning that there is always enough training and other project materials and ARK commodities for distribution to the beneficiaries. This process should be accomplished at the early planning stages of the project; otherwise this affects the good reputation of ARK, and WV in general.  Provide the peer educators should be equipped with adequate working tools to facilitate their work with the youth in order to achieve set goals in a timely manner.  Standardize distribution for different activities and be open about it.  Improve IEC messages coverage by using more mass media vehicles like billboards. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 74  IEC material developed should be integrated within youth and child health policies at the national level and target set to achieve those with adequate budgetary line item to ensure it is implemented. 6. Create opportunities that will sustain behavior change Income Generation Activities (IGAs) were suggested as one way of sustaining behavior change of vulnerable groups (youth and sex workers). Lack of IGAs continues to render them vulnerable. Recommendations:  Support people’s behavior change by creating opportunities that will sustain their new chosen behaviour.  Compile a resource book of activities that youth and others on alternatives to risky life style for those who decide to change their behavior can be involved in. 7. Increase the accessibility of VCT & access to combination comprehensive HIV prevention packages. Participants said people who were willing to go for VCT after ARK sensitized them could not test in some cases because there were not enough testing kits for us at the time of counseling. Men’s uptake of VCT was also viewed as generally low. It was noted that in the later years of ARK project, the focus extended to condom use promotion. Participants also suggested that they should be given information on other combination comprehension prevention packages including condoms even though they understood that ARK’s intervention was based on abstinence and be faithful message to young people. Recommendations  Improve VCT uptake in ARK Project by working closely with and informing service providers about your plan before sensitizing campaigns on VCT at the community level. There is need to balance demand creation with supplies of such services.  Develop ways of reaching men with VCT information.  Work with local health authorities to provide both mobile VCT and home￾based HIV testing for individuals and household within ARK project  Explore opportunities of giving more information on comprehensive HIV prevention services including condoms. 8. Operations Research Studies: A number of operations research studies could be conducted to assess the effect of different combinations of interventions under different programme settings. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 75  The use of controls has ignored and yet it could offer a very powerful comparison to investigate quantitatively the effect of interventions. Youth in schools offer an easy to work with captive audience which would reduce costs normally associated with quantitative impact surveys. As the programme expands clear indicators before intervention and after interventions should be measured.  World Vision should seek to collaborate with agencies conducting social, health and demographic surveys. In such circumstances a few questions may be added to gather data on key impact indicators.  World Vision should continue to take advantage of its regional presence to share data gathering tools, processes and lessons learnt from different site and country settings.  At some point in future, World Vision needs to evaluation the different strategies of interventions and their combinations; for example, TOTs and voluntarism, the understandability and effect of materials. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 76 VIII.APPENDICES Case Studies Survey Instrument or evaluation questionnaire Qualitative instrument ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 77 Case Study 1: Matumaini Group, Misenyi This case study demonstrates how ARK is forging useful partnerships to benefit the communities. Secondly, it demonstrates the use of VCT services by some of the participants. After its inception in 2005 ARK started sensitizing people in communities in Misenyi. After these campaigns, ARK realized that a lot of people came to their offices for testing and they were not able to provide this service. ARK then approached the ADP manager for World Vision and through their HIV/AIDS directorate; more testing reagents were budgeted for. However, they realized that they still could not meet the huge demand. Then the ADP manager initiated a link with the local service provider ‘Kyaka Youth Friendly Centre’ to assist in providing VCT services to those approaching ARK for it. Those who tested also wanted to find out how ARK could support them in terms of food security. ARK referred them to the ADP projects that provided support with food and medication for opportunistic diseases. The people who had tested then joined Matumaini support group that already existed but had no funding as the NGO that was supporting it had just closed down. The people joined the support group as a result of the stigma that they initially experienced after their status was confirmed. Once they received support of how to generate their own food and how to sell the excess, they were confident enough to declare their status openly. They also sought ways of supporting the ARK project and decided to join in ARK campaigns to give testimonies of their experiences to the communities; how they were infected, viewed and discriminated. The chairperson of the support group was trained about ARK activities. Kyaka Youth Friendly services continue to work closely with ARK as a VCT service provider. Member serial number Ever tested for HIV Yes = 1 No=2 How many years ago did you test? Decision influenced by ARK? Are you practicing any abstinence? Yes =1 No=2 Decision influenced by ARK? 01 1 4 1?? 2 2 02 1 6 2 2 2 03 1 1 1 2 2 04 1 4 1 1 1 05 1 2 1 1 1 06 1 3 1 1 1 07 1 4 2 1 1 08 1 3 2 1 1 09 1 4 2 2 2 10 1 2 1 2 2 11 1 3 1 1 1 N=11 11 6 tested within ARK existence 7 decision influenced by ARK 6 yes 5 No During the end term project evaluation, some members of the group were approached to find out some of their experiences with VCT. The group started with 9 members ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 78 and currently had a membership of over 37 working together as a support group. Data was gathered from 11 of them who could be assessed on the day of the visit to the offices. Of this group, 7 of them said their decision to test had been influenced by ARK. Six of the 11 said they are practising abstinence and this decision was a result of the relationship with ARK. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 79 Case Study 2: Monduli, VCT uptake The youth in Monduli knew about ARK and its activities and understood that the purpose of ARK was to reduce the rate of HIV infections amongst them. They showed deep understanding of how risky behaviour could lead to HIV infection. Once they got to understand this they took control of their lives by changing and eliminating the risky behaviours. They also mentioned that through ARK they were now able to respect their parents, themselves and each other as boys and girls. They kept on saying “It has taught us to value life and to love ourselves” (FGD, Naiti). They thought involving parents in ARK activities was a very good idea and reported change in their homes as well. They said “There were things that were going on at home that are no longer happening, for example, female circumcision is no longer practiced despite its having been a cultural practice for a long time” (female respondent, Naiti). Parents were viewed as “now faithful to each other”. “Most of the Maasai men used to have two wives but through ARK they have stopped” (male respondent, Naiti). For example, at first a wife or a husband could die and a member of the family would have to take over either as a husband to the widow (widow inheritance). This has stopped. VCT ARK is the one organization that has worked with them over time and sensitized them about HIV/AIDS. As a result of sensitization about HIV they said they have sought a fresh start in their lives by going for VCT. Here are some of their testimonies: I am now able to fight against risky behaviours because of ARK. I am abstaining and waiting to get someone special. I also know that I will behave when I am married. Male respondent. I used to engage in ezembe (practicing sex without a condom) for a long time and had not been using condoms and now through ARK I have changed. After testing I have decided to stop and now I use my time to educate youth about HIV and AIDS. Male respondent. Before ARK I was involved in a lot in sexual activities, I had a lot of boys that I was sleeping around with but after getting the message from ARK I decided to change and abstain and now I am involved in groups that are educating the community about the impact of HIV. I realized that what I had been doing was wrong and it put my life at risk. I went to test VCT and after that I stopped sleeping around. ARK rescued a lot of us I wish it can go to other places that are far from here and where there is no one who educate them about these things. Female respondent. Some cultural practices were very dangerous to us because they exposed us to dangerous practices that could have led to getting HIV. We used to meet with boys at night for singing and dancing and sex. At first it was taken this was a normal thing and because of that we have been engaging in sex again and again. But now after knowing the truth about HIV and AIDS and how people get it we have decided to change. Female respondent. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 80 Youth in this group said after getting enough information and knowledge from ARK, after watching the videos and going to training, abstaining was not a difficult decision for them. They now knew that AIDS is there, it happens and it can happen to them as well. When asked what made it hard for them to abstain in their environment one said “In the past there were many challenges and things that promoted risky behaviour in our community and culture. But knowledge as a tool has helped us to avoid those practices. Our parents know now why some practices are no longer acceptable”. The pressure is not so much because our parents understand when we do not want to do some of the things anymore. The youth from Naiti, Monduli were asked about how they had got to use VCT. Nearly, all had tested for HIV and the whole groups said they had been influenced by ARK in their decision to test. They also said they were practising abstinence and again this was a result of the influence of ARK. Monduli: Maasai youth ARK influence for VCT uptake Member serial number Ever tested for HIV Yes = 1 No=2 How many years ago did you test? Decision influenced by ARK? Are You practicing any abstinence? Yes =1 No=2 Decision influenced by ARK? 1 1 2 1 1 1 4 1 2 1 1 1 13 1 2 1 1 1 3 1 2 1 1 1 12 1 2 1 1 1 2 1 2 1 1 1 5 1 2 1 1 1 8 1 2 1 1 1 6 1 2 1 1 1 10 1 2 1 1 1 7 1 2 1 1 1 9 1 2 1 1 1 11 1 2 1 1 1 14 1 2 1 1 1 13 1 2 1 1 1 8 1 3 1 1 1 Total 16 16 within ARK’s inception 16 decision influenced by ARK 16 practicing abstinence 16 decision to abstain influenced by ARK ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 81 Case Study 3 (Partnerships and Religious): Monduli Participants from Faith Based Organizations related stories of how ARK brought two major religious groups of the area together, that is, Muslims and Christians. It helped them to break stereotypes about each other’s religion. Together the leadership from the two groups confronted their cultural and religious beliefs and practices that are harmful (e.g., polygamy). “I have changed my perception and that of my fellow Muslims towards Christians and also address the issue of polygamy which is contradicting the “Be Faithful” message” (Sheikh in Makuyuni Mosque narrated). Through ARK they have found ways of having their voices as community members and believers heard “ I have been able to prepare plans to meet the morans to discuss openly about HTPs and make a follow up especially Esoto and marrying un-mutilated girls something which is working very well now” (Maasai youth out of school, Catechist Catholic Church). Youth in church have been able to talk openly about sexuality when they give testimonies “ARK has helped in the evangelization process as youth in my church have given personal testimonies and declared that they will stop risk behaviours and stay away from sex” (Pastor PCA). ARK gave courage and a platform for a female pastor to tackle the issue of Multiple Concurrent Partners with married women in her church through seminars. The participant in the meeting summed up the discussion with the FBOs when she said “The strong Unity among the Christians and Muslims has been cemented by ARK, that is, why we can interchange for the Sheikh to go to Church and Pastors to the Mosque to talk neutral about ARK Messages”. What ARK has been able to do through many platforms was described by many participants as “simply wonderful”. It has given people power and the tools to deal with issues in their communities in an amicable and peaceful way. Knowledge has been turned into power. Partnerships and collaborations have emanated from unlikely groups as ARK has brought them together. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 82 Final Evaluation of ARK Project in Kenya, Tanzania and Haiti (March-June, 2010) YOUTH ENDLINE QUESTIONNAIRE: 15 – 24 Years Introduction: Hello, Greetings. My name is …… I am here for a study on behalf of World Vision, Inc and we are conducting a survey to assess the accomplishments of the ARK project that has been running for 5 years. The ARK project seeks to expand and strengthen HIV/AIDS prevention efforts through changes in behaviour that will reduce the risk among young people of becoming infected with the HIV/AIDS virus –primarily abstinence and mutual monogamy while creating a supportive family and community environments. We would like to understand what you think and what your experience are with respect to the project. Are you willing to participate in this interview? [Then read and discuss the informed consent in detail!] Country 1 = HAITI 2 =TANZANIA 3 = KENYA DISTRICT/REGION 1. 2. 3. 4. 5. 6. Organisation Target Youth Group 15-24 Years Purpose & Objectives of the Evaluation Tool 1. To determine the tangible accomplishments of the ARK project over the five years. 2. To assess the operations and impact of the ARK project. 3. To identify best practices and lessons learned in the project. Date: Section Theme No. of Questions ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 83 1 Background information 8 2 Attitudes 5 3 Reproductive Health Knowledge and Sexual Behaviour 47 4 Life Skills and Livelihood 2 Total number of questions 62 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 84 SECTION 1: BACKGROUND INFORMATION Let’s begin by talking about some basic details: No. Coding Categories SKIP To Q101 RECORD SEX OF THE RESPONDENT Male Female 1 2 Q102 How old are you? Age in years Don’t know No response |__]__] 88 99 Q103 Who is your guardian where you live? Mother Father Father &Mother Other Relative (Specify) Other Non Relative (Specify) 1 2 3 4 5 Q104 What is your relationship status? Single Have boyfriend/girlfriend Separated / Divorced Widowed Married No response 1 2 3 4 99 Q105 Do you have any children of your own? include only biological children Yes No Don’t know No response 1 2 88 99 If No, SKIP to Q107 Q106 How many children do you have? Write number of children (can be 0) [_____] Now I would like to ask you some questions about school. Q107 Are you currently in school? Yes No No response 1 2 99 Q108 What grade (class/form) or level are you in? If respondent is out of school, record highest grade attended Record the grade (Gov’t sch) Record the level (community sch) College/University (highest level completed) Don’t know No response [____] [____] [____] 88 99 SECTION 2: ATTITUDES TOWARDS THOSE LIVING WITH HIV Now I’d like to ask you about the attitudes of people in this community towards people who have been affected by HIV/AIDS. And I am going to read out some statements about HIV/AIDS. I would like you to tell me what you think about them. You can say “Strongly agree”, “Agree”, “Disagree” or “Strongly disagree”. If you are not comfortable answering any one question or section, feel free to let me know. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 85 Q201 AIDS is a punishment for bad behaviour. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q202 People with AIDS deserve what they get. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q203 I would feel ashamed if someone in my family got AIDS. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q204 I am happy to show compassion for people I love, whether they have HIV/AIDS or not. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q205 I do not discriminate or stigmatize people living with HIV and AIDS. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 SECTION 3: HIV/AIDS, REPRODUCTIVE HEALTH KNOWLEDGE AND SEXUAL BEHAVIOUR Now I would like to talk to you about what you think about and know about HIV/AIDS, reproductive health and sexual behaviour, and the choices you make in your own life. Q301 How does someone with HIV/AIDS look like? Multiple responses possible. Thin Fat Their skin looks different Their mouth looks different Coughs Always sickly Hair looks different You can’t really tell Other (specify)_______________________ __ Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Q302 What are some of the ways that youth can adopt to avoid getting HIV/AIDS? Multiple responses possible. There is nothing we can do Avoid mosquito bites/sleeping under a net Marrying a young girl Sleeping with a minor Abstinence Be faithful Condomise/Use condoms Avoid sharing sharp instruments (e.g. razors) Not having sex with prostitutes Not having sex with many people Getting married Not getting married Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 86 Q303 Which of these methods (if any) do you currently use? (now) Multiple responses possible. Probe “any other?” None / Avoid mosquito bites/sleeping under a net Marrying a young girl Abstinence Be faithful Use condoms Avoid sharing sharp instruments (e.g. razors) Not having sex with prostitutes Not having sex with many people Getting married Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 Q304 Which of these methods (if any) do you intend to use in future to prevent HIV/AIDS infection? Multiple responses possible. Probe “any other?” None / There is nothing we can do Avoid mosquito bites/sleeping under a net Marrying a young girl Sleeping with a minor Abstinence Be faithful Use condoms Avoid sharing sharp instruments (e.g. razors) Not having sex with prostitutes Not having sex with many people Getting married Not getting married Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 Q305 What do you think are the chances that a person like yourself will get HIV/AIDS? Probe. Absolutely no chance Not likely Likely Very likely Don’t know No response 1 2 3 4 88 99 Q306 Why do you think you have that chance to get HIV? Multiple responses possible. Be sensitive to use of local language. I am not having sex I am having sex I don’t use condoms / having unprotected sex I use condoms I do not have sex with prostitutes I am faithful to my partner I am unfaithful to my partner I avoid sharp instruments (needles, razors) I will avoid a blood transfusion I avoid mosquito bites Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 87 Q307 Where can you get information on HIV/AIDS? Multiple responses possible. Home School Clinic Church Radio/ TV Books/Magazines Posters/leaflets/brochures Anti-AIDS Club Home-based care Counselling / VCT centre Youth-friendly corner ARK Other community-based organisation (CBO) Internet Other (specify) _____________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 99 Q308 Where can you get information on reproductive health (i.e contraceptives, condoms, circumcision, and menstrual hygiene)? Multiple responses possible. Home School Clinic Church Radio/ TV Books/Magazines Posters/leaflets/brochures Anti-AIDS Club Home-based care Counselling / VCT centre Youth-friendly corner ARK Other community-based organisation (CBO) Internet Other (specify) _____________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 99 Now I am going to ask you about some behaviour that youth sometimes engage in. All the things we talk about will be private between you and me. You can tell me if there are any questions you prefer not to answer. Q309 Do you smoke cigarettes? Yes No No response 1 2 99 If No or NR SKIP to Q311 Q310 Have you smoked a cigarette in the last week? Yes No No response 1 2 99 Q311 Do you drink alcohol? Yes No No response 1 2 99 If No or NR SKIP to Q313 Q312 Have you drunk alcohol in the last week? Yes No No response 1 2 99 Q313 Do you have a boyfriend/girl friend? Yes No No response 1 2 99 Q314 Have you ever had sexual intercourse? Yes No No response 1 2 99 If No or NR, SKIP to Q327 Q315 How old were you when you first had sex? ________________Years Old Q316 Think about the last person you had sex with. How old were they? (approximate age) Record age in years Don’t know No response ____ _ 88 99 Q317 Have you had sexual intercourse within the last 3 months? Yes No No response 1 2 99 Q318 How many sexual partners have you had in the last 6 months? Record number (Could be 0) ____ ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 88 Don’t know No response _ 88 99 Q319 Have you ever had sex without a condom? Yes No No response 1 2 99 Q320 Have you ever been pregnant or made anyone pregnant? Yes No 1 2 Q321 Have you ever used a contraceptive method to prevent pregnancy? Yes =1 No=2 IF No SKIP TO 325 Q322 What contraceptive method was it? Pills = 1 Injectable = 2 Condom = 3 Other=4 None=5 Q323 Are you currently using any contraceptive method to prevent a pregnancy? Yes = 1 No =2 Q324 What contraceptive method is it? Pills = 1 Injectable = 2 Condom = 3 Other=4 None=5 Q325 What signs and symptoms of sexually transmitted infections do you know? Discharge in private parts=1 Ulcers in private parts=2 Lower abdominal pains=3 Other (specify)......................................4 Don’t know 88 No response 99 If DK or NR SKIP TO 327 Q326 Have you ever had a sexually transmitted infection? Yes=1 No=2 Now I will ask you some questions about “staying away from sex”, which is sometimes also called abstinence / avoidance of risky behaviour. Q327 Have you heard of the word “abstinence”? Yes No No response 1 2 99 If No or NR SKIP to Q339 Q328 What do you understand by the word “abstinence”? Multiple responses possible. Use probes but DO NOT lead. E.g. “any other?” Be careful to remember and use the terms used by the respondent. Staying away Staying away from sex Staying away from drugs Staying away from alcohol Staying away from risky behaviour Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 88 99 Q329 Have you heard any messages about abstinence in the past 6 months? Yes No No response 1 2 99 Q330 Q330 Are you currently “staying away” from sex? Use respondent’s terms. Yes No Don’t know No response 1 2 88 99 If No or NR SKIP to Q332 Q331 Why are you “staying away” from sex? Multiple responses possible. To avoid pregnancy To avoid HIV and AIDS To avoid diseases Want to complete education 1 2 3 4 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 89 No partner Will wait until marriage Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 5 6 7 8 9 88 99 Q332 Have you heard any messages about abstinence / risky behaviour in the past 1 year? Yes No No response 1 2 99 If No or NR, skip to Q335 Q333 If YES to Q330: What are the abstinence / risky behaviour messages you have heard about? Multiple responses possible. Write messages in the space provided. Abstinence is 100% safe Virgin power virgin pride ……….. ………… Other (specify) ________________________ Other (specify) ________________________ Other (specify) ________________________ Don’t know No response 1 2 3 4 5 6 7 88 99 Q334 Where did you get the abstinence /risky behaviour messages from? Multiple responses possible. Radio Television Fliers/posters Church School Health Centre Community Health Worker Traditional Birth Attendant Anti-AIDS club Community-based organisation (CBO) Programme (name)____________________ Other (specify)______________________ _ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 90 Q335 In your opinion, are abstinence / avoiding risky behaviour a beneficial message for you personally? (I.e. is the message helpful relevant, appropriate…?) Yes No Don’t know No response 1 2 88 99 If No DK or NR SKIP to Q337 Q336 If Q334 is Yes: So, why is “abstinence” / “avoiding risky behaviour” a helpful message for you? Can you name any ways in which abstinence brings benefits to your life? Record up to 3. 100% disease protection (STIs, HIV/AIDS) Have a healthy state of mind, body & soul No unwanted pregnancies Can focus on other goals (school, sports) Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 88 99 SKIP to Q338 Q337 If Q334 is No: So, why is “abstinence” / “avoiding risky behaviour” not a helpful message for you? Can you give me reasons why it is not relevant to your life? Record up to 3. I want to have sex It is not realistic to avoid sex I am married Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 88 99 Q338 In your opinion, do you think that people your age can abstain from sex until marriage? Yes, everyone can Some can, some can’t No, nobody can It depends on (specify) _________________ Other (specify) ________________________ Don’t know No response 1 2 3 4 5 88 99 Now I will ask you some questions about “being faithful”. Q339 Have you heard of the word “being faithful”? Yes No No response 1 2 99 If No or NR SKIP to Q401 Q340 What do you understand by the word “being faithful”? Multiple responses possible. Use probes but DO NOT lead. E.g. “any other?” Be careful to remember and use the terms used by the respondent. Staying with one partner Staying away from sex Having protected sex with numerous partners Other (specify)______________________ __ __ Don’t know 1 2 3 4 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 91 No response Q341 Are you currently being faithful to your partner? Yes No Don’t know No response 1 2 88 99 If No or NR SKIP to Q344 Q342 Why are you being faithful to your partner? Multiple responses possible. To avoid pregnancy To avoid HIV and AIDS To avoid diseases Want to complete education It is the right thing to do No partner Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 88 99 Q343 For how long have you been faithful to your partner? For the last year For the last 6 months For the last 3 months Since we were partners Other (specify)______________________ __ Don’t know 1 2 3 4 5 6 7 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 92 No response Q344 Have you heard any messages about being faithful in the past 1 year? Yes No No response 1 2 99 If No or NR, skip to Q401 Q345 If YES to Q339: What are the “being faithful” messages you have heard about? Multiple responses possible. Write messages in the space provided. Being faithful is 100% safe Virgin power virgin pride ……….. ………… Other (specify) ________________________ Other (specify) ________________________ Other (specify) ________________________ Don’t know No response 1 2 3 4 5 6 7 88 99 Q346 Where did you get the being faithful messages from? Multiple responses possible. Radio Television Fliers/posters Church School Health Centre Community Health Worker Traditional Birth Attendant Anti-AIDS club Community-based organisation (CBO) Programme (name)____________________ Other (specify)______________________ _ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 88 99 Q347 In your opinion, is being faithful a beneficial message for you personally? (I.e. is the message helpful relevant, appropriate…?) Yes No Don’t know No response 1 2 88 99 If No DK or NR SKIP to Q401 Q348 If Q347 is Yes: So, why is “being faithful” a helpful message for you? Can you name any ways in which abstinence brings benefits to your life? Record up to 3. 100% disease protection (STIs, HIV/AIDS) Have a healthy state of mind, body & soul No unwanted pregnancies Can focus on other goals (school, sports) Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 88 99 SKIP to Q401 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 93 Q349 If Q347 is No: So, why is “being faithful” not a helpful message for you? Can you give me reasons why it is not relevant to your life? Record up to 3. I want to have sex It is not realistic to avoid sex I am married Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 88 99 Q350 Have ever tested for HIV? Yes No Don’t know No response 1 2 88 99 If NO, DK or NR skip to Q401 Q351 Whom did you tell your result? (Do not include health workers or counsellors who conducted the test) No one Family member Sexual partner Someone from ARK Teacher, religious or community leader Don’t know No response 1 2 3 4 5 88 99 Q352 What was the result of the HIV test? HIV positive HIV negative I am not willing to tell the result Don’t know No response 1 2 3 88 99 SECTION 4: LIFESKILLS Now let’s talk about the skills and training that you have. Q401 In the last one year have you attended any life skills training? Yes No Don’t know No response 1 2 88 99 If NO, DK or NR skip to Q405 Q402 What life skills have you learnt about? Give hints but DO NOT READ OUT Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Q403 Has the life skills training brought changes to your life? Yes No Don’t know No response 1 2 88 99 If No, NR or DK SKIP to Q408 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 94 Q404 Mention 3 ways in which your life has changed since life skills training. Can be positive or negative Helps me to be independent I have a lot of friends I have improved my skills Makes me think wiser Helps me to think critically I have learnt to respect people No one can bully me Helps me not to be lazy Helps me to decide on right and wrong Helps me to get what I want Other (specify) ___________________ Other (specify) ___________________ Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 Q405 Would you recommend the skills to your other friends? Yes = 1 No = 2 Q406 Which of these skills would you teach to others? Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Q407 What skills will you be using in the next 12 months? Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Finally, I would like to conclude by talking with you about your goals for the future. Q408 During the last 1 year, have you thought about your future plans? Yes No Don’t know No response 1 2 88 99 Q409 What would you like to do? (open response) Use this as an opportunity to give encouragement as you conclude. ____________________________________ ________ Q410 (During the past 1 year,) Have you talked about these future plans with anyone? Yes No Don’t know No response 1 2 88 99 If No, DK or NR SKIP to end ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 95 Q411 If yes to Q407: With whom have you talked with future plans? Friend (close in age) Parent/guardian Brother/sister Relative (close in age) Relative (older) Church leader Program staff Teacher Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 88 99 Q412 Would you have thought about these future plans in a similar or different way if you had not known ARK programme? 1. Different ways 2. Similar way End with an encouraging message for the youth about their future: e.g. I wish you the best of luck in achieving the goals that you have just told me about. Thank you very much for talking with us today, especially about some of these difficult issues. I assure you that the things we discussed today will stay private between you and me. I have finished my questions to you. Is there anything you would like to ask me? Explain what happens next: As I explained at the beginning, we’re asking young people from different places in Kenya/Tanzania/Haiti the same questions. When our activities here are finished, we will collect all the responses we have received and add them together to find out what young people are saying. We hope you will be able to find out about the results by contacting WV (identify local partner and make sure youth knows how to contact them). ………………….THE END ………………. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 96 Final Evaluation of ARK Project in Kenya, Tanzania and Haiti March-June, 2010 YOUTH ENDLINE QUESTIONNAIRE: 10 – 14 Years Introduction: Hello, Greetings. My name is …… I am here for a study on behalf of World Vision, Inc and we are conducting a survey to assess the accomplishments of the ARK project that has been running for 5 years. The ARK project seeks to expand and strengthen HIV/AIDS prevention efforts through changes in behaviour that will reduce the risk among young people of becoming infected with the HIV/AIDS virus –primarily abstinence and mutual monogamy while creating a supportive family and community environments. We would like to understand what you think and what your experience are with respect to the project. Are you willing to participate in this interview? [Then read and discuss the informed consent in detail!] Country 1 = HAITI 2 =TANZANIA 3 = KENYA DISTRICT/REGION 1. 2. 3. 4. 5. 6. Organisation Target Youth 10 - 14 years Purpose & Objectives of the Evaluation Tool 1. To determine the tangible accomplishments of the ARK project over the five years. 2. To assess the operations and impact of the ARK project. 3. To identify best practices and lessons learned in the project. Date: Section Theme No. of Questions 1 Background information 6 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 97 2 Attitudes 4 3 Reproductive Health Knowledge and Sexual Behaviour 31 4 Life Skills 12 Total number of questions 53 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 98 SECTION 1: BACKGROUND INFORMATION Let us begin by talking about some basic details about yourself No. Coding Categories SKIP To Q101 RECORD SEX OF THE RESPONDENT Male Female 1 2 Q102 How old are you? Age in years Don’t know No response [__| __] __] 88 99 Q103 Who is your guardian where you live? Mother Father Father &Mother Other Relative (specify)………… Other non-relative (specify)…………….. 1 2 3 4 5 Q104 Are you currently in school? Yes No No response 1 2 99 If Yes SKIP to Q106 Q105 If NO to Q104: Have you ever been to school? Yes No No response 1 2 99 If No or NR skip to next sectio n Q106 What grade (class/form) or level are you in? If respondent is out of school, record highest grade attended. Grade (Gov’t sch) Level (community sch) Don’t know No response [__ __] [__ __] 88 99 SECTION 2: ATTITUDES TOWARDS THOSE LIVING WITH HIV Now I’d like to ask you about the attitudes of people in this community towards those who have been affected by HIV/AIDS. I am going to read out some statements about HIV/AIDS. I would like you to tell me what you think about them. You can say “Strongly agree”, “Agree”, “Disagree” or “Strongly disagree”. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 99 Q201 AIDS is a punishment for bad behaviour. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q201 I would feel ashamed if someone in my family got AIDS. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q202 I am happy to show compassion for people I love, whether they have HIV/AIDS or not. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 Q203 I do not discriminate or stigmatize people living with HIV and AIDS. Strongly agree Agree Disagree Strongly disagree No response 1 2 3 4 99 SECTION 3: HIV/AIDS, REPRODUCTIVE HEALTH KNOWLEDGE AND SEXUAL BEHAVIOUR Now I would like to talk to you about what you think about and know about HIV/AIDS reproductive health and sexual behaviour, and the choices you make in your own life. Q301 How does someone with HIV/AIDS look like? Multiple responses possible. Thin Their skin looks different Their mouth looks different Coughs Always sickly Hair looks different You can’t really tell Other (specify)_______________________ __ Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 88 99 Q302 What are some of the ways that young people can avoid getting HIV/AIDS? Multiple responses possible. There is nothing we can do Avoid mosquito bites/sleeping under a net Marrying a young girl Sleeping with a minor Abstinence Be faithful Condomise/Use condoms Avoid sharing sharp instruments (e.g. razors) Not having sex with prostitutes Not having sex with many people Getting married Not getting married Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 100 Q303 Which of these methods (if any) do you intend to use in future to prevent HIV/AIDS infection? Multiple responses possible. Probe “any other?” None / There is nothing we can do Avoid mosquito bites/sleeping under a net Marrying a young girl Sleeping with a minor Abstinence Be faithful Use condoms Avoid sharing sharp instruments (e.g. razors) Not having sex with prostitutes Not having sex with many people Getting married Not getting married Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 Q304 What do you think are the chances that a person like yourself will get HIV/AIDS? Probe. Absolutely no chance Not likely Likely Very likely Don’t know No response 1 2 3 4 88 99 Q305 Why do you think you have that chance to get HIV/AIDS? Multiple responses possible. Be sensitive to use of local language. I am not having sex I am having sex I am faithful to my partner I am unfaithful to my partner I avoid sharp instruments (needles, razors) I will avoid a blood transfusion I avoid mosquito bites Other (specify)_______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 88 99 Q306 Where can you get information on HIV/AIDS and reproductive health? Multiple responses possible. Home School Clinic Church Radio/ TV Books/Magazines Posters/leaflets/brochures Anti-AIDS Club Home-based care Counselling / VCT centre Youth-friendly corner ARK Other community-based organisation (CBO) Internet Other (specify) _____________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 99 Now I am going to ask you about some behaviour that youth sometimes engage in. All the things we talk about will be private between you and me. You can tell me if there are any questions you prefer not to answer. Q307 Do you smoke cigarettes? Yes No No response 1 2 99 Q308 Do you drink alcohol? Yes No No response 1 2 99 Q309 Do you have a boyfriend/girl friend? Yes No No response 1 2 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 101 Q310 Have you ever had sexual intercourse? Yes No No response 1 2 99 If No or NR, SKIP to Q314 Q311 How old were you when you first had sex? ________________Years Old Q312 How old was the person you had sex with the first time? (approximate age) Record age in years Don’t know No response ____ _ 88 99 Q313 Have you ever been pregnant or made anyone pregnant? Yes No 1 2 Q314 Do you know of any method to prevent pregnancy? Yes =1 No=2 IF No SKIP TO 316 Q315 What contraceptive method s do you know? Multiple responses possible Pills = 1 Injectable = 2 Condom = 3 None=4 Other (specify)........................................=4 Q316 Do you know any sexually transmitted infections? Yes=1 No=2 If No SKIP TO 318 Q317 What signs and symptoms of sexually transmitted infections do you know? Discharge in private parts=1 Ulcers in private parts=2 Lower abdominal pains=3 Other (specify)......................................4 Don’t know 88 No response 99 Now I will ask you some more questions about “staying away from sex”, which is sometimes also called abstinence / avoidance of risky behaviour. Q318 Have you heard of the word “abstinence”? Use the language of the respondent to describe abstinence: e.g. “keeping away”, “staying away”, “avoiding sex”, etc. Yes No No response 1 2 99 If No or NR SKIP to Q327 Q319 What do you understand by the word “abstinence”? Multiple responses possible. Use probes but DO NOT lead. E.g. “any other?” Be careful to remember and use the terms used by the respondent. Staying away from sex Staying away from drugs Staying away from alcohol Staying away from risky behaviour Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 88 99 Q320 In your opinion, do you think that people your age can abstain from sex until marriage? Yes, everyone can Some can, some can’t No, nobody can It depends on (specify) _________________ Other (specify) ________________________ 1 2 3 4 5 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 102 Don’t know No response Q321 Are you currently “staying away” from sex? Use respondent’s terms. Yes No Don’t know No response 1 2 88 99 If No or NR SKIP to Q323 Q322 Why are you “staying away” from sex? Multiple responses possible. To avoid pregnancy To avoid HIV and AIDS To avoid diseases Want to complete education No partner Too young Will wait until marriage Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 7 8 9 88 99 Q323 Have you heard any messages about abstinence / avoidance of risky behaviour in the past 1 year? Yes No No response 1 2 99 If No or NR, skip to Q327 Q324 What are the abstinence / avoiding risky behaviour messages you have heard about? Multiple responses possible. Write messages in the space provided. Abstinence is 100% safe Virgin power virgin pride ……….. ………… _____ Other (specify) ________________________ Don’t know No response 1 2 3 4 6 7 88 99 Q325 Where did you get the abstinence /avoidance of risky behaviour messages from? Multiple responses possible. Radio Television Fliers/posters Church School Health Centre Community Health Worker Traditional Birth Attendant Anti-AIDS club Community-based organisation (CBO) ARK Programme Home Other (specify)______________________ _ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 11 88 99 Q326 If YES to Q330: Through what means or activities did you get the abstinence /risky behaviour messages? Multiple responses possible. Group discussion Discussion one-to-one Drama Songs/poems Training Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 88 99 Now I will ask you some questions about “being faithful”. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 103 Q327 Have you heard of the word “being faithful”? Yes No No response 1 2 99 If No or NR SKIP to Q401 Q328 What do you understand by the word “being faithful”? Multiple responses possible. Use probes but DO NOT lead. E.g. “any other?” Be careful to remember and use the terms used by the respondent. Staying with one partner Staying away from sex Having protected sex with numerous partners Other (specify)______________________ __ Other (specify)______________________ __ Other (specify)______________________ __ Don’t know No response 1 2 3 4 5 6 88 99 Q329 Have you heard any messages about being faithful in the past 1 year? Yes No No response 1 2 99 If No or NR, skip to Q401 Q330 If YES to Q331: What are the “being faithful” messages you have heard about? Multiple responses possible. Write messages in the space provided. Being faithful is 100% safe Virgin power virgin pride ……….. ………… Other (specify) ________________________ Other (specify) ________________________ Other (specify) ________________________ Don’t know No response 1 2 3 4 5 6 7 88 99 Q331 Where did you get the being faithful messages from? Multiple responses possible. Radio Television Fliers/posters Church School Health Centre Community Health Worker Traditional Birth Attendant Anti-AIDS club Community-based organisation (CBO) Programme (name)____________________ Other (specify)______________________ _ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 88 99 Q332 Have ever tested for HIV? Yes No Don’t know No response 1 2 88 99 If NO, DK or NR skip to Q401 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 104 Q333 Whom did you tell your result? (Do not include health workers or counsellors who conducted the test) No one Family member Sexual partner Someone from ARK Teacher, religious or community leader Don’t know No response 1 2 3 4 5 88 99 Q334 What was the result of the HIV test? HIV positive HIV negative I am not willing to tell the result Don’t know No response 1 2 3 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 105 SECTION 4: LIFESKILLS Now let’s talk about the skills and training that you have. Q401 In the last one year have you attended any life skills training? Yes No Don’t know No response 1 2 88 99 If NO, DK or NR skip to Q408 Q402 What life skills have you learnt about? Give hints but DO NOT READ OUT Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Q403 Has the life skills training brought changes to your life? Yes No Don’t know No response 1 2 88 99 If No, NR or DK SKIP to Q408 Q404 Mention 3 ways in which your life has changed since life skills training. Can be positive or negative Helps me to be independent I have a lot of friends I have improved my skills Makes me think wiser Helps me to think critically I have learnt to respect people No one can bully me Helps me not to be lazy Helps me to decide on right and wrong Helps me to get what I want Other (specify) ___________________ Other (specify) ___________________ Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 11 12 13 88 99 Q405 Would you recommend the skills to your other friends? Yes = 1 No = 2 Q406 Which of these skills would you teach to others? Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 Q407 What skills will you be using in the next 12 months? Communication Goal setting Decision-making Avoiding peer pressure Negotiation skills Assertiveness Self esteem Role models Critical thinking Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 10 88 99 ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 106 Finally, I would like to conclude by talking with you about your goals for the future. Q408 During the last 1 year, have you thought about your future plans? Yes No Don’t know No response 1 2 88 99 Q409 What would you like to do? (open response) Use this as an opportunity to give encouragement as you conclude. ____________________________________ ________ Q410 (During the past 1 year,) Have you talked about these future plans with anyone? Yes No Don’t know No response 1 2 88 99 If No, DK or NR SKIP to end Q411 If yes to Q410: With whom? Friend (close in age) Parent/guardian Brother/sister Relative (close in age) Relative (older) Church leader Program staff Teacher Other (specify) ___________________ Don’t know No response 1 2 3 4 5 6 7 8 9 88 99 Q412 Would you have thought about these future plans in a similar or different way if you had not known ARK programme? 1. Different ways 2. Similar way End with an encouraging message for the youth about their future: e.g. I wish you the best of luck in achieving the goals that you have just told me about. Thank you very much for talking with us today, especially about some of these difficult issues. I assure you that the things we discussed today will stay private between you and me. I have finished my questions to you. Is there anything you would like to ask me? Explain what happens next: As I explained at the beginning, we’re asking young people from different places in Kenya/Tanzania/Haiti the same questions. When our activities here are finished, we will collect all the responses we have received and add them together to find out what young people are saying. We hope you will be able to find out about the results by contacting WV (identify local partner and make sure youth knows how to contact them). ………………….THE END ………………. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 107 Final Evaluation of ARK Project in Kenya, Tanzania and Haiti March-June, 2010 KEY INFORMANT INTERVIEW Country 1 = HAITI 2 =TANZANIA 3 = KENYA DISTRICT/REGION 1. 2. 3. 4. 5. 6. Organisation Target Group/Individuals Teachers Group representatives from PAG, YAGs Gov’t representatives NGO representatives Representatives from WV Johns Hopkins Bloomberg School of Public Health. Purpose & Objectives of the Evaluation Data Collection Tool 1. To determine the tangible accomplishments of ARK project over the five years. 2. To assess the operations and impact of ARK project. 3. to identify the strengths, weaknesses, opportunities and constraints of the project implementation. 4. to identify best practices and lessons learned in the project. 5. To make recommendations on effective ways to improve upon and model a more sustainable, community-owned and active youth-participatory HIV prevention programs. Date: 1. Briefly describe the ARK project a. Ask how much the persons / groups interviewed know about the project and b. how they have been involved. ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 108 2. Do you think the different groups [youth, parents, and teachers] know adequately about the project? Has the project fostered desired change in KABP? [This is not a quantitative questionnaire but try to focus on the issues as indicated in the Table.] For Question: 2 Group Knowledge about Project Desired Change from project (KABP) Youth Parents Teachers 3. What are the accomplishments / results of the ARK project over the past five years? Probe for the following issues: How were these results achieved? What other ways could have made ARK to realise more results? 4. This question is for WV staff, JHUCC and ARK two sub-grantees Explore the following issues a. Training, outreach, media and communication) b. Structure and organisation; (In terms of Donor, National office, field offices, community level (e.g.: PAGs, YAGs, radio listening groups, Health clubs). c. Did it meet all the needs of the project? d. Changes / Effect with respect to the knowledge attitude, behaviour and practices of youth. e. Changes / Effect with respect to the knowledge, attitude, behaviour and practice of parents and community organisations f. Effect on interaction between different key role players (youth, parents, community organisations). 5. Taking into account the functioning of the ARK project, what can we use to improve the future quality of HIV youth prevention programs? a. What are new thematic areas that we can add? b. What are old aspects that we should retain and expand upon? c. Other issues? ARK Final Evaluation in Haiti, Kenya and Tanzania August 2010 Page 109 6. What do you believe are the strengths and weaknesses of the project? 7. What type of constraints affected the implementation of the project? a. How can these be addressed? 8. What do you believe are the best practices and lessons that can be learned from the project? 9. How can these be integrated and scaled-up by local partners and within government infrastructures for youth HIV prevention? 10. What are your recommendations on the following? a. Effective ways to improve upon the ARK way of programme in addressing HIV/AIDS prevention for young people? b. A community-owned and active youth-participatory HIV prevention programs? 11. To what extent have the linkages with stakeholders including local govt, CBOs, FBOS been enhanced? a. How have they been made sustainable?