Formative Evaluation: Presidential Initiative on AIDS Strategy for Communication to Youth Formative Evaluation: Presidential Initiative on AIDS Strategy for Communication to Youth Netsayi N. Mudege Chi-Chi Undie African Population and Health Research Center Acknowledgements The authors would like to thank the study participants from the Ministry of Education and Sports (MOES), Uganda Initiative for Teacher Development Management System and PIASCY (UNITY), Uganda Programme for Human and Holistic Development (UPHOLD), as well as the pupils, teachers, and school administrators who took their time to accommodate us and answer our questions. We would also like to thank Margaret Kemigisa, our consultant in Uganda, for making initial contacts with key informants, facilitating discussions, and assisting with field logistics. Special thanks to the field workers for a job well done. We also thank Sam Kalibala and Deborah Weiss of the Population Council for reading our report and providing invaluable comments. Special thanks also go to our APHRC colleagues, Joseph Inungu, Chimaraoke Izugbara, Akaco Ekirapa, Ziraba Kasiira, and Joseph Gichuru for their contributions to the project. This report was made possible through support provided by the United States Agency for International Development (USAID)/Uganda under the terms of Contract No. GHH-I-02-07-00034-00. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. APHRC conducts research in Africa focusing on health, urbanization, population studies, and education. The organization contributes to science through high impact research projects and publications. APHRC brings together African scholars to take the lead in developing priority research programs and enhancing use of research findings for policy formulation and program improvement in sub-Saharan Africa. www.aphrc.org The Population Council conducts research worldwide to improve policies, programs, and products in three areas: HIV and AIDS; poverty, gender, and youth; and reproductive health. www.popcouncil.org Published in September 2009. Copyright © 2009. United States Agency for International Development. Suggested citation: Mudege, Netsayi N. and Chi-Chi Undie. 2009. “Formative evaluation: Presidential Initiative on AIDS Strategy for Communication to Youth,” Final Report. New York: Population Council. This document may be reproduced in whole or in part without permission of the Population Council provided full source citation is given and the reproduction is not for commercial purposes. Telegram: Telephone Fax: "EDUCATION" 23445118 234920 In any correspondence on This subject please quote No. ADM/105/136101 'TlfE 11 REl'UIUC Of IXlANDA FOREWORD Ministry of Education & Sports Embassy House P.O. Box 7063 Kampala, Uganda In 2002, the President of Uganda, Yoweri Museveni, proposed a way to improve communication on HIV and AIDS among young people in an effort to sustain declining trends in HIV prevalence rates. The President's vision was for head teachers to address assemblies on HIV and AIDS every two weeks, after which other teachers could continue the discussion in classrooms and clubs. The Uganda AIDS Commission responded to the President's call and brought together line ministries, civil society organizations, the private sector, and individuals working in HIV to forge a way forward, which led to the inception of the Presidential Initiative on AIDS Strategy for Communication to Youth (PIASCY). PIASCY is a national, holistic programme designed to provide all school-going children and teachers with information on HIV/AIDS both to cope with the disease￾for those infected and affected-and to prevent further infections. The program is spearheaded by the Ministry of Education and Sports (MOES) and covers all primary schools in the country. Targeting parents and communities reinforces activities delivered in the school. Most recently, PIASCY has expanded to cover post primary institutions under the Uganda Initiative for Teacher Development Management System and PIASCY. The formative evaluation of PIASCY, described in this report, examines the successes and challenges of the programme, and highlights a variety of lessons learned, best practices, and sustainability issues. MOES intends to use these findings to inform the continued roll-out and expansion of PIASCY in the years ahead. It is also hoped that evaluation findings will prove valuable for school-based HIV prevention programmes in other countries, given the unique opportunities such programmes afford to reach large proportions of young people. We thank all those who supported, conducted, and participated in this important evaluation. Table of Contents Acronyms i I. Executive Summary 1 Key Findings 1 Recommendations and Conclusion 4 II. Introduction 8 Background 8 Objective and Research Questions 9 III. Methodology 10 Study Design 10 Selection and Description of the Study Sample 10 Ethical Considerations 10 Data Collection Methods 11 Data Management and Analysis 11 Limitations of the Study 12 IV. Results 13 IV.A. To what extent is PIASCY achieving its planned goals and objectives? 13 IV.B. What are the strengths and limitations of the design, organisational structure, and rollout of PIASCY? 21 IV.C. What are the lessons learned and best practices for continued rollout of the programme? 28 IV.D. What are the unintended consequences of the development and implementation of PIASCY? 30 IV.E. What are the sustainability issues that will need to be addressed in handing over the programme? 33 V. Discussion 35 VI. Recommendations and Conclusion 40 VII. References 43 VIII. Annexes 45 Annex 1: Respondent Categories and Sample Size 45 Annex 2: Study Instruments 46 i Acronyms AIDS Acquired Immune Deficiency Syndrome ARV Anti-retroviral APHRC African Population and Health Research Center BEPS Basic Education Policy Support CCT Coordinating Centre Tutor CIE Community Involvement in Education CPTC Core Primary Teachers College FGD Focus Group Discussion GOU Government of Uganda HIV Human Immunodeficiency Virus MDD Music, Dance and Drama MOES Ministry of Education and Sports NACP National AIDS Control Programme PIASCY Presidential Initiative on AIDS Strategy for Communication to Youth PPTC Private Primary Teachers College PTC Primary Teachers College TDMS Teacher Development and Management System UNEB Ugandan National Examination Board UNITY Uganda Initiative for Teacher Development Management System and PIASCY UPE Universal Primary Education UPHOLD Uganda Programme for Human and Holistic Development USAID United States Agency for International Development PIASCY Formative Evaluation 1 I. Executive Summary HIV prevention programming is increasingly taking place in school settings, which provide an expansive population of young people and offer immense potential for making a large and much-needed impact in the lives of this target group. The Presidential Initiative on AIDS Strategy for Communication to Youth (PIASCY) is an ambitious, school-based programme that has sought to address the HIV/AIDS epidemic in Uganda in a holistic manner since 2002, targeting young people, school personnel, parents, and the wider community. A formative evaluation was conducted by the African Population and Health Research Center (APHRC) in February 2009 to provide an understanding of: the extent to which PIASCY is achieving its planned goals and objectives (e.g., increased capacity to deliver learning resources and materials, increased skills and knowledge of chief actors, and promotion of stigma-free school environments); the strengths and limitations of the design, organisational structure, and rollout of PIASCY; the lessons learned and best practices for continued rollout of the programme; the unintended consequences of the development and implementation of the programme; and the sustainability issues facing the programme‘s continued implementation. The study findings highlight the positive elements of the programme that can serve as a foundation as PIASCY is rolled out to post-primary institutions, as well as challenges that need to be addressed in preparation for this process. Study Methods The study was carried out in the four main regions of the country (Northern, Central, Eastern, and Western). Its design was qualitative in nature, drawing on 250 focus group discussions (FGDs) with pupils, teachers, and parents/community members; 28 key informant interviews with relevant stakeholders and programme administrators; and rapid appraisals to record key PIASCY-related observations within the schools. Secondary sources—data available from government bodies and non-governmental organizations (NGOs)—were also used to supplement the interview and FGD data. Key Findings Child-centric, interactive PIASCY activities are regarded by most teachers and pupils as having the strongest impact. Respondents cited child-centric activities such as assemblies; Music, Dance, and Drama (MDD); and Safety Friends Networks as having the strongest impact. These activities imparted knowledge to pupils and their communities in a provocative and lasting way, and provided space for pupils to discuss, address, and/or reflect on their HIV/AIDS-related concerns. These activities are designed to be interactive and contain elements that inspire and empower young people. Textual PIASCY materials are regarded by most pupils and teachers as having the least impact. Despite the time and creativity invested in developing textual PIASCY materials such as readers, posters, and Talking Environments, overall, these were viewed as having the least 2 impact on pupils. Respondents identified a number of barriers to using these materials. Some schools did not have PIASCY readers; if they did, only a few were available. To preserve these texts, pupils in some schools were not allowed to handle them. In other cases, teachers suggested that low literacy of pupils (particularly at the lower primary level) prevented them from reading the books. Posters were also considered difficult for pupils (particularly at the lower primary level) to understand, as comprehension depended on English literacy abilities, which were lower at the primary school level, particularly in rural areas. Finally, posters and Talking Environments ran the risk of being overlooked because their messages had not been changed over time. There was also an apparent misunderstanding of the purpose of PIASCY readers and handbooks. The latter were designed specifically for teachers; nonetheless, the use of teacher handbooks by students highlights the demand for PIASCY pupil reading material by both students and teachers. Some teachers promote condom use as a PIASCY message. PIASCY‘s focus is on abstinence for primary school pupils, and condom use is not an intended message for young people under this programme. However, several teachers in the Eastern and Western regions indicated that they talked about condom use under PIASCY. This act seemed to be an honest misunderstanding of the PIASCY objectives, rather than an attempt to ―rebel‖ against them. Some teachers also felt this practice was logical, given that some older pupils in upper primary school were sexually active. This finding is in contrast to that of the Northern region, where teachers were unequivocal about not teaching condom use under PIASCY. Some teachers censor PIASCY messages. There was evidence that some teachers who taught PIASCY at the lower primary level censored PIASCY information. They spoke of revising the content of their PIASCY messages for lower primary pupils according to their personal beliefs, noting for example that ―the children are so young.‖ The data from many schools demonstrate that a good number of teachers at the lower primary level tend to emphasise topics such as personal hygiene, the importance of living in a clean environment, and of hand-washing after using the toilet, rather than placing an emphasis on life skills education, such as saying ―no‖ to sex. Guidance and counselling procedures under PIASCY need to be better understood and standardised. As part of the expansion of PIASCY, the current implementer of the programme [the Ugandan Initiative for Teacher Development and Management System and PIASCY (UNITY)], intends to enhance guidance and counselling activities in primary schools by producing materials on the subject and disseminating them nationally. UNITY also expects to improve the quality of guidance and counselling offered by training more teachers. However, guidance and counselling as currently practised in many schools is teacher-driven and takes the form of group instruction. Teacher-driven counselling involves teachers deciding what they want to talk to the pupils about depending on what the teachers define as the problem, and not problems as defined by pupils. This is problematic as guidance and counselling issues are meant to derive from the pupils themselves. Female teachers are also under-represented among guidance counsellors in schools; yet, female pupils may feel more comfortable obtaining guidance and counselling on certain issues from teachers of the same sex. PIASCY Formative Evaluation 3 PIASCY has successfully contributed to a stigma-free school environment, but inadequately addresses the needs of those who are HIV-infected. The PIASCY programme has achieved its intended result of creating more openness with regard to HIV/AIDS issues among teachers and pupils. A common perception among school personnel is that HIV-positive pupils are now more likely to disclose their status in school. At the same time, current messages under PIASCY do not take into account pupils and school staff that are already living with HIV, as they focus on HIV prevention. There is also widespread demand among school personnel for the provision of anti-retrovirals (ARVs) in schools for pupils and teachers. Structures are needed to ensure consistent implementation and longevity of the PIASCY programme in schools. The establishment of structures in schools (such as ―PIASCY teachers‘ committees‖ and ―PIASCY clubs‖ for pupils) is necessary to ensure that periodic PIASCY activities are planned and actually take place. In the absence of such structures, the continuity of PIASCY is threatened by the transfer of PIASCY-trained teachers (who are viewed as the sole possessors of the institutional knowledge on PIASCY) to other schools, or the lack of compensation for teachers‘ perceived ―extra work‖ of engaging with PIASCY. The role of communities in the PIASCY design is not well understood. Although communities are intended to be a key component of the PIASCY programme, community members appear to be the least engaged of all the target groups. Unlike other programme actors who are directly targeted through training, such as teachers and pupils, community members are targeted through more indirect channels such as attendance at assemblies, prize-giving days, and parents‘ days, during which HIV/AIDS-related messages are displayed and conveyed orally. Perhaps as a result, community members, including parents, were the least articulate about the PIASCY programme, as well as their role in it. Private schools do not function seamlessly within PIASCY’s organisational structure. PIASCY‘s organisational structure aligns with the Teacher Development and Management System (TDMS), which is situated within the Ministry of Education and Sports (MOES). Public schools were already situated within the TDMS prior to PIASCY‘s inception. Thus, both the public schools that were selected as Centres of Excellence/Model Schools and the Core Primary Teacher Colleges (CPTCs) under PIASCY played roles that were already familiar to them. Under PIASCY, two or three teachers from the Centres of Excellence/Model Schools were trained, and in turn these teachers were meant to train other teachers at their school and beyond. This is referred to as the knowledge cascade approach. CPTCs were tasked with introducing PIASCY training modules into their curriculum to ensure that all teachers receiving instruction at these institutions were trained in PIASCY. For instance, the Model Schools supported schools within their own catchments with regard to PIASCY-related issues, while the CPTCs focused on pre-service and post-service training of teachers on PIASCY. The country‘s two Private Primary Teacher Colleges (PPTCs), however, are not part of the TDMS, and are therefore absent from PIASCY‘s organisational structure. The vast majority of key informants noted that private schools, though designated as satellite schools, are disconnected from the programme. 4 Some districts and District Education Officers are not well-integrated into the PIASCY structure. Some District Education Officers (DEOs) are left out of the PIASCY structure despite the fact that they are vital decision-makers in the education system. The DEOs were not involved in the conception and implementation of PIASCY, as this followed the TDMS structure, which does not include the DEOs. Yet DEOs were expected to guide programme monitoring activities. This oversight reportedly affected the monitoring and evaluation of PIASCY activities, which was not carried out in some districts because the respective DEOs felt excluded from the TDMS structure used to roll out PIASCY. The monitoring of PIASCY is in need of improvement. The monitoring of the PIASCY programme was generally considered as poor across all regions except for the North. Monitoring was often noted by school personnel as either being non-existent or rare. The Centre Coordinating Tutors were supposed to monitor the project (under the guidance of the DEOs), but were often overworked and therefore ignored the monitoring of PIASCY in favour of monitoring ―core‖ school activities. One area in particular need of monitoring is PIASCY‘s knowledge cascade approach. Teachers pointed out that in most cases teachers trained under PIASCY were not training other teachers, either because they were not willing to do so, or because they faced time limitations and/or lacked resources to facilitate the training. The lack of monetary compensation is regarded by most teachers as a major weakness of the programme. The opinion that PIASCY has inordinately increased teachers‘ workload is widespread; therefore, teachers across all regions voiced their expectations for some form of financial compensation for the extra time they spend on integrating PIASCY into their regular classes or activities. There are more similarities than variances in participants’ experiences with and perceptions of PIASCY across the country. As a national programme targeting all primary schools in Uganda, one would expect some differences in participants‘ experiences with or perceptions of PIASCY, across the country‘s four regions and/or between urban and rural regions. However, narratives from the qualitative data collected in each region regarding the PIASCY programme were largely similar, with only a few markedly regional and urban-rural distinctions. For instance, compared to other regions, teachers in the North felt that monitoring of PIASCY programmes was strong. Teaching children about condoms also varied by region as indicated above. Rural schools were more likely to report language barriers to pupils understanding PIASCY material compared to urban schools. Despite the negative perceptions expressed about some aspects of the programme, in general, respondents where overwhelmingly in favour of the continuance of the PIASCY programme, underscoring that its overall benefits outweighed any limitations of the programme. Recommendations and Conclusion Several recommendations based on the formative evaluation are offered for the enhancement of the PIASCY programme. They are targeted at two levels of programme implementation: (1) schools and (2) MOES, UNITY, and USAID. PIASCY Formative Evaluation 5 Schools 1. Monitor the use of PIASCY readers (for schools that register them with the school library) to determine how often and by whom the books are being checked. This will help assess whether putting books in the library results in their optimum utilisation. 2. Address the language barriers posed by PIASCY reading materials by having teachers read PIASCY texts aloud to pupils; creating forums for peer-to-peer reading, or for upper primary pupils to read to their lower primary counterparts; and by involving pupils more closely in the development of messages to be posted within the Talking Environments. Teachers can also explain and clarify some of the messages posted on Talking Environments to pupils during assembly. 3. Establish a formal hand-over process for PIASCY-trained teachers that are being transferred. To avoid gaps in institutional knowledge, PIASCY committees can facilitate this process. MOES, UNITY, and USAID 1. Continue to build the capacity of teachers to seamlessly mainstream PIASCY across the curriculum through periodic refresher training. The widespread opinion that PIASCY has increased teachers‘ workload points to the need to ensure teachers have the skills to mainstream the curriculum in a way that does not overburden them. Refresher courses for PIASCY-trained teachers will also help mitigate the censorship of PIASCY information during teaching and the delivery of inappropriate information. 2. Consider involving teachers well-versed in PIASCY in monitoring schools‘ progress with the programme, especially given that there are too many schools for the Centre Coordinating Tutors to properly handle this responsibility. Instead, monitoring can be conducted by school PIASCY committees. Alternatively, the Centre Coordinating Tutors‘ monitoring tool could also be redesigned to include the monitoring of PIASCY activities. 3. Train more female guidance counsellors, as they are currently under-represented, and female pupils may feel more comfortable having specific issues addressed by guidance counsellors of the same sex. There is also a need to incorporate HIV/AIDS counselling for both students and teachers that are infected with or affected by the disease. 4. Explore ways of directly targeting parents and community-members to maximise their involvement in the PIASCY programme. Trainings developed specifically for these target groups can be conducted with strategically-placed community leaders, for example, using the knowledge cascade approach. This responsibility can be given to school PIASCY committees. 5. Explore ways to establish links between schools and health facilities to facilitate access to ARVs and counselling for pupils, school personnel, and community￾members living with HIV. Horizontal linkages with health workers and NGOs need to be established with schools because teachers may not have the skills to deal with certain issues. At the same time, NGOs have more resources, experience working with schools and communities, and highly trained and skilled personnel in the field of HIV/AIDS. There may be a need to fully adopt a multi-sectoral approach and advocate for a school health policy that will define ways of dealing with HIV as a chronic epidemic within schools. This may also serve to strengthen the implementation of PIASCY activities within schools. 6 6. Diversify current HIV-prevention PIASCY messages by taking into account pupils and school staff living with HIV. For example, PIASCY could teach about treatment, emphasising the importance of drug adherence, as well as inform both teachers and pupils about where to get assistance if needed. 7. There is a need to hold a meeting every two years to refocus the education sector strategy on HIV/AIDS. This will enable the MOES to evaluate its achievements and strengthen the coordination and implementation of PIASCY and other HIV/AIDS activities in schools. This could be turned into a regional meeting involving stakeholders and experts from other countries. This will help generate new ideas to rejuvenate and refocus PIASCY. However, it is noteworthy that this strategy will also have budgetary implications. MOES 1. Monitor the MOES policy to put books in the hands of children. Although this policy is intended to ensure that children have direct access to books, there is evidence that some schools are not adhering to it. Monitoring of the entire PIASCY programme by the Education Standards Agency may be more effective. 2. Strengthen school structures such as ―PIASCY committees,‖ as they appear to be instrumental in the programme‘s continuity within schools. PIASCY committees could be given resources to sponsor activities they identify as necessary for proper implementation of the programme at their schools. They can also be involved in monitoring and evaluation. Establishment of PIASCY committees could be mandatory for all schools. 3. Develop an operational definition of ―guidance and counselling‖ for the purposes of PIASCY, and determine to what extent it should be student- versus teacher-driven. There is a need to help school administrators, teachers, and pupils reconstruct the notion of guidance and counselling to include pupil-driven, individual-level engagement that goes beyond didactic instruction. A uniform training guide should be developed or adopted, and the required duration of training should be specified for one to be certified as a guidance counsellor. 4. Make concerted efforts to integrate private schools into the programme. Private schools should be brought on board as equal partners, not just as satellite schools to the Centres of Excellence/Model Schools. Holding deliberative meetings to explore ways of incorporating the PPTCs into PIASCY‘s organisational structure would also be useful. Integrating PIASCY into the private school curriculum could be made a requirement for licensing, and could be enforced through regular monitoring. PIASCY can also be integrated into policy through the university level. 5. Make concerted efforts to integrate districts and DEOs into the programme by involving them in decision making as well as the implementation of the programme, rather than just at the monitoring stage. 6. Monitor the knowledge cascade process regularly and evenly across schools and provide feedback to schools on how they are performing. 7. Tailor PIASCY instructions to the needs of older students within Free Primary Education schools, by focusing PIASCY instructions on both the grade and the age of the pupil, so that older students in lower grades can be included. 8. Include occasional studies to inform the programme implementation. 9. Identify empirical evidence on behaviour change that is resulting from the PIASCY programme. This evidence could be generated by conducting school-based behaviour PIASCY Formative Evaluation 7 change surveillance surveys to help PIASCY target and respond to arising issues and training needs. 10. Conduct a countrywide study of adolescents that passed through the PIASCY programme to ascertain whether knowledge in formative years has a positive impact on behaviour later in life. Such a study will be valuable in designing future school￾based programmes and in improving PIASCY design, delivery, and content. 8 II. Introduction Background The early 1990s in Uganda were characterised by a remarkable and now much-cited decline in national HIV prevalence, from a national average prevalence of 15 percent in 1992 (Cohen 2006)—and as high as 30 percent in the hardest hit regions (GOU 2003)—to 6.7 percent in 2005 (Uganda AIDS Indicator Survey 2004/2005). This success resulted from a coordinated effort between the Government of Uganda (GOU) and international and local non￾governmental organisations (NGOs). In 1986, high level political support led to the creation of the National AIDS Control Programme (NACP), which organised countrywide public campaigns against HIV, including HIV education programmes in schools. In 1987, the AIDS Support Organisation (TASO) was created to promote HIV care and to advocate against discrimination and stigma toward people living with HIV/AIDS. Early mobilisation of NGOs, including the Straight Talk Foundation and Naguru Teenage Information and Health Centre, led to more intensive education of youth in and out of school on HIV/AIDS and sexuality. Despite these achievements, recent evidence suggests that HIV prevalence rates have stagnated or even increased in some areas (GOU 2003 and 2007). According to the Uganda AIDS Indicator Survey, HIV prevalence rates are higher in urban areas (10.2 percent) than in rural areas (5.7 percent), while women have a higher prevalence rate (7.5 percent) compared to their male counterparts (5.0 percent). Several reasons have been put forward to explain these changes, including increased risky sexual behaviour, decreased intensity of HIV prevention programmes (UNAIDS 2006a), and ―individual and organisational complacency in responding to the epidemic, emanating from fatigue and false impressions that the epidemic was under control, following reports of declining trends‖ (GOU 2003). To sustain declining trends in the HIV prevalence rates among youth, President Yoweri Museveni launched the Presidential Initiative on AIDS Strategy for Communication to Youth (PIASCY) in 2002. PIASCY is a national programme designed to provide all school-going children and teachers with information on HIV/AIDS in order to prevent further infections, and to help both the HIV-infected and affected cope with the disease. Currently, about 7 million pupils are enrolled in primary schools with a total of about 130,000 teachers (Kibenge 2009). The broad objectives of the programme are to: a. Increase the capacity of a network of institutions (public and private) to continuously increase behaviour change; b. Increase the skills and knowledge of chief actors—teachers, parents, community leaders, and pupils—that culminate in the practice of behaviours that delay sex until marriage; and, c. Promote a stigma-free school environment in support of children infected and affected by HIV/AIDS (MOES 2008). In 2003, the United States Agency for International Development (USAID)/Uganda, through its Basic Education Policy Support (BEPS) programme, supported the MOES to develop two PIASCY handbooks and spearhead the orientation of teachers in their use. In 2005, the Uganda Programme for Human and Holistic Development (UPHOLD) took over support to roll out and implement PIASCY. By the end of 2008, UPHOLD handed over its PIASCY PIASCY Formative Evaluation 9 activities to the Ugandan Initiative for Teacher Development and Management System (TDMS) and PIASCY (UNITY), the current implementer of the PIASCY programme. PIASCY is a multifaceted programme that involves the use of textual material such as posters, readers, suggestion box notes, and HIV-prevention messages placed strategically around school compounds (―Talking Compounds/Environments‖). It also includes a variety of child-centred ―edutainment‖ activities, such as ―Music, Dance, and Drama,‖ that are often incorporated into school assemblies. Objective and Research Questions The objective of this formative evaluation is to assess the effectiveness of PIASCY to provide the GOU, the MOES, USAID/Uganda, UNITY, and other stakeholders with an understanding of the lessons learned and recommendations for improving programme strategies and/or activities for continued implementation of PIASCY. The African Population and Health Research Center (APHRC), through a subcontract from the Population Council and with funding from USAID/Uganda, carried out the activity in the four regions of Uganda (Northern, Central, Eastern, and Western) in February 2009. The evaluation was guided by the following research questions: 1. To what extent is PIASCY achieving its planned goals and objectives (e.g., increased capacity to deliver learning resources and materials, increased skills and knowledge of chief actors, and promotion of stigma-free school environments)? 2. What are the strengths and limitations of the design, organisational structure, and rollout of PIASCY? 3. What are the lessons learned and best practices for continued rollout of the programme? 4. What are the unintended consequences of the development and implementation of PIASCY? 5. What are the sustainability issues that will need to be addressed in handing over the programme? The study report first describes the evaluation methodology, data collection tools, and data analysis methods. It next presents the study results by research question. This is followed by a discussion of the study findings. Finally, a set of recommendations are offered. 10 III. Methodology Study Design The study design was qualitative in nature, drawing on focus group discussions (FGDs) and key informant interviews with programme actors. Patton (2002: 10) observes that ―Qualitative methods are often used in evaluations because they tell the program‟s story by capturing and communicating the participants‟ stories…. Understanding the program‘s and participants‘ stories is useful to the extent that they illuminate the processes and outcomes of the program for those who must make decisions about the program.‖ FGDs were conducted with pupils, teachers, and parents/community members, and key informant interviews were carried out with relevant stakeholders from entities such as UNITY, MOES, and UPHOLD. Secondary sources—data available from government bodies and NGOs—were used to supplement the FGD and interview data. A rapid appraisal tool was used to record key PIASCY-related observations within the schools. Selection and Description of the Study Sample Data collection occurred at both the school and the community levels. Using a stratified purposeful sampling approach, 80 primary schools were selected for participation based on their geographic location (North, Central, East, and West) and on their classification as either ―public‖ versus ―private‖ schools or ―rural‖ versus ―urban.‖ Twenty schools were selected from each region (10 in urban areas and 10 in rural areas), resulting in a total of 80 schools across the four regions. The majority of the schools selected (60) were ―Model Schools‖, as defined by the PIASCY programme, while a few (20—i.e., one per district) were private schools. Ethical Considerations The research team first obtained permission from headmasters of each school to conduct the study within their school setting, and to have those pupils that were willing and eligible participate in the evaluation. The team then worked with teachers to raise awareness about the study, identify potential pupils for participation, and answer questions as needed. To ensure that the respondents represented the range of PIASCY programme experiences, the team worked with teachers in schools to identify pupils that represent one of two levels of engagement: ―engaged‖ (visibly involved in PIASCY programme activities such as participating in PIASCY school club activities), and ―less engaged” (less visibly involved in PIASCY programme activities except for class lessons). Prior to each FGD, participants were asked to read an informed consent statement that assured confidentiality and the right to refrain from answering any question posed by the discussion moderator. All respondents were at liberty to refuse to participate in the study with the understanding that there would be no punishment for such refusals. Respondents were assured that their responses would remain anonymous unless they indicated that they could be identified. Adult respondents that were willing to participate in the study signed the informed consent form. There were no inducements for participation in this study, nor were PIASCY Formative Evaluation 11 any promises made to participants in this regard, apart from the fact that their responses could provide an understanding of the aspects that had worked well under PIASCY, so that these could be maintained, as well as to identify the negative aspects to inform recommendations on improvement. Data Collection Methods As noted above, this formative evaluation included desk reviews, FGDs, key informant interviews, and naturalistic observation via rapid appraisals. The evaluation team designed qualitative field guides with minor variations depending on the respondent category. Within each of the 80 schools, one FGD was conducted with ―senior men/women‖ (a term used to denote teachers that had been specifically trained to deliver PIASCY instruction), along with two FGDs with pupils. FGDs with pupils were conducted in sex-segregated groups at either the upper primary or lower primary level for each school. This resulted in a total of approximately 40 FGDs with boys and 40 FGDs with girls at the upper primary school level, and approximately 40 FGDs with boys and 40 FGDs with girls at the lower primary school level. One FGD per school was also conducted with parents/community leaders from a sub￾sample of 10 schools that were selected from the larger sample of 80 schools, using a convenience sampling strategy. The rapid appraisals were also carried out within these 10 schools, using an observation form—a checklist of observations made by the data collector to describe a specific school activity structure, or to note the presence or absence of PIASCY￾related materials. Finally, key informant interviews were conducted with several categories of stakeholders, including personnel from the MOES, UNITY, and UPHOLD. This process resulted in a total of 246 FGDs conducted with pupils, teachers, and parents/community members, and a total of 24 interviews with key informants. (See Annex 1 for a table summarising the data collection; see Annex 2 for study instruments.) Data Management and Analysis Interviewers took detailed, handwritten notes during the FGDs/interviews. The note-taking was enhanced by the use of a standardised contact summary sheet (see Annex 2 for an illustration), which contained focusing or summarising questions about each field contact. The corresponding notes and contact summary forms were labelled appropriately and typed in Microsoft Word format. The analysis of the interview and FGD data was conducted via a two-tiered process. Initially, first-level coding (Miles and Huberman 1994) was carried out by reading the field notes and contact summary sheets, and identifying key themes that emerged, or by summarising segments of data. Pattern coding (ibid) then followed by conducting close and repeated readings of the field notes and contact summary sheets to identify patterns within the emerging themes. The themes and patterns generated through this process were subjected to repeated cross-checking and comparison with other field notes in the data set, as well as with data from the rapid appraisal forms, to ensure their empirical grounding. Results were then written up thematically, organised around the five main research questions. Consistent with qualitative approaches to evaluation, findings are a reflection of the respondents‘ perspectives. Thus, the report includes several quotations that ―give voice‖ to 12 approximately 1,400 respondents, and that signify the main themes that emerged from the discussions and interviews. Where multiple quotations are used to highlight a theme, an attempt is made to draw from different regions and interviewee categories (if applicable) to further demonstrate the generalisability of each theme. Limitations of the Study A limitation of the study is that teachers helped identify the pupils for participation in the FGDs. This process could have introduced a selection bias. Pupils that participated in this evaluation were certainly aware of PIASCY and able to comment on its perceived impact; however, teachers might have been more likely to select pupils whom they knew well and who may have been more satisfied with the programme. This limitation is moderated by the use of multiple sources of data collected from multiple categories of participants, and by the use of triangulation to validate findings. PIASCY Formative Evaluation 13 IV. Results IV.A. To what extent is PIASCY achieving its planned goals and objectives? In order to provide all school-going children and teachers with information on HIV/AIDS, prevent new infections, and to help those infected with and affected by HIV cope with the disease, the PIASCY programme seeks to achieve a number of goals, including increased capacity to deliver learning resources and materials, increased skills and knowledge of chief actors, and the promotion of stigma-free school environments. This section illustrates the extent to which the PIASCY goals and objectives outlined above are being achieved. A general observation that emerged during the analysis of data on the above research question was that some schools were more actively engaged in PIASCY activities than others. While the engagement of some schools was limited to the creation of Talking Compounds/Environments1 and the existence of suggestion boxes and/or PIASCY readers, other schools demonstrated stronger and more proactive participation in the PIASCY programme through periodic activities (e.g., assemblies; open days; parent days; and Music, Dance, and Drama) deliberately incorporating PIASCY activities/messages. These schools invariably seemed to have structures (such as ―PIASCY teachers‘ committees‖ and ―PIASCY clubs‖) to support the programme. PIASCY committees consisted of teachers who usually met to plan for periodic PIASCY activities, prepare timetables, and assign specific roles to teachers. Such structures were instrumental in ensuring the continuity of the programme, especially in the event of the transfer of a PIASCY-trained teacher, since they diffuse knowledge of PIASCY across several personnel. PIASCY clubs were made up of pupils who would meet under the guidance of a teacher to engage in PIASCY activities. These clubs included but were not limited to drama clubs, dance clubs, and debate clubs. IV.A.1. Increased capacity to deliver learning resources and materials Key informants consistently mentioned that all PIASCY materials designed for teachers and pupils were printed and delivered to the target schools as intended. This finding supports information from the desk review, which highlights the reach of the PIASCY programme. Specifically, PIASCY reports indicate that, since its inception, the programme has distributed 113,616 copies of PIASCY materials countrywide including PIASCY Teacher‘s Handbooks, Guidance and Counselling Manuals and Charts, Community Involvement in Education (CIE) Toolkits, and Teacher‘s Guides for School Talking Environments (MOES 2008). As one MOES key informant notes, however, ―Delivering materials to schools is one thing and having people who are able to utilise the materials is another.‖ The study findings suggest that the efforts to distribute materials to schools are moderated by four issues of access to them: language barriers, insufficient supplies in light of high primary school enrolment rates, censorship on the part of teachers, and preservation of texts. As a result, in most of the FGDs with pupils, respondents spoke about the paucity of PIASCY reading materials. 1 See page 18 for additional information. 14 Language barriers to access PIASCY texts (e.g., readers, posters) are written in English. However, both teachers and pupils pointed out that pupils were unable to read these texts independently, as they had not yet acquired strong English language skills. This is an issue that affected all regions in both urban and rural settings, but was noted most often as a problem for lower primary pupils by teachers in rural areas. For example, when asked ―In your opinion, which [PIASCY dissemination] means has had the least impact on pupils?‖ FGD respondents noted: Some words in English cannot be easily translated into the local language. For instance, the words on the posters can be hard to interpret. The language used in the reading material is too technical for the pupils, and it‟s even harder for us teachers to help them understand. (Teachers, Northern region) R1: Talking environments, as pupils fail to read them and more so they do not have explanations as they just phrase words and most of the words are new to us. R6: The readings [have the least impact] because they leave the burden to us to read, and not every child can read since the readings are in English. (Boys, Lower Primary, Western region) When asked to elaborate on such responses, pupils tended to speak in general terms about the particular inaccessibility of posters to their fellow pupils who struggled with reading English. The specific terms that some pupils may have found difficult to understand were not mentioned in the FGDs. Some school personnel, such as head teachers, explained that posters were either bought by the school with grants they had received from PIASCY, or delivered to them by the Centre Coordinating Tutor (CCT), suggesting that some personnel and pupils of individual schools may have limited input into the kind of posters developed. However, MOES officials pointed out that materials had been field tested and revised before being adopted for generalised use. It is therefore likely that schools that pointed to limited input into the development of PIASCY materials had not been part of the sample of schools involved in the pilots. Insufficient supplies In some instances, the number of assigned PIASCY texts could not adequately serve the number of pupils, given high enrolment rates, particularly in Universal Primary Education (UPE) schools. A key informant from the MOES corroborated this point, which was also raised in several FGDs with teachers, saying, ―They also claim that because of UPE, the schools are overcrowded, so the books are not enough for the children. One book sometimes may have to be shared by between five to eight children.‖ A related issue is access to PIASCY materials for older pupils. As a result of UPE, some older pupils who were previously unable to afford primary school education are now able to attend school. Some UPE schools (mainly those situated in rural areas) have older/mature pupils in primary 1 and 2 (P1 and P2) who would otherwise qualify for PIASCY education; however, PIASCY is targeted at pupils in P3 and upwards: ―[PIASCY] was designed like [for] P3 to P4. [Mature pupils] are left out of the curricular and are therefore at risk of behaving naively with regards to sex and HIV‖ (Teachers, Eastern region). PIASCY Formative Evaluation 15 Censorship There was evidence of censorship with regard to PIASCY information on the part of some teachers who taught at the lower primary level. Some teachers spoke of revising the content of their PIASCY messages for lower primary pupils according to their personal beliefs, noting for example that ―the children are so young.‖ The data from many schools demonstrate that a good number of teachers at the lower primary level tend to emphasise topics such as personal hygiene, the importance of living in a clean environment, and of hand-washing after using the toilet, rather than life skills education, such as saying ―no‖ to sex. Keeping texts “new” or preserved The data also suggest that there is a strong perception among some school personnel that the PIASCY texts need to be preserved and kept looking ―new,‖ even if this means that pupils are prevented from reading them: We also have big problems with utilisation of books since the teachers want to keep the books new. To overcome this we have developed a policy of “books in the hands of children.” However, to some extent, the distribution of the materials did not go as we had expected. There are materials which we sent to all primary schools under PIASCY, but they are not being used. Instead, they are kept in cabinets by the head teachers… [who] claim that the children cannot borrow the books because they don‟t have bags, so if it rains, the books will be spoilt. (Key Informant, MOES) The textbooks are few. We cannot lend them out because we may lend a pupil a book and then they leave the school. (Teachers, Central region) Reading is the least effective [PIASCY-related activity] because teachers do not let us read the books. (Boys, Upper Primary, Eastern region) IV.A.2 Increased skills and knowledge of chief actors Overview of the perceived impact of PIASCY In all study regions, most respondents perceived PIASCY as having a positive influence. Specifically, many credited the programme with changing the perception of HIV/AIDS among PIASCY beneficiaries. Respondents described the disease as a normalised feature of daily living that is integrated into other aspects of regular life (such as child protection and walking home from school, relating with relatives, having confidence in oneself, and sanitation). In essence, respondents credit the programme with constructing HIV/AIDS as something to live with—related holistically with other aspects of life: It‟s about more than just giving people information about HIV/AIDS. PIASCY is unique because it is about saving lives. The information and skills we get are about saving life. (Teachers, Central region) We now have children who are assertive. They can say no to sexual advances, they can report when faced with difficult situations, and they can get help or guidance on how to deal with such situations. (CCT, Eastern Region) 16 PIASCY has encouraged parents to bring even the sick children to school. In the past, the children who were positive were left for dead—no one bothered to take them to school. (Parents, Eastern Region) In response to questions about what they had learned through PIASCY activities, pupils were fairly consistent and accurate in naming key PIASCY messages (e.g., abstinence, refusing gifts from strangers, taking care of the infected), which suggests the successful retention of knowledge gained from PIASCY activities. The following quotation represents a common sentiment expressed by pupils across the FGDs: ―[Before PIASCY started] I did not know that HIV spreads through sexual intercourse‖ (Boys, Upper Primary, Northern Region). Other quotes further suggest that knowledge of HIV is increasing as a result of the programme: Before, the HIV negative students feared to interact with positive ones because they thought it could be spread through body contact. (Boys, Upper Primary, Eastern Region) We used to refuse their [HIV positive pupils‟] food if they gave us some because we thought AIDS could pass through food but these days we don‟t discriminate against them. (Boys, Upper Primary, Western Region) Teachers also indicated a perceived increase in knowledge: I treat all children equally, whether you are positive or negative, unlike before where we used to see no need of HIV-positive students even studying because they will die soon. (Teachers, Eastern Region) PIASCY enriched knowledge in addition to life skills, like people being assertive. People comfortably go for HIV tests hence people are free unlike in the past where it was hard… We have a positive attitude towards AIDS. We no longer assume that whoever has AIDS gets it through sex. (Teachers, Central Region) This increase in knowledge seems to have also positively impacted the behaviour of programme participants. For example, a commonly expressed perception among school personnel was that HIV-positive children are increasingly being enrolled into school by their parents. HIV-positive pupils were also noted to be more open about their status, compared to the period before the PIASCY programme began. Other behaviour changes such as condom use among teachers and a reduction in cases of pupil molestation by teachers were also highlighted: It has been positive. As you all know, whatever you preach, you must practise. Thus, with that, our behaviour has changed, too. For example, we, too, use a condom. (Teachers, Eastern Region) Teachers have reduced the behaviour of “sending pupils in the houses.” [laughter] (Teachers, Western Region) The statement ―sending pupils in the houses‖ refers to teachers engaging in sexual relations with their pupils in their (teachers‘) homes. This is certainly an illegal and negative behaviour PIASCY Formative Evaluation 17 of teachers that should be condemned. While ―reduction of the behaviour‖ is a step in the right direction, the aim should be zero tolerance and complete eradication. Indeed, as shown below, PIASCY has empowered pupils to seek help when teachers make such advances. Guidance and counselling According to a key informant from MOES, ―Guidance and counselling is an integral part of HIV. There has to be counselling for vulnerable children.‖ Guidance and counselling is expected to play a key role in UNITY‘s future PIASCY activities. UNITY is also expected to introduce PIASCY to the post-primary level of education by producing PIASCY manuals and guidance and counselling materials, and disseminating them nationally but with a focus on the Northern Region. In addition to the distribution of guidance and counselling tool kits, more than 8,000 primary school teachers (two teachers per school, representing about 4,000 schools) were trained under PIASCY to offer psycho-social support to school children living with HIV/AIDS (UPHOLD 2008). At the time of fieldwork (February 2009), not all schools were covered, and the recent increase of registered primary schools to 17,008 complicates the issue. In addition, the desk review indicated that only 37 percent of teachers trained in guidance and counselling were women and yet, women teachers are preferred by female pupils. The exact duration of the guidance and counselling training is also not clear; the session is three weeks long, but includes several different subjects, guidance and counselling being only one of them. Findings suggest this training is insufficient: Teachers need to be trained on how to counsel students who are infected. For example, when conducting career guidance and counselling, students who are infected with HIV/AIDS may have different needs. (Key Informant, UNITY) There is need for more training or refresher courses. We need training on how to handle the infected and affected pupils in our schools. (Teachers, Central Region) Guidance and counselling is needed not only for the pupils, but for teachers as well. It was noted in FGDs with school personnel that some teachers were equally susceptible to emotional distress related to HIV/AIDS through caring for others, losing spouses and other loved ones (including their pupils), as well as through being infected. A previous evaluation of PIASCY activities conducted by the MOES (2008) indicated that although counselling of pupils existed in schools, it was mostly carried out in the form of group counselling activities. The current evaluation corroborated this finding. Group counselling is sex-segregated and focuses on issues such as body changes, menstruation, discipline, respect for parents, and staying away from ―sugar mummies/daddies.‖ Although sex-segregated counselling may encourage students to open up to discuss certain issues (e.g., menstruation) compared to when they are mixed, school group counselling sessions appeared to be more instructional in nature. This raises the issue of whether such sessions should instead be student-driven (i.e., focusing on issues collectively raised by pupils as representing their own counselling needs). There may also be a need for more individual counselling to address needs that may be difficult to express in a group. A disturbing observation emerged from one school in which pupils reported that one group counselling session was used to 18 examine female pupils for pregnancy. This was not only inappropriate, but it violated the pupils‘ right to confidentiality. Given the acknowledged importance of guidance and counselling, there is a need to help school administrators, teachers, and even pupils to include individual-level guidance and counselling, and to ensure that group counselling goes beyond regular pedagogical-type instruction to become a child-centred, interactive forum based on needs identified by the pupils. Pupil participation in PIASCY activities The PIASCY programme uses a variety of child-centric activities/strategies, including Music, Dance, and Drama (MDD), Talking Environments/Compounds, suggestion boxes, and the Safety Friends Network/Family System. This section briefly examines the extent to which these activities/strategies are advancing the objectives of the programme, particularly, that of increasing the skills and knowledge of pupils. Overall, the evaluation indicated that participation in these activities gave pupils ownership of the programme while simultaneously enhancing their knowledge of HIV, their coping skills (for those infected/affected), and their agency in proactively addressing situations that could expose them to contracting HIV. Music, Dance, and Drama According to an UPHOLD report (2008), ―School clubs are avenues that foster peer-to-peer learning, because the majority of adolescents feel more comfortable discussing sexual issues with peers rather than with teachers.‖ In the MDD clubs, children compose their own drama scripts and songs with the help of their teachers to address factors that put children at risk of HIV. MDD thus provides an opportunity for children to discuss their issues in their own words. Children that participated in MDD perform before their entire school and parents. Some MDD club members also travel to satellite schools to perform. According to most respondents, this activity is not only popular among pupils, but is also perceived as advancing PIASCY‘s objectives. The songs and dramas were typically composed in the local languages and were thus easily accessible to most audiences. Key informants and pupils emphasised the impact of MDD via the following narratives: Drama clubs are more popular. At assemblies, the AIDS messages can be imparted through a poem, play, or skit. Imparting of information is made entertaining to make it more interesting. (Key Informant, UNITY) I think music, dance, and drama have made the strongest impact. When we participate in a play, we act as if it‟s real and the message people get affects them permanently. (Boys, Upper Primary, Eastern Region) Up to now, the MDD are still in demand. I think they should be replicated, promoted, and supported. (Key Informant, MOES) MDDs are [good]. Students come up with issues that affect them. The children are happy and sometimes cry when the events are being dramatised. (Key Informant, MOES) PIASCY Formative Evaluation 19 When acting, pupils get emotional. If a child acts a part where s/he loses a parent, the pupils get emotionally caught up. (Teachers, Central Region) Talking compounds Talking compounds were initiated in Phase 1 of the PIASCY rollout process. Schools composed messages about HIV/AIDS and posted them in their compounds and surrounding environments to communicate to both pupils and communities. These messages are painted on highly visible target areas such as large boulders, water tanks, and school toilets (see cover page of this report for an example). Messages range from encouraging pupils to abstain from sex in order to stay safe to discouraging pupils and community members from stigmatising those infected or affected by HIV. While pupils and community members regarded the talking compounds as a useful tool, many schools noted the need to periodically alter the messages within the talking compounds, as there was the risk of no longer ―seeing‖ the messages when they had been displayed for too long. Suggestion boxes Suggestion boxes were introduced as part of the PIASCY programme in order to provide an opportunity for pupils to ask questions anonymously about issues that they might otherwise not broach. The suggestion boxes were noted by respondents as having enriched the school assemblies. Teachers would typically pick out an anonymous question from the suggestion box and answer it for the whole school. Pupils asked a range of questions via suggestion boxes, demonstrating the need for information not only on HIV but also on broader sexual and reproductive health issues. Examples include: ―Is it true that if I don‘t [have] sex, it means I am barren?‖ ―If you don‘t [have] sex, is it true your organ does not grow?‖ ―Is it normal to feel pain during menstruation?‖ ―Will my breasts not grow if I don‘t [have] sex?‖ Safety Friends Networks The Safety Friends Network is a system that enables children to protect themselves, defend their rights, and minimise or eliminate predisposing factors that put them at risk of acquiring HIV. Pupils choose three or four friends to accompany them to and from school, and to other places, such as teachers‘ houses. Each friend is supposed to watch out for the others and to remain aware of what is happening to them. Respondents were generally of the opinion that Safety Friends Networks were particularly empowering for pupils, as being in a group granted them a level of security they would be less likely to have if unaccompanied. A parent from the Eastern Region expressed the usefulness of Safety Friends Networks as follows: ―These days, children know that they can be attacked or raped by strangers if they move alone or in isolation, so these days they move in groups‖. An MOES key informant gave a compelling example of how the Safety Friends Network had functioned successfully in a school in Jinja District: A teacher took a female student to use her at his house, but the children had known the teacher was using that girl. The girls moved from their dormitory, went to get the boys from their dormitory, and proceeded to the teacher‟s house. They demanded that the teacher release the girl. The children became wild, so the teacher was afraid to come out. The teacher on duty called the headmaster who was away at that time. The head teacher came back with the police. As we speak now, the teacher is in prison. Now, everyone knows that you cannot touch children because if you do they will take action and you can end up in prison. 20 Of the four child-centric activities described above, different schools expressed a preference for different activities. Teachers, however, tended to point out that conveying HIV/AIDS information required a multi-pronged approach; therefore, ranking one method as better than the other was not a useful exercise. However, it was apparent in FGDs with all respondent categories that interactive methods such as MDD were highly valued and seen as particularly effective. IV.A.3. Promotion of stigma-free school environments PIASCY played a major role in reducing stigma against people living with HIV/AIDS in schools. The vast majority of pupils clearly articulated that they had been taught not to discriminate against HIV-positive individuals because the disease could not be contracted by, for example, playing with them. Pupils gave several examples of playing with young people living with the virus, and of assisting them in various ways: When they get sick at school, we escort them back home and we remind them to swallow their drugs. (Boys, Lower Primary, Eastern Region) R1: Some positive pupils can be sick and their parents can‟t afford treatment so teachers take them to hospital. R4: I have a friend who is positive. I remind her to take tablets whenever she forgets but she has not reported this term and her sister died of AIDS. R3: When one of the pupils who are positive is down and sick, we contribute vegetables and fruits as two pupils and a senior woman go to visit her/him. (Girls, Upper Primary, Western Region) Teachers voiced similar comments on pupils‘ experiences with stigma. When asked to rate their school in terms of being stigma-free, teachers from schools that were actively engaging with PIASCY made the following observations: R5: I rate it high because no one segregates pupils depending on whether they have HIV or not. R4: Recently, during our march, pupils that have HIV comfortably participated in the march. R3: Teachers draw closer to provide help and support once they learn that one of their pupils has HIV. (Teachers, Central Region) R2: A child can come and say to you, “My father died last year and my mother died six months ago and I am feeling sick.” In this regard, I have to counsel the child and eventually take him or her for testing or treatment. R5: On admission, it is important to know the status of each child, whether he or she has both parents, a single parent, or [s/he is] an orphan. (Teachers, Northern Region) A recurring theme among the majority of schools was to never insult or be mean to pupils living with HIV/AIDS since it was not their fault that they were HIV-positive. This mentality was also used as a basis to guard against discrimination. PIASCY Formative Evaluation 21 Despite the apparent success in addressing stigma in many schools, a few schools—notably, the less engaged ones—still seemed lag behind: We had a child whose skin was full of rashes and she was in isolation. She was accused of [having] AIDS, but on testing several times… she was found negative. Today, she is friendly with most children. (Teachers, Central Region) The issue of stigma seemed to place a clear demarcation between schools that were minimally involved in PIASCY and schools that were more involved. For example, in the schools that were more clearly engaged with PIASCY, the notions of love and care for people living with HIV were often mentioned by pupils. On the other hand, pupils from less actively engaged schools often indicated that their schools were not ―good places‖ for children living with HIV/AIDS. For instance: R7: It‟s not a good school for people who are suffering from HIV/AIDS because nobody cares [agreement from others]. (Girls, Upper Primary, Central Region) While concerted efforts seem to have been made to combat stigma at the level of individual pupils, at the school-level, there appears to be room for improvement. According to an MOES key informant, ―Some headmasters are taking infected teachers off the payroll or transferring them haphazardly to schools where they may have problems accessing ARVs.‖ It was also noted that some head teachers discriminate against HIV-positive teachers by removing them from positions of authority such as the senior teacher position. The attitudes of teachers and school administrators toward stigma are particularly important, as they may play a role in shaping pupils‘ attitudes. IV.B. What are the strengths and limitations of the design, organisational structure, and rollout of PIASCY? IV.B.1. PIASCY design Strengths of PIASCY design The formative evaluation suggests that the greatest strength of PIASCY‘s design is that it is a comprehensive, holistic programme in which all actors are simultaneously imparters and recipients of PIASCY knowledge, and in which HIV/AIDS is innovatively mainstreamed into various aspects of daily living. The PIASCY design is based on the ―Whole Schools Approach‖—a strategy devised by the MOES to offer continuous engagement with HIV￾related issues across schools. One key informant from the MOES elaborated: For example, while conducting P.E. [physical education], a teacher will be interacting with a number of pupils and can use that opportunity to engage them on issues to do with sexual and reproductive health, sexuality, and HIV/AIDS. The teacher can also request children to write about HIV/AIDS and describe what they see as part of a composition lesson. In an art lesson, the teacher can ask children to sit and think about a family that has been left desolate as a result of HIV and put an image on paper. This will teach the child creativity and at the same time allow the children to think and talk about HIV/AIDS. As part of comprehension skills, children 22 can be given passages to read describing situations that border on HIV/AIDS and related issues and then are asked questions and can discuss these things in class. A second strength of the PIASCY design is its inclusion of a variety of actors who had a vested interest in the issue of sex education in schools. Following the development of the PIASCY readers, the MOES held meetings with a range of stakeholders to discuss the appropriateness of the content for pupils in primary school to ensure there would be no resistance at implementation stage. These stakeholders included UNICEF, Straight Talk Foundation, TASO, Uganda AIDS Commission, the Ministries of Health and Gender, the local government, and the National Council for Children. Given that most primary schools in Uganda have a religious affiliation, religious issues were also taken into account during the design of PIASCY. The Protestant, Catholic, and Seventh Day Adventist churches; the Church of Uganda; and the Uganda Moslem Education Association (UMEA) were also represented at the stakeholder consultation meetings. Notably, some religious stakeholders objected to content such as instruction on the correct use of condoms, expressing concerns about encouraging pupils to engage in sexual activity. A compromise was eventually reached between the MOES and these stakeholders, and according to an MOES key informant, ―We again added in a whole chapter on morals, ethics, and virginity so that our children can abstain. We follow the ABC model. ‗A‘ is for Abstinence, and we state that all children should abstain. ‗B‘ and ‗C‘ are for teachers.‖ Without close consultation with a variety of stakeholders, and particularly, with religious stakeholders, the very existence of the PIASCY programme may have been threatened. Limitations of PIASCY design The formative evaluation also revealed several limitations of the PIASCY design. First, the fact that PIASCY was not an examinable subject discouraged some teachers from engaging with it. Secondly, teachers pointed out that the design of PIASCY did not include financial incentives for motivating human resources. These sentiments were widespread among all teachers interviewed regardless of region or whether they were urban or rural. It also appears that the Model School approach—in particular, the disbursement of school incentive grants to Model Schools to facilitate their leadership roles—was not well￾understood by some schools that did not receive these grants, fostering suspicion and perception that Centres of Excellence/Model Schools receive ―all the praise‖ and resources, as well as favours from the government. Even when the grant scheme ended, suspicion and resentment continued: Some schools refused to participate [in PIASCY] because we had adopted a strategy of giving a small incentive to the Model Schools. Those who did not receive the financial incentive felt marginalised and started saying that they would not do anything without also getting something. Now, we have resolved to treat all schools as at par. (Key Informant, UNITY) Since we are a Model School, when we invite other schools to come, they expect transport, food, and an allowance, which are not provided for, so they think we ate the money. (Teachers, Eastern Region) There needs to be transparency on the funding and materials that are sent to particular schools and all these need to be sent directly, otherwise, that‟s the PIASCY Formative Evaluation 23 weakness because not all funds and materials reach the final destination—the schools. (Teachers, Northern Region) The PIASCY administration should be lowered down to the level of schools rather than at the districts where the PIASCY material is given selectively. (Teachers, Central Region) Another limitation of the PIASCY design noted by several respondents (primarily teachers and parents/community members) was that parents and communities were not engaged as much as they could have been during the stakeholder consultation stages. Some teachers attributed the resistance of some parents toward the programme to this factor: R2: The design is upper-centred. The people at the top designed PIASCY without consultation with stakeholders. R3: They sent us books according to classes. We are supposed to read and teach children, but their parents were never catered for or sensitised; thus, there is a gap between parents and teachers. (Teachers, Central Region) The programme should have been rolled out to communities before being implemented by ensuring that parents/communities are fully involved and trained. By doing all this, much more could have been realised than what has been achieved so far. (Parents, Central Region) We do not know exactly what happens in these PIASCY activities apart from the Talking Compound where we, too, can have an opportunity to read what is displayed. (Parents, Eastern Region) It [PIASCY] has not included parents who are [key] in bringing up these pupils because they spend almost half of their entire lives with them. But the programme has not looked into that. (Teachers, Northern Region) Indeed, out of all the respondent categories, parents/community members were the least verbose about PIASCY. Most did not seem to fully understand their role as actors within the programme. Unlike other target groups that have clear roles outlined within the programme (i.e., teachers and pupils), parents and community members play an indirect role as recipients of PIASCY knowledge (rather than as both recipients and imparters). Parents, for instance, receive PIASCY-related information if they attend school assemblies and through interacting with the Talking Environments/Compounds, but there was no deliberate effort to ensure that parents understood the programme and saw themselves as an integral part of it. In the rare instances where schools collaborated with communities for the purposes of PIASCY (e.g., by having key community members, such as the police or doctors, give talks at the school; or by having pupils visit the sick in hospitals) this collaboration was a result of efforts on the part of the school, rather than of the community.2 2 Despite this general perception, there is some evidence of engagement of parents and community members with PIASCY. For example, one teacher noted that ―There has been appreciation by parents. Today, parents thank us for teaching their children things like not all relatives are good.‖ In addition, a number of schools indicated that community members looked to their schools for HIV counselling services, and would occasionally visit the schools for this purpose. 24 Teachers also raised specific limitations of the programme. Some expressed a desire to provide input into the kinds of activities that should be carried out under PIASCY. Several teachers mentioned that there were no opportunities for them to develop their own PIASCY budgets and work plans to address the particular needs of their school. Rather, their perception was that they were expected to carry out pre-specified activities with little attention to the specific context of their individual schools. A final limitation of the PIASCY design had to do with lack of information on antiretroviral therapy (ART) support and care for children with HIV. The vast majority of key informants, as well as many teachers and pupils, expected PIASCY to incorporate linkages to services or actual provision of anti-retrovirals (ARVs) for HIV-positive pupils and teachers: We can no longer have messages on prevention alone when we are faced with huge numbers of children who are living with HIV/AIDS. These children should have access to ARVs and should be able to bring ARVs to school. We need to promote VCT [HIV voluntary counselling and testing] for teachers. (Key Informant, MOES) I think PIASCY should also include other things apart from just giving information. PIASCY has been very successful at disseminating information; now, there is need to equip the informed people with the resources and skills to practise what they know. For example, we may need to provide care and support for affected students and teachers. We need a clear plan on what to do with orphans and vulnerable children, and on how to link people with services. In any given community or school, we can find people who are affected or infected—we need to provide them with support. (Key Informant, UNITY) IV.B.2. PIASCY organisational structure and rollout The organisational structure and eventual rollout of PIASCY are closely interconnected. The two will therefore be discussed in tandem. PIASCY‘s organisational structure, which guided the rollout of the programme, is illustrated in Figure 1: PIASCY Formative Evaluation 25 Figure 1 PIASCY organisational structure The Teacher Development and Management System (TDMS) is a structure that caters to government teacher development and training within the MOES. Within this system, (government-affiliated) Core Primary Teacher Colleges (CPTCs) handle teacher training tasks, and the Coordinating Centre Tutors (CCTs) supervise and monitor the delivery of school curricula. The organisational structure that UPHOLD, in collaboration with the MOES, adopted for the PIASCY programme was deliberately aligned with these existing structures. Therefore, the CPTCs were provided with a PIASCY module that was used to train PIASCY teachers. The CCTs were charged with supervising and monitoring the delivery of the PIASCY curriculum in schools. Between 2006 and 2008, UPHOLD encouraged the transformation of government-aided primary schools into Model Schools/Centres of Excellence. The Model Schools were meant to be foci of HIV prevention interventions that promoted practices and environments to enable pupils abstain from sex. They were selected based on a number of criteria, such as the capacity to provide leadership to other schools and presence of PIASCY-trained teachers (UPHOLD 2008). A total of 1,078 Model Schools/Centres of Excellence were created from among the 15,680 primary schools that existed at the time. Through a knowledge cascade approach, the Model Schools were expected to influence other schools within their area of influence (both public and private) by training other teachers on the implementation of PIASCY. 26 Strengths of PIASCY’s organisational structure and rollout A major strength of PIASCY‘s organisational structure and rollout is its alignment with the pre-existing structures of the MOES. For example, the involvement of the various bodies outlined in the Figure 1 was critical for the inclusion of HIV/AIDS in the PIASCY curriculum. It also encouraged the Uganda National Examination Board (UNEB) to include HIV questions in national examinations for school children, which helped to mainstream HIV/AIDS within the classroom. Reliance on the TDMS also benefited the rollout process. It served to decentralise PIASCY activities, and, as they were already situated within the TDMS, the Model Schools and the CPTCs played roles that were familiar to them. UPHOLD was initially a regional programme working in about 20 districts to cater to the national PIASCY programme. The UPHOLD programme was then obligated under the PIASCY programme to extend its reach to other districts in which it had no prior relationships. Using the MOES structures for the rollout enabled UPHOLD to forge strong partnerships in the new districts while simultaneously relying on its regional offices where it had been operating prior to PIASCY‘s inception. The UPHOLD end of programme report notes that: ―This national rollout was accomplished in a record time of five months… It was probably among the most ambitious rollout efforts of its kind in Africa‖ (UPHOLD 2008:78). A final strength of the PIASCY organisational structure and rollout is the adaptability and creativity of the programme implementers, such as the MOES, BEPS, and UPHOLD, particularly in the face of funding limitations. For instance, as the available funding did not permit the provision of PIASCY training to all teachers, the project implementers devised a strategy through which three teachers per school would undergo training, who in turn would train other teachers within their schools and satellite schools. The creation of the Model Schools/Centres of Excellence was also partly a response to funding limitations. Under other circumstances, such funding limitations could have undermined programme functioning. Limitations of PIASCY organisational structure and rollout A recurrent weakness of the PIASCY structure is that the number of Centres of Excellence/Model Schools was insufficient to serve the number of satellite schools that needed their support, leading Centre of Excellence staff to feel overwhelmed. In an attempt to alleviate this problem, in 2007 each of the 1,078 existing centres was encouraged by the MOES to create one more Centre of Excellence, resulting in 2,156 Centres of Excellence by the end of the year. However, the total number of primary schools (both private and public) increased exponentially during that time, undermining the potential impact of this increase in Centres of Excellence. According to a key informant from the MOES, ―The schools also increased from to 15,000 to 17,008. The Centres of Excellence are still too few to cover all schools effectively.‖ Another limitation of the PIASCY organisational structure and rollout is a general lack of human resources: The structure of funding—one CPTC for the whole—is not workable, and a CCT can [be expected to] coordinate four sub-counties. This is a big workload… I would say the programme design has not been effective because for Jinja, Iganga, Kamuli, or maybe even for the rest of Busoga Region, we have one CPTC and in each district or county, one CCT. There are so many schools that a CCT has to cater for; even transparency about funds does not exist. We don‟t know how much we are given PIASCY Formative Evaluation 27 because these CPTCs who receive the money are far. And one model school is given to manage and supervise other schools. In the case of [a particular district in the Eastern Region], there are 21 other schools. So how can we supervise 21 other schools? (Headmaster, Eastern Region) The worst part is that only few schools were chosen as Model Schools, leaving many schools outside [neglected], hence the multiplier-effect is still minimal. (CCT, Northern Region) As mentioned previously, the Centres of Excellence were intended to produce a critical mass of trained teachers who would then train other teachers in the satellite schools. This strategy was not entirely foolproof, however. At rollout, teachers trained under PIASCY were sometimes transferred to other schools, leaving a gap in institutional knowledge about PIASCY. In one case when a CCT who was committed to PIASCY was transferred, teachers pointed out that they no longer received the support they needed to implement the programme. Pupils and teachers described this issue as follows: They used to tell us about HIV/AIDS [during assemblies], but these days, they don‟t because the [PIASCY] teacher went away to another school. (Girls, Lower Primary, Northern Region) The programme at the beginning used to have other players like the CCT and people from the district who used to come and give support in implementing this programme, which was very good but nowadays, it is no longer done… The CCT who used to give us support was transferred, and for those people at the district, I don‟t know what has happened. (Teachers, Central Region) The teachers who first initiated PIASCY here were all transferred, so I don‟t personally know the work plan. (Teachers, Northern Region) Another limitation of PIASCY‘s structure and rollout is its failure to fully integrate private schools. While private schools were indirectly included in PIASCY‘s centrifugal approach through the satellite schools, their personnel were not directly trained, nor were these schools selected as Centres of Excellence. Private schools were described by respondents as being integrated at ―the end of the chain‖ (as satellite schools), rather than as potential ―centres of first contact.‖ Moreover, Uganda‘s two Private Primary Teacher Colleges (PPTCs) were omitted from the PIASCY organisational structure. These institutions have not as yet been integrated into the TDMS structure and do not offer training on PIASCY. In the words of an MOES key informant, ―We say that after training, public school teachers should go out there and train their private counterparts. But the impact is too little. If we train them and they go there and the environment is not conducive, they can‘t operate. Private schools also don‘t get PIASCY books.‖ In other words, the exact process through which public schools are expected to engage with private schools (and vice-versa) remains unclear. The lack of proper integration of the district authorities within the PIASCY rollout structure was another limitation highlighted in the respondents‘ narratives. Under the (amended) 1997 Local Government Act, the management of primary schools is devolved to districts, which was the source of some struggles between Ministry-level versus district-level roles in the PIASCY rollout. As an MOES key informant explained: 28 This programme has been pushed from the Ministry-level; however, districts are semi-autonomous government entities. Therefore, we can‟t simply direct them on what to do. We had a few problems with the districts. In response to the idea that some district authorities were not well-integrated into PIASCY‘s organisational structure, another informant explains: [In regard to] education, the country is divided into zones. Therefore, the Education Ministry operates according to zones. These zones are divided according to Core Primary Teacher Education Colleges. We have 23 CPTCs meaning the country is divided into 23 zones. However, we have 80 administrative districts in Uganda. I am not saying that the districts were not well-integrated, but it is possible that some districts may have felt that [way] because they are operating at the district level whilst we were operating through the education zones and the TDMS structures were only 23 compared to 80 districts. (Key informant, UPHOLD) The lack of integration for some districts caused further problems given that the districts were expected to monitor and evaluate PIASCY activities, but, according to an MOES key informant, some did not do so. Teachers in several schools underscored the comment that: PIASCY did not involve the District Education Officers. So it looks like we are running a parallel programme from our bosses. This leads to conflict, so this needs to be harmonised. (Teachers, Eastern Region) A related weakness, as reported by school personnel, was that the monitoring and follow-up of the PIASCY programme were inadequate. Few schools were satisfied with the monitoring of their activities, excluding schools in the Northern Region. Some school personnel mentioned that they had received one monitoring visit over the life of the programme, while others reported not being monitored at all: ―There has been no monitoring of the programme since it started‖ (Coordinating Tutor, Western Region). As with any programme, the PIASCY design, organisational structure, and rollout seem to have experienced a combination of both strengths and constraints. It is noteworthy that the limitations discussed around these areas seem to have stemmed primarily from insufficient resources—both human and financial. Additionally, the role of parents and community￾members in the PIASCY programme appear to have been less clearly defined than that of other target groups. IV.C. What are the lessons learned and best practices for continued rollout of the programme? The study results suggest several lessons learned and best practices for the continued rollout of the PIASCY programme. These have been developed with post-primary institutions in mind, given the intention for UNITY to introduce PIASCY within these contexts. PIASCY Formative Evaluation 29 IV.C.1. Lessons learned 1. Language barriers may prevent effective use of PIASCY material. 2. Lack of an operational definition of ―guidance and counselling‖ has precluded some schools and teachers from effectively implementing this component of the PIASCY programme. 3. The involvement of an extensive range of stakeholders in the design of PIASCY from the planning phase and during the development of readers has been critical to the existence of the programme. This is a practice that should be carried forward, with particular attention to stakeholders from religious bodies, who serve as powerful gatekeepers for parochial schools. 4. Direct methods of targeting programme beneficiaries/actors (such as the methods used to target school personnel and pupils) make a stronger impact than indirect methods (the methods used for targeting parents and community members). 5. Lack of monitoring of the knowledge cascade approach as well as lack of feedback to teachers and schools on their performance has led some teachers and schools to be de￾motivated and not participate in teaching other teachers and schools. Training three teachers in a school may also be insufficient, particularly when teachers are transferred to other schools. IV.C.2. Best practices 1. The establishment of supportive structures within schools ensures the continued implementation of PIASCY (e.g., PIASCY teachers‘ committees and PIASCY school clubs) as these encourage ownership of the programme by a wide variety of staff members and pupils. Without these sorts of structures, PIASCY programmes run the risk of being championed by a sole teacher, which could undermine sustainability. 2. Key PIASCY child-centric school activities (e.g., assemblies, suggestion boxes, Safety Friends Networks), are functioning successfully and enable pupils to gain valuable HIV/AIDS information as well as have their issues addressed in a non￾threatening atmosphere. School personnel and parents (who attend assemblies and other school functions) also gain knowledge from these activities, and some are inspired to change their own risky behaviour. 3. The TDMS structure helps school-based interventions to ensure that training occurs seamlessly. It is noteworthy, however, that this structure does not exist within the secondary school system (which forms an integral part of post-primary institutions). 4. The development of school-community partnerships/collaborations (e.g., in the form of pupils visiting the sick in hospitals, and medical personnel or police officers giving talks to the pupils in school) can lead to the acceptance of PIASCY within communities and can also play a role in giving schools more visibility in the community, or in improving a school‘s image in the eyes of its community. 30 IV.D. What are the unintended consequences of the development and implementation of PIASCY? IV.D.1.Positive unintended consequences The implementation of the PIASCY programme yielded a number of positive, albeit unintended, consequences. The programme reached populations for whom PIASCY was not originally designed. Although pupils in P3 to P7 were one of PIASCY‘s main target groups, some teachers observed that secondary school students, who had not formally gone through the PIASCY programme, could nonetheless gain PIASCY-related knowledge by reading the messages posted on school compounds. PIASCY has also enabled pupils to receive information beyond HIV/AIDS. Other related issues, such as child molestation, sexuality, and general safety were also discussed in the context of the programme. An UPHOLD key informant mentioned that ―because of sensitisation in one rural area… communities volunteered to clear up the paths and remove all hanging trees so that their children could be safe on the way to school.‖ The PIASCY programme also prompted its beneficiaries to become involved in activities outside of the school setting. In one school, teachers and pupils began to do outreach work, visiting the sick in hospitals. These efforts were noted by teachers as having improved the image of the school within the community. Other schools spoke of how they had developed good relationships with surrounding hospitals, which often sent their staff to talk to the pupils about HIV/AIDS. The PIASCY programme was also noted as increasing the confidence of teachers and pupils. For example: PIASCY also made teachers more confident. In the past, it was only the headmaster who addressed assemblies, but with PIASCY, teachers started alternating. The teacher on duty would be responsible for heading the PIASCY assembly. Children‟s confidence also increased as they were given opportunities to give testimonies and share with others in the assembly. In one district in Eastern, although it was not during the assembly, a pupil openly told the headmaster that her guardian was sexually abusing her and that the guardian had HIV. The guardian was reported and arrested and as we speak he is in jail. However, the girl was tested and she had already been infected. However, this also sent a message to the community that if you do something to the children, they will report you to the school teachers. (Key Informant, MOES) Some teachers noted that PIASCY training had improved their communication with their own children about sexuality issues and HIV/AIDS. The confidence-level of pupils was also said to have been boosted by their regular participation in plays and skits at school and through their involvement in addressing other pupils during assembly especially by giving testimonies. In one school, teachers suggested that enrolment in their school could have increased because of their outstanding performance in PIASCY: ―PIASCY has acted as an advertisement for the school as we have gone to PIASCY activities. Other children get to know of our school and PIASCY Formative Evaluation 31 subsequently come for vacancies‖ (Teachers, Eastern Region). The teachers that participated in this particular discussion generally viewed this as a positive development. Suggestion boxes have also led to the discussion and resolution of issues that may have nothing to do with PIASCY at schools. In one school, the suggestion box led to the resolution of a particularly interesting issue: according to the headmaster, some pupils in lower primary dropped a note in the suggestion box complaining that boys in Primary 7 were not bathing. Further investigations proved this allegation to be true and it turned out that upper primary boys felt self-conscious bathing in the presence of the younger boys. The school authorities ended up putting a demarcation between the upper and lower primary bathrooms. Several schools in the study also noted a reduction (and, for some, the complete cessation) of love letters from boys to girls. Teachers were concerned that love letters between boys and girls could encourage pupils to experiment with sex and other risky behaviours. In one school, love letters ceased because girls would simply place received letters into the school‘s suggestion box. IV.D.2. Negative unintended consequences On the other hand, respondents noted a number of unintended, negative consequences as a result of programme implementation. As mentioned previously, teachers commonly cited the perception of a higher workload, given their new PIASCY-related responsibilities. This issue emerged in all FGDs with teachers. Some complained that they were not being remunerated for their PIASCY efforts and felt that these activities were above and beyond their call of duty. In one FGD, for instance, teachers remarked: ―Under PIASCY, the role of teacher has extended to that of counsellor and doctor, yet we had originally thought we were simply going to teach PIASCY.‖ Other examples include the following: R1: In my view, it consumes a lot of time. Training the girls takes time. Assembly time has expanded from 30 minutes to two hours. R4: It takes a lot of teachers‟ time. It is tiresome. (Teachers, Central Region) The programme has added more workload on the teachers, which was not the case in the past. (Teachers, Western Region) Teachers who participate should be motivated to boost their morale in supporting the PIASCY programme. Since in most cases it‟s an outside class activity, this would mean [an extra] workload for the teachers involved. (Teachers, Northern Region) Various other unintended, negative consequences were also mentioned by respondents. For example, a FGD with teachers revealed that ―some games which children used to enjoy have been banned [under PIASCY in their schools]; for example, ‗hide and seek,‘ because some students use this game to engage in sexual activities.‖ Another set of teachers remarked that ―at some schools, PIASCY has now been limited to singing and drama and students are not given more information.‖ Finally, a statement by one teacher during a FGD suggests that some pupils may be using the gravity of PIASCY messages to their own advantage: ―since children know that they have to care for their sick [HIV] positive relatives, they have learnt to 32 be absent from school, and when they come back, they give an excuse of having gone to Kakiso to pick medicine or to escort a sick parent.‖ Additionally, several school staff noted that PIASCY had been instrumental in helping them teach (both upper and lower primary) pupils about condom use, or that this topic fell within the PIASCY curriculum: Before training, it was difficult for us to demonstrate how to use a condom, but now, we do demonstrate. (Teachers, Western Region) As mentioned previously, abstinence is the intended focus of PIASCY for pupils, while being faithful and condom use are concepts reserved for teachers. The notion that condom use was to be taught as part of PIASCY was most prevalent in the Eastern and Western Regions, but was non-existent in the Northern Region. It is also possible that for some teachers, the process of integrating PIASCY messages with their regular science classes began to blur the boundary between PIASCY and other subjects: For upper primary, we have a topic on sexually transmitted diseases, so we take advantage of this topic to integrate these PIASCY messages into the standard topic… The community attitude is negative to the practical method that we use for teaching the children. For example, teaching the use of condoms to the children is resisted by parents. (Teachers, Eastern Region) There was also a perception that the motivations behind the establishment of the PIASCY programme were political in nature. In some cases this perception fostered a high engagement level of teachers; one teacher from the Western Region said, ―It is the initiative of the President [or else] people would have ‗dodged‘ it like they do other programmes.‖ In other cases, however, the association of the programme with political figures could serve as a deterrent to participation: The President‟s picture is on all the PIASCY material. If the President goes, the whole project may be regarded as political, and I think this is a problem for us. There may be a backlash against the project… We need to remove the picture of the President from the materials in the new revised versions so that people can see the programme instead of seeing the whole programme as merely political. We do not want a backlash. (Key Informant, MOES) The President attributes the successes of PIASCY to his political party, which annoys other would-be PIASCY trainers. The programme should be made neutral to all people in Uganda. (Teachers, Central Region) In the preliminary data interpretation meeting for this formative evaluation, held at MOES, participants noted that as a result of PIASCY training some teachers have left the teaching profession and joined NGOs working on HIV and AIDS. Such departures have depleted the pool of PIASCY trained teachers to train other teachers in the knowledge cascade model. PIASCY Formative Evaluation 33 In sum, the PIASCY programme yielded several unintended but positive outcomes, such as the increase in confidence for both teachers and pupils, as well as some negative outcomes, such as a perceived burden by teachers. IV.E. What are the sustainability issues that will need to be addressed in handing over the programme? Sustainability of PIASCY could be affected by funding. The integration of the PIASCY programme with all the departments in the Ministry of Education, as well as with other line Ministries (e.g., Ministry of Gender) within the rollout structure has played a key role in increasing the sustainability potential of the project. Nonetheless, as a key informant acknowledged: If funding ceases, projects tend to die off. However, the Ministry should identify and see which practices they should keep and provide supervision support to schools to ensure that they still practice PIASCY activities, even without funding. (Key Informant, UPHOLD) Several interviewees observed that although there was high-level political support for PIASCY, the government traditionally had not invested large amounts of funding in HIV/AIDS projects. This observation raised concerns over financial sustainability: The only big problem I see with PIASCY is its dependence on donor funding. PIASCY is PEPFAR-funded, and I do not see the Ugandan government being able to sustain it if USAID pulled out. We need to be integrated into the system so that the programme becomes sustainable. The government is supposed to put in money; however, with the competing priorities, it may not be able to do so. For example, now we have free primary and secondary education, so a lot of money in the national budget has to cover these things. The HIV budget is also very small; therefore, projects like PIASCY have had to rely strongly on funders. (Key Informant, UNITY) The sole informant that expressed full confidence in the government‘s ability to independently sustain the PIASCY programme nonetheless suggested that with government funding alone, the delivery of the programme could still be susceptible to delays: With or without [donor funding], PIASCY has to go on… However, USAID funding helped us achieve our goals faster than if they had not funded us. Without USAID funding, progress would have been much slower. (Key Informant, MOES) Another MOES informant pointed to the risk of complacency on the part of actors, such as MOES, USAID, teachers, and pupils as a factor that needs to be addressed to ensure PIASCY sustainability. In his words: ―People say there is a lot of information fatigue and yet there is always a new generation of students, and we need to constantly [bombard] them with information‖. This issue could be mitigated by periodic PIASCY refresher courses for teachers, which take into account current realities and issues. It is noteworthy however that such an endeavour has its own financial implications. 34 It is evident that the sustainability of PIASCY activities is threatened by the insufficient number of trained PIASCY teachers to take over in cases when the driver of the programme leaves a school (e.g., transfer, career change, or death). Several respondents alluded to this problem: ―…You can get a very good teacher and headmaster, and they are transferred. When they are transferred, that school may stop performing well and the programme is disorganised.‖ Finally, the shortage of PIASCY texts—an issue raised repeatedly by respondents—merits attention. An inadequate number of books per school, coupled with some school personnel placing more value on preserving the texts than on allowing pupils to read them, could pose limitations to the sustainability of the programme. While the majority of key informants were unequivocal about the role of donor funding in sustaining the PIASCY programme, as one key informant suggested, there are actions that can be taken to foster sustainability, irrespective of funding. In particular, best practices that incur minimal costs (such as the utilisation of pre-existing structures) can be identified, strengthened, and maintained to guarantee the continuity of PIASCY. PIASCY Formative Evaluation 35 V. Discussion School-based HIV/AIDS prevention programmes have been heralded as an efficient and effective way to combat the epidemic, given the large proportion of young people that can be reached (UNESCO 2005; UNAIDS 2006b). PIASCY is an ambitious programme that has sought to address the HIV/AIDS epidemic in a holistic manner, targeting young people, school personnel, parents, and the wider community. The programme was built through a partnership between the Government of Uganda, civil society, schools, and communities. This partnership is arguably responsible for many of the programme‘s achievements. However, the findings of this evaluation indicate that despite PIASCY‘s accomplishments, it has not been immune to the challenges that have historically been associated with the implementation of school-based HIV prevention programmes. A review of the literature (Griffiths 2005; UNAIDS 2006) highlights a number of issues that can affect the implementation of such programmes, namely: problems associated with teachers, schools, and curricular; financial constraints; and poor political leadership. The discussion that follows is informed by these issues and also focuses on additional concerns arising from the study‘s findings. V.A.1. Teachers The lack of teacher training is a major obstacle to the success of school-based HIV prevention programmes. To effectively teach young people about HIV and reproductive health, teachers must have a good understanding of the subject, possess strong pedagogical skills, and be cognizant of the developmental and cultural appropriateness of teaching materials. In many ways, PIASCY has overcome this obstacle—the vast majority of teachers interviewed expressed enhanced confidence in PIASCY-related teaching because of the training that they had received. This is in contrast to their level of confidence prior to the introduction of PIASCY. Many talked about feeling uncomfortable teaching HIV issues before they were trained because they felt ill-equipped to do so. PIASCY has also supported teachers in using a variety of methods not only to instruct their pupils, but also to provide them with an opportunity to share in creating the learning process (e.g., through composing HIV-prevention messages, poems, songs, stories, writing, and acting in plays), rather than relying on conventional didactic methods. Although a few teachers talked about still being selective with their teaching to lower primary school pupils—only teaching them about things they personally felt were appropriate for that age group—most were comfortable delivering the curriculum specially designed by the PIASCY programme for both lower and upper primary school pupils. High teacher attrition rates are another limitation of school-based HIV-prevention programmes. While the literature often focuses on the loss of teachers to HIV/AIDS (see, for example, World Bank 2002; Gallant and Maticka-Tyndale 2004), teacher attrition in the PIASCY programme is reported to have stemmed largely from the small number of teachers who were initially trained in each model school. These teachers were trained with the hope that a knowledge cascade approach would eventually result in a much larger group of PIASCY-trained teachers across the country. Where teachers did proceed to train other teachers at their schools, however, they were sometimes still regarded as the repositories of PIASCY knowledge and the drivers of the school-based programme. This meant that whenever they were transferred, the programme often came to a standstill. 36 V.A.2. Community engagement Another challenge of the programme has to do with engaging with local communities under PIASCY. Although communities are a key target of the programme, the formative evaluation found that parents and community members were the least engaged out of all programme actors. This seems to be linked to the fact that, unlike other actors such as teachers and pupils, parents and community members were targeted in more indirect ways. Teachers were trained and then passed on teachings to pupils, who were then expected to relay information to their parents. There is evidence that this occurred, but not in a systematic way. Parents also attended parent days, sports days, and assemblies at which PIASCY information was disseminated. Nonetheless, these activities were held infrequently and were not compulsory for parents. Many teachers wondered why parents were not directly trained in the manner that teachers were, feeling that this would have been beneficial. Indeed, the International Academy of Education strongly advises close collaboration between Ministry of Education‘s work and the target groups of children, local communities, and school administrators in the course of the ―development, planning and implementation, evaluation, and redesigning of the programmes‖ (IAE 2005). As a result of these factors, of all respondent categories, parents were the least articulate about what PIASCY involved, and expressed a lack of clarity over their role in the programme. There was also no systematic way for teachers and schools to engage their broader communities; therefore, while a few schools spoke of involving community members as resource people (e.g., to give talks to the pupils), the role of the community in PIASCY was not clear in most schools. Because of this lack of systematic engagement of parents and community members, teachers were in some instances blamed when communities or community members disapproved of the dissemination of HIV information to children. At least one teacher in the present study narrated experiences of physical assault from community members for his engagement with PIASCY. V.A.3. A conducive school environment Schools are entrusted with ensuring that young people have a safe place to learn. This role becomes particularly important when implementing HIV prevention programmes or carrying out HIV/AIDS education. Schools across the continent have often not lived up to this expectation, however. The literature notes that schools are often highly sexualised sites in which school personnel take advantage of pupils (Humphreys, Undie, and Dunne 2008), increasing risks of HIV/AIDS and undermining the effectiveness of school-based HIV prevention programmes. However, the PIASCY evaluation provides evidence that creating safe school environments for primary school pupils was one of the achievements of the programme. Interviewees from all respondent categories spoke of how involvement with the PIASCY programme had empowered young people to resist advances from both school personnel and their peers. PIASCY provided mechanisms for addressing this behaviour, including talking with teachers, using the suggestion boxes, and joining a Safety Friends Network. Teachers themselves spoke of a deliberate change in their own behaviour as a result of exposure to PIASCY information. PIASCY Formative Evaluation 37 V.A.4. Structures and incentives within the schools The study shows that some schools were more actively engaged in PIASCY activities than others. Teasing out the exact reasons for this variation is complex, but a number of commonalities emerged from schools that were more engaged. These schools invariably had set up structures (such as ―PIASCY committees,‖ ―committees of teachers,‖ and ―PIASCY clubs‖) whose role was to plan for periodic PIASCY activities, prepare timetables, and assign specific roles to teachers. Such structures were instrumental in ensuring the continuity of the programme, even in the event of the transfer of a PIASCY-trained teacher, since they diffuse knowledge of PIASCY across several personnel. Schools that had structures for ensuring the continuity of PIASCY had the advantage of creating incentives or benefits, which in turn served as motivating factors for teachers. For instance, such schools talked about how they had gained popularity in their communities as a result of their PIASCY activities. These activities (e.g., assemblies and national performances) helped to enhance the visibility and image of some schools in their communities, fostering pride amongst teachers that compelled them to continue their engagement with the programme. Such positive perceptions of the programme may explain why, although all schools talked about the dwindling funding for PIASCY or the complete lack thereof, some were purchasing their own PIASCY materials. It is also true that some schools were simply better resourced than others, and therefore may have been in a better position to purchase such materials. V.A.5. The role of monitoring and evaluation The role of monitoring and evaluation (M&E) with regard to school-based HIV-prevention programmes cannot be over-emphasised, as M&E not only helps ensure that programme targets are being met, but also can help programme implementers make strategic decisions about improving the programme. In the case of PIASCY, study findings indicate that there is need to actively monitor and manage the knowledge cascade process. Apart from the Northern Region, in which school personnel were satisfied with the monitoring of their PIASCY activities, there was widespread concern across the regions (especially in the East and West) that there was no M&E of schools‘ PIASCY activities, with some respondents indicating that no follow-up of the programme had occurred in their schools since the programme‘s inception. Regular and consistent monitoring can ensure that diffusion of PIASCY training and knowledge among teachers occurs evenly across schools. V.A.6. Funding issues Funding was a recurrent theme of the evaluation. Teachers across all regions voiced their expectations for some form of financial compensation for the extra time they spent integrating PIASCY into their regular classes or activities. At the same time, many respondents showed a lack of understanding with regard to funding of the programme. The first set of Model Schools under PIASCY received (school incentive) grants for the implementation of the programme. However, this practice was not continued as the number of Model Schools increased. A number of schools erroneously suspected that they were being short changed, thinking that some schools were still receiving grants while they were 38 overlooked. This led to resentment and disenchantment among some teachers. In addition, several private schools stopped attending PIASCY training sessions when they were neither given transport refunds nor materials, unlike the public schools. This differential treatment between public and private schools was also noted in interviews with many public schools. Financial constraints can also hinder the effective implementation of school-based HIV￾prevention programmes, despite the best intentions of many African governments. In similar school-based programmes in the region, the necessary teaching materials are often not available (Gachuhi 1999). Likewise, the PIASCY programme experienced a shortage of PIASCY reading materials for pupils. Some teachers circumvented this issue, for instance, by reading the PIASCY materials aloud to the class, or registering them with the library so that each pupil would have an opportunity to check them out. It was not clear whether pupils were actually doing so, however. In other schools, there was a clear objective to preserve the PIASCY readers, as it was feared that pupils would either ruin or lose them. In these schools, the readers were therefore kept away from pupils in places such as locked cabinets or the headmaster‘s office. V.A.7. Integration within the school curriculum A major challenge for Ministries of Education that attempt to implement school-based HIV prevention programmes is the integration of such information into an already overloaded curriculum. There are arguments in the literature that teaching separate HIV/AIDS courses or lessons in schools has a better chance of impacting students than integrating HIV/AIDS information into pre-existing courses (Gachuhi 1999). However, the training that teachers received under PIASCY has clearly helped them with this task of integration, as they described many innovative ways in which they successfully incorporated PIASCY messages into both their curricula and other school activities/structures, such as assemblies and suggestion boxes. The fact that the majority of pupils were able to speak knowledgeably about HIV/AIDS is proof that this integration is working well. Nonetheless, a good number of teachers expressed the view that PIASCY ought to be an examinable subject in order to guarantee its long-term sustainability. There is also a need to ensure that all schools are aware of the approved and standardised PIASCY messages to be relayed to pupils. While condom use is not a focus of the PIASCY curriculum, FGDs with school personnel in the Eastern and Western Regions indicate that some teachers and administrative staff consider this topic as part of the PIASCY curriculum for upper primary school pupils, and sometimes for lower primary school pupils who were older and already sexually experienced. This finding is in contrast to the Northern Region, where teachers were consistent in saying that condom use by young people was strongly discouraged and therefore not taught at all. V.A.8. Political will Political will is a final ingredient necessary for the effective implementation of school-based HIV prevention programmes. Uganda is well-known for its committed efforts toward combating the spread of HIV/AIDS, and PIASCY is one of several Presidential initiatives. The study findings suggest, however, that political will can be a double-edged sword. A few PIASCY Formative Evaluation 39 teachers attributed PIASCY‘s achievements to its connection with Uganda‘s President, which was perceived as a motivator for some teachers to participate. Conversely, this connection was seen by others as a deterrent for would-be participants who were not proponents of the ruling political party. There were also fears that the close association of PIASCY with the President could actually be detrimental to the programme in the long-run, as it might be viewed as a purely political endeavour, regardless of the gains made. School-based HIV-prevention programmes are not a new phenomenon. In comparing the PIASCY programme to those discussed in the literature, it is evident that in some ways, PIASCY has experienced parallel challenges. On the other hand, PIASCY has in many ways overcome challenges enumerated in the literature — a testament to its strengths. PIASCY‘s teacher training component is particularly strong and has had a spill-over effect to teacher behaviour and the general school environment. 40 VI. Recommendations and Conclusion Based on the results of this study, the evaluation team offers two sets of recommendations to enhance PIASCY programme delivery: for (1) schools and for (2) MOES, UNITY, and USAID. This division is made for organisational purposes; clearly it represents an artificial boundary, given that these entities are closely interconnected within the PIASCY programme. V1.A.1. Schools Recommendations at the school level include: 1. Monitor the use of PIASCY readers (for schools that register them with the library) to determine how often and by whom the books are being checked. This will help assess whether putting books in the library results in maximum utilisation. 2. Address the language barrier posed by PIASCY reading materials by having teachers read PIASCY texts aloud to pupils; creating forums for peer-to-peer reading, or for upper primary pupils to read to their lower primary counterparts; and by involving pupils more closely in the development of messages to be posted within the Talking Environments. Teachers can also explain and clarify some of the messages posted on Talking Environments to students during assembly. 3. Establish a formal hand-over process for PIASCY-trained teachers who are being transferred to avoid gaps in institutional knowledge; the PIASCY committees can facilitate this process. VI.A.2. MOES, UNITY, and USAID 1. Continue to build the capacity of teachers to seamlessly mainstream PIASCY across the curriculum through periodic refresher training. The widespread opinion that PIASCY has increased teachers‘ workload points to the need to ensure teachers have the skills mainstream the curriculum in a way that does not overburden them. Periodic refresher courses for PIASCY-trained teachers will also help mitigate the censorship of PIASCY information during teaching and the delivery of inappropriate information. Alternatively, the Centre Coordinating Tutors‘ monitoring tool could also be redesigned to include the monitoring of PIASCY activities. 2. Consider involving teachers well-versed in PIASCY in monitoring schools‘ progress with the programme, especially given that there are too many schools for the Centre Coordinating Tutors to properly handle this responsibility. Instead, monitoring can be conducted by school PIASCY committees. Alternatively, the Centre Coordinating Tutors‘ monitoring tool could also be redesigned to include the monitoring of PIASCY activities. 3. Train more female guidance counsellors, as they are currently under-represented, and female pupils may feel more comfortable having specific issues addressed by guidance counsellors of the same sex. There is also need to incorporate HIV/AIDS counselling for both students and teachers that are infected with or affected by the disease. PIASCY Formative Evaluation 41 4. Explore ways of directly targeting parents and community-members to maximise their involvement in the PIASCY programme. Trainings developed specifically for these target groups can be conducted with strategically-placed community leaders, for example, using the knowledge cascade approach. This responsibility can be given to school PIASCY committees. 5. Explore ways to establish links between schools and health facilities to facilitate access to ARVs and counselling for pupils, school personnel, and community￾members living with HIV. Horizontal linkages with health workers and NGOs need to be established with schools because teachers may not have the skills to deal with certain issues. At the same time, NGOs have more resources, experience working with schools and communities, and have highly trained and skilled personnel in the field of HIV/AIDS. There may be a need to fully adopt a multi-sectoral approach and advocate for a school health policy that will define ways of dealing with HIV as a chronic epidemic within schools. This may also serve to strengthen the implementation of PIASCY within schools. 6. Diversify current HIV-prevention PIASCY messages by taking into account pupils and school staff living with HIV. For example, PIASCY could teach about treatment, emphasising the importance of drug adherence as well as inform both teachers and pupils about where to get help and assistance if needed. 7. There is a need to hold a meeting every two years to refocus the education sector strategy on HIV/AIDS. This will enable the Ministry of Education and Sports to evaluate its achievements and strengthen the coordination and implementation of PIASCY and other HIV/AIDS activities in schools. This could be turned into a regional meeting involving stakeholders and experts from other countries. This will help in generating new ideas to rejuvenate and refocus PIASCY. However, it is noteworthy that this strategy will also have budgetary implications. Given the structure of the PIASCY programme, the MOES has a particularly prominent role to play in building and monitoring the capacity of teachers, and in ensuring that key stakeholders, such as private schools and district authorities, are properly integrated into the programme. Thus, the activities that are recommended specifically for the MOES are as follows: 1. Monitor the MOES policy to put books in the hands of children. Although this policy is intended to ensure that children have direct access to books, there is evidence that some schools are not adhering to it. Monitoring of the entire PIASCY programme by the Education Standards Agency may be most effective. 2. Strengthen school structures such as ―PIASCY committees,‖ as they appear to be instrumental in the programme‘s continuity within schools. PIASCY committees could be given resources to sponsor activities they identify as necessary for proper implementation of PIASCY at their schools. They can also be involved in M&E of school-based programmes. Establishment of PIASCY committees could be mandatory for all schools. 3. Develop an operational definition of ―guidance and counselling‖ for the purposes of PIASCY, and determine to what extent it should be student- versus teacher-driven. There is a need to help school administrators, teachers, and even pupils re-construct the notion of guidance and counselling to include pupil-driven, individual-level engagement that goes beyond mere didactic instruction. A uniform training guide should be developed or adopted, and the required duration of training should be specified for one to be certified as a guidance counsellor. 42 4. Make concerted efforts to integrate private schools into the programme. Private schools should be brought on board as equal partners, not just as satellite schools to the Centres of Excellence/Model Schools. Holding deliberative meetings to explore ways of incorporating the Private Primary Teachers Colleges into PIASCY‘s organisational structure would also be useful. Integrating PIASCY into the private school curriculum could be made a requirement for licensing, and could be enforced through regular monitoring. PIASCY can also be integrated into policy through to university level. 5. Make concerted efforts to integrate districts and DEOs into the programme by involving them in decision making as well as the implementation of the programme and not just at the monitoring stage. 6. Monitor the knowledge cascade process regularly and evenly across schools and provide feedback to schools on how they are performing. 7. Tailor PIASCY instructions to the needs of older students within Free Primary Education schools, by focusing PIASCY instructions on both the grade and age of the pupil, so that older students in lower grades can be included. 8. Include occasional studies to inform the programme implementation. 9. Identify empirical evidence on behaviour change that is resulting from the PIASCY programme. This evidence could be generated by conducting school based behaviour change surveillance surveys to help PIASCY target and respond to arising issues and training needs. 10. Conduct a countrywide study of adolescents that passed through the PIASCY programme to ascertain whether knowledge in formative years has a positive impact on behaviour later in life. Such a study will be valuable in designing future school￾based programmes and in improving PIASCY design, delivery, and content. In conclusion, this formative evaluation has highlighted both PIASCY‘s innovative approaches that should be continued during the expansion of the initiative, as well as the areas in need of improvement. In subsequent phases of PIASCY, it is important to keep in mind that even the best of school-based programme designs can only flourish with strong mechanisms for monitoring, evaluation, and feedback, followed by specific actions to address arising issues. Large-scale, multi-faceted programmes such as PIASCY are not without their challenges, yet education and HIV-prevention programmes continue to be the best options in the absence of a cure for HIV. PIASCY Formative Evaluation 43 VII. References Cohen, J. 2008. ―A tale of two Presidential Initiatives,‖ Human Rights Watch. Retrieved December 29, 2008, (http:www.hrw.org). _______. 2006. ―The less they know, the better: Abstinence-only education programs in Uganda,‖ Reproductive Health Matters 14(28): 174–178. Gachuhi, D. 1999. ―The impact of HIV/AIDS on education systems in the Eastern and Southern Africa Region, and the response of education systems to HIV/AIDS: Life Skills Programmes,‖ prepared for UNICEF ESARO. Retrieved March 18, 2009 (http://www.unicef.org/lifeskills/files/gachuhi.pdf). Gallant, M. and E. Maticka-Tyndale. 2004. ―School-based HIV prevention programs for African youth,‖ Social Science & Medicine 58(7): 1337–1351. Government of Uganda. 2008. ―UNGASS country progress report, Uganda, January 2006– December 2007.‖ Kampala: Government of Uganda. ———. 2007. ―End of term PIASCY review meeting report.‖ Kampala: Government of Uganda. ———. 2003. ―Implementing the Presidential Initiative on the AIDS Strategy for Communication to Youth: Enhancing HIV awareness and dialogue among youth leadership in Uganda. Summary of the proposal.‖ Kampala: Government of Uganda. Griffiths, J. 2005. ―HIV/AIDS intervention programs for youth in Africa: The role of grassroot soccer,‖ unpublished M.A. thesis. Department of International Comparative Education, School of Education. Stanford University. Retrieved March 5, 2009, (http://www.grassrootsoccer.org/docs/griffith_standford.pdf). Humphreys, S., C. Undie, and M. Dunne. 2008. ―Gender, sexuality and development: Key issues in education and society in sub-Saharan Africa,‖ in Mairead Dunne (ed). Gender, Sexuality and Development: Education and Society in Sub-Saharan Africa, 7–38. Rotterdam: Sense Publishers. Ibale, A.M. 2007. Do „Talking Environments‟ talk? Experience from Uganda‟s primary schools. Unpublished UPHOLD manuscript. Ibale, A.M. and M. Nakibirige. 2004. ―End of tour report.‖ USAID-UPHOLD. Kibenge, A. 2009. ―Overview of PIASCY,‖ presented at Commonwealth Conference of Education Ministers, Kuala Lumpur Malaysia, June. Miles, M.B. and A. M. Huberman. 1994. Qualitative Data Analysis: An Expanded Sourcebook. Sage: Thousand Oaks, CA. Ministry of Education and Sports. 2008. PIASCY Primary UPHOLD Supported Activities: Status and Achievements. Kampala: Government of Uganda. 44 Office of the President. 2004. ―President‘s Program for Enhancing HIV/AIDS Dialogue among Youth Leadership,‖ Summary Progress Report. Kampala: Government of Uganda. __________________. 2003. ―Implementing the Presidential Initiative on the AIDS Strategy for Communication to Youth: Enhancing HIV/AIDS awareness and dialogue among youth leadership in Uganda—Summary of the Proposal.” Kampala: Government of Uganda. Patton, M. Q. 2002. Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage. UNAIDS. 2006a. ―AIDS epidemic update.‖ Retrieved December 30, 2008, (http://www.unaids.org/en/KnowledgeCentre/HIVData/EpiUpdate/EpiUpdArchive/2006/defa ult.asp). ———. 2006b. ―Linking EDUCAIDS with other ongoing initiatives: An overview of opportunities, an assessment of challenges.‖ Retrieved May 4, 2009, (http://unesdoc.unesco.org/images/0014/001479/147916E.pdf). UNESCO. 2005. ―UNESCO‘s response to HIV and AIDS.‖ Retrieved May 4, 2009, (http://unesdoc.unesco.org/images/0014/001414/141417e.pdf). UPHOLD. 2006. ―Quarterly report, April to June 2006.‖ Kampala: Government of Uganda. World Bank. 2002. Education & HIV/AIDS: A Window of Hope. Washington, D.C.: World Bank. PIASCY Formative Evaluation 45 VIII. Annexes Annex 1: Respondent Categories and Sample Size Respondent Category No. of FGDs per site Total No. of FGDs No. of respondents per FGD Sample size Pupils 2 FGDs x 80 schools 160 FGDs 160 FGDs x 6 pupils 960 pupils Teachers 1 FGD x 80 schools 80 FGDs 80 FGDs x 4 teachers 320 teachers Parents 1 FGD x 10 communities 10 FGDs 10 FGDs x 8 parents 80 parents 250 FGDs in total Total no. of FGD respondents = 1,360 Key Informant Category No. of Key Informants Headmasters 8 Core Primary Teachers College representative 6 Coordinating Centre Tutors 6 Ministry of Education representative 3 UNITY representative 1 Total number of Key Informants = 24 46 Annex 2: Study Instruments Guide for Focus Group Discussions with Pupils PIASCY Project African Population and Health Research Center Introduction Thank you so much for your willingness to take part in this group discussion. My name is [Name] and I work with [Institution]. We would like to have a group discussion with you based on the PIASCY program that is being implemented in your school. We‘re interested in learning about what goes on in the PIASCY program and how the PIASCY program activities have affected things at your school. Many times, people from outside think they know what students are experiencing when they really don‘t. To us, you are the real experts, and there‘s a lot we can learn from you. So today I‘d like to ask you a few questions about your experiences with PIASCY as students of this school. This is very informal; you can talk about anything you think is important for us to know. We hope this discussion will help us better understand what you‘ve experienced and how the program has functioned so that the great things about the program can continue, and so that any other aspects of the program can be improved in the future, if necessary. Please note that this is a general discussion which should not involve the sharing of personal, private information. I also want to remind you that everything we talk about today is confidential. We will be taking down notes as we discuss, but no one will see our notes except for people working on the project. Whenever we write a report, we will use numbers rather than names in the report so no one can identify you. If there are any questions you‘d rather not answer, just let me know—that‘s fine.  Explain the role of note-takers  Give a few minutes for answering any questions regarding the interview  Provide ground rules for the discussion Finally, I would like to point out that your frank responses and discussion will be most helpful to us as we try to really understand the PIASCY program. Remember, your answers to our questions will not be considered ―right‖ or ―wrong.‖ They are merely information you will provide based on your experiences, observations, or feelings. Before we begin, let‘s go round the room and introduce ourselves. You could just tell everyone your name and how long you‘ve been at this school. PIASCY Formative Evaluation 47 Perceptions of PIASCY’s Impact 1. Let‘s start by hearing from you what the PIASCY program is all about. Kindly describe to me what the PIASCY involves. What are some of the key things you have learnt through the PIASCY program? What did you not know about HIV/AIDS until the PIASCY program started? 2. Let‘s imagine that today I‘ll be sitting in during one of your PIASCY activities (it could be a class, club meeting, assembly, etc). Tell me what I would observe during the PIASCY activity. [Let respondents choose two PIASCY activities to talk about in this section] How would you describe the way in which students and teachers typically relate to one another during the PIASCY activity? Kindly elaborate. How free would you say students feel to ask questions or contribute their views during these activities [do you feel they ask any question you want, or do you feel they limit themselves to certain kinds of questions]? What is the typical content of the HIV-related messages that are disseminated during activities such as school assemblies? Do students discuss things they learned during PIASCY activities with teachers or parents outside the PIASCY context? If so, how often would you say this occurs, and what kinds of things are discussed? If not, why not? 3. HIV info is disseminated in different ways in your school (e.g., assemblies, posters, clubs, readings, talking environments, etc.). In your opinion, which means has made the strongest impact on students? Kindly elaborate. Which means has made the least impact on students? Kindly elaborate. 4. What would you say students at your school have gotten out of participating in PIASCY? What changes do you see in your fellow students as a result of the PIASCY program? What are some of the things students have learned through the PIASCY program that have carried over to their lives (i.e., influenced your lives)? Kindly elaborate. What plans have students made, if any, to change anything or do anything differently as a result of the PIASCY program? How have your fellow students‘ thoughts/behaviour in regard to abstinence changed, if at all, since the PIASCY program started? What about your fellow students‘ thoughts on having unprotected sex—how have they changed, if at all, since PIASCY? 5. As a result of the PIASCY program, how have the feelings of students in this school about HIV changed, if at all? Kindly give an example of what you mean. 48 How have students in this school come to understand HIV differently, if at all? How have students in this school come to understand sexual behaviour differently, if at all? 6. As a result of the PIASCY program, how have the feelings of students in this school changed in regard to people living with HIV, if at all? Kindly give an example of what you mean. To what extent would you say your school is a nice place to be for people that are HIV positive? What role do students play in the lives of those in your school that are HIV+? What role do students play in the lives of others in your community that are HIV+? How is this role different from the role students played before PIASCY was introduced at your school, if at all? Reasons behind PIASCY’s Impact 7. So far, you‘ve provided a lot of useful information about how PIASCY has impacted the students in this school, and your school at large. Now, what is it about PIASCY that makes it have the effect it has? What happens within the PIASCY program that makes a difference? What do students see as the important parts of PIASCY that make the program what it is? What has been the high point of the program for most students? Kindly elaborate. What has the low point been? Kindly elaborate. Likes & Dislikes about PIASCY 8. Let us turn now to people‘s general likes and dislikes about the program. What are some of the things that students really like about the program? What are some of the things people don‘t like so much about the program? How can these things be improved upon—what would you like to see happen instead? Recommendations 9. Okay, you‘ve given us a lot of information about your experiences in the program, and some of the things you‘ve liked and haven‘t liked so much. Now I‘d like to ask you about your recommendations for the program. If you had the power to change things about the program, what would you change or do differently? What would you focus on first? PIASCY Formative Evaluation 49 10. Suppose you were asked by a government agency for your honest opinion on whether or not they should sponsor a program like PIASCY. What would you say? What arguments would you give to support your opinion? Closing That covers the things I wanted to ask. Is there anything you care to add? Thank you so much for you time. I‘ve really learned a lot from you today and I really appreciate your insights. 50 Guide for Focus Group Discussions with Teachers PIASCY Project African Population and Health Research Center Introduction Thank you so much for your willingness to take part in this group discussion. My name is [Name] and I work with [Institution]. We would like to have a group discussion with you based on the PIASCY program that is being implemented in your school. We‘re interested in learning about what goes on in the PIASCY program and how PIASCY program activities have affected things at your school. Today I‘d like to ask you a few questions about your experiences with PIASCY as teachers at this school. This is very informal; you can talk about anything you think is important for us to know. We hope this discussion will help us better understand what you‘ve experienced and how the program has functioned so that the great things about the program can continue, and so that any other aspects of the program can be improved in the future, if necessary. I also want to remind you that everything we talk about today is confidential. We will be taking down notes as we discuss, but no one will see our notes except for people working on the project. Whenever we write a report, we will use numbers rather than names in the report so no one can identify you. If there are any questions you‘d rather not answer, just let me know—that‘s fine.  Explain the role of note-takers  Give a few minutes for answering any questions regarding the interview  Provide ground rules for the discussion Finally, I would like to point out that your frank responses and discussion will be most helpful to us as we try to really understand the PIASCY program. Remember, your answers to our questions will not be considered ―right‖ or ―wrong.‖ They are merely information you will provide based on your experiences, observations, or feelings. Before we begin, let‘s go round the room and introduce ourselves. You could just tell everyone your name, how long you‘ve been at this school, and what your role in the PIASCY program at this school has been so far. Perceptions of PIASCY’s Impact 1. Let‘s start by hearing from you what PIASCY is all about. What is the PIASCY program about, and how does it feature in your lives as teachers and in the lives of students at your school? 2. Now, I would like to ask you to kindly walk me through the way the PIASCY program works, step-by-step. When you receive the PIASCY materials, for instance, what happens next? PIASCY Formative Evaluation 51 To what extent are the materials being translated and adapted into action? What are the barriers to this happening, if any? What happens next (i.e., after materials are received)—how exactly does the program work at your school? 3. To what extent has the PIASCY program been what you expected it to be? In what ways has it met your expectations so far? In what ways has it been different from what you expected? To what extent did the things you were concerned about before the PIASCY program come true? o Which things came true? o Which didn‘t come true? 4. How would you describe your skills and knowledge in relation to HIV/AIDS before you got involved with the PIASCY program? Kindly elaborate. How would you describe your skills and knowledge in relation to HIV/AIDS after your PIASCY orientation and beyond? Kindly elaborate. 5. Let‘s imagine that today I‘ll be sitting in during one of your PIASCY activities (it could be a class, club meeting, assembly, etc). Tell me what I would observe during the PIASCY activity. [Let respondents choose two PIASCY activities to talk about in this section] How would you describe the way in which the students and teachers typically relate to one another during the PIASCY activity? Kindly elaborate. How free would you say students feel to ask questions or contribute their views during these activities [do you feel they ask any question they want, or do you feel they limit themselves to certain kinds of questions]? What is the typical content of the HIV-related messages that are disseminated during activities such as school assemblies? Do you discuss things you taught during PIASCY activities with students outside the PIASCY context? If so, how often would you say this occurs, and what kinds of things are discussed? 6. Which particular topics, if any, do the students feel less comfortable discussing/being taught during PIASCY activities, and why do you think this is the case? What about you as teachers—which topics, if any, do teachers typically feel uncomfortable teaching/discussing, and why do you think this is the case? How are these topics dealt with, given the discomfort you mention on the part of students/teachers? What do you think could have been done in advance to help ensure students and teachers feel comfortable addressing these kinds of topics? 52 7. HIV info is disseminated in different ways in the school (e.g., assemblies, posters, clubs, readings, talking environments, etc.). In your opinion, which means has made the strongest impact on students and staff? Kindly elaborate. Which means has made the least impact on you? Kindly elaborate. 8. What would you say you, as teachers, have gotten out of participating in PIASCY? What changes do you see in yourselves/others at this school as a result of the PIASCY program? What are some of the things you have learned through PIASCY that have carried over to your lives (i.e., influenced your lives)? Kindly elaborate. What plans have you made, if any, to change anything or do anything differently as a result of the PIASCY program? What are some of the things students have learned through PIASCY that they have carried over to their lives? Kindly elaborate. How have students‘ thoughts/behaviour in regard to abstinence changed, if at all, since the PIASCY program began? What about students‘ thoughts on having unprotected sex—how have they changed, if at all, since PIASCY? 9. As a result of your experience with the PIASCY program, how have your feelings about HIV changed, if at all? Kindly give an example of what you mean. How have you/other staff at this school come to understand HIV differently, if at all? How have you/other staff at this school come to understand sexual behaviour differently, if at all? 10. As a result of your experience with the PIASCY program, how have your feelings/the feelings of students in this school changed in regard to people living with HIV? Kindly give an example of what you mean. To what extent would you say your school is a stigma-free environment when it comes to HIV? What role do you/other students play in the lives of students in your school that are HIV+? What role do you/other students play in the lives of others in your community that are HIV+? How is this role different from the role you/other students played before PIASCY was introduced at your school? Reasons behind PIASCY’s Impact 11. So far, you‘ve provided a lot of useful information about how PIASCY has impacted teachers, students, and your school at large. Now, what is it about PIASCY that makes it have the effect it has? What happens within the program that makes a difference? PIASCY Formative Evaluation 53 What do you see as the important parts of PIASCY that make the program what it is? What has been the high point/best part of the program for you as teachers? Kindly elaborate. What has the low point/worst part been? Kindly elaborate. Unintended Consequences of PIASCY 12. I‘d like to ask you to think over the PIASCY program and any implications that running it at your school has had for the ways things usually operate here. Has running the PIASCY program brought up any issues at your school that you didn‘t expect? If yes, kindly tell me more about that. Let‘s start with any issues it may have brought up. What are the positive issues, if any? How have they affected the way you used to do things at the school? What are the negative issues, if any? How have they affected the way you used to do things at the school? PIASCY Design, Organizational Structure, & Roll-out 13. I would now like you to reflect on how the PIASCY program is designed. How well or effectively would you say the program design has worked so far? Kindly elaborate. What would you say are the strengths of the program? What would you say are the weaknesses of the program, and how can they be modified/improved? What would you say are the strengths of the program‘s organizational structure? What would you say are the weaknesses of the program‘s organizational structure, and how can they be modified/improved? 14. I would like to ask you to reflect on the design of the roll-out of the PIASCY program in Uganda. How well or effectively would you say the design is working? Kindly elaborate. How were the private schools integrated into the roll-out? To what extent was this process effective? What would you say are the strengths of the roll-out design? What would you say are the weaknesses of the roll-out design and how can they be modified/improved? 54 Recommendations 15. Okay, you‘ve given us a lot of information about your experiences in the program, and your impressions of how things are running. Now I‘d like to ask you about your recommendations for the program. If you had the power to change things about the program, what would you change or do differently? What would you focus on first? 16. Suppose you were asked by a government agency for your honest opinion on whether or not they should sponsor a program like PIASCY. What would you say? What arguments would you give to support your opinion? Closing That covers the things I wanted to ask. Is there anything you care to add? Thank you so much for you time. I‘ve really learned a lot from you today and I really appreciate your insights. PIASCY Formative Evaluation 55 Guide for Focus Group Discussions with Parents/Community Members PIASCY Project African Population and Health Research Center Introduction Thank you so much for your willingness to take part in this group discussion. My name is [Name] and I work with [Institution]. We would like to have a group discussion with you based on the PIASCY program that is being implemented in your child‘s school/the school you are involved in. We‘re interested in learning about what goes on in the PIASCY program and how the PIASCY program activities have affected things at the school. Today I‘d like to ask you a few questions about your experiences with PIASCY as parents/community members involved in this school. This is very informal; you can talk about anything you think is important for us to know. We hope this discussion will help us better understand what you‘ve experienced and how the program has functioned so that the great things about the program can continue, and so that any other aspects of the program can be improved in the future, if necessary. I also want to remind you that everything we talk about today is confidential. We will be taking down notes as we discuss, but no one will see our notes except for people working on the project. Whenever we write a report, we will use numbers rather than names in the report so no one can identify you. If there are any questions you‘d rather not answer, just let me know—that‘s fine.  Explain the role of note-takers  Give a few minutes for answering any questions regarding the interview  Provide ground rules for the discussion Finally, I would like to point out that your frank responses and discussion will be most helpful to us as we try to really understand the PIASCY program. Remember, your answers to our questions will not be considered ―right‖ or ―wrong.‖ They are merely information you will provide based on your experiences, observations, or feelings. Before we begin, let‘s go round the room and introduce ourselves. You could just tell everyone your name and how long you‘ve been involved with this school. Perceptions of PIASCY’s Impact 1. Let‘s start by hearing from you what the PIASCY program is all about. Kindly describe to me what the PIASCY involves. What are some of the key things you have learnt through the PIASCY program? What did you not know about HIV/AIDS until the PIASCY program started? 56 2. Let‘s imagine that today I‘ll be sitting in during one of your PIASCY activities (it could be a school assembly, school open days, talking environments etc). Tell me what I would observe during the PIASCY activity. [Let respondents choose two PIASCY activities to talk about in this section] How would you describe the way in which students and teachers typically relate to one another during the PIASCY activity? Kindly elaborate. How would you describe the way in which you and students typically relate to one another during the PIASCY activity? Kindly elaborate. How free would you say students feel to ask questions or contribute their views during these activities [do you feel they ask any question they want, or do you feel they limit themselves to certain kinds of questions]? What is the typical content of the HIV-related messages that are disseminated during activities such as school assemblies? Do students discuss things they learned during PIASCY activities with you outside the PIASCY context? If so, how often would you say this occurs, and what kinds of things are discussed? 3. HIV info is disseminated in different ways in the school you‘re involved with (e.g., assemblies, posters, clubs, readings, talking environments, etc.). In your opinion, which means has made the strongest impact on students? Kindly elaborate. Which means has made the least impact on students? Kindly elaborate. 4. What would you say students at the school have gotten out of participating in PIASCY? What changes, if any, have you observed in the community as a result of the PIASCY program? What about changes in yourselves? What are some of the things students have learned through the PIASCY program that have carried over to their lives (i.e., influenced your lives)? Kindly elaborate. What about yourselves? What plans have you made, if any, to change anything or do anything differently as a result of the PIASCY program? 5. As a result of the PIASCY program, how have the feelings of community members about HIV changed, if at all? Kindly give an example of what you mean. How have community-members come to understand HIV differently, if at all? How have community-members come to understand sexual behaviour differently, if at all? 6. As a result of the PIASCY program, how have the feelings of people in this community changed in regard to people living with HIV, if at all? Kindly give an example of what you mean. To what extent would you say this school is a nice place to be for people that are HIV positive? What about this community? PIASCY Formative Evaluation 57 What role do students play in the lives of others in your community that are HIV+? How is this role different from the role students played before PIASCY was introduced in this school, if at all? Reasons behind PIASCY’s Impact 7. So far, you‘ve provided a lot of useful information about how PIASCY has impacted the school and the community. Now, what is it about PIASCY that makes it have the effect it has? What happens within the PIASCY program that makes a difference? What do you see as the important parts of PIASCY that make the program what it is? What has been the high point of the program for you? Kindly elaborate. What has the low point been? Kindly elaborate. Likes & Dislikes about PIASCY 8. Let us turn now to your general likes and dislikes about the program. What are some of the things that you, as parents/community members, really like about the program? What are some of the things that you don‘t like so much about the program? How can these things be improved upon—what would you like to see happen instead? Recommendations 9. Okay, you‘ve given us a lot of information about your experiences in the program, and some of the things you‘ve liked and haven‘t liked so much. Now I‘d like to ask you about your recommendations for the program. If you had the power to change things about the program, what would you change or do differently? What would you focus on first? 10. Suppose you were asked by a government agency for your honest opinion on whether or not they should sponsor a program like PIASCY. What would you say? What arguments would you give to support your opinion? Closing That covers the things I wanted to ask. Is there anything you care to add? Thank you so much for you time. I‘ve really learned a lot from you today and I really appreciate your insights. 58 Guide for In-depth Interviews with Ministry of Education and Sports (MOES), Uganda PIASCY Project African Population and Health Research Center Introduction Thank you so much for meeting with me today. My name is [Name] and I work with [Name of Institution]. We would like to conduct an interview with you based on the PIASCY program that is being implemented in primary schools in Uganda by the Ministry of Education and Sports in partnership with other organizations. We‘ve requested an interview with you because we believe that in your position as a [Position/Job Title] in [Name of Institution], we can learn a lot from you about the PIASCY program, how your institution/office has engaged with it, and the kinds of issues your institution/office and the other organizations involved may be dealing with. We hope this interview will help us better understand what your institution has experienced and how the program has functioned so that the great things about the program can continue, and so that any other aspects of the program can be improved in the future, if necessary. Is there anything you‘d like to ask me at this point? [Answer any questions regarding the interview]. Overview of PIASCY & MOES’s Involvement 1. Perhaps we can start by getting an idea of your understanding of the PIASCY program. What is your understanding of the PIASCY program and what it was designed to achieve? What is your assessment of how well the PIASCY program is achieving its objectives? What are some of the factors that have helped the program achieve its objectives? What have the barriers been to the program achieving its objectives, in your opinion? How can these barriers be addressed? 2. Some of the things that the PIASCY program hoped to do were to: 1) increase the capacity of key providers (BEPS and UPHOLD) to deliver learning resources and materials to train teachers and students, 2) increase the number of trained (PIASCY) teachers, and 3) increase the number of schools involved in the PIASCY program in all the regions. To what extent would you say this has been achieved? [Let‘s begin with the first one] What are some of the factors that have helped the program achieve this objective? What have the barriers been to achieving this objective? How can these barriers be addressed? [Repeat questions for all 3 objectives above] PIASCY Formative Evaluation 59 3. What has been the nature of MOES‘s engagement with PIASCY schools over the years? Whom within the schools does the MOES engage with, how, and how often? In addition to the orientation provided to school teachers by the MOES, are there ways in which the MOES has followed-up with teachers/schools to see the PIASCY program is working? Kindly elaborate. What, in your opinion, have been the commendable things about the MOES‘s engagement with schools? What areas in the MOES‘s engagement with schools could have been improved, and how? Based on the MOES‘s engagement with schools, what do you see as the areas of strength that the schools bring to the PIASCY program? What are the areas of weakness that schools need to improve upon in regard to the PIASCY program? Reasons behind PIASCY’s Impact 4. What would you say that school teachers have gotten out of participating in the PIASCY program? What changes have you been able to observe in teachers as a result of the PIASCY program? What are some of the things school teachers have learned through the PIASCY program that have carried over to their lives (i.e., influenced your lives)? Kindly elaborate. What plans, if any, have you observed school teachers make to change anything or do anything differently as a result of the PIASCY program? What is it about the PIASCY program that makes it have the effect it has? What do school teachers see as the important parts of PIASCY that make the program what it is? Unintended Consequences of PIASCY 5. I‘d like to ask you to think over the PIASCY program and any implications that implementing it has had for the ways things usually operate here at the MOES. Has implementing the PIASCY program in the country brought up any issues at the MOES that you didn‘t expect? If yes, kindly tell me more about that. Let‘s start with any positive issues it may have brought up. What are the positive issues, if any? How have they affected things at your institution? What are the negative issues, if any? How have they affected things at your institution? PIASCY Design, Organizational Structure, & Roll-out 6. I would now like you to reflect on how the PIASCY program is designed. 60 How well or effectively would you say the program design has worked so far? Kindly elaborate. What would you say are the strengths of the program? What would you say are the weaknesses of the program? What would you say are the strengths of the program‘s organizational structure? What would you say are the weaknesses of the program‘s organizational structure, and how can they be modified/improved? 7. I would like to ask you to reflect on the design of the roll-out of the PIASCY program in Uganda. How well or effectively would you say the design has worked? Kindly elaborate. How were the private schools integrated into the roll-out? To what extent was this process effective? What would you say are the strengths of the roll-out design? What would you say are the weaknesses of the roll-out design and how can they be modified/improved? Recommendations 8. Okay, you‘ve given us a lot of valuable information about the program from your perspective as someone within the MOES. Now I‘d like to ask you about your recommendations for the program. If you had the power to change things about the program, what would you change or do differently? What would you focus on first? 9. Suppose you were asked by a donor for your honest opinion on whether or not they should sponsor a program like PIASCY. What would you say? What arguments would you give to support your opinion? 10. Finally, as you are aware, the Uganda Program for Human and Holistic Development (UPHOLD) took over support to roll out and implement PIASCY, and now, UPHOLD will be handing over its PIASCY activities to the Ugandan Initiative for TDMS and PIASCY (UNITY). Given the MOES‘s experience as the first implementer of PIASCY, what are some of the sustainability issues that will need to be addressed in handing over the program from one establishment to another? Closing That covers the things I wanted to ask. Is there anything you care to add? Thank you so much for you time. I‘ve really learned a lot from you today and I really appreciate your insights. PIASCY Formative Evaluation 61 Guide for In-depth Interviews with Funders PIASCY Project African Population and Health Research Center Introduction Thank you so much for meeting with me today. My name is [Name] and I work with [Name of Institution]. We would like to conduct an interview with you based on the PIASCY program that is being implemented in primary schools in Uganda by the Ministry of Education and Sports in partnership with other organizations. We‘ve requested an interview with you because we believe that, given your affiliation with a donor institution that has helped to fund PIASCY, we can learn a lot from you about the initiative, how your institution has engaged with it, and the kinds of issues your institution and the other organizations involved may be dealing with. We hope this interview will help us better understand your institution‘s impressions of how the program has functioned so that the great things about the program can continue, and so that any other aspects of the program can be improved in the future, if necessary. Is there anything you‘d like to ask me at this point? [Answer any questions regarding the interview]. Overview of PIASCY & Funder’s Involvement 1. Perhaps we can start by getting an idea of your understanding of the PIASCY program. What is your understanding of the PIASCY program and what it was designed to achieve? 2. What has been the nature of your institution‘s engagement with the PIASCY program over the years? Whom within has your institution engaged with, and how? In addition to what you‘ve described, are there ways in which your institution has followed-up with those implementing the program to see PIASCY is working? Kindly elaborate. Based on the MOES‘s engagement with schools, what do you see as the areas of strength that the schools bring to the PIASCY program? What are the areas of weakness that schools need to improve upon in regard to the PIASCY program? 3. To what extent has the PIASCY program so far been what you expected it to be? In what ways has it met your expectations so far? In what ways has it been different from what you expected? 62 To what extent did the things you were concerned about before the PIASCY program come true? o Which things came true? o Which didn‘t come true? 4. Some of the things that the PIASCY program hoped to do were to: 1) increase the capacity of key providers (BEPS and UPHOLD) to deliver learning resources and materials to train teachers and students, 2) increase the number of trained (PIASCY) teachers, and 3) increase the number of schools involved in the PIASCY program in all the regions. To what extent would you say this has been achieved? [Let‘s begin with the first one] What do you feel are some of the factors that have helped the program achieve this objective? What do you feel the barriers have been to achieving this objective? How can these barriers be addressed? [Repeat questions for all 3 objectives above] Reasons behind PIASCY’s Impact 5. What would you say that PIASCY beneficiaries (teachers, pupils, parents, community-members, etc.) have gotten out of participating in the PIASCY program? In your opinion, what is it about the PIASCY program that makes it have the effect it has? Kindly elaborate. PIASCY Design, Organizational Structure, & Roll-out 6. I would now like you to reflect on how the PIASCY program is designed. How well or effectively would you say the program design has worked so far? Kindly elaborate. What would you say are the strengths of the program? What would you say are the weaknesses of the program? What would you say are the strengths of the program‘s organizational structure? What would you say are the weaknesses of the program‘s organizational structure, and how can they be modified/improved? 7. I would like to ask you to reflect on the design of the roll-out of the PIASCY program in Uganda. How well or effectively would you say the design has worked? Kindly elaborate. What would you say are the strengths of the roll-out design? What would you say are the weaknesses of the roll-out design and how can they be modified/improved? PIASCY Formative Evaluation 63 Recommendations 8. Okay, you‘ve given us a lot of valuable information about the program from your perspective as a funder. Now I‘d like to ask you about your recommendations for the program. If you had the power to change things about the program, what would you change or do differently? What would you focus on first? 9. Suppose you were asked by another donor for your honest opinion on whether or not they should chip in and help sponsor a program like PIASCY. What would you say? What arguments would you give to support your opinion? 10. Finally, as you are aware, the Uganda Program for Human and Holistic Development (UPHOLD) took over support to roll out and implement PIASCY, and now, UPHOLD will be handing over its PIASCY activities to the Ugandan Initiative for TDMS and PIASCY (UNITY). Given the your experience as a funder of PIASCY, what are some of the sustainability issues that will need to be addressed in handing over the program from one establishment to another? Closing That covers the things I wanted to ask. Is there anything you care to add? Thank you so much for you time. I‘ve really learned a lot from you today and I really appreciate your insights. 64 Contact Summary Form: Illustration CONTACT SUMMARY SITE: Boma Primary School (rural) WRITTEN BY: MM TODAY‘s DATE: 1/20/09 Type of contact: FGD with Teachers Library 1/20/09 Who, what group place contact date Instructions: Pick out the most salient points in the contact. Number in order on this sheet. Attach relevant study question or theme to each point in CAPITALS. Invent themes where no existing ones apply and asterisk these. Comment may also be included in double parentheses. SALIENT POINTS STUDY QUESTIONS/THEMES 1. Students not able to comprehend many PIASCY MEETING ITS OBJECTIVES [‗increased technical terms in the English PIASCY capacity to deliver learning resources‘]. (Maybe manual, so teachers have developed a this is also an ‗unintended consequence‘??) manual in the local language to complement the official manual. 2. Teachers are generally enthusiastic about *RESISTANCE teaching upper primary classes, but vary in their willingness to introduce PIASCY concepts to lower primary students lower primary school students. 3. One teacher says, ‗Now that we have to UNINTENDED CONSEQUENCES meet our PIASCY obligations, we have done away with the P.E. program at school. There‘s no time for it.‘ (All others agreed.) 4. PIASCY orientation was fondly remem- STRENGTHS OF PIASCY PROGRAM bered; gave teachers lots of confidence, which has positively affected how they teach their other subjects. 5. Would really like to have refresher trainings RECOMMENDATIONS and to have new teachers with potential to undergo PIASCY orientation, as teachers get transferred often. PIASCY Formative Evaluation 65 Rapid Appraisal Tool _____Talking Environments Notes: What messages are posted and the location of messages (e.g. toilet block, water tank, gate etc is it a place that students can easily access) _____Assembly and what happens during the Assembly Notes: (if there is an assembly at the time of visit please observe and note down what happens) _____Material/Readers Notes: (which ones do they have and who is using them. Have they been issued to student or are locked up if locked up give reason why. Are they enough for students and teachers) _____Guidance and Counseling Notes: (Do they have the files. How are they used. Please check for content of files to determine what guidance and counseling occurs, e.g. group counseling/individual counseling. Do they have a counseling room, how often is it used) _____Suggestion Box files, issues and how they are handled Notes: (Do they have the suggestion box files. Get a sample of messages and write down their content. How are issues handled and recorded) _____Clubs Notes: (Names of Clubs, Activities, Who is involved, play titles, debates topics etc. Have they participated in the Music Dance and Drama festival) _____Safety Friends and Networks Note: (Do they have safety friends and networks at the school. Are all the children in the SFN or particular children) USAID FROM THE AMERICAN PEOPLE