24 November 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Southern Hemisphere USAID Botswana Performance Evaluation - Tsela Kgopo Gender and OVC Project Final Report (RFTOP number SOL-674-15-000007) i ACKNOWLEDGEMENTS Southern Hemisphere would like to thank the individuals and organizations that made this evaluation possible. We would like to thank USAID Botswana and Project Concern International for entrusting Southern Hemisphere with this task, and providing ongoing support during this evaluation process. We would also like to extend our thanks to the implementing partners sampled for this evaluation: Hope Worldwide, Kagisano Women’s Shelter, Humana People to People, Bakgotla Bolokang Matshelo and Catholic Relief Services, for their openness to the evaluation process and commitment under tight time constraints. Our appreciation goes to all the participants of this evaluation who shared their knowledge and experiences with us, and in so doing, contributed to generating knowledge about OVC and gender programming. Lastly, a special thanks to all the team members, specifically Carol Mnqayi, Cathy Chames, Gasegale Sengwaketse, Gethwana Mahlase, Ivor Williams, Marina Vayanos, Dr. Onalenna Selolwane, and Tracey Phillips for their dedication and hard work during this period. Wilma Wessels-Ziervogel (Project Manager) and Dena Lomofsky (Team Leader) Southern Hemisphere ii EXECUTIVE SUMMARY Introduction The Tsela Kgopo Orphans and Vulnerable Children and Gender Project is funded by USAID and managed by PCI, who provides technical support to ten Implementing Partners and the Government of Botswana. This is meant to contribute to creating an enabling policy environment and improved services for OVC, youth and women and ultimately is meant to improve the quality of life of OVCs. The uniqueness of this project lies in its use of a comprehensive family care (CFC) approach. Methods A performance evaluation of this project was conducted by Southern Hemisphere between March and October 2015. The evaluation sort to determine whether the project had been implemented as planned; what the strengths and challenges were in implementation; whether there were signs of the intended results (outcomes); and how programme effectiveness could be enhanced. A qualitative research design, comprising semi-structured interviews (SSIs) and focus group discussions (FGD) was used for this performance evaluation. Interviews were conducted with key informants at USIAD and PCI, national and district government, five implementing partners and beneficiaries. Findings Programme design and approach The TK Project has been well designed to meet the needs of OVCs and their families and has taken services to hard to reach areas. However the demand for services has not been met as there are vulnerable areas, and particular sub-groups of OVCs (e.g. children on farms and streets) that are still not being reached. The CFC approach is a holistic model of care that facilitates the provision of a range of services to meet the needs of the family. The key strengths of this approach are the identification of vulnerable households and service provision at household level. Home visits have been a key enabler in this regard and should be used to compliment group based interventions, whose comparative advantage is building community cohesion. The CFC approach is dependent on having a good referral system; this has been a challenge for most organisations sampled. Another challenge is the perceived demand that the CFC approach places on IPs (in terms of number of staff and skills) to provide a range of specialised services. Objective 1: Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women improved Most of the Objective 1 activities captured in the cooperative agreement and work plans were implemented as planned according to activity design, although some remain inconclusive due to incomplete reporting. Community structures were capacitated, through the use of community mobilisation tools (e.g. JoL and CCE) and the CFC approach, to identify vulnerable community members and refer them to relevant service providers. These community mobilisation tools are well designed to encourage community ownership and sustainability. In terms of increasing the participation of vulnerable households in economic strengthening activities, the findings are generally positive. Basic skills development in financial literacy and household financial management is being iii rolled out, particularly via the formation of GROW groups. The Aflateen and YEP programmes offer capacity building in financial literacy and business management skills for youth, while YEP also offers youth participants job readiness training. Partnerships are being formed with local and private businesses and financial organisations to facilitate the provision of skills development and mentoring services to beneficiaries. Participation and successful implementation of youth economic strengthening activities are constrained by a lack of popularity amongst youth, which perhaps also suggests that the programme is not sufficiently contextualised. A number of positive outcomes were identified for economic strengthening which are reportedly as a result of the TK interventions. Women and youth have improved knowledge and understanding of saving, budgeting and business development; as well as improved communication and leadership skills. However, a substantial number of respondents noted that despite these changes in knowledge, as well as self-confidence and self-esteem, the programme had yet to make any fundamental difference to their income levels or livelihoods. Objective 2: Developmental interventions for children and adolescents Some of the successes of this objective area is the reach of IECD services, improved perceptions of parents about the importance of ECD, the use of creative models (such as the mobile model) for engaging in IECD stimulation groups. Although some communities have mobilised around nutrition and stimulation materials, given the economic context within which IPs are working, the exclusion of this from the TK project budget has been a challenge in terms of promoting quality IECD services. Furthermore, government regulations around minimum requirements for ECD centres has meant that a number of centres have been forced to close or operate illegally. Positive outcomes for IECD include improved access to IECD services, improved school readiness, and the instilling of a passion for learning among children. Parents have also benefited from the knowledge gained on holistic childcare, and as a result are more vigilant about the health, hygiene and emotional needs of children. The most significant changes for youth in terms of lifeskills was around their improved self￾confidence which resulted in better communication. This has contributed to improved communication within families. Youth also seem to be making better decisions regarding their friends and substance use. HIV/AIDS and health interventions have contributed to related positive changes in awareness, knowledge and practice, including HIV testing. The positive changes have been supported by a dedicated and committed staff, who received appropriate training, teaching and support from PCI. The very approach used in these programmes has been credited with their success. Community support has also been an important factor in the outcomes of the programmes; positive support has been instrumental in any successes and a lack of support has hindered changes that may have occurred. A lack of government support resources may also have reduced the positive effects of the programme. Objective 3: Neglect, exploitation and abuse of vulnerable children, adolescents and women Implementing partners have undertaken a range of activities to reduce the risk of neglect, exploitation and abuse of vulnerable children, adolescents and women. These included the establishment, strengthening and sensitisation of community structures; strengthening the response iv services to protect abused women and children particularly by KWS which expanded its outreach services into six districts; and the facilitation of numerous child protection and gender related training activities targeting implementing partners, teachers, social workers. Most of the planned activities in the work plan to increase the involvement of men as peer educators, mentors and caregivers and role models were not undertaken. However, the Men’s Sector was engaged as a strategic partner and implementing partners targeted men in the gender awareness raising activities. Although gender is a new concept for most partners, they have started mainstreaming gender into their programme activities. Key strengths for this objective include the multi-sectoral partnership approach; establishment of local structures to coordinate prevention and response services; the good quality outreach, counselling and shelter services provided particularly by KWS; and the use of available and existing platforms, commemorative days and community dialogue methodology to raise awareness around GBV and child protection. Main challenges with implementation include the delay in gender training for partners; limited training interventions for local coordination structures; poor oversight of the VCPCs due to poorly functioning DCPCs; delay in piloting and roll-out of GBV referral system; limited legal and shelter services particularly in rural, outlying areas; insufficient targeting of men with programme activities; and the weak response services to cope with the increase in reporting and demand for services after awareness raising events. Some emerging outcomes were identified at community level. Women report an increase in GBV￾related knowledge and attitudes which has influenced behaviour change particularly around their ability to negotiate safer sex. Men also report changes in knowledge and some shifts in attitude. Furthermore, those men who have participated in the KWS life skills programme report that they are no longer abusive towards women which suggests that this type of targeted intervention with men can lead to behaviour change. Changes in knowledge, attitude and practice in relation to child abuse, neglect and exploitation were also noted, although these changes were not as significant as those mentioned for gender. Objective 4: National frameworks and policies An assessment of the extent to which each activity has been implemented according to activity design reveals that many of the activities under objective 4 have been achieved. The main cause of deviations or shortfalls was due to lengthy duration of government processes and the limited staff capacity at PCI to carry out all the activities on time. PCI has provided a significant amount of technical and financial support to a range of national government departments including DSP and GeAD. This includes support for the implementation of the Children’s Act and development of the new Gender and Development Policy, strategy and implementation plan. Whilst there was no mention of activities related to the OVC policy implementation, a number of additional activities were undertaken which were not contained in the project work plans. For example, PCI has reviewed aspects of the ECD policy and developed IECD training materials for 0-6 year olds. The interviews reveal that PCI is well-regarded by national government stakeholders because of their technical expertise; experience in the sector; and good understanding of government systems and processes. v PCIs work at national level has made an important contribution to the development, implementation and coordination of national frameworks and policies that address the needs of children and women improved. The capacity building and partnership approach adopted by PCI with their good understanding of government systems and strong training skills have enabled the positive changes. In addition, the funding provided by PCI for meetings and conferences has supported government coordination activities and processes, which have often not been budgeted for by departments. Whilst PCI has MoUs and joint TK work plans in place with some of the national departments, there was no lead department or coordinating body within government at national level leading the TK Project. This problem is mirrored at district level where coordination with government stakeholders is weak although some of the partners are working through the DMSAC. At local level, coordination is stronger as there is evidence of joint working together and referrals, however, the functionality of local structures varies from village to village. Capacity development support provided by PCI PCI training and capacity development was undertaken via a ‘cascade’ approach. This worked well in terms of budget and timeframe parameters, but it appears as though IP capacity and resources to adequately equip and follow up on non-attending staff and volunteers was often lacking. A key strength of PCIs approach to capacity development and support is that it was regarded as being relevant and highly collaborative. As such it encouraged PCI-IP and inter-IP contact and consultation. The allocation of a focal TA to oversee (backstop) specific IPs also appears to have been a sound strategy in that it personalised PCI contact with the IPs and encouraged relationship￾building. However, challenges in terms of TA capacity for follow up and mentoring were noted in this regard as well. Feedback regarding the content, facilitators and training methods utilised for capacity development was overwhelmingly positive. However, a number of challenges were noted by respondents regarding planning. These include a lack of adequate notification time prior to training sessions as well as limited supply and hence access to teaching and learning materials. The TK Project has definitely provided organisations with platform to grow in terms of reach, but also in terms of their organisational and technical capacity. The TK Project has led to an increase in knowledge and capacity amongst IPs, particularly around the facilitation, creation and support of economic strengthening interventions. This knowledge has led to improved levels of confidence and motivation to engage in such activities. However, this was discerned more amongst staff involved in GROW initiatives as opposed to those responsible for running Aflateen or YEP groups. The biggest shift for IPs in relation to IECD is that staff now understand the importance of providing integrated IECD services, and this had led to more holistic service provision around IECD. The TK Project has led to an increase in knowledge of implementing partners, particularly around gender related concepts and child and gender related legislation as this is a new area of work for them. There have also been a shift in attitudes and through the gender related programme and activities they now recognise the importance of linking gender and HIV. This has resulted in all partners integrating gender into their programme activities; being more skilled at identifying and vi responding to cases of GBV and child abuse; and generally improving the quality and reach of their services. Project management The TK project was quite ambitious in what it set out to achieve, given the human resource capacity afforded to this project at the level of PCI and IPs and the time frame of five years. The TK project would have benefited from more structured and combined programme planning which focussed on implementation and results. Monitoring and evaluation requirements and changes in these requirements was a major challenge that hindered implementation. Conclusion In evaluating the strategic approach and theory of change, the findings show that the TK Project is relevant to the needs of OVCs and their families1 and complements the services of government. The programme effectiveness could have been strengthened through the inclusion of particular vulnerable children like those on farms and streets; strengthening the support for establishing referral systems of IPs; and improving the capacity of the grantee to provide better support to IPs. Despite these capacity building challenges, the TK Project has provided organisations with a platform to grow in the quality and quantity of it services. The evaluation questioned whether the project has been implemented according to design. Generally, performance against activities set out in work plans and the co-operative agreement was varied. This is also hindered by reporting that is not aligned with work plans. However, in terms of achievement against Key Result Areas, results are positive. In terms of the likelihood of the project achieving their desired end results, there are signs of positive outcomes, across all objective areas, but there is insufficient evidence to be able to conclude on whether this will in fact lead to improved quality of life of OVCs. Key recommendations • There needs to be a purposeful effort by future grantees, IPs and the GoB to target children with disabilities, children on farms, children on streets and children who have dropped out of school in future OVC programmes. • Programme strategies for including men and boys need to be implemented or strengthened by future grantees, IPs and the GoB where they do exist. • The GROW model demonstrates a lot of potential and can be strengthened through consolidating linkages with existing government economic and livelihood programmes; the selection of appropriate IGAs; and better mentoring support for IGAs. • The Aflateen programme must be adapted by grantees to ensure that the content is relevant to the needs and interests of the youth. • For programmes such as the YEP, grantees should ensure that linkages with appropriate government and private sector youth empowerment interventions. 1 There is no formal theory of change or logic model, and so envisaged changes refers to those represented in Figure 2: TK OVC and Gender Project Theory of Change. vii • Grantees in partnership with implementing partners and the Department of Community Development should strengthen efforts to support DCPCs and VCPCs in the future. • There is still a critical need for community based safe spaces for GBV survivors particularly in rural, outlying areas. • It is recommended for PEPFAR 3 that a lead Ministry or department take responsibility for the strategic direction and coordination of the programme on policy level and programme level. • Nationally formulated plans are implemented at district and village/ward level. It is therefore recommended that, when replicating programmes of this nature, it is important to work midstream at district level. viii TABLE OF CONTENTS 1. INTRODUCTION.......................................................................................................................... 1 2. BACKGROUND............................................................................................................................. 1 2.1 THE TSELA KGOPO GENDER AND OVC PROJECT......................................................... 3 2.2 THE PERFORMANCE EVALUATION................................................................................... 3 3. EVALUATION METHODOLOGY AND SAMPLE........................................................................ 4 3.1 DATA COLLECTION............................................................................................................ 5 3.2 DATA ANALYSIS ................................................................................................................... 5 3.3 SAMPLING.............................................................................................................................. 5 3.4 LIMITATIONS......................................................................................................................... 7 4. FINDINGS: PROGRAMME DESIGN AND APPROACH.............................................................. 7 4.1 RELEVANCE ........................................................................................................................... 8 4.1.1 Addressing needs of vulnerable children and women........................................................ 8 4.1.2 Alignment to needs of government.................................................................................... 8 4.1.3 Alignment to needs of sub-grantees................................................................................... 8 4.2 THEORY OF CHANGE ......................................................................................................... 9 4.3 THE COMPREHENSIVE FAMILY CARE APPROACH ........................................................10 4.3.1 Description........................................................................................................................11 4.3.2 Implementation strengths and challenges .......................................................................12 4.4 LESSONS LEARNT...............................................................................................................15 5. FINDINGS: IMPLEMENTATION AND OUTCOMES................................................................. 15 5.1 OBJECTIVE 1: LIVELIHOODS.............................................................................................16 5.1.1 Project Implementation....................................................................................................16 5.1.2 Outcomes for beneficiaries...............................................................................................24 5.1.3 Key lessons learnt for objective 1 .....................................................................................26 5.2 OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS................................................................................................................................26 5.2.1 Project Implementation....................................................................................................26 5.2.2 Outcomes for beneficiaries...............................................................................................33 5.2.3 Key lessons learnt for objective 2 .....................................................................................35 5.3 OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN.....................................................................................................36 ix 5.3.1 Project Implementation....................................................................................................36 5.3.2 Outcomes for beneficiaries...............................................................................................46 5.3.3 Key lessons learnt for objective 3 .....................................................................................48 5.4 OBJECTIVE 4: NATIONAL FRAMEWORKS AND POLICIES............................................49 5.4.1 Project Implementation....................................................................................................49 5.4.2 Outcomes for objective 4..................................................................................................54 5.4.3 Key Lessons learnt for objective 4 ....................................................................................55 5.5 ENABLERS AND BARRIERS TO CHANGE........................................................................55 6. FINDINGS: STRENGTHENING THE CAPACITY OF IPS TO PROVIDE SERVICES.................57 6.1 OBJECTIVE 1: LIVELIHOODS..............................................................................................58 6.1.1 Capacity building support provided by PCI to IPs.............................................................58 6.1.2 Outcomes for IPs...............................................................................................................58 6.2 OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS................................................................................................................................59 6.2.1 Capacity building support provided by PCI to IPs.............................................................59 6.2.2 Outcomes for IPs...............................................................................................................59 6.3 OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN.....................................................................................................60 6.3.1 Capacity building support provided by PCI to IPs.............................................................60 6.3.2 Outcomes for IPs...............................................................................................................60 6.4 OTHER SUPPORT PROVIDED TO IPS...............................................................................62 6.5 STRENGTHS AND CHALLENGES OF THE SUPPORT PROVIDED BY PCI....................62 6.5.1 PCI capacity development and support structure ............................................................62 6.5.2 PCI approach .....................................................................................................................63 6.5.3 Content covered during capacity building sessions..........................................................64 6.5.4 Facilitation of capacity building sessions..........................................................................64 6.5.5 Provision of mentoring and support.................................................................................65 6.5.6 Logistics and planning.......................................................................................................65 6.6 KEY LESSONS LEARNT FOR CAPACITY DEVELOPMENT AND SUPPORT ..................65 7. FINDINGS: SUSTAINABILITY .....................................................................................................66 7.1 EFFORTS TO IMPROVE SUSTAINABILITY ........................................................................66 7.2 PERCEPTIONS OF SUSTAINABILITY.................................................................................66 8. FINDINGS: PROJECT MANAGEMENT ......................................................................................67 8.1 HUMAN RESOURCE CAPACITY OF THE TK TEAM .......................................................67 8.2 PLANNING AND CO-ORDINATION...............................................................................67 x 8.2.1 Planning process...............................................................................................................68 8.2.2 Documenting the Programme Logic Model......................................................................68 8.2.3 Internal co-ordination and integration.............................................................................68 8.3 MONITORING, EVALUATION, REPORTING AND LEARNING.....................................69 8.4 PARTNERSHIP BETWEEN PCI AND USAID .....................................................................70 8.5 CO-ORDINATION WITH GOVERNMENT STAKEHOLDERS ........................................71 8.5.1 National.............................................................................................................................71 8.5.2 District...............................................................................................................................72 8.5.3 Village/Ward .....................................................................................................................72 8.6 LESSONS LEARNT FOR PROJECT MANAGEMENT.........................................................73 9. CONCLUSION ............................................................................................................................74 10. RECOMMENDATIONS ...........................................................................................................77 10.1 PROGRAMME DESIGN AND APPROACH........................................................................77 10.2 OBJECTIVE 1: LIVELIHOODS..............................................................................................77 10.3 OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS................................................................................................................................79 10.4 OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN.....................................................................................................79 10.5 OBJECTIVE 4: NATIONAL FRAMEWORKS AND POLICIES...........................................81 10.6 CAPACITY DEVELOPMENT SUPPORT PROVIDED BY PCI ............................................82 10.7 PROJECT MANAGEMENT ..................................................................................................82 TABLE OF FIGURES Figure 1: Evaluation Process Diagram..................................................................................................... 4 Figure 2: TK OVC and Gender Project Theory of Change........................................................................ 9 Figure 3: The CFC Approach..................................................................................................................11 TABLE OF TABLES Table 1: Sample and data collection methods........................................................................................ 5 Table 2: Key indicators for TK project..................................................................................................16 Table 3: Key indicators for Objective 1 ................................................................................................16 Table 4: Key indicators for Objective 2 ................................................................................................27 Table 5: Key indicators for Objective 3 ................................................................................................36 xi Table 6: Summary of national frameworks, guidelines and policies that have been developed with the assistance of PCI .............................................................................................................................49 xii LIST OF ACRONYMS BAPR Botswana Association for Psychosocial Rehabilitation BB Building Bridges BBM Bakgatla Bolokang Matshelo BNSC Botswana National Sports Council BOCODOL Botswana College of Distance Learning CFC Comprehensive Family Care CITF Construction Industry Trust Fund CRS Catholic Relief Services CSI Child Status Index CSO Civil Society Organization DAC District AIDS Coordinator DC District Commissioner DCD Department of Community Development DCPC District Child Protection Committee DHMT District Health Management Team DMSAC District Multisectoral HIV/AIDS Committee DSP Department of Social Protection DSS Department of Social Services DYC District Youth Council ECCE Early Childhood Care and Education GeAD Gender Affairs Department GBV Gender Based Violence GROW Grass Roots building our Own Wealth HOH House of Hope HPP Humana People to People HWW Hope World Wide Botswana IECD Integrated Early Childhood Development IP Implementation Plans JAB Junior Achievement Botswana JOL Journey of Life xiii KWSP Kagisano Women’s Shelter Project KyT Kgetsi Ya Tsie LEA Local Enterprise Authority MCCNT Marang Child Care Network Trust MLHA Ministry of Labour and Home Affairs MLG Ministry of Local Government MLGRD Ministry of Local Government and Rural Development. MOE&SD Ministry of Education & Skills Development MoH Ministry of Health MU Mother’s Union MYSC Ministry of Youth, Sports and Culture NACA National AIDS Coordinating Agency NGO Non-Governmental Organization OVC Orphans and Vulnerable Children TA Technical Assistance TOT Training of Trainers YAL Young Africa Live xiv 1. INTRODUCTION USAID/Botswana’s PEPFAR (Presidency Emergency’s Plan for AIDS Relief) programme seeks to strengthen local health service-provider organisations in Botswana to improve health outcomes among beneficiaries and communities2. To this end, USAID supported two key activities in Botswana: The Maatla Civil Society Strengthening Project, and the Tsela Kgopo Orphans and Vulnerable Children (OVC) & Gender Project. In June 2011, Project Concern International (PCI)3 was awarded a five year grant to implement the Tsela Kgopo (TK) Project. The goal is to improve the quality of life of OVCs and their caregivers and improve gender based responses and programming in HIV activities in Botswana. Southern Hemisphere was commissioned to conduct a performance evaluation of the Maatla and TK Projects. The evaluation was conducted between March and October 2015. The findings of the TK evaluation are contained in this report. The report starts with a brief overview of the Tsela Kgopo project. The evaluation design, methodology and limitations are then briefly described. The findings are covered in the following five sections (PROGRAMME DESIGN AND APPROACH, IMPLEMENTATION AND OUTCOMES, STRENGTHENING THE CAPACITY OF IPS TO PROVIDE SERVICES, SUSTAINABILITY, PROJECT MANAGEMENT). The report ends with conclusion and recommendations. 2. BACKGROUND While HIV prevalence (18.5%) and HIV incidence (1.35%) remains high amongst the sexually active population in Botswana, progressive public health interventions and economic policies have propelled the country to Upper Middle Income Status4 . These gains are however not equally enjoyed across the entire population of Botswana, and certain groups remain vulnerable. Botswana’s income inequality remains high, with the poorest 40% earning only 11% of the country’s total income. Poverty is more concentrated in rural areas and in female (and sometimes girl child) headed households5 . HIV is also spread differently across the country. Urban areas have higher incidence rate of HIV than rural areas; estimates suggest 17.5% across all types of urban areas compared to 15.8%. Districts also vary in the HIV prevalence, with the following having the highest HIV prevalence: Selebi-Phikwe, Francistown, Central-Mahalapye, Kweneng East, Barolong, Central-Boteti, Kgatleng, Sowa, Central-Bobonong and Ngwaketse West6 . In 2007, almost 22% of children in Botswana had lost one or both of their parents. HIV and AIDS has been one of the contributing factors to the number of children left orphaned. Children (under the age of 18 years) are also made vulnerable to neglect, abuse, violence and exploitation through 2 RFTOP Botswana Evaluation, 2014. Page 3 3 Sometimes referred to as the Grantee. 4 Statistics Botswana (2014) BAIS IV, Human Development Report, 2014. 5 UNICEF Botswana, 2011; Annual Report 6 Statistics Botswana (2013), Preliminary AIDS impact survey IV (BAIS IV), 2013 [online: http://www.cso.gov.bw/templates/cso/file/File/BAIS%20IV%20PRELIMINARY%20RESULTS%20Nov%202013(1). pdf] 1 living in an abusive environment; a poverty-stricken family that is unable to access basic services; a child-headed household; living with sick parents or outside family care; or living with HIV7 . These vulnerable children are included in the group of OVC. About half (49%) of households with vulnerable children receive assistance from the state through the National Orphan Care Programme, which ensures that these children are provided with food baskets and have access to education, security, provision of psychosocial support and alternative care. Aside from government efforts, NGOs and CBOs are also assisting OVC8 . Youth are another vulnerable group. In 2011, national unemployment was at 18%; however, this figure was almost double (34%) for those 20-249 . Substance abuse is a challenge for the youth in the country. According to a study by Campbell (2003), the consumption of alcohol leads to increased frequency of sex among youth in the country; males who drink at least once per week having double the amount of sex than those who are teetotal. Additionally, alcohol intoxication negatively influences the use of condoms10 Women are also particularly vulnerable, with unemployment for females at 21%, compared to 15% for males. 46% of female headed households suffer from poverty; this can again be compared to male-headed households in which poverty is a concern in 27% of households11. Women and girls are often the main victims of gender based violence; that is “violence that … includes …physical, mental or sexual harm or suffering, threats of such acts, coercion and other deprivations of liberty…” (UN definition). A study conducted by Kagisano Women’s Shelter in 2012 indicated that 67% of women in Botswana have experienced some form of GBV during their lifetime12. Although there is increased emphasis placed on the importance of IECD, relatively few are able to access early childhood education. For males of pre-primary school going age, the gross enrolment ratio of pre-primary school between 2008 and 2012 was 18.7%; this figure was 19.2% for females13. Furthermore, between 2002 and 2012, 9% of those between 9 and 14 years were involved in child labour14. Policy restrictions that hinder the set-up of ECD centres, a limited understanding of the value of early learning and stimulation, limited investment in IECD, and inadequate skills to provide IECD are among some of challenges in the provision of quality IECD services (evaluation interviews). The vulnerability faced by the groups highlighted above is compounded when living in remote or hard to reach areas, as services are more difficult to access due to geographic distance and also lack of information about services. 7 Feranil, I., Herstad, B., Jallow, W., & Mbuya-Brown, R. (2010). Assessing implementation of Botswana’s program for orphans and vulnerable children. 8 Malinga, T., & Ntshwarang, P. N. (2011). Alternative Care for Children in Botswana: a Reality or Idealism? Social Work & Society, 9(2). 9 UNDP Botswana, 2014 10 Campbell, E. (2003). Note on Alcohol Consumption and Sexual Behaviour of Youths in Botswana, African Sociological Review, 7(1), pp.146-161. 11 UNICEF Botswana, 2011; Annual Report 12 Baseline Assessment Report on Gender Based Violence in (Sebina, Ghanzi, Kassane and Ramotswa) 2013, Kagisano Women’s Shelter, 2013 13 UNICEF Botswana Statistics, 2013 [Online: Retrieved 13 October 2015 http://www.unicef.org/infobycountry/botswana_statistics.html 14 UNICEF Botswana Statistics, 2013 [Online: Retrieved 13 October 2015 http://www.unicef.org/infobycountry/botswana_statistics.html 2 It is within the context of these challenges, that the TK OVC and Gender Project, targeting particularly vulnerable women, youth, children, men (all referred to as beneficiaries) and households in remote areas were developed. 2.1THE TSELA KGOPO GENDER AND OVC PROJECT The goal of the Tsela Kgopo Project is to improve the quality of life of OVC and their parents/caregivers and improve gender-based responses and programming in HIV/AIDS activities in Botswana. The programme seeks to achieve this through four key objectives: 1) Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women, improved. 2) Developmental interventions for vulnerable children and adolescents improved. 3) Risk of neglect, exploitation and abuse of vulnerable children, adolescents and women reduced. 4) Development, implementation and co-ordination of national frameworks and policies that address the needs of children and women improved. PCI provides technical and financial support to ten Implementing Partners (also referred by USAID as sub-grantees) operating in nine districts with a total of 16 implementation sites. These implementing partners are: Catholic Relief Services (CRS), Hope Worldwide Botswana (HWW), Humana People to People (HPP), Marang Child Care Network (MCCNT), Kagisano Women’s Shelter (KWS), Bakgatla Bolokang Matshelo (BBM), House of Hope, Mother’s Union, Kgetsi Ya Tsie and Botswana Association of Psychosocial Rehabilitation. PCI develops the capacity of IP staff; who are then tasked with transferring this capacity to implementers (volunteers, community facilitators, etc.) who then support OVCs, youth and their caregivers. PCI also collaborates with key government ministries and departments including Gender Affairs Department (GeAD); Department of Social Protection (DSP) and Department of Community Development (DCD); Ministry of Education & Skills Development; and coordinating structures at the district and village levels. PCI also partners with private sector organizations to support and sustain project activities. 2.2THE PERFORMANCE EVALUATION The purpose of the evaluation is 15: 1. to assess how well the activities are being implemented so far, whether according to plan 2. to inform programme design for next round of USAID supported programmes, and 3. to contribute to evidence of what works or doesn’t work in improving health outcomes among beneficiaries and communities – lessons learned. The key evaluation questions are: • What modifications, if any, to the strategic approach and theory of change have potential to substantially increase program effectiveness? 15 Inception report, 22 July 2015. 3 • To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? • How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? • To what extent has USAID fulfilled its role in the activities by providing adequate funding, responsive managerial oversight and technical support, and assistance in solving problems in implementation as they arise? • What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The primary audience of the evaluation is USAID/Botswana and its implementing partners. A secondary audience of the evaluation is the Government of Botswana and other stakeholders, such as other US government agencies, bilateral and multilateral donors, and other NGOs. The evaluation statement of work can be found in Appendix 1. 3. EVALUATION METHODOLOGY AND SAMPLE This section provides an overview of the methods used and sampling undertaken for the TK evaluation. The detailed evaluation methodology can be found in Appendix 2. A qualitative research design was used for this performance evaluation. The process diagram below outlines key steps in the implementation of this evaluation: Figure 1: Evaluation Process Diagram An initiation workshop was conducted by Southern Hemisphere with USAID and PCI. The purpose was to allow for discussion of overall evaluation objectives, research questions, methodology and logistics. This informed the design and roll out of the evaluation. The document review then commenced16 and a total of 11 research instruments were designed. On the first day of fieldwork an evaluation planning meeting was conducted with USAID and PCI to finalize all fieldwork arrangements. This was followed by fieldworker training and data collection, which is further detailed below. Data was then analyzed using data analysis software and findings presented to USAID and PCI. The report was then finalized based on verbal and written input from USAID and PCI. 16 List of documents can be found in Appendix 3. Initiation workhop and evaluation design Document review Instrument Design Evaluation planning meeting with PCI & USAID Fieldwork Analyse data (NVIVO) Draft report writing Presentation of findings Final report writing 4 3.1DATA COLLECTION Instruments were developed in consultation with USAID and PCI. Instruments can be found in Appendix 4. Ethical clearance was obtained for the study from the Ministry of Health Chief Research Officer, Health Research Division, Department of Health Policy Development, Monitoring and Evaluation. Ethical guidelines followed included informed consent to participate in the evaluation (which was obtained through consent and assent forms), confidentiality of information shared, anonymity in reporting process and counseling, which was offered where necessary to research participants who felt emotionally sensitive following interviews. Data was collected using semi-structured interviews (SSIs) and focus group discussions (FGD). Generally focus group discussions were held with between five and ten participants, and were conducted at implementing staff and beneficiary levels. Table 1: Sample and data collection methods provides an overview of the key stakeholders interviewed, the data collection methods used, and the purpose of interviews. Data was collected by two consulting teams, each made up of one South African and one Motswana consultant. 3.2DATA ANALYSIS Data was coded using NVIVO 10 for qualitative data analysis. This data analysis software was used to conduct a thematic analysis of qualitative data. Themes, classified as nodes in NVIVO, were based on the objectives of the project and key evaluation questions. Coding was done deductively based on predetermined themes, but with some flexibility to add nodes to fit emerging data. The findings are discussed in light of the strengths and challenges for each objective area. The frequency of mention informed how findings were prioritized for inclusion into the report; however, if an issue was deemed by the evaluator as being important (i.e. offering a unique or pertinent point of view), having lower frequency did not necessarily mean that it was omitted. 3.3SAMPLING The table below provides an overview of the sample and data collection methods. As illustrated, interviews were conducted at a national level with USAID, PCI and National Government of Botswana stakeholders. At a district level, interviews were conducted with key community structures and district government stakeholders whom PCI was meant to have interacted (through support or partnerships) with. Implementing partners were interviewed both at the level of management and implementers. Lastly, 63 beneficiaries (of which five were male and 58 were female) targeted through the programme were also interviewed through SSIs and FGs. Table 1: Sample and data collection methods Level Stakeholder Method Purpose of interview No. of interviews National Key informants: USAID, PCI SSI17 To understand the programme design and key strengths and 12 17 Semi-structured interviews. 5 challenges in implementation. Government (Department of Community Development; Ministry of Education and Skills Development; GeAD) SSI To explore the strengths and challenges in the support provided to key GoB partners as well as the policy or other outcomes achieved. 3 District Government/stakeholders (S&CD, District Health Management Teams, Gender Officers, District Child Protection Committee Co-ordinator, District AIDS Co-ordinator, Men’s Sector, Police) SSI To explore the strengths and challenges in the support provided by PCI to District structures. 15 Implementi ng IPs Management staff (Directors, Programme Managers, Programme Officers/Co-ordinators) SSI To understand the strengths and challenges in the capacity building support provided by PCI to IPs, as well as the enablers and barriers to implementing services/models advocated for by PCI. To explore the organizational outcomes achieved through the programme. 13 Implementing staff (e.g. facilitators and volunteers) SSI To understand the strengths and challenges in the capacity building support provided by PCI to IPs, as well as the enablers and barriers to implementing services/models advocated for by PCI. To explore the organizational outcomes achieved through the programme, as well as the observed outcomes for beneficiaries. 318 FGD19 Same as above. 520 Beneficiarie s Recipients of home based, economic strengthening, developmental and gender interventions. SSI To understand the strengths and challenges in the models used and support provided by IPs. To explore the emerging outcomes for beneficiaries. 821 FG Same as above. 822 TOTAL FG = 13 18 Two GROW Facilitators and one Youth Facilitator. 19 Focus Group Discussions. 20 IECD Volunteers, Economic Strengthening staff (GROW and youth), GROW Facilitators, Youth Volunteers, IGA Volunteers. 21 2 x Economic Strengthening, 2 x Home-Based Care, 2 x IECD, 2 x GBV. 22 2 x HBC, 3 x GROW, 2 x YEP and Aflateen, 1x GBV. 6 SSI = 59 A non-probability sampling method was used for the selection of implementing partners and participants of this evaluation. This sampling method allowed for key interest groups/partners to be selected based on their ability to best fulfil the information needs of evaluators. Districts were purposefully selected due the density of implementing partners (sub-grantees) sites in these areas, possibility of covering both rural and urban areas23, and cost. Implementing partners were purposefully selected based on: Coverage of the Districts (based on what is logistically possible in order to save cost); partners who had the most coverage in terms of numbers reached; and ability to cover the three objective areas relevant for IPs (i.e. objectives 1, 2 and 3). The five IPs covering these criteria were: Hope Worldwide (HWW), Kagisano Women’s Shelter (KWS), Bakgotla Bolokang Matshelo (BBM), Humana People to People (HPP) and Catholic Relief Services (CRS). 3.4LIMITATIONS The use of a non-probability sample means that the findings from this evaluation cannot be generalized to the entire project or all beneficiaries. Two interviews were not conducted at Ministry level (with Ministry of Health and Department of Social Protection) due to lack of availability of the sampled interviewees. Interviews planned for District Child Protection Committees did not take place due to District CPCs either being non-functional or having minimal interaction with the Implementing Partner. As beneficiaries who had “dropped out” of the programme were not interviewed; the evaluation lacks their perspectives on the effectiveness of the TK Project. The Theory of Change is not adequately documented through a TK Project logic model. This makes it challenging to report on key deliverables and outcomes. Annual reports are not always aligned to work plan activities, making it challenging to report on planned vs actual activities. There is no work plan available for the second reporting period making it difficult to assess planned versus actual achievement of activities for this period. Technical capacity assessments (TCA) of the five sampled organizations were conducted to validate the quality of previous capacity assessments, and compare change in the capacity of implementing partners over time. However, this data was excluded from the report, as there is no TCA data previously collected to compare current results to. The design of questions in the TCA data collection tool also had limitations which made its usefulness, beyond using it for comparative purposes, questionable. 4. FINDINGS: PROGRAMME DESIGN AND APPROACH This section focusses on whether the TK Project was well conceptualized to bring about change in the quality of life of OVC. It assesses project relevance, overall theory of change and the use of the comprehensive family care approach. The key evaluation question is: 23 Five villages (Molepolole, Makanke, Mosokotso, Muchidi village, Sebina Site) and three urban sites (Gaborone, Selebi Phikwe, Francistown) were covered in total. 7 What modifications, if any, to the strategic approach and theory of change have potential to substantially increase program effectiveness? 4.1RELEVANCE This section assesses whether the project was aligned to the needs of community members, Government of Botswana and Implementing Partners. 4.1.1 Addressing needs of vulnerable children and women Interviews considered the TK Project interventions as relevant to the challenges facing women and OVCs. The project reportedly reaches some of the most vulnerable (in terms of poverty, HIV and OVC) districts. There is still however more demand to reach other vulnerable groups, which will only be possible with more financial resources. Three of the organisations sampled have reached and benefited children with disabilities, although this is not consistently done across all organisations. People with disabilities are still stigmatised by their families and communities, and staff do not always feel adequately skilled to deal with this. Children living on farms and on streets (particularly in Gantsi District) are considered to be among the most vulnerable children, however they are not targeted through the programme. Likewise children who have dropped out of school, for example young teenage mothers, are not considered by a few interviewees to be adequately targeted by the TK Project. While some organisations are targeting men, this is still a fairly new area being explored, and will need to be strengthened in other OVC and Gender Programmes. 4.1.2 Alignment to needs of government Across national and district government interviewees, the TK Project is considered to be well aligned with government priorities, particularly Vision 2016, the National Home Based Programme, and the National Programme on Poverty eradication. It is based on limitations in policy and gaps in service delivery, as identified through the situation analysis commissioned by PCI. This was a good foundation for the plan and helped facilitate relationships at national level. Both government officials and IPs felt that they had a complementary working relationship, and that IPs assisted government to reach areas that were difficult to access: “Hope implements programme that are key priorities identified by Government. They go beyond the basic interventions to cover needs that are not sufficiently reached by Government …all their interventions augment government initiatives and fill in gaps where government officers cannot effectively reach the communities.” (Gov_DAC) “It is aligned because it is deriving its activities and objectives from the national priorities and programmes. It is very relevant to us as government” (GeAD National). 4.1.3 Alignment to needs of sub-grantees The TK Project included IPs already working in the area of HIV, OVCs or Gender. These organisations were assessed to determine their capacity needs, and the programme was then designed to meet their organisational and technical capacity needs. 8 Many of the IPs lacked sufficient resources and staff to expand their services and the TK Project provided the opportunity to upscale. The introduction of new approaches and models (e.g. CFC and GROW) strengthened the quality of work of IPs, thus meeting their technical capacity needs. 4.2THEORY OF CHANGE A Theory of Change is an expression of the connection between programme activities and outcomes, and the articulation of the theoretical and contextual assumptions underlying this connection24. No formal theory of change exists, and so the description below is based on interviews and the evaluator’s interpretation of the Theory of Change. Figure 2: TK OVC and Gender Project Theory of Change PCI increases the technical capacity of Civil Society Organisations (CSO) and the Government of Botswana (GoB) to provide services to improve the quality of life of orphans and vulnerable groups (children, adolescents, and women) in Botswana. The support to implementing partners (IP) is provided through: • developing strategies (e.g. Comprehensive Family Care) and tools (e.g. Journey of Life; etc.) for community engagement; • providing capacity development support (training and mentoring) to CSOs who are their IPs and; • providing funding support. PCI also provides technical assistance and financial support to the GoB around the development and implementation of national frameworks relevant to women and OVC. 24 Southern Hemisphere, Theory of Change Training Manual. USAID via PCI provides technical capacity support to CSOs and GOB including training and mentoring CSOs and GOB have increased technical capacity (knowledge, skills, tools, funding) Better policies by GOB CSO improve quantity and quality of services provided improved access and quality of services Improved quality of life for OVCs 9 Through strengthening of civil society organisations, PCI expects to see an improved capacity among CSOs to provide support to OVCs, youth and women. This should improve the quality and reach of services to these vulnerable groups. The technical support provided to the GoB seeks to contribute to a policy environment that is sensitive to the needs of OVCs and women. The Tsela Kgopo Project thus seeks to intervene at different levels of the ecological system, working with civil society organisations to reach individuals, families and communities, as well as with the Government of Botswana to create an enabling policy environment25. This perspective “recognises that violence is not the result of a single factor, but rather the outcome of a complex interplay of individual, relationship, community, and societal factors.”26 In this way the Tsela Kgopo Project seeks to have a more systemic and sustained impact. Two key assumptions underlie the attainment of the change the Tsela Kgopo Project wishes to bring about at the level of Government and Civil Society. The one is that building supportive partnerships are necessary at the level of Government to bring about the necessary policy changes. The other assumption is that a comprehensive approach is required to bring about changes to the quality of life of OVCs as “HIV affects families not just individuals, services need to be targeted to the entire family rather than infected/affected individuals” (Country Director 2007 – 2012, PCI Botswana). When assessing a Theory of Change it is important to look at whether the right components were in place to bring about the envisaged changes. The TK Project has four key causal pathways for addressing the needs of OVCs which are well aligned to the ecological systems approach: • Micro level: OVCs and women are at the centre of interventions, as most services either target them directly or seek to change the environment towards benefiting them. • Meso level: Families are targeted through the CFC approach which follows OVCs and women into their homes to help facilitate services and support to meet the needs of the family. • Exo level: Community structures (including government structures, traditional structures, co￾ordination structures, schools and clinics) are targeted to form a collaborative relationship with IPs towards creating a supportive and involved community around OVC and gender issues. • Macro level: Policy level interventions to ensure an enabling environment that puts the needs of women and OVCs at the centre of policy decisions. These causal pathways are well conceptualised to tackle challenges facing OVCs from a social￾ecological perspective. 4.3THE COMPREHENSIVE FAMILY CARE APPROACH This section provides an overview of the Comprehensive Family Care approach and its strengths and challenges. The CFC model informs the approach taken by PCI and IPs in interacting with communities across the first three TK Project objectives. 25 Dawes, A, Willenberg, I and Long, W. (2006) “Indicators for child protection” Report for the research directorate, Department of Social Services and Poverty Alleviation, Provincial Government of the Western Cape, Child Youth Family and Social Development, Human Sciences Research Council (HSRC); Preventing Child Maltreatment, WHO and ISPCAN, 2006 26 South African Child Gauge 2014, Children’s Institute, University of Cape Town 10 4.3.1 Description The Comprehensive Family Care (CFC) approach was developed by PCI with the intention of improving the quality of HIV and AIDS prevention, care and support to families. It is “a holistic approach to provision of services that address the multiple needs of vulnerable individuals and their families. Vulnerable individuals may include OVC as defined by the OVC National Guidelines, HIV infected and affected adults, the elderly, people with disabilities and people living in absolute poverty.”27 The approach was based on the Family Care Model (FCM) developed by Ministry of Local Government, which was simplified and adapted to suit the requirements of civil society organisations. The diagramme below summarises the CFC process28. Figure 3: The CFC Approach 27 Draft CFC Description, PCI. 28 Draft Comprehensive Family Care: A guideline for Civil Society Organisations. November 2014. 11 The CFC process starts with identification of vulnerable families through home visits (identified through door to door), existing “index clients” who are receiving services (for example, GROW participants), community mobilisation activities and referrals from other service providers (e.g. schools, VDCs, S&CD, etc.). The needs of all family members are then assessed using the Household Assessment (Tool 1), Child Status Record (Tool 2A), and Adult Status Record (Tool 2B) which relies on interviewing and observation methods. The family needs are then prioritised and an action plan is developed to address these. This leads to direct service provision or referrals for service provision, and then quarterly (or as required) follow up home visits thereafter. Direct service provision either takes place within the home or through group interventions. The effective implementation of CFC is dependent on the following29: • Skills development opportunities for family members (in the home or through groups) • Understanding and linking with community support systems (including both CSOs, community structures and government) for the provision of services • Community mobilisation to ensure that communities drive the care for vulnerable families (often the Journey of Life is used here) • Comprehensive set of process and technical skills for CSO staff A key feature of the CFC, as illustrated in the points above, is that communities have strengths and resources that should be harnessed towards addressing their own developmental needs. 4.3.2 Implementation strengths and challenges Perceptions of the CFC model There is generally an appreciation for the usefulness and effectiveness of the CFC model among all stakeholders (IPs, PCI and USAID). The strengths of the methodology is perceived to lie in: • The ability to access vulnerable people using the home visits • The identification of the holistic needs of families, not just individuals, which is perceived to be more effective • the inclusion and empowerment of communities The following extract expresses this appreciation for the methodology: “Comprehensive family care model: this has been the best project I have worked in. HWW has been working in Botswana for the past 10 years. It was the first time that we felt we are making a difference for the family and not just the child…CFC is a star model for us. I have seen that with CFC you can reach everyone in the community. Even GROW women will talk about CFC without naming it…even our women are thinking holistically about their families’ needs” (IP2) Capacity development for IPs to use CFC As the CFC approach places high demands on the skillset of CSO staff/volunteers, capacity development was provided by PCI over one to five weeks in the use of this approach. 29 Draft Comprehensive Family Care: A guideline for Civil Society Organisations. November 2014. 12 PCI trained IP staff (site manager, projects officer, etc.) with the expectation that this will be cascaded to those responsible for implementing the CFC (e.g. volunteers, community mobilisers, etc.). When cascaded to implementer levels though, training was sometimes conducted in under days, with 30 min being reported as the minimum time. Where a low dosage of training was administered, this is a concern for the quality of support provided by volunteers. Identifying families that need support Identification of vulnerable households is done by community volunteers (or facilitators, mobilisers) operating under the management of IPs, or community structures (including VDC, church, clinics, hospitals, schools guidance and counselling teachers, etc.). One of the five IPs sampled however show that this is not always consistently done, as some facilitators and beneficiaries report still supporting individuals in isolation of their families: “Sometimes in a group setting we can forget that there are vulnerable and disadvantaged children among a group. Sometimes it’s because we don’t know a child’s home situation, where they come from.” (IP1) This may suggest that there has been insufficient capacity development around the CFC, or that facilitators see the “home visit” service as entirely separate to the group based services. Organisations, such as HWW, identify vulnerable families using various entry points, one of which is home visits, but also through group interventions. This should be adopted by all IPs. Home visits are widely considered to be useful for identifying vulnerable families that would otherwise not have had access to services: “TK had these assessments, house to house, for identification of vulnerability. That, to me, is something that needed to be scaled up. When people are affected by social issues, it is very difficult for them to come out voluntarily and seek a service. That is why to go there (to those need the service) and identify them and facilitate access to services, is good.” (IP5_Implementing staff) Volunteers play an important role in the identification of families in need of support. In most instances volunteers are an important and trusted resource in the community, and are approached by community members for advice or services. Volunteers sometimes struggle with gaining acceptance by communities due to not being liked, or being young. The reputation of IPs in communities plays a role in facilitating access to households. IP are generally well known in the community and this enables service delivery. There are however cases where the lack of community knowledge of the programme hinders participation. Assessing family members, identifying priority needs and develop an intervention plan All Implementing Partners sampled indicate using the CFC tools and home visits to assess and prioritise family needs. The CFC tools enable a holistic assessment of all members of a vulnerable family. Volunteers consistently raise two key challenges about CFC assessment tools: the assessment process, especially Life Changing Index tool (15 pages) is quite lengthy and some questions are considered repetitive. The questions, particularly those in relation to food security, are perceived as 13 being insensitive to poor households and also raise expectations about how Implementing Partners can assist. “It is very long. It asks about food and people are starving. Other families go to bed without eating. It is embarrassing to ask…it feels like we have no consideration for these families… If it has to ask about food, ”IP 1” must have food parcels for desperate families” (IP1_Implementing staff) Provision of services Three mechanisms are predominantly used for the provision of services: 1) direct services through group interventions; 2) direct services through home based interventions; and 3) facilitation of service provision through referrals. The table below provides a description of home based and group interventions. Home based interventions take the form of psychosocial support, education (particularly around HIV prevention, testing and treatment and healthcare) and health monitoring. Group based interventions take the form of GROW, YEP and Aflateen Groups for economic strengthening; Kids Clubs for HIV, gender and life skills; Stimulation and play groups and preschools for IECD; and gender and child protection is offered through community conversations, gender norms, men’s group interventions. While group based interventions provide the opportunity for building social cohesion, home based interventions offer clients the opportunity for dealing with more personal and household challenges. Kagisano is the only organisation that provides individual centre based interventions (psychosocial support) in relation to Gender Based Violence (GBV). Although the intention of the CFC approach is to directly provide services or refer as needed, most organisations seemed to feel pressurised to directly provide services. It is unclear where exactly the message was lost, that is whether expectations were misinterpreted or miscommunicated. The requirement of IPs to report on PEPFAR indicators also created a sense of pressure to directly provide services. Organisations were often overwhelmed by the task of “trying to address the entire family’s needs.” (IP2_Management). Referrals and follow up IP most commonly refer to clinics and hospitals for HIV and health issues; Social and Community Development Officers for poverty alleviation and food baskets; and the police for cases of Gender Based Violence. Very few IPs mention referring to other CSOs and other IPs. A referral form is used, although it seems that forms are not standardised across partners. Only one organisation mentions a good work relationship with organisations in their referral network. Other organisations report challenges with referrals especially to government structures. Often a family is assessed using the CFC tools, and referred to the S&CD Officer who then reassesses using their own tools, or has a different idea about the needs and priorities of the community. Service delivery is then delayed which is not only frustrating for volunteers and the client, but also gives clients little faith in the support that is provided by volunteers: “I referred her to S&CD 2 years ago. There has been no help. They ask her to come back the following days or the following week. She has stopped going there. She is tired and has given up on life” (IP3) 14 The movement of government officers into different departments or positions undermines efforts made to solidify relationships between IPs and government. To facilitate good referral networks, PCI invited partners to training, encouraged partners to refer to government partners, and has facilitated introductions between IPs and government. Follow up and feedback on referrals is also problematic. Clients lose referral forms, which makes it challenging to determine whether and how a service was delivered. Feedback from service providers is sometimes only possible through formal processes like the multisectoral meetings. The HWW model of ‘taking government services to the community’ is an innovative approach addressing these gaps and improving access to services. Once per quarter, different departments visit communities to provide a range of services such as registration or clinic services. 4.4LESSONS LEARNT The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to the strategy and approach:  Children with disabilities, children on farms, children on streets and children who have dropped out of school are not easily accessible, but are particularly vulnerable and therefore need to be targeted in future OVC and gender programmes.  Developing OVC and gender programmes based on the needs and gaps in policy and service delivery, as identified by government, enables relevant programme design and is a good platform for initiating relationships at national government level.  Home visits are crucial in the identification of vulnerable households.  There needs to be clear strategy for how comprehensive services are to be delivered. The mechanisms for service delivery (i.e. home-based interventions, group-based interventions, and referrals by IPs) and core service areas need to be clear to avoid IPs from being overstretched.  The provision of comprehensive services is dependent on a good referral system with government and other CSOs.  Community involvement – through community mobilisation methods – is time-consuming.  Relying on clients to keep a paper trail of referrals is a challenge as clients very often lose referral forms, which is a problematic for tracking service delivery. 5. FINDINGS: IMPLEMENTATION AND OUTCOMES This section assesses the achievement of planned activities and emerging outcomes for each Objective under the TK project. Even though outcomes are reported per objective area, it is important to acknowledge that the integrated approach of PCI means that outcomes cannot necessarily be attributed to the 15 intervention offered within one objective area only. Outcomes, particularly for youth and those related to gender, should thus be understood as the combined effort of the team. The table below provides the data for key indicators for the TK project. Disaggregated data can be found in Appendix 5. Data for each objective area can be found in the sub-sections that follow. Table 2: Key indicators for TK project Indicator Cumulative target Cumulative total % of target reached Number of eligible adults and children provided with a minimum of one care service 31783 3083430 97% Number of HIV positive adults and children receiving a minimum of one clinical service31(PEPFAR C2.1.D) 2931 5056 172.5% 5.1OBJECTIVE 1: LIVELIHOODS 5.1.1 Project Implementation This section on project implementation evaluates the actual activities against planned activities, and also assesses the strengths and challenges experienced in the implementation of Objective 1: Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women improved. The key evaluation question is: To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? The data on planned vs. actual activities is mostly extracted from annual reports, and to some degree from primary data collection. The detailed list of planned and actual activities for each KRA for Objective 1 can be found in Appendix 6. Primary data collected through evaluation has mainly provided the data on strengths and challenges. The table below provides the data for key indicators for Objective 1. Disaggregated data can be found in Appendix 5. Table 3: Key indicators for Objective 1 Indicator Cumulative target Cumulative total Total % of target reached Number of eligible children provided with education and/or vocational training. (PEPFAR C5.4D) 11179 8976 80.2% Number of eligible adults and children provided 3958 4054 102% 30 The data is exclusive of HPP Mabutsane which had not submitted by the reporting deadline for the 1st year 31 Not reported on in all Annual Reports 16 with economic strengthening activities. (PEPFAR C5.7D) Number of persons trained in a community mobilization methodology (Tsela Kgopo 1.1.4) 1265 1025 81% Number of people reached by an individual, small group, or community-level intervention or service that explicitly aims to increase access to income and productive resources of women and girls impacted by HIV/AIDS32 (PEPFAR P12.4.D) 3411 5553 162.79% Key result area 1.1: Community structures and linkages to relevant services Planned activities versus actual activities The aim of key result area (KRA) 1.1 is to ensure that community structures, including Dikgosi33, Village Development Committees (VDCs), VCPCs, DCPCs, volunteers, and faith-based organisations, are equipped to identify the most vulnerable households and children, and refer / link them to relevant CSO and government service providers. The cooperative agreement notes that activities within this KRA would be underpinned by a number of different strategies and methodologies. These are listed below: • Community mobilisation methodologies including the Journey of Life (JoL)34 and Community Capacity Enhancement through Community Conversations35 (CCE-CC), which were later combined. • The Comprehensive Family Care model (as described in previous section). • Facilitation of service partnerships between CSO and government through a) joint training of CSO and district government and b) the facilitation of regular service provider dialogues. Other planned activities in the cooperative agreement include: • Capacity building of DCPCs and VCPCs to further strengthen identification and referrals, and • The development of criteria for the identification and selection of the most vulnerable clients for economic strengthening interventions. 32Only reported on in two Annual reports 33 Traditional leaders 34 Journey of Life (JoL) aims to improve the identification and support of vulnerable children and households, plus referral of vulnerable community members for relevant service provision. Through the use of this methodology, communities are empowered to organise and mobilise resources within their communities, and compile community action plans for sustainable local development. 35 Community Capacity Enhancement (CCE) is a tool for the facilitation of community dialogues on HIV/AIDS, gender and OVC-related issues. It aims to facilitate community-level identification and support of vulnerable children and households, plus link them with services. The CCE incorporates other community mobilisation tools, such as ‘In Her Shoes’ (a tool for addressing GBV) and ‘Go Girls’ (a tool that addresses girls’ vulnerability to HIV/AIDS). 17 When compared with implemented activities, as noted in the Annual Progress Reports, it is evident that many of the planned activities (according to the co-operative agreement and work plans) for this KRA were rolled out. The annual reports note that the JoL and CCE methodologies were revised and contextualised to suit community needs. Training on these methods was provided by PCI to a variety of government, community and CSO stakeholders. The majority of the training of civil society and government stakeholders appears to have taken place jointly. However, there is no reporting on whether or not regular service provider dialogues were facilitated by PCI. According to the work plans, an evaluation of the JoL methodology was commissioned by PCI in consultation with REPSSI to document lessons learned. However, it is not clear from the reports or interviews whether the evaluation took place. An implementation shortfall was experienced with regard to capacity building of DCPCs and VCPCs to strengthen identification of those in need36. Constraints included delays with DCPC establishment guidelines37. Progress reports and input obtained via interviews and focus group discussions also indicate that the DCPCs were non-functional for most of programme roll-out. Nevertheless, PCI offered JoL and CCE training to potential members of DCPCs and VCPCs38, plus partnered with MCCNT in an effort to re-constitute and strengthen DCPCs. This led to a number of DCPCs being established in the TK districts of operation. Interviews reveal that IPs were also involved in strengthening community structures to identify and support vulnerable households and conduct community mobilisation activities. A number of KRA 1.1 activities were not reported on, making it difficult to assess the level of achievement in these areas. These include work plan activities such as the integration of JoL into school camps for OVC. There is also a limited level of reporting on the implementation of community action plans, which are meant to be compiled as part of the JoL process. Therefore, it is not possible to gauge if this aspect of the TK Programme was implemented as planned. There is also no reporting to indicate whether a set of criteria was developed by PCI and IPs to guide the identification and selection of the most vulnerable clients. However, the development and use of the CFC assessment tools suggest that this was achieved. Strengths and challenges in implementation In terms of general factors facilitating the strengthening of community structures to identify and support vulnerable households, the JoL and CCE methodologies emerge as a key programme strength. Respondents noted that the tools are highly effective, adaptable, easy to understand and easy to use. For example, the JoL methodology was adapted for use with children and was also used with GROW groups. Affecting the level of effectiveness of community structures in their provision of support to vulnerable community members, is the slow response rate to referred cases, particularly amongst government service providers. The delay in establishing DCPCs – and their reported lack of functionality – was also cited as a constraint in terms of the implementation of KRA 1.1 activities. 36 Note that DCPCs and VCPCs are described in greater detail under Objective areas 3 and 4. 37 TK Annual Progress Report for October 2012 – September 2013. 38 It is not clear whether these were eventually members of DCPCs and VCPCs. 18 Key result area 1.2: Participation of vulnerable households, including men, in economic strengthening activities Planned activities versus actual activities As noted in the cooperative agreement, KRA 1.2 activities were formulated with the aim of assisting vulnerable families / households to improve their livelihoods, through the provision of: • Basic skills development in financial literacy and household finance management. • The formation of partnerships with local, private businesses and financial organisations by PCI and IPs with the aim of encouraging financial professionals to provide their services pro￾bono through their corporate social responsibility programmes. • The roll-out of small business development interventions, such as the Food for Life and Honey for Life initiatives, by PCI and IPs. • Assisting programme participants with accessing of loans / grants through GOB programmes such as Citizen’s Entrepreneurial Development Agency (CEDA) and the Ministry of Youth, Sport and Culture (MYSC) and GeAD’s grant programmes. • Establishing linkages with business resource organisations, such as the University of Botswana’s The Business Clinic and the Local Enterprise Authority (LEA), as a means of offering vulnerable community members the necessary level of technical support and mentorship to enable entrepreneurial success. • The formation of GROW groups by PCI and IPs, including training of community-based facilitators who would be responsible for supporting and capacitating GROW group members in the basics of saving, lending and business management procedures39. The GROW model was an existing PCI model was piloted and rolled out in many African and other countries. A comparison of KRA 1.2 planned activities stipulated in the cooperative agreement (as noted above) and in the TK Project work plans indicates that these were overall well-aligned. Furthermore, a review of implemented activities reported in the Annual Progress Reports, reveals that a high number of the planned activities were rolled out. However, there are also a number of activities that were not reported on, making it difficult to assess levels of programme achievement in these areas. 39 The cooperative agreement also notes that male GROW groups would be encouraged once female-led groups are well-established and successful. Grass Roots building Our Own Wealth (GROW) is a model that aims to empower the most marginalised women (with a special emphasis on those infected and affected by HIV/AIDS) to use their existing resources to save and invest in income-generating initiatives to improve their wellbeing. Women, from the same community, form groups of 15 to 20 members and meet weekly to discuss social issues, plan individual and group economic empowerment activities, and contribute towards a savings and loan fund. 19 In terms of establishing partnerships to ensure ongoing support to beneficiaries of livelihoods interventions, PCI held a number of discussions with training and education providers, private businesses, banking institutions and other resource organisations. In addition to this, PCI held meetings with district officials to identify programmes and services available to beneficiaries that might assist them with their IGAs and micro businesses. PCI provided training on the GROW model to IPs and supported their implementation thereof; for example, through the hosting of a GROW Learning Forum in 2014. IPs provided support to group and individual IGAs through facilitation of linkages with government and private sector partners for capacity building and funding, while assisting beneficiaries to explore marketing opportunities for their products. The following was noted during a focus group discussion with HWW GROW beneficiaries: “The facilitators were very good but not only that – where they did not have the necessary knowledge or skill they brought in subject experts, example people from the bank, business people, people who could train us in beekeeping. A lot of useful information was shared…they told us how we could open a bank account. The bank was coming regularly to give us training on financial literacy.” The above quote also indicates that basic skills development in financial literacy and household financial management was implemented as specified in the cooperative agreement. There is no reporting on the roll-out of small business development interventions, such as the Food for Life and Honey for Life initiatives, nor on the levels of success achieved in terms of assisting programme participants with accessing of loans / grants through GOB programmes. Similarly, specific input on the establishment of linkages with business resource organisations, such as the Business Clinic and LEA, is not provided in progress reports. Strengths and challenges in implementation The Annual Progress Reports note that IPs used their capacities, gained through participation in the TK Project, to strengthen the income generation activities of existing women’s groups as well as those of individual clients. Examples of these IGAs include poultry farming, beading, baking, brick making, and vegetable gardening. Primary data collection yielded a number of reports regarding the strengths of the GROW model specifically. A number of IP staff felt that the model is relevant plus easy to understand and implement. This perception seems to be enhanced by the translation of the GROW manual into Setswana. Furthermore, it was noted that GROW groups provide an effective platform for integrating discussions around relevant social issues, such as gender, child protection, and HIV and AIDS; as well as facilitating women’s knowledge of (and access to) other services such as medical care. In this way, the model facilitates the implementation of activities related to other TK Project objectives. The partnership approach adopted by IPs – particularly for securing training, mentoring, resource allocations, and financial support of beneficiaries – was also noted as a key implementation strength. The high level of financial and technical support offered to TK economic strengthening interventions by Barclays Bank, is commendable. This included teaching economic strengthening beneficiaries about financial literacy, savings, budgeting, and business management. Barclays, through the Grants Programme, has also provided seed capital to individual and group businesses between 2013 and 20 2015. BiZkid provided mentoring support for six IGAs linked to GROW groups. Although having this technical expertise was valued, the time frame of six months was too short to allow for any meaningful support to be offered as the reality is that IGAs are involved in long processes (for example registration). A number of general implementation challenges were noted during primary data collection in connection with the GROW model. For example, all of the IPs working with GROW groups noted substantial drops in group numbers. Reasons cited include the movement of members to other areas, a lack of commitment amongst some of the group members, internal conflict and a lack of cooperation, and the long-term investment required of participants before noticeable benefits might be enjoyed. The following quote refers: “Another problem is the drop in numbers of the groups…start with 20 and now many groups only have four members. Also difficult to get all the members of a group to agree on a particular idea – if they don’t like an idea, then they simply don’t come to the meetings.” (IP2 beneficiaries_FGD) Despite attempts to establish GROW groups with members living in close physical proximity, low population density – particularly in rural areas – and high attrition rates mean that groups are often comprised of women living in different areas. Being geographically dispersed further hinders group attendance and – some argued – consensus and collaboration. Another factor affecting the implementation and effectiveness of economic strengthening interventions is what appears to be a lack of sustainable IGAs. Beneficiaries’ reported a lack of access to markets for produced items; as well as a high degree of vulnerability to environmental threats (for example, lack of access to water) which suggests that IGAs sometimes lack sufficient relevance, strategizing and planning and IPs need more support with this. These constraints appear to be exacerbated by limited skills amongst community facilitators and beneficiaries as well as other contextual challenges such as limited capital, which hampers beneficiaries’ management of - and recovery from - setbacks, such as disease or theft affecting their livestock. Limited government support and poor coordination between government and IP economic strengthening interventions are other challenges affecting the successful roll out of KRA 1.2 activities. As opposed to establishing linkages with government poverty eradication programmes, some IP staff reported these as conflicting with programme initiatives. The following quote – by an IP Manager - elaborates: “Iphelegeng is a government programme aimed at poverty alleviation. When they are on duty in Iphelegeng, then they cannot attend the GROW group. The Iphelegeng is on a rotational basis and so the people come and go from a GROW group and this is not good for the group.” Finally, the limited number of male community members involved in economic strengthening initiatives was cited by a number of respondents as an implementation shortfall. The precise reason for the lack of male interest in programme participation remains unclear. The following quote elaborates: “HWW also tried to form male groups but this did not work – was a total failure. The men just failed to get themselves together. We are not too sure why…they are unemployed and have very limited income, but they were just not interested.” (GROW beneficiaries_FGD) 21 Key result area 1.3 Livelihoods opportunities for vulnerable youth increased Planned activities versus actual activities The aim of the activities rolled out under key result area 1.3 is to provide support to vulnerable youth on a number of different levels. These include: • Support with formal education; • The addressing of any challenges that may impact negatively on youth’s personal development and progress, including the provision of psycho-emotional support; • Capacity building in basic financial literacy and business management skills plus job readiness training to prepare those who wish to apply for formal sector employment or establish their own businesses. Other planned activities noted in the co-operative agreement include: • Conduct a mapping of vocational training programmes offered and entry-level requirements of potential employment providers and use these to inform initiative training provision, internship and apprenticeship opportunities. • Facilitation of small business development as well as youth access to credit and grants, similar to that proposed for adult community members40. • Through partnerships with Botho College’s E-Skills Foundation and focal CSOs, PCI aimed to pilot an ICT café scheme. Through this scheme, marginalised communities would be offered access to Internet connectivity as well as practical training and skills development in ICT41. • PCI and partners, Botho/E-Skills and Positive Innovations for the Next Generation (PiNG), were to explore ways of incorporating cell phone technology to increase Internet access in remote areas. These activities were delivered though the YEP and Aflateen models, which was sometimes combined and delivered as one model. The Aflateen model was adopted from a local NGO (Stepping Stones) while the YEP was developed for the TK project. The planned activities stipulated in the cooperative agreement (as noted above) and in the TK Project work plans are generally aligned. A review of implemented activities, as reported in the 40 See section for KRA 1.2. 41 This would focus more specifically on youth. Aflateen is a financial and business skills building and social development programme that targets OVC and youth aged 14 - 24 years. By equipping the youth with these skills, it aims to empower them to become productive community members and agents of social change. Youth meet weekly to discuss issues including saving and budgeting, and personal development as well as HIV/AIDS prevention. The Youth Employment Program (YEP) targets youth aged 18-35 who are out-of-school, unemployed or under-employed. YEP includes a job readiness component coupled with experiential learning through job placements and access to vocational training opportunities, resulting from programme partnership agreements. 22 Annual Progress Reports, reveals that a number of the planned activities were implemented. This includes PCI’s roll out of job readiness, life skills and financial management capacity building methods and materials to IPs as well as the organisation’s facilitation of partnerships with local financial/other businesses for assistance with training, mentoring, provision of vocational opportunities, and financial support of youth economic strengthening activities. While the mapping study is noted in the work plan for reporting period June 2011 – May 2012, there is no reporting on the implementation of this study. There is also no reporting on the ICT café scheme42. The use of cell phone technology to enhance Internet access in rural areas was abandoned due to a lack of support by USAID. A number of Aflateen-related activities, specified in the work plans, are also not reported on in the Annual Progress Reports. This includes the integration of Aflateen into Botswana Association for Psychosocial Rehabilitation (BAPR) youth programmes, the organisation of Aflateen camps and an Aflateen learning forum, and the development of marketing materials to strengthen Aflateen implementation. However, respondents involved in primary data collection did note that an Aflateen consortium, comprised of members of all IPs offering such interventions, was established to facilitate discussion and exchange of ideas regarding programme implementation. Furthermore, the consortium was perceived as a highly effective, useful and motivational programme intervention. Strengths and challenges in implementation A good practice noted amongst IPs working under objective 1.3 was the strategic use of existing interventions to identify youth for participation in economic strengthening activities. IPs involved in these activities reported recruiting programme participants for Aflateen through their kids and teens clubs. Aflateen and YEP members were often recruited via the adult interventions such as the GROW groups. Home visits also offered opportunities for the recruitment of youth for financial literacy programmes. While some facilitators noted capacity gaps in their roll out of the Aflateen and YEP, IPs demonstrated a sound level of effectiveness in establishing partnerships with private enterprises for the purpose of programme support. For example, HWW engaged BiZkid, a company specialising in providing business and entrepreneurship support, to conduct a five-day entrepreneurship programme with 15 Aflateen youth in Borakalalo Ward. The organisation also invited Barclays Bank staff to conduct a session on entrepreneurship with 12 Aflateen youth (under Barclays Bank’s Unlocking Youth Potential initiative). Topics covered included challenges associated with entrepreneurship, establishing a small business, marketing, and how to conduct a SWOT Analysis. A number of general challenges were reported in the course of this evaluation apropos the offering of livelihoods opportunities for vulnerable youth. • Respondents argued that the implementation of youth economic strengthening initiatives is being hampered by insufficient skills amongst facilitators, which then limits the level of input and assistance that can be offered to Aflateen and YEP participants43, particularly when they wish to apply their new financial or business knowledge. Not having staff dedicated to 42 PCI input to the draft report indicates that the ICT Project is being implemented - by HOH. 43 Where skills levels are higher; for example, in the case of BBM, programme effectiveness was reportedly higher. 23 economic strengthening also hindered implementation. Where dedicated staff was in place, for example with BBM, there seems to be more positive results. • Compounding this is the availability of Aflateen and YEP manuals in English only. This means that facilitators have to translate and explain concepts that they themselves are not particularly familiar or comfortable with. • Despite the high level of emphasis placed on TK youth programme articulation with other, affiliated youth empowerment initiatives, some of the latter programmes appear to remain difficult for youth to access. For example, it was noted that certain programmes, such as LEA, require financial resources, which some youth do not have at their disposal, or high levels of skill, for example in the compiling of comprehensive business plans, that the economic strengthening programmes do not appear to be providing to the youth. • Another challenge noted by a number of evaluation respondents in terms of implementation is the lack of youth appeal. While this was noted of both YEP and Aflateen, it appears to apply particularly to the latter programme. Respondents felt that programme content was insufficiently engaging to sustain youth’s interest and attendance at sessions. Challenges within the school context (e.g. limited time for extra curricula activities, teacher strikes, etc.) also reportedly affected the roll out of the Aflateen programme. • The YEP, on the other hand, was seen as offering more tangible benefits. This may be as a result of the programme’s emphasis on job placements and assisting youth to access vocational training opportunities – an area where secondary data sources report a measure of success. • Respondents also noted that youth often held high, and somewhat erroneous, expectations of the YEP and Aflateen programmes. The quotes below elaborate: “The one on youth has challenges…it raises expectations…they come expecting jobs quickly but when they cannot get them, they disappear. So the numbers keep dwindling.” (Youth beneficiaries_FGD) “The programme is not working well…it doesn’t seem to be well-received. And also, it’s a demanding area. It requires a lot of dedication because …the youth are a difficult population. So they require somebody who has the proper skills for that and the dedication.” (IP4 centre coordinator) 5.1.2 Outcomes for beneficiaries The evaluation sought to establish outcomes at community level as a result of the TK Project. While it is too early to establish the achievement of some outcomes (due to delay in the roll-out of services), the reported outcomes provide an indication of whether results may be achieved. The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? Interviews with programme staff and beneficiaries reveal that women and youth targeted for economic strengthening have improved their knowledge and understanding of how to identify business opportunities; how to go about setting up a small or micro business; how to save and bank money; and how to draw up a budget. In addition to these areas of skill development, a number of GROW beneficiaries noted that their participation in the programme had facilitated their development of communication skills as well as leadership capacity. 24 Substantial shifts in feelings of self-confidence and self-esteem were also noted by a high number of respondents involved in GROW and the YEP. This is described in the quote that follows: “I like that I am the managing director of my own business! That has given me a lot of self-esteem and the confidence to stand on my own two feet as a young person.” (IP4 youth beneficiary) Feelings of empowerment and self-sufficiency were also noted by a high number of respondents, together with a more positive outlook on life in general. Much of this was attributed to the use of methodologies such as JoL, where “…people are responsible for identifying their own problems and so can identify their own solutions…”. (IP2 Programme Manager). Changes in knowledge and attitude have led to changes in beneficiaries’ expenditure and also their prioritisation of - and contribution towards - household needs44. Consequently, GROW and YEP beneficiaries noted that they are now buying groceries and school uniforms, and providing better care for - and support of - other family members. This is improving family relations as well as contributing towards higher levels of food security within the home. The following quotes elaborate: “I can now support my family and provide for their needs…for example, I am providing my siblings with money for school transport.” (IP4 youth beneficiary) “I give money to my mother since I work now. I have some responsibility…like to buy food for the house. I never did that before.” (IP4 youth beneficiary) However, despite these changes, beneficiaries report that the programme is yet to make any fundamental difference to their income levels or their ability to substantially improve their livelihoods. Of interest is the number of respondents that also noted how participation in GROW and youth groups was enabling feelings of belonging, security and community cohesion. As noted by a GROW beneficiary: “I have found myself a family that has helped me in all areas of life…socially, emotionally, economically…We counsel and comfort one another when we are not feeling too good”. This appears to be encouraging higher levels of community engagement, tolerance and support of fellow community members. This is reflected in the quote below: “The changes in me…I have learnt to be sympathetic towards others. Like there is a lady who cannot take care of herself due to illness that has disabled her. Together with the group we are planning to surprise her with a gift of some sort…maybe help financially.” (IP3 GROW beneficiary) Youth have also reportedly been more active in community outreach activities such as assisting other youth and the elderly in their community and implementing social campaigns. This was however not verified by the youth or IPs sampled. As indicated in the introduction to the findings, differentiating between the outcomes for youth that resulted from economic strengthening interventions and those that resulted from youth developmental interventions is a challenge. This is particularly because very often the same group of youth were targeted with an economic strengthening activity and a youth development activity. It is therefore suggested that the Outcomes for beneficiaries under Objective one and two be 44 This was confirmed by a government stakeholder and also by a number of IP staff; for example, the HWW economic strengthening staff, IECD volunteers, Peace Corps volunteer and gender officer noted that requests for food baskets, uniforms and other forms of assistance had dropped subsequent to the introduction of the GROW groups. 25 understood as a reflection of youth outcomes as a whole, rather than strictly differentiating between the subsets of youth. 5.1.3 Key lessons learnt for objective 1 The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to objective 1:  Community mobilisation initiatives are an innovative means of creating community awareness as well as facilitating community level identification and support of vulnerable households.  Systematic post-mobilisation interventions must be put in place to ensure that there is adequate service provision as well as on-going mobilisation and support for the implementation of community action plans.  The GROW model is an effective means of facilitating a measure of financial security and stability through collaboration, as well as building individual capacity for sound day-to-day financial management.  Economic strengthening programmes need to be implemented by highly skilled staff, to allow for effective skill / capacity development as well as effective mentoring.  Economic strengthening programmes need to be carefully planned and suitably aligned to local market opportunities and environmental/social conditions.  Income generation and youth livelihood interventions need to be concretely linked with affiliated public and private enterprises to heighten levels of success and sustainability. 5.2OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS 5.2.1 Project Implementation This section on project implementation evaluates the actual activities against planned activities, and also assesses the strengths and challenges experienced in the implementation of Objective 2: Developmental interventions for vulnerable children and adolescents improved. The key evaluation question is: To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? The data on planned vs. actual activities is mostly extracted from annual reports, and to some degree from primary data collection. The detailed list of planned and actual activities for each KRA for Objective 2 can be found in Appendix 6. Primary data collected through evaluation has mainly provided the data on strengths and challenges. To achieve objective 2, three broad results areas were planned. These are reported on in this section. 26 The table below provides the data for key indicators for Objective 2. Disaggregated data can be found in Appendix 5. Table 4: Key indicators for Objective 2 Indicator Cumulative target Cumulative total % of target reached Number of children accessing IECD services (Tsela Kgopo 2.1.2) 2809 3000 106.8% Number of children ages 0-3 accessing structured play stimulation activities through community groups.45 (Tsela Kgopo 2.1.3) 912 304 33% Number of adolescents participating in life skills activities. (Tsela Kgopo 2.3.2) 8090 5921 73% Number of eligible children provided with health care referral46 (PEPFAR C5.3D) 850 1119 131% Number of targeted population reached with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet the minimum standards required47 (PEPFARP8.1.D) 18355 18557 101% Number of active beneficiaries served by PEPFAR OVC programs for children and families affected by HIV/AIDS.48 (PEPFAR OVC_SERV) 13268 8750 66% Number of HIV positive adults and children receiving care and support outside of the health facility49 (PEPFAR Care Comm) 2,567 1,208 47% Number of active beneficiaries receiving support from PEPFAR OVC programs to access HIV services.50 (PEPFAR OVC_ACC) 3200 86 3% Key Result Area 2.1 Access of most vulnerable children to quality early childhood development services increased Planned activities versus actual activities The aim is to ensure that the most vulnerable children have increased access to quality early childhood development services. The concept document discusses the activities, strategies and methodologies within this key result area. These are listed below: • Building IECD skills of CSO staff and volunteers, parents and caregivers including standardized IECD modules and PCI’s Say and Play methodology • Home-based outreach and care 45 No target set in 2012 Annual Report 46 No target set in 2012 Annual Report 47 Only reported in 2014-2015 Annual Report 48 Only reported in 2014-2015 Annual Report 49 Only reported in 2014-2015 Annual Report 50 Only reported in 2014-2015 Annual Report Say and Play Methodology Developed by PCI, this simple, low￾cost set of techniques enhances the ability of those working with children and their parents/caregivers to understand and communicate with the young children better as well as support their psychosocial needs. 27 • Strong referral linkages with S&CD and MOE • Strong linkages with health and nutrition sectors • Collaborating with Botswana Television on using the life skills radio and television programs they have produced There is some misalignment between the cooperative agreement and the work plans. Areas such as television and radio and nutrition and health, and referral linkages have not been mentioned in the work plans or annual reports, and so it is challenging to ascertain performance against activities in co-operative agreement. Comparing planned activities in the work plans with activities reported in the Annual Progress Reports demonstrates partial achievement of planned activities. Much of the focus was on the creation, distribution and roll-out of the IECD training package, which relates to skills building. Generally IECD is implemented through three models: • Mobile playgroups, where community members gather with their children a couple of times per week to stimulate children. • Centre based ECD where IECD is either implemented at a fixed centre intended for ECD or at other community centres such as libraries. • Home-based ECD where caregivers are educated on how to nurture and stimulate children in simple ways. The goal of these groups are to “promote sharing, playing and learning in children” (IP1 Program Manager). IPs mostly seem to offer IECD through mobile play and stimulation groups and centre-based services. Integrated ECD services are meant to include health, nutrition, education, child protection and environmental provisions. IPs show that they are implementing holistic services through a number of different activities51: • Provision of a meal for children attending ECDs • Encouraging and ensuring children are registered at birth and receive birth certificate so that they may access government services. • Linking abused or neglected children with social workers • Health assessments, including ears and eyes and if a child is HIV+, support parents and children with self-acceptance and medication adherence. • Psycho-social support for OVC • Education and encouragement for parents and caregivers in ECD on issues such as: never sending kids to school hungry, the importance of talking to children with love, encouraging good relations between parents and children, and how to keep homes hygienic • Checking whether parents are immunising their children and maintaining their growth monitoring card • Checking for signs of abuse 51 Please note that the extent of integrated services varies from IP to IP. All activities are not implemented by all IPs. 28 Educational activities include teaching colours, shapes, sizes, facilitating interaction with the environment using their senses, and encouraging socialising with other children. Strengths and challenges in implementation A major strength of this objective has been the increase in access of vulnerable children to IECD. The numbers of children who attend have increased and centres are reaching their targets for attendance. Through interaction with parents and community conversations, parents seem to be more appreciative of the importance of IECD. However, there is still uncertainty among some parents about the purpose of playgroups as they consider them to be schools and question the nature of stimulation and play activities for children. As an intervention, the mobile playgroup is promising as an IECD model. It does not require a structure; it can be facilitated by community members/caregivers; and it can be conducted anywhere in the community and so has potential to expand the reach of IECD services. The challenges are common to all models of ECD service provision, i.e. cost of equipment, toys, providing a nutritious meal etc. Likewise the use of existing community spaces, such as libraries, for IECD activities work well to work around the structural challenges facing the provision of IECD services. The training and manuals provided by PCI enabled good quality service provision. Resource mobilization had some success in organizing private funds as well as other resources. Donations were received from US private funders, individuals and the Catholic Church in Francistown. Donations included $10 000, as well equipment such as blankets, mats, crayons and doors. Resource mobilization has generally gone well. IPs have taken initiative and connected with organizations to mobilize necessary resources. However, there are still challenges with resources, with some centres not operating in a building or providing snacks for the children. “We are trying to get the community to supplement the meals – can provide a simple meal and we will supplement that - at some stage they did this but if it is always the same parents doing this then they eventually get tired of doing it – not doing it anymore” (IP2_IP staff_Prog Manager). Although IPs take the lead on resource mobilization, communities themselves have also assisted with meals/snacks where possible. In some instances, parents are also charged a small fee, which is then used on transportation and food. Although some communities have mobilised around nutrition and stimulation materials, this has not been the case for all IECD services. Given the economic context within which IPs are working, the exclusion of this from the TK project budget has been a challenge in terms of promoting quality IECD services. As described in the Relevance Section, there are challenges with access to IECD services for children with disabilities as parents choose to keep them home. There are also issues concerning distance and location with minority groups and children on farms settling too far away to access services. Distance to service is a concern for parents in the community as well. Furthermore, government regulations around minimum requirements for ECD centres has meant that a number of centers have been forced to close or operate illegally as they do not have the 29 required structural facilities, such as junior toilets. PCI has done much work around the amendment of these requirements, although this has clearly not yet translated into action at a District level. The delays of the IECD resource pack and the guidelines (See Objective 4) are a challenge for implementation. It limits the level of standardization across different centers. However, both the resource pack and the guidelines are due to be completed in 2015. Key Result Area 2.2 Strengthened collaboration between parents/caregivers and IECD centers for improved child development Planned activities versus actual activities The aim of key result area 2.2 is to strengthen collaboration between parents/caregivers and IECD centers to improve child development to ensure that there is consistency in care. The concept document discusses the activities, strategies and methodologies within this key result area. These are listed below: • Parent teacher groups to be formed to act as a bridge between CSO center services and home. Special focus on motivating men to become active in these groups and to work with women to achieve goals for their children. • Establishment of structured play/stimulation groups for children under 3 years old. This is a low-cost, less formal service. • Engaging men in support of CSO center activities as part of promoting gender equity Comparing planned activities in the Annual Work Plans with activities reported in the Annual Progress Reports appears to demonstrate partial achievement of planned activities. However this is not to say that the KRA has not been achieved; as demonstrated by the previous KRA and the outcome of IECD, there has been much collaboration with parents. PCI has supported partners to strengthen or establish PTAs using Ministry of Education Guidelines. Mothers Union and House of Hope have both strengthened their PTAs and now meet regularly to discuss pertinent issues. There have also been instances where partners have worked to include the activities related to gender, specifically including men and their role in families. HwwB organized a family day in 2014 that focused on teaching fathers or male caregivers about childcare and the important role fathers play in their children’s lives. In October 2012-September 2013 they mobilized parents and caregivers to clean the informal play/stimulation area Mmakanke. In October 2014- March 2015, there were numerous trainings and workshops on IECD for communities. However, in this period the TK Project only reached 64% of its target for number of children 0-3 accessing structured play stimulation. Strengths and challenges in implementation As mentioned, PCI has done well in supporting partners to strengthen or establish PTAs using Ministry of Education Guidelines; however, only 2 of the IPs have set up PTAs. The format of engaging parents through formal structures such as PTAs was too prescriptive in the communities that IPs were engaged. Thus engagement with parents took other forms, including through management committees. Furthermore, male-specific focused activities are not mentioned in reports aside from a family day in November 2014 that HwwB organized in Borakalalo, attended by both males and females. The focus was on teaching fathers or male caregivers in the family about child care and the importance of fathers’ presence in their children’s lives. Although there are 30 positive signs of collaboration, positive results around caregiver-school relations is not widespread. Parents do get involved with the centers where in certain cases they provide nutrition. There have also been numerous workshop targeting parents and caregivers about the importance of IECD and parenting skills. Key result area 2.3: Access of adolescents to mentoring, life skills and HIV prevention programs increased. Planned activities vs actual activities The aim of key result area 2.3 is to increase adolescents’ access to mentoring, life skills and HIV prevention programs. PCI and partners worked with stakeholders to review existing toolkits and approaches in Botswana to help develop pre-adolescents and adolescents with regards to life skills, healthy sexual and reproductive health practices, which includes avoiding HIV and early pregnancy and gender related attitudes and behaviours. The concept document discusses the activities, strategies and methodologies within this key result area. These are listed below: • PCI and partners to establish a relationship with Lovelife South Africa and adapt elements for Botswana’s youth • PCI will partner with PiNG and PCI South Africa to integrate Young Africa Live (YAL) and iAMP, two ICT methodologies, into one ICT approach to sexual and gender behaviour development and behaviour change • HIV+ Teen Clubs to promote healthy lifestyles and coping with HIV • PCI to provide technical assistance to the MOE to help ensure the effectiveness of a new life skills curriculum as a school-based intervention • A mentorship program for vulnerable adolescents will be established with the partners and the University of Botswana and other local colleges. Comparing planned activities in the annual work plans with activities reported in the Annual Progress Reports appears to demonstrate a fair achievement of planned activities. Delays in the first year of implementation were due to the focus on the development and institutionalising of life skills materials. Lovelife was not mentioned in any of the work plans or annual reports. Young Africa Live was mentioned in 2011-2012 where discussions were ongoing and it was meant to be awarded Quarter 4 to start implementation. A decision was however taken by USAID to exclude PiNG from the TK project. Due to delays in the approval of the life skills tool kit, the draft adolescent toolkit was used for training purposes. The adolescent life skills tool kit was not completed according to schedule and finalising the tool is now planned for Quarter 3 of the current financial year. Interviews reveal that youth developmental interventions are implemented through Kids Clubs, Teen Clubs and Peer Education Groups. In addition YEP and Aflateen groups are also used as a platform to discuss issues of HIV, Lifeskills, Sexual Reproductive Health and other topics related to this KRA. Furthermore, while the objective speaks only about HIV interventions for youth, it is important to note that HIV education and support is provided across all target groups. Kids Clubs take place in schools and in villages and topics covered in the Kids Club includes sensitizing children on gender issues, gender norms and disparities, the meaning of HIV, hygiene, parental communication and recognizing and responding to abuse. Many of the children who are part 31 of the Kids Clubs were identified through home visits and identification by teachers. The clubs meet weekends and there are also camps for the children to attend. The regularity of Teen club sessions changes according to the IP, for example sometimes they are held once per month and sometimes once per quarter, however this is not considered enough. Teens are identified through Clinics as well as hospitals. The focus of life skills is to empower youth, particularly to prevent teenage pregnancy and then how to take care of children, for those who already have children. Life skills discourage girls from having boyfriends for financial gain. And they are also taught sexual reproductive health; contraceptives; condom use; and completion of school. In addition, they are encouraged to avoid vandalism and the destruction of government entities. There is also PACT (Peer Approach Counselling by the Teens) and mentoring of the youth which takes place one per week. A benefit of working with the youth is that they seem to be less restricted by culture, however despite the discussions and focus sexual reproductive health, there are still girls as young as twelve who are falling pregnant. As a result IPs question the effectives of youth interventions. HIV has been integrated into the various different approaches taken by the TK Project. It is discussed often with IECD, and PCI has greatly supported and strengthened HIV+ Teen Clubs; providing training for facilitators. Strengths and challenges in implementation A key strength of objective 2.3 is the reach of the life skills programs. They are able to operate in villages which governments do not reach. “Lifeskills is working well. They have done a lot in this field. They cover many villages which government would never reach”. (DG_DAC) The life skills programs take place in schools, through teachers as well as through IPs who visit the schools. The children enjoy the activities and it is relevant for their age group. The volunteers are trained and so can monitor and assess any problem areas the children may have. Although the draft toolkit acted as a good training guideline, an aspect lacking in the draft toolkit that was in use was the lack of coverage of sexual orientation which needs to be added. Educators in some instances still struggle to understand the relevance and importance of lifeskills as building a good foundation for well balanced and healthy children and this hinder project implementation: “Teachers … don’t really understand that you need to build a child before you can teach them abcd – curriculum is very important to them – have to follow it. If NGOs go to the schools, then they say that we are wasting their time …” (IP2_ staff_Prog Manager_ SSI). The Kids Clubs Camps are a significant strength of the program, although only one IP mentions implementing these. This platform allows children to build cohesion and trust and as a result they speak openly and support each other, 32 “Had these discussions during the camps. Camps are where the kids really start to open up. In a group, feel that they are the same and can support each other, can share. Away from home environment and so they feel more comfortable”(IP2 Program Manager). 5.2.2 Outcomes for beneficiaries The evaluation sought to establish outcomes at community level as a result of the TK Project. While it is too early to establish the achievement of some outcomes (due to delay in the roll-out of services), the reported outcomes provide an indication of whether results may be achieved. The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? IECD The IECD programmes are “geared towards family, community, and national level interventions for early childhood development” (IP2 District management programme officer). As such, these programs benefit not only children, but their parents as well. There is a big demand for IECD, and the provision of IECD through stimulation and play groups (as well as centre-based services) have increased access for those who previously had no access to such services. There is reportedly still however a challenge with few 0-3 year olds accessing ECD centres. The provision of a safe space for children, who would otherwise sometime be left at home under the supervision of siblings or on their own, has also allowed caregivers to be freed up to take advantage of employment opportunities such as Ipelegeng. The programs have increased the knowledge of parents, who “know about child development and they know that a child needs certain things to grow” (IP4 Volunteers). There has also been a shift in parents’ attitude towards nurturing and disciplining children: “I used to be violent and hit children randomly - now have learnt to show children love” (IP2 Beneficiaries). Parents also show that they are addressing the holistic needs of their children more. There is more consideration for the health, hygiene, social protection and emotional needs of children. “The evidence of our success is the way parents now treat their children, and the rate at which they immunize, weigh, register birth and clean their children” (IP2 District management programme officer). Children reportedly show better school readiness as a result of good quality ECD, and a growing passion for learning: “the little ones have learnt recitations and numbers; coming home to share their knowledge” (IP2 Beneficiaries). 33 “the children come home to show parents what they have learnt at school – they are learning a lot. When the children come home, want to show off – want their parents to join them in their games” (IP1_Prog Man) The program also is “encouraging good relations between parents and children, also how to keep homes in good hygienic condition where kids can grow happily” (IP2 Beneficiaries). Life skills For the older children, the biggest attitude change that parents have noticed is the increase in their self-confidence, self-esteem, and self-worth. “The children were very shy and very reserved- but through the programme have gained confidence-are not so shy anymore” (IP2 Beneficiaries). This view is shared by the youth themselves. The increased confidence has also helped to improve the communication skills of the participants: “There have been huge benefits and changes in their ability to articulate their thoughts and ideas, good communication skills” (IP2 Volunteers). Other positive attitudes noticed were that those who participate “have a lot more courage and determination” (IP2 Volunteer). A number mentioned that they have less anger and bitterness and are more open. Since participating in the programme, fewer young people are drinking alcohol, “a lot of the children were drinking alcohol. But now as I speak, they don’t even enter through the door of a bar” (IP1 facilitator). Beneficiaries agree with this assessment, a beneficiary of CRS, a boy, now knows that he must “stay single, drug, and alcohol free” (IP1 Beneficiary). This reduction in alcohol use is linked to making more positive life choices. This includes attending religious services, “[I] used to be an alcohol abuser-now I go to Church” (IP2). The positive decision making by youth and the improved communication has also had some benefit in terms of improving family relationships. This is because youth display less “problematic” behaviour; but also because there is more freedom to discuss ssubjects that they hadn’t previously discussed. Cleanliness has also improved among beneficiaries, “Learning that it is a good thing to be hygienic and clean, despite the challenges of water-extra effort now made by the family and the children… no longer have dirty kids running around”(IP2 Beneficiary). HIV and AIDS The results of the HIV/AIDS activities are spread across youth and adults. The TK program has also increased knowledge on HIV/AIDS which has positively shifted attitudes and behaviour. A strong focus is placed on acceptance. “They do HIV counselling and offer support. They teach about acceptance of those that are infected with HIV because they are still people” (IP3 Beneficiary). 34 Acceptance and understanding have benefited families, “The programme renewed relationships in the family because they understood HIV better” (IP4 Beneficiary). Furthermore, it has encouraged families to support each other, “Children are giving a lot of support [to mother] tell her to go to the clinic and to get her medication and to also adhere to the treatment - are all supporting each other” (IP2 Beneficiary). This acceptance can also be linked to a decrease in stigma, “They have made HIV/AIDS a problem that people talk openly about and thus have taken the stigma out of it” (IP2). Many believe that stigma has reduced because community members now take responsibility for alerting volunteers when people living with HIV have defaulted on their medication. However, this view is not shared by all, “We had a client who no longer wanted to be associated with us as she didn’t want people concluding she is HIV positive” (IP3 Volunteer). Although stigma is not reported to be widespread, this may indicate that community members are reluctant to use services provided by some IPs as a result of its association with HIV. The knowledge gained from the program has shifted the behaviour of many. People, particularly youth, are more cautious with their sexual behaviour even delaying their sexual debut. Furthermore, a greater number people are being tested, “Even the whole family, the teenagers who don’t want to go to school and what-have-you-all are being tested for HIV” (IP2 Beneficiary). It also has increased adherence to medication in general, “People now appreciate and take their medication” (IP2 Volunteer). The program has been able to provide assistance to youth who previously did not have the necessary support: “In one home there was a young girl of around 13/14 years. She was heading the household - had 2 siblings that she was looking after, but was very ill - has HIV. HWW got her on ARVs and back into school-she is doing well now, thanks to the programme” (IP staff volunteer). As indicated in the introduction to the findings, differentiating between the outcomes for youth that resulted from economic strengthening interventions and those that resulted from youth developmental interventions is a challenge. This is particularly because often the same group of youth were targeted with an economic strengthening activity and a youth development activity. It is therefore suggested that the Outcomes for beneficiaries under Objective 2 and Objective 1 be read as a reflection of youth outcomes as a whole, rather than strictly differentiating between the subsets of youth. 5.2.3 Key lessons learnt for objective 2 The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to objective 2: 35  Active community support, particularly from parents/caregivers, is vital for successful implementation of IECD, it helps to ensure that the most vulnerable children receive services. The TK project have found ways to do this outside of PTA structures.  Resource mobilization is an important aspect of the project as is government support. Partnerships are vital for resource mobilization.  It is vital to manage the community’s expectations, with the ECD centres there is the expectation of it being a pre-school so parents are expecting more than what is delivered.  Stimulation and play groups work really well in taking IECD services to poorly resourced and hard to reach areas. The mobile IECD stimulation and play groups as well as those IECD services linked to other community facilities are good models for the provision of IECD services.  Kids club camps seem to work well for the provision of life skills education.  The provision of nutritious meal/snack as well as stimulation materials is important for the provision of IECD. 5.3OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN 5.3.1 Project Implementation This section on project implementation evaluates the actual activities against planned activities, and also assesses the strengths and challenges experienced in the implementation of Objective 3: Reduced neglect, exploitation and abuse of vulnerable children, adolescents and women. The key evaluation question is: To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? The data on planned vs. actual activities is mostly extracted from annual reports, and to some degree from primary data collection. The detailed list of planned and actual activities for each KRA for Objective 3 can be found in Appendix 6. Primary data collected through evaluation has mainly provided the data on strengths and challenges. The table below provides the data for key indicators for Objective 3. Disaggregated data can be found in Appendix 5. Table 5: Key indicators for Objective 3 Indicator Cumulative target Cumulative total % of target reached Number of eligible clients who received food and/or other nutrition services (PEPFAR C5.1D) 15090 18182 120.5% Number of eligible children provided with shelter and care52(PEPFAR C5.2D) 3103 3086 99.45% Number of eligible adults and children provided with protection 6750 6992 103.58% 52 No target set in second report 36 and legal aid services (PEPFAR C5.5D) Number of eligible adults and children provided with psychological, social or spiritual support. (PEPFAR C5.6D) 19379 24095 124% Number of people reached by an individual, small group, or community-level intervention or service that explicitly addresses norms about masculinity related to HIV/AIDS (PEPFAR P12.1.D) 6034 8537 141% Number of people reached by an individual, small group or community-level intervention or service that explicitly addresses gender-based violence and coercion related to HIV/AIDS53 (PEPFAR P12.2D) 16285 15926 97.8% Number of people reached by an individual small group, or community-level intervention or service that explicitly addresses the legal rights and protection of women and girls impacted by HIV/AIDS54 (PEPFAR P12.3.D) 9565 8942 93.48% Number of community health and para – social workers who successfully completed a pre-service training program within the reporting period (PEPFAR H2.2.D) 335 907 270% Number of Health care workers who successfully completed an in-service training program within the reporting period55 (PEPFAR H2.3.D) 1620 683 42% Number of people receiving post GBV care56 822 306 37% Number of people completing an intervention pertaining to gender norms57 (PEPFAR GEN_NORM) 1,900 653 34% The section below reports on the five key results areas planned for this objective. KRA 3.1 Strengthened systems to prevent abuse and meet the needs of those neglected, exploited and abused Planned activities versus actual activities The cooperative agreement specifies that, under result area 3.1, PCI planned to work firstly with the Gender Affairs Department (GeAD) District Officers, District Child Protection Committees/Village Child Protection Committees (DCPC/VCPCs) and partner CSOs to strengthen systems and skills for preventing and addressing the abuse, neglect and exploitation of girls, boys and women; and secondly with the National Aids Coordinating Agency (NACA), African Comprehensive HIV/AIDS Partners (ACHAP) and GeAD to determine how the Interactive Amplification Platform (iAMP) discussed under 2.3 could be used to tackle gender norms. Other activities planned include: service mapping and the production of a simple referral booklet on services; intervention partnerships with district stakeholders to improve referral systems for prevention and response; and educating Women’s support groups under objective 2 about gender and child rights. All except two of the above mentioned activities were covered in the workplans and these were: the use of the iAMP to tackle gender norms; and service mapping and production of a referral booklet on services. This shows a good alignment between the original activity design and workplans. A comparison of planned activities in the annual workplans versus actual activities 53 No target set in 2012 Annual Report 54 No target set in 2012 Annual Report 55 Includes para-social workers in 2012 Annual Report (2nd) 56 Only reported in 2014-2015 Annual Report 57 Only reported in 2014-2015 Annual Report 37 reported in the Annual Progress Reports reveals a mix of achievement, non-achievement and partial achievement of activities. It should be noted that some activities were not reported on, making it difficult to assess whether they were achieved or not. As planned, KWS together with GeAD, DSP and PCI identified community structures to be strengthened. A range of community coordination structures were put in place by implementing partners to identify and provide support to most vulnerable households. These included ward gender committees, VCPCs and DCPCs. There is no clarity on the exact numbers of VCPCs put in place by the TK Project but the annual reports indicate the following: • HWWB, established two VCPCs at Mogonono and Maitseta, KYT established one in Lerala, while CRS Sebina formed a child protection committee which will be formalized into a VCPC once the DCPC is established and functional. • KWSP established Lekgwapheng ward gender committee which works closely with Kweneng District Gender Committee. With membership of 20 (9 males; 11 females), the Lekgwapheng committee aims to enhance community capacity to respond to gender based violence at community level. • In collaboration with DSP, Marang spearheaded the formation of DCPCs in six of the TK districts. • Marang and other TK partners continued to form and strengthen the lower level coordinating body – VCPCs. Marang also supported various DCPCs; Charleshill Tsabong and Ghanzi with different forms of assistance. 58 Progress reports lacked detail on whether assistance was provided to GeAD to conduct national validation workshops on the draft national GBV referral system; and to GeAD to develop the 365 days GBV campaign strategy and action plan. KWS trained social workers and CSO staff in life skills and GBV as planned but it is unsure if a training of trainer (TOT) approach was used as specified in the workplan. The activities related to working with CRS to advocate and ensure mass birth registration were rolled over to Quarter One FY 15. During the final year of the project (2014-2015), those activities which were planned and which were not reported on include: rapid district GBV response assessment in Gantsi, Mabutsane and Selibe Phikwe; media and social communication training on reporting GBV; and support GeAD to coordinate implementation of referral system. Planned activities which were achieved include: strengthening of DCPCs, VCPCs and a range of community structures through sensitisation activities; establishing networks with traditional leadership to make referrals; awareness raising activities during commemoration days; strengthening training teachers at MU and HoH pre-schools on recognizing and reporting abuse with children, in partnership with Childline Botswana; advocating for the extension of the waiver for birth registration penalty fees and launch of a birth registration campaign targeting vulnerable and hard to reach populations; and working with GeAD and other stakeholders to develop the National Gender Based Violence Strategy aimed at ending GBV by 202059. 58 TK OVC and Gender Project Semi Annual Reports, FY 2013-2014 59 Although this is an activity under objective 3 it is also reported on under objective 4 which focuses on development, implementation and coordination of national frameworks and policies that address the needs of women and children 38 As stated in the cooperative agreement, the overarching focus of KRA 3.1 was to strengthen systems at local level to prevent and address the abuse, neglect and exploitation of girls, boys and women. Interviews and focus groups with key stakeholders reveal that implementing partners have been actively engaged in this process. HPP identifies cases of abuse through CFC assessments, makes relevant referrals using a district referral book and accompanies clients on referrals to ensure proper response and follow-up. The organisation has set up a Gender Committee in Selebi-Phikwe which works closely with Dikgosi to raise awareness and address issues of GBV in the community. Furthermore, Ward Action Groups have been set up to identify the needs of children, and HPP works closely with government officials to facilitate the registration of children. HWW has worked towards referral system strengthening and has also brought key stakeholders together, including justice, police, and clinics, to discuss common protocols for dealing with cases of child abuse. Although the HWW and CRS have set up child protection committees at a number of sites and provided capacity building of these structures, some have not been successful. KWS operates through a community partnership model where ‘community mobilisers’ establish Community Gender Committees. Made up of six members who are trained, the Community Gender Committees develop action plans to assess community needs and respond to GBV and HIV using locally available resources for prevention and response. Committee members hold collective discussions to assess strengths, constraints and lessons learnt. Strengths and challenges in implementation The focus of KRA 3.1 is to strengthen systems at local level and the DCPCs and VCPCs were identified in the concept document as the targeted structures for strengthening. This was based on the assumption that these structures were already established in the targeted TK districts prior to project implementation. However they were not in place and much effort was placed on establishing these structures as the annual reports state that there was a ‘delay in the establishment of DCPCs’. It was also found that most of the DCPCs are not functional due to a lack of support from social workers who are responsible for supervising these structures. This results in VCPCs, which were put in place by the project partners, ending up operating in a vacuum with little oversight and support from the district.60 The functionality of the VCPCs is also a challenge as they are dependent on unpaid volunteers and are often dependent on one or two active and committed members. When these ‘champions’ leave, the structure collapses, raising the critical need for community ownership of such structures to ensure sustainability. The delay in piloting and rolling out of the GBV referral system has been a barrier to integrated service delivery, although the reasons for this delay are unclear. Despite this, HWW developed its own referral forms and are in the process of developing a standardised referral form. KWS has also developed their own referral system including a referral form with a feedback slip to be filled out by the service provider (counter-referral). The partnership approach used by implementing partners to strengthen systems as well as the formation of locally based Community Gender Committees to ensure locally owned solutions for addressing GBV are strengths of the programme. The inclusion of councillors and Dikgosi in these 60 TK OVC and Gender Project Semi Annual Reports, FY 2013-2015 39 structures is a sound approach as they are important gatekeepers of the community. A gap, however, is the insufficient training of committee members in child abuse and child protection. Key results area 3.2: Services that cater for and protect abused children and women increased. Planned activities versus actual activities Based on the co-operative agreement, PCI’s planned activities under results area 3.2 focused on strengthening service delivery for abused women and children and included partnering with key stakeholders to: develop and replicate models for improving response to gender based violence; establish a referral point for abused children; develop guidelines for recognising and managing child abuse; explore low-cost options for short-term safe haven to women and children, including setting up ‘safe houses’ at community level; and mobilise lawyers willing to provide pro bono legal aid for abused women and children. There is no clear alignment between the programme workplan and those activities contained in the cooperative agreement. The only activity which is clearly reflected in the workplans is the exploration of low-cost options for short term safe havens. A review of the programme work plans and annual reports reveals that a considerable number of planned activities under key results area 3.2 were delayed or not achieved, whilst some planned activities were not reported on. Activities related to exploring and creating low-cost safe havens in implementation districts were planned in the first year but initiated in the third year when PCI, Women Against Rape (WAR), KWSP and GeAD embarked on a partnership to create community safe spaces for GBV survivors. The model was to be piloted in Mochudi and Maun but no further reporting was provided on this activity. Interviewees reveal that activities related to establishing Community Safe Havens were initiated by PCI and GeAD in collaboration with KWS and WAR in Maun; however, follow-up work was still being planned at the time of the evaluation.61 Planned activities related to the development of a national tool to assess GBV prevalence at community level were initiated in the third year when PCI, in partnership with GeAD, finalized procurement for recruitment of a consultant to develop national GBV assessment tools. However, there was no further reporting on this activity in 2015.62 Although a number of planned activities have not been implemented, the programme has shown positive achievements in increasing services. Interviews and focus group discussions with key stakeholders reveal that implementing partners have provided a range of services for protection of abused women and children and men. Most organisations sampled mention that it was the first time they had been exposed to information on GBV and so service provision in this area is relatively new. CRS and HPP provide psycho-social support, counselling services and, through their home visits and referral networks, assist women and children with accessing relevant support services. KWS provides a comprehensive GBV service which includes temporary shelter to women survivors of GBV and their children; a drop in centre and counselling service for individuals, couples, families 61 Feedback from PCI is that this activity is progressing well at the time of the reporting period. 62 TK OVC and Gender Project Annual Progress Report FY2014 40 and men who are perpetrators of violence; and workshops and training sessions on GBV. Women are referred for other services including HIV treatment, economic strengthening activities and legal aid; and men have recently been targeted for awareness raising activities. Outreach activities are also carried out and there is an emergency support system through an SMS helpline. The findings from the TK Technical Capacity Assessment (TK TCAT) reveal that all organisations were fully rated on GBV, with the exception of HPP (66%) who could not claim a comprehensive reaction to GBV. Strengths and challenges in implementation There are a number of broad strengths and challenges related to achievement of KRA 3.2. It was mentioned frequently that ‘insufficient resources’ is one of the main challenges faced by implementing partners when tackling GBV in the community. In particular there is a shortage of legal services and there is also a shortage of shelters for abused women and children. This highlights the critical need for the implementation of the PCI, WAR, KWSP and GeAD partnership to create safe spaces for GBV survivors. KWS’s 30 years’ experience working in the gender sector; the quality of their services; and the inclusion of male counsellors are all strengths. Although this is not necessarily as a result of the TK Project, the project provided the opportunities for KWS to extend its reach. Male beneficiaries are positive about the service and the resultant shift away from the approach of labelling all men as ‘bad’. HWW’s services were also commended by beneficiaries as being both sensitive and an integrated package of services, which is critical for female survivors of GBV who are in particularly vulnerable situations. However, it is also a challenge for those providing the service as this type of provision takes time and they are often ‘spread too thin’ which compromises the quality of their work and leads to staff burnout. Key results area 3.3: Child welfare workforce skills development. Planned activities versus actual activities The cooperative agreement specifies that planned activities under results area 3.3 included: working with CSOs and University of Botswana to review minimal skills set and existing training programmes of the child welfare workforce to improve their performance in preventing and handling abuse; and working with DSS to review and improve the national psycho-social support programme and to integrate lay counselling programmes and the Social Work training programme at University of Botswana. Of these activities, the only one which is clearly reflected in the workplan is the review of minimal skill sets and existing training programmes. Whilst a number of activities contained in the workplans for key result area 3.3 were implemented, an equal number were either delayed or not reported on further. During the IECD pilot training of 22 pre-school teachers and selected TK partners, sessions were facilitated on “recognizing signs and symptoms of child abuse”. However, the planned activity to support DSP to conduct TOT for selected child care workforce on skills to recognise abuse among children was not reported on further. Skills development activities were picked up in the fourth year when PCI, in partnership with Childline Botswana, conducted targeted training with children and teachers at Mothers Union and 41 House of Hope pre-schools in recognizing and reporting abuse. Planned activities around sensitising DCPCs and DGCs on GBV referral systems were not implemented. Implementing partners (BBM, CRS, HPP, HWW, KWS) have cascaded training to community level. Interviews with target groups of the programme all confirm that they have attended workshops covering the theme of gender. A range of training topics were covered and target group members include: youth volunteers, IECD volunteers, gender committee members, Dikgosi and Headsmen, VDCs, DHMTs, Village Health Committees, school staff (guidance and counselling teachers) and male beneficiaries. HWW have sensitised the Crime Prevention Clusters and staff at NGOs and CBOs working in the GBV sector. KWS has been particularly active in training government officials including magistrates, district commissioners, health officials, S&CD officials and police gender focal persons. Strengths and challenges in implementation The capacity building approach used by implementing partners was highlighted as a strength as regular workshops are held with community level stakeholders to impart knowledge around GBV. The inclusion of a mentoring component is also critical and HPP’s mentorship of gender committees was highlighted as a particular strength. Implementing partners also stated that those staff members and volunteer,s who had previous training on CFC and JOL, are better equipped to tackle issues related to GBV. However, a challenge has been that the gender training was implemented late and only took place in 2014 and 2015, so this is a new concept for most of the organisations. Interviews with government officials reveal that they have not received sufficient training on GBV prevention, response and relevant legislation. A key challenge with skills development of government officials is that they are often transferred to other parts of the country. On the other hand, this could be viewed as an advantage as the skills are preserved within the government workforce. Key results area 3.4: Involvement of men as peer educators, mentors and caregivers and role models increased. Planned activities versus actual activities According to the cooperative agreement and workplans, the activities planned under objective 3.4 would focus on three initiatives: encouraging men to participate in activities and services related to maternal and child health, HIV and gender through “Men make a difference” groups ; engaging the sports fraternity where PCI and partners planned to work with Botswana National Sports Council (BNSC) under MSYC to adapt their Sports Role Modelling/Heroes programme in training peer educators with new skills which challenge gender norms and attitudes, and mobilise male involvement in support of OVC and gender project objectives; and identifying ways to involve the Botswana Defence Force (BDF) in the Sports Role Model/Heroes work by PCI building on already existing HIV prevention partnerships with BDF. A comparison of the work plans with the annual reports reveal that the majority of the planned activities were either not reported on or delayed. Despite this, the annual reports and interviews 42 reveal that a number of activities were undertaken by PCI and implementing partners to increase the involvement of men as peer educators, mentors, caregivers and role models. During the first year PCI worked with GeAD to establish strategic alliances with the Men’s Sector and Gender Links and agreed to develop a concept note on the 365 Days’ Campaign on GBV against women and children. HPP and HWW implemented activities targeting men to increase their role in the prevention of GBV. Discussions were also initiated between PCI, OVC & Gender and DHAPP programs on how to collaborate to address gender issues within the BDF - as a membership to the men’s sector. However, there was no reporting on the planned consultation meetings with MSYC and BNSC to build on their Sport Role Modelling/Heroes programme that was meant to take place in the first year. During the second year HPP, BBM and KWS all implemented some activities specifically targeting men, including workshops and GBV awareness raising sessions with expectant fathers, taxi drivers and football teams. Commemorative days were also used to reach out to men with gender messages. Activities in the work plan focused on collaboration with Men and Boys for Gender Equality (MBGE) to conduct media training and develop a GBV harassment campaign for IMD and conduct CEO’s male dialogue. The men as role models strategy was also planned. Whilst none of these activities were reported on in the annual reports, a number of activities were implemented over commemorative days to raise awareness among communities on child abuse and GBV related issues. Interviews with implementing partners reveal that men were actively targeted and engaged in their programme activities. The CRS has used community conversations to engage boys and men on gender and they have used innovative ways of reaching groups of men such as targeting football teams and taxi drivers. KWS worked closely with the Men’s Sector to target men with community outreach and education. In addition, they have targeted sports coaches and, more recently the UB rugby club as an ally in the fight against GBV. Strengths and challenges in implementation Although implementing partners have identified the importance of bringing men on board as strategic partners in addressing GBV, this has not been the main focus of the programme and a challenge has been the insufficient targeting of men with programme activities: “Gender equality has not been addressed fully. The focus has always been on the women and children. We seem to be leaving men behind because somewhere the men are seen to be the perpetrators of all these gender issues, gender violence… .in most forums, the address is focusing on women and the girl child …. It has left the men behind. In future we have to incorporate them. Bring them on board”, (IP1) A key overall challenge is that gender is a new concept for almost all of the implementing partners and it took time to get all implementing partners on board and to programme from a gender perspective. However, there is evidence from this evaluation that implementing partners are mainstreaming gender into their programme activities and this has led to a more holistic approach and to the integration of gender and HIV . Key results area 3.5: Awareness of the rights and ability to seek help by vulnerable children, adolescents and women increased 43 Planned activities versus actual activities The cooperative agreement states that to achieve key results in area 3.5, PCI planned to: develop and disseminate a user-friendly guide on major laws affecting women and children and how to get help when rights are violated; and liaise with Childline and Lifeline to strengthen existing toll-free hotlines for abuse. Whilst the first of these activities is reflected in the programme workplan, there is no mention of the second activity (strengthening toll-free hotlines). A review of the annual reports reveals that the PCI advisor met with the Head of Child Protection Unit to finalize the DCPC’s guidelines and also inaugurate the DCPCs. Implementing partners worked with the Department of Social Protection (DSP), Gender Affairs Department (GeAD), Men’s Sector and other NGOs at the district level, to disseminate both the National Children and Gender related Acts to the existing structures. The main activity planned for the third and fourth year was to provide technical assistance to the partners to disseminate gender and children’s laws and acts, targeting traditional leaders. Although there is no detail about the technical assistance provided, it is reported in the previous year that a powerpoint presentation developed by PCI on women and child related acts in Botswana was utilised and Marang, BAPR, Mothers Union, House of Hope and HPP all conducted seminars to disseminate Acts to different audiences. It was also reported that the PCI Gender Advisor “presented on the marital power act”. During the fourth and fifth year TK partners used all available platforms (e.g. GROW groups, PTA meetings, PACT school clubs) to disseminate children and gender related acts, and deliberately organized dissemination meetings targeting community gate keepers to include traditional and religious leaders.63 In early 2015 a session was held on the Children’s Act and Marital Power Acts for Mohembo and Shakawe caregivers and traditional leaders. In addition, KWS used various media platforms to encourage GBV survivors to utilise their toll free hotline number. Again although there were mixed results on planned activities, interviews with implementing partners and beneficiaries confirmed that a range of activities are being implemented to raise awareness on the rights of vulnerable children, adolescents and women. • BBM has partnered with Men’s Sector at district level to run awareness workshops on gender norms and gender based violence. • CRS has integrated gender issues and children’s rights awareness raising into their lifeskills activities with youth in collaboration with guidance and counselling teachers at schools, which include the Gender Mainstreaming Curriculum and the Go Girls toolkit. They have also run GBV workshops with the women’s income generating groups. • HPP confirmed that they hold community conversations to raise awareness and make use of the gender norms tool to stimulate discussion with the GROW group. The Gender Committee established by HPP plays a major role in raising awareness in the community as people are often not aware that “when they perpetrate abuse they are committing an offence”, (HPP_Gender Committee Chair). 63 For further activities in dissemination of acts please see activities 4.1.1, 4.1.3 and 4.1.5. 44 • HWW also integrates awareness, raising activities with their GROW groups including how to communicate with children without physical and verbal violence. The IECD volunteers confirm that they raise awareness with VDCs and traditional authorities. • KWS also integrates GBV awareness raising into all of their current activities. For example, the organisation works through the Community Gender Committees and VDCs which mobilise chiefs, church leaders and others to actively support women, children and men who have experienced gender based violence. • KWS creates awareness through radio jingles and is converting a book — “Stories of Courage” — containing 14 stories of women who have experienced GBV in Botswana, into a TV drama series running on BTV twice per week. A number of female beneficiaries in the focus group discussion confirmed that they had heard about the KWS via the TV programme and radio. As highlighted above, the activities being implemented were more generalised in nature and not specifically focused on major laws affecting women and children. However, both KWS and HWW indicate that awareness raising on laws and policies are integrated into their activities. The child abuse, neglect and exploitation domain of the TK TCAT assessed the organisations’ promotion of general awareness of national policies and guidelines on care and protection of children and it was found that all organisations, except for HPP, cite compliance with relevant policy such as the National Children’s Act with regard to awareness promotion. While organisations meet with the criteria in the domain, internal capacity to deal with the challenge of child abuse, neglect and exploitation appears bounded in cases. For example, in the case of CRS and BBM, measures taken to ensure protection of rights are handled through referrals only. Strengths and challenges in implementation A strength of the programme has been the use of commemoration days and campaigns and the use of available and existing platforms such as the GROW groups to raise awareness around GBV and rights. However, the challenge is that these are once-off events and there is a need for a more systematic and sustained approach to awareness raising interventions. Whilst the KWS TV series on GBV is an innovative approach to raising awareness, it is not accessible to the poorest and most marginalised communities, particularly in rural, hard to reach areas. In contrast, the community dialogue and/or conversation methodology is an effective way to mobilize for social change at community level and is a strength of the programme. Community outreach activities, particularly if they are carried out by locally recruited volunteers who understand the local language and culture are a strength. However, a challenge with community outreach and awareness raising is that it is costly and most organisations lack sufficient resources to expand their reach into all areas. It is critical, when facilitating awareness raising activities that links to response services are strengthened to address the increase in reporting and demand for GBV services. As one respondent noted: “The strength is that people participate, but the challenge is lack of resources. Where can you send the people you have identified? All we can do right now is continue the conversation on awareness of these issues”, (IP1_programme officer) 45 This challenge was also highlighted in the TK Annual Report (2014-2015) which states that “the program has created high levels of awareness in regard to child protection and gender-related issues. However, there are still weak service provision systems, especially in regard to child abuse and GBV”. It was also frequently mentioned that, even if community members do access these services, they are not always “user friendly” or able to offer the proper protection and response required. 5.3.2 Outcomes for beneficiaries The evaluation sought to establish outcomes at community level as a result of the TK Project. While it is too early to establish the achievement of some outcomes (due to delay in the roll-out of services), the reported outcomes provide an indication of whether results may be achieved. The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? Changes in the experience of neglect, exploitation and abuse require a more substantial amount of time than the TK programme has allowed before it can be claimed that reduction in the risk of neglect, exploitation and abuse of vulnerable children, adolescents and women is, indeed, taking place. Despite this, some emerging outcomes at community level were identified. Changes in knowledge, attitudes and practice in relation to GBV Interviews with programme staff and beneficiaries reveal that women targeted by TK interventions have increased knowledge and understanding of GBV; the legal protection they are entitled to; and the importance of identifying and reporting cases of GBV. There are also shifts in attitudes as female beneficiaries no longer see GBV as a ‘norm’ as the following respondents explain: “There was a time when some of the women didn’t know that a man was treating them unfairly or abusing them. After discussions and us pointing their rights out to them, they realize that they have been in abusive relationships with men”, (IP3 _GROW facilitator) “The issue of abuse – violence that is gender-based - some women don’t report it or even see it as abuse. The men also see it as being ‘okay’. This is seen as a big challenge. HWW have addressed this – people are more aware now that they should be reporting - thanks to the work of HWW”, (IP2_S&CD Officer) The inclusion of social issue discussions in the various economic empowerment groups has led to higher levels of awareness of HIV/AIDS, gender norms, GBV and gender equality. This has also contributed towards shifts in attitude and self-confidence, particularly for the women and girls involved in economic strengthening activities, as outlined below: “For me now at GROW, we discuss gender rights. We know that women can be anything these days…that women are leaders too. After this, I can become chairlady at the VDC and then further after that.” (IP4 GROW beneficiary) Some report that knowledge of their rights has also empowered them to negotiate safer sex practices with their partners: “We have increased knowledge regarding GBV – and to understand that this is not only physical, but includes knowledge about women’s rights as well – and a woman’s needs; even in bed (sexual rights) – so I can also put a condom on him. We have learnt to negotiate sex with our husbands 46 and quite enjoy our relationships now because we understand a lot and use the knowledge to encourage our partners on safe sex and safe behaviour”, (IP2_GROW beneficiaries_FGD) This statement highlights the critical importance of GBV prevention and response as a key tool in efforts to prevent the spread of HIV. Male beneficiaries also report improved understanding of ‘what constitutes GBV’ and the link between GBV and the spread of HIV. They report an increase in knowledge on: ‘how culture positions men as people to be feared’; the rights of women and children; GBV related legislation; access to legal aid and the importance of constructive communication. Men who have participated in the lifeskills programme at KWS report that they have changed their attitudes towards gender norms and their abusive behaviour. Some of the of these men are now advising and referring GBV related cases for counselling at the centre and some report that they are creating awareness on gender norms in their communities in their role as football coaches or community leaders. Changes in knowledge, attitudes and practice in relation to child abuse, neglect and exploitation Although mentioned less frequently, changes in relation to child abuse, neglect and exploitation were also mentioned. HWW IECD volunteers state that, as a result of the programme intervention, parents are conscious that they should no longer use their older children as minders of the younger ones. Rather that they must be allowed to go to school. They also report that parents no longer shout abuse or indiscriminately use corporal punishment on children or siblings. Some caregivers confirm that they have learnt positive discipline and non-violent techniques as an alternative to spanking their children: “We have had a shift to positive disciplining of children – we used to beat them for the smallest thing; but now we have learnt how to relate to children and how to discipline them without violence using the robot – stop-think-then act”, (IP2_GROW beneficiaries_FGD) “I used to be violent and hit children randomly but now I have learnt to show children love”, (IP_GROW beneficiaries_FGD) Respondents also mention that there is greater awareness of abuse in the community as the following quote reflects: “I have also observed that Tsela Kgopo has helped. There were abuse cases that were high in numbers and were also hidden, where you usually find that the man of the house sexually abuses the kids of the house or the uncle sexually abuses the kids but it is always hidden. But these days you can enter these homes and talk to them, and these issues are exposed and talked about. In the past when the father or maternal uncle sexually abused a child in the house, beating the child whilst raping her, mothers kept quiet because the father is the one who brings food to the house. It has helped a lot because things like that can now be exposed. When you approach people they can talk openly that yes indeed this is how things are and then you can give them the support they need and see what their next step is”, (IP4_volunteers_youth FGD) This quote highlights the important role that community awareness raising plays in exposing sexual abuse which is the most hidden and under-reported form of violence against children. 47 For those children who have access to the KWS shelter, they are now being protected from the abusive situation at home and are showing improvements in performance at school and in their behaviour. Furthermore, the awareness campaign on birth registration has resulted in more people understanding the importance of these documents and registering their children: “We would not have birth cards and IDs. HPP helped to bring Home Affairs to this community and they offered these services”, (IP3_HBC beneficiaries_FGD) This quote highlights how the programme has contributed towards fulfilling children’s fundamental right to birth registration. 5.3.3 Key lessons learnt for objective 3 The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to objective 3:  VCPCs and Community Gender Committees play a critical role in the prevention, identification and response to child protection and gender based violence at community level and they also ensure that solutions to these problems are relevant to the context and are locally owned.  VCPCs cannot function in a vacuum, they need to mirror district level structures – DCPCs – which provide support, oversight and a vertical linkage to the National Child Protection Committee.  The inclusion of Dikgosi in local structures is a good programme strategy given their standing in local communities and position as community-based mediators and gatekeepers. Their involvement also improves buy-in of key stakeholders which contributes to programme sustainability.  Working with a range of implementing partners, each with its own prevention approach, can strengthen implementation and facilitate up-scaling and reach.  The deployment of well-trained community mobilisers to facilitate outreach and the coordination of GBV and child protection related services is a sound strategy for maximising the reach and impact of the programme at community level.  The use of local CBOs also helps to improve programme coverage, particularly in rural, hard-to-reach communities.  Whilst the provision of an integrated package of services is important to deal with the multiple needs of GBV survivors, it is important that implementing partners do not run the risk of being ‘spread too thin’ and compromise the quality of services on offer.  Inclusion of male counsellors in a psycho-social support programme makes services more accessible to men who are both victims and perpetrators of GBV.  The approach of focussing on men and boys specifically, and targeting them via entry into ‘masculine spaces’ such as football clubs and taxi ranks, is a highly innovative and original method of reaching men and boys in places where they feel comfortable to talk.  Workshops and interventions with men should accompany any women-centred development initiatives to ensure that women are adequately supported by their partners. 48  Making use of commemoration days to raise awareness on GBV and child protection is a good approach for maximising impact of messages.  The community dialogue methodology is an innovative means for mobilising social change around gender norms and GBV; however, dialogues and once off awareness raising events alone are insufficient for sustainable social change, therefore systematic post-mobilisation interventions need to be planned and implemented to ensure that effects are sustained.  Awareness raising activities should include a component that focuses on strengthening the response to GBV and child abuse in order to deal with the increase in reporting and increased demand for services.  Intensive workshop and counselling interventions with men, such as those offered by KWS, have the ability to shift negative gender attitudes and change abusive behaviour.  Community based child protection structures play a critical role in awareness raising of child abuse and, in particular, the identification of sexual abuse - the most hidden and under￾reported form of violence against children. 5.4OBJECTIVE 4: NATIONAL FRAMEWORKS AND POLICIES 5.4.1 Project Implementation This section on project implementation evaluates the actual activities against planned activities, and also assesses the strengths and challenges experienced in the implementation of Objective 4: Development, implementation and co-ordination of national frameworks and policies that address the needs of children and women. The key evaluation question is: To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? The data on planned vs. actual activities is mostly extracted from annual reports, and to some degree from primary data collection. The detailed list of planned and actual activities for each KRA for Objective 4 can be found in Appendix 6. Primary data collected through evaluation has mainly provided the data on strengths and challenges. The section below reports on the five key results areas planned for this objective. The following is a list of 16 national frameworks, guidelines and policies that have been developed with TK Project assistance and support. Details of PCI’s role in the development of these documents are discussed under each KRA in the section below. Table 6: Summary of national frameworks, guidelines and policies that have been developed with the assistance of PCI Sector Document Status IECD and Life skills/HIV IECD Resource Guide Printing and distribution planned for Quarter 3 Play & Stimulation Guidelines Final copy of guidelines submitted to PCI and undergoing editing and dissemination to different stakeholders for final input 49 National life skills toolkit Developed Gender GeAD strategic plan PCI Gender Advisor participated in finalization Develop 20/20 National Action Plan towards ending GBV with GeAD PCI worked with GeAD and other stakeholders to develop the National Gender Based Violence Strategy aimed at ending GBV by 2020. National GBV Assessment Tool Finalizing procurement for recruitment of a consultant Gender and Development Policy, The GAD Policy Strategy and the GAD Policy National Operational Plan. The policy has recently been submitted for the next parliament sitting for adoption. Best practice Step by Step guide for establishing District Gender Committees Documented Gender norms facilitation guide Developed and implemented Community safe spaces guideline Developed Gender Analysis and Mainstreaming curriculum Developed Develop a popular version of gender and Children related acts and laws (communities, NGOs and the GOV partners) In October 2013, the PCI OVC Advisor participated in a two-day meeting with DSP to review and finalize the Child Protection Protocol. During the meeting, the team also reviewed and finalized the implementation framework of the Children’s Act. OVC Guidelines on the destitute housing programme to support the shelter and care needs of vulnerable families Developed. Child Protection Protocol Pending approval. Children’s Act implementation framework to support the Children’s Act. Pending approval. Probation and aftercare guidelines Developed. Results area 4.1: Support to implementation of Children’s and Domestic Violence Acts/Plans. According to the cooperative agreement, PCI planned to work with GeAD, Department of Social Protection (DSP) and CSOs to develop and disseminate a user friendly guide to the Domestic Violence Act and the Children’s Act and to disseminate it through various platforms at district and 50 community levels; and work with DSP and other stakeholders to further conceptualise and possibly expand DCPCs to become District Family Protection Committees to address issues (including gender) affecting vulnerable women, children and families. Almost all of these activities are reflected in the programme workplans except for those related to the development of a user friendly guide for the Domestic Violence Act. The majority of activities planned under this results area have been successfully implemented (see annexure 2). During the first reporting period a training workshop on gender concepts for Maatla project was undertaken; a 2 year GeAD/PCI workplan was finalised; and the PCI OVC Advisor met with the Head of Child Protection Unit to work on the DCPC’s guidelines and also inaugurated the DCPCs – this process has taken three years. PCI developed standardized presentation and facilitation guidelines on Children and Gender Acts and shared these with the CSO partners to disseminate. PCI also provided financial and technical support to DSP to conduct a two-day meeting so as to review the Probation and After Care Guidelines. In addition PCI provided support for consultative meetings for the validation of the Gender and Development Policy and planned to support its dissemination once the policy was approved. However, delays in approval of the policy hampered the plan for PCI to support GeAD with the dissemination and implementation of the policy during FY14. In October 2013, the PCI OVC Advisor participated in a two-day meeting with DSP to review and finalize the Child Protection Protocol. During the meeting, the team also reviewed and finalized the implementation framework of the Children’s Act. During the final reporting period, PCI and BONELA agreed to work together to finalize the already existing simplified Children’s Act and translate it into Setswana. In January 2015 the Children’s Act was disseminated to caregivers at Mabele. PCI has also provided technical support to review the ECD Policy with a particular focus on adapting the stringent guidelines for ECD service delivery in rural areas in partnership with the Ministry of Local Government. PCI has also supported the development of play and stimulation guidelines for 0- 6 year olds; an IECD resource training guide, including a facilitator guide which integrates information on the Children’s Act; and an accredited national training curriculum for ECD practitioners. These materials have been handed over to Bokamoso, an NGO which runs an ECD teacher training centre. PCI is also conducting visits to school hostels in remote areas to develop best practice guidelines for the protection of children in hostels, some of whom are particularly young and at risk; and has provided support for the development of the national life skills toolkit and framework for adolescents. Two child related documents were developed in 2013 – the Child Protection Protocol which guides stakeholders on how to handle child protection issues; and the Children’s Act implementation framework to support the Children’s Act. Both documents are still pending approval. Results area 4.2: Support to development and implementation of OVC policy and draft national policy in Gender and development improved. 51 The cooperative agreement indicates that PCI planned to assist DSPwith the implementation of the OVC policy and to develop an implementation strategy for the Gender and Development Policy based on a gender analysis. The project also planned to assist GeAD to align the National Gender Programme Framework to the policy; and provide GeAD with technical assistance to GeAD in establishing a National Gender Commission or Council. Two of these activities are not reflected in the programme workplans: technical assistance with the OVC policy and establishment of the National Gender Commission or Council. A review of workplans and annual reports found that most of the planned activities under this key result area have been implemented with some delays due to length of time it has taken to finalise the Gender and Development Policy. PCI worked with GAD and other development partners on the Gender and Development Policy, The GAD Policy Strategy and the GAD Policy National Operational Plan. In 2014 the Cabinet reviewed the draft policy on Gender and Development and comments from the Cabinet were incorporated by GeAD and stakeholders and re-submitted to cabinet which finally approved it in March 2015. The policy has recently been submitted for the next parliament sitting for adoption. During the first year, the PCI Gender Advisor participated in the finalisation of GeAD strategic plan. PCI also provided technical and financial assistance to development of National Gender Analysis and Mainstreaming curriculum; and documented a best practice Step by Step guide for establishing District Gender Committees. PCI also supported DSP in the development of guidelines on the destitute housing programme to support the shelter and care needs of vulnerable families. During the 2014/2015 reporting period PCI developed the CFC assessment tools and the government (Social Protection) assessment tools and recommended harmonization of the two. Results area 4.3: National management of data for OVC and gender services strengthened. The cooperative agreement indicates that PCI planned to provide technical assistance to help DSS either expand the existing database or create a new one that is more comprehensive in scope and linked to the NACA system; the incorporate a GIS component for mapping OVC services; and to review the status of existing data on OVC services to determine national priorities for research so as to inform policy. The concept document also states that PCI and partners planned to work with GeAD to conceptualise a national gender database; and to review and strengthen the National Health Information System (NHIS) to improve the availability, analysis and use of data for gender programming. All except one of these activities — the reviewing and strengthening of the NHIS — are reflected in the programme workplans A review of planned versus actual activities under this key result area shows that very few activities were implemented as planned during the first three years. However, during the 2014/2015 reporting period, PCI conducted a full assessment of the GeAD M&E system in October 2014 and compiled a report. Unfortunately the report produced for this exercise was limited due to inadequate information provided by GeAD. Despite this, significant work was conducted in FY15 on the GeAD M&E system, including training, development of an M&E framework and plan and an accompanying deskbook to operationalize the plan, developed in partnership with Peace Corps64. During this year, the PCI Country Director also met with the Director of DSP to discuss the way 64 TK OVC and Gender Project Semi Annual Reports, FY 2015 52 forward on the OVC database. The DSP Director confirmed that this matter is still pending before MLGRD. Results area 4.4: National curriculum on Gender Analysis and Mainstreaming developed and implemented. The cooperative agreement specifies that PCI would work with GeAD and international resource organisations to develop a national curriculum on gender analysis and gender mainstreaming, and then roll out the national gender analysis and gender mainstreaming training programme through a team of Master Trainers. These activities are reflected in the programme workplans. There has been successful achievement of planned activities under this key result area. During the initial year PCI and GeAD established a Reference Group and Technical Working Group for gender analysis and mainstreaming and the gender analysis and mainstreaming training curriculum was completed anda technical review pilot training was conducted in August 2013. The facilitators and participant guides and TOT workshop materials were also completed and finalised. During 2014/2015 the curriculum underwent an internal review and PCI worked with GeAD to develop a rollout strategy for the Curriculum. In collaboration with GeAD and USAID, PCI conducted the Gender Analysis and Mainstreaming training workshop for PEPFAR implementers in 2015 and it was planned that Training of Trainers (TOT) will be done in Quarter Three. Results area 4.5: Logistical support to DSS and Women’s Affairs Department Annual meetings. According to the cooperative agreement PCI planned to provide technical assistance to DSS and GeAD in the design and implementation of their annual meetings. This is reflected as an activity in the programme workplans. A review of the workplans and annual reports reveals that many activities planned under this key results area were either not achieved or not reported on. However, many of the activities reported on in the annual reports reveals that a significant amount of technical and coordination support has been provided by PCI to GeAD around planning and implementation of activities for national campaigns and commemorative days related to gender and child protection. PCI has also provided financial and technical support for documentation and finalisation of the proceedings for the National Child Care Forum in February 2013 and organized the first ever Psychosocial Support (PSS) Conference in partnership with with DSP, Marang and Regional Psychosocial Support Initiative (REPSSI). Strengths and challenges in implementing objective 4 Most of the activities under this objective focused on working closely with national government partners and the main challenge experienced here is the ‘bureaucracy’ or lengthy duration of government processes which has led to delays in implementation. This is in conflict with the donor requirement to implement activities within a five year timeframe. For example, there was a delay in the completion of the Gender Analysis and Mainstreaming Curriculum due to unforeseen competing responsibilities of the drafting team, composed mainly of the officials from GeAD. This affected subsequent implementation of the gender component of the TK program at district and community level. Government stakeholders also have competing priorities, making it difficult to commit to planned activities. However, it was noted that this improved after the MoU between PCI and the DCD were signed in 2013/2014. The MoUs defined areas of collaboration, a joint plan was developed and the 53 partnership was formally launched. After this, breakfast meetings were held every quarter and this assisted stakeholders to discuss implementation on a regular basis. Another frequently mentioned challenge affecting implementation is the allocation of government funding for certain activities. For example, there are insufficient financial and human resources for the implementation of the M&E system and database which was conceptualised and planned in partnership with PCI but requires computers and personnel to implement. PCI’s expertise and knowledge of government systems was identified by government officials as a key strength when working at national level as one government official noted, “it is easy for them to work with us”. The PCI staff members are described as having good expertise in the sector and are also skilled and experience trainers and this was identified as a strength together with their ability to assist in analysing gaps and using this to identify relevant interventions. As one respondent noted: “Before we do any activity we go with PCI to analyse the gender dimensions so we have an informed knowledge of the area”, (National government_GEAD) A challenge, however, is that there is not enough staff at PCI to carry out all activities, particularly since activities under output 4, such as materials development, are time consuming. This has resulted in capacity building not being adequately undertaken, especially at district government level. 5.4.2 Outcomes for objective 4 The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? PCI’s intervention and technical support at national level has made a significant contribution to the development of national policies, plans, and guidelines as highlighted in the Project Implementation section for objective four: “The national policy on gender and development has just been passed, PCI was involved through their Technical assistance…..they were part of the team and making sure it goes to parliament and now we have a policy we didn’t have because of the valuable contribution of partners such as PCI”, (National government_GEAD) The National Curriculum on Gender Mainstreaming will provide a standard package for practitioners to deliver comprehensive training on gender and development in line with the National Gender and Development Policy; and the national data management system for gender has also been strengthened as the following respondent highlights: “What worked well is that our data management system has been strengthened. It involved even those in the districts – contributed to better management of information in the districts”, (National government_GEAD) Respondents noted that PCI’s work on the ECD policy and a review of the stringent guidelines will contribute to improving access to ECD services in remote areas for 0-3 year olds whilst at the same time make provision for quality services and child protection. Furthermore, the IECD training guide will fill the gap in a national teacher training programme for ECD. Preschool teacher training for government was halted in 2008/2009 and private training institutions are costly and have minimum requirements for entry. 54 5.4.3 Key Lessons learnt for objective 4 The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to objective 4:  Signing MoU and developing joint workplans are important procedures to follow when working with government as they formalize relationships and clarify expectations, roles and responsibilities. It also ensures that programme objectives and activities are integrated into government workplans and are not viewed as ‘add-ons’ by government officials.  Joint planning and regular review of programme implementation with government stakeholders builds capacity and leads to joint ownership and problem solving of barriers and challenges encountered along the way.  Building partnerships with government requires time and effort and should be underpinned by a participatory and capacity building approach to create buy-in and ownership for the programme.  The approach of working both upstream and downstream is a good approach; however, the programme should be sequenced so that enough time is allocated to upstream work at the beginning of the programme to build an enabling environment for successful programme implementation. 5.5ENABLERS AND BARRIERS TO CHANGE Besides these strengths and challenges mentioned in the FINDINGS: IMPLEMENTATION AND OUTCOMES section, the following list indicates those that were mentioned the most frequently during discussions with IP staff and programme beneficiaries: • The IP facilitators and volunteers were mentioned by the majority of respondents as being a key enabler of positive outcomes. Contributing towards this assertion is the perception that IP staff are dedicated, compassionate and motivating. The good relations that they have established with community members, coupled with high levels of trust, were also noted as enablers by the beneficiaries. • The teaching, training and support from PCI has also been mentioned by many implementing staff members as they feel that due what they had learned they were able to implement their programmes most effectively thereby creating a positive change. • Another enabler that was frequently mentioned by both adult and youth beneficiaries was the positive influence and support of other group members, plus the support offered to them by family and community members. This not only motivated beneficiaries to stay in their relevant programmes, but also encouraged personal change as they felt that they had to set an exemplary example to the rest of the community. • Determination, motivation and the drive to succeed on the part of the beneficiaries was also a frequently mentioned key enabler. Unfortunately, this was linked with the perception that their options were limited and any future potential eroded by high levels of poverty, vulnerability and a lack of access to resources. Instrumental in creating these feelings of 55 empowerment and independence is the JoL methodology. It was reported that the JoL encourages and allows community members to make decisions about their own lives. As a community-orientated tool for awareness-raising and personal development, it has proven to be highly successful. • Finally, the private partnerships formulated by IPs with financial institutions, vocational training institutes and local business, were also regarded as being a major contributor to the positive outcomes reported above, in that they provide opportunities for income generation, work experience and preparation, as well as additional skills development. Attitudes around gender norms and gender roles were also noted by some of the women involved in GROW groups as a barrier to change. For example, it was noted that domestic and childcare responsibilities often limit the amount of time and energy that women can devote to their IGAs. While some IPs, for example, HWW, have made successful efforts to sensitise men regarding the benefits of women-centred economic empowerment, a number of respondents noted that there is still a measure of resistance amongst some of the male community members regarding such initiatives. Community support and buy-in to a programme seems to mitigate the impact of IP efforts. Where there was community support, staff felt that people were more excited about the programme and so the programme was able to reach many more people. There are, however, instances where the lack of community support hindered outcomes “the major factor that prevents me from changing is the community. You find that the way you want to live is not the way the community appreciates or wants you to live” (IP1 Beneficiary). This highlights that community acceptance and support plays an important part in mitigating the impact of the programme. Finally, both IP and beneficiary respondents noted that some community and group members lacked the necessary level of commitment and perseverance in the face of slow accumulation of funds. They would then leave their groups, thus limiting the amount of funds and capacity available for group enterprises. Internal conflict and a lack of consensus amongst group members were also noted as barriers to change. Again, it was observed that this demotivated group members, thus further contributing to group attrition. The deeply entrenched attitudes around gender norms are a continuous barrier to change and still require a significant amount of time for change to happen. Added to this are the cultural conventions that discourage people to seek external help for problems, resulting in them dealing with the situation ‘behind closed doors’. When cases are reported, they are often pending for a long time because of reliance on pro bono legal assistance and this is a systemic barrier to change. The limited funding and resources for scaling up the reach of gender services is a challenge. It was also mentioned that, because gender training was only conducted in the final two years of the project, there was insufficient time for partners to properly integrate gender into their programmes, thus affecting the achievement of outcomes. This partnership approach is adopted at both national and local level as it was noted, for example, the PCI technical advisors at local level work closely with the technical officers of the DCD which facilitates transfer of knowledge and expertise. PCI noted that enough time should be allocated to build trust and buy-in from government. When working with government it is important to respect people, know your boundaries, know when to ‘step back’ and be prepared to work at their pace. These have all enabled a good working partnership with government and interviewees from government confirm that the relationship with PCI is ‘working well’ and that they ‘work within 56 government processes and systems’. PCI’s participatory approach to planning and developing joint workplans enables good buy-in and ownership from government: “They worked with us right from the start – we sat with GA and PCI and said ‘how do we go about and develop the curriculum?’ and we made a contribution. This encouraged ownership – it is not a document for PCI – we were involved from the start”, (Government_GEAD) Furthermore, a key barrier to coordination and consultation activities with government is that they take time and there are the related costs for food, venue, and accommodation, which are very often not included in government budgets. Therefore, the funding provided by PCI to support these activities was highlighted as a key enabler for change. It was also mentioned that the additional human resources of the programme supplements government staff, which is sorely needed on the ground, especially in hard to reach areas. PCI’s work focuses upstream at policy level and downstream at grassroots implementation. This enables the use of evidence from programming on the ground to inform policy which, in turn, contributes towards the creation of an enabling environment at national level for programme implementation at local level. This is reflected in the following quote: “Their district presence across all districts has helped to understand the situation on the ground which has informed the policy”, (National government_GeAD) To enable the implementation of national policies on the ground it is important to undertake proper vertical communication and capacity building of implementation staff on the ground. This should often involve working through local structures as the following quote reflects: “If you want to disseminate information on a new policy or new act – have to go to the kgotla and consult – hear what the traditional leaders say, identify where the people are maybe missing something and then try to organise the workshops accordingly”, (Government_SC&D officer) 6. FINDINGS: STRENGTHENING THE CAPACITY OF IPS TO PROVIDE SERVICES While the previous section assessed the achievement, strengths and challenges related to service provision; this section turns to the efforts made by PCI to develop the capacity of IPs, and the resulting outcomes for IPs. Capacity development in relation to Objectives one to three is covered, as there was no intention to develop the capacity of IPs to engage in advocacy work (i.e. objective 4). The graph below shows the reported changes in the level skill of IP staff. Respondents were asked to rate their “level of skills” on a scale of 1 to 5 before and after PCI support, where one is poor and five is excellent. Skills in economic strengthening, IECD and Gender seemed to have improved the most. The results are discussed in each subsection below. 57 6.1OBJECTIVE 1: LIVELIHOODS 6.1.1 Capacity building support provided by PCI to IPs PCI support to IPs and programme staff focused on training on the models and tools for community mobilisation and economic strengthening. Following this, PCI supported the roll-out of these interventions by conducting weekly follow ups via phone or email as well as quarterly on-site observation and feedback sessions. Regular management meetings were also held where IP staff could meet to present on their programme roll-out as well as any achievements or challenges. Lessons learnt were shared for improvement of programme effectiveness. 6.1.2 Outcomes for IPs The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? The findings of the quantitative assessment reveal that respondents’ level of skill in economic strengthening has increased as a result of their participation in the TK Project. Similar findings emerged during qualitative data collection, where the majority of the respondents, including IP staff and volunteers, reported having limited knowledge about economic strengthening initiatives prior to their involvement in the TK Project. For example, IP staff noted that, while they had been offering IGA support prior to TK, they were not well-equipped to offer any form of follow up to the beneficiaries of such initiatives. This limited beneficiary outcomes as well as IP monitoring of effectiveness levels and good practice. Similarly, another IP respondent noted that – prior to their introduction to the GROW model specifically – the organisation had been dealing at a “…surface level…” with communities in terms of economic strengthening initiatives. Areas of improved knowledge noted by IP respondents include the ability to facilitate and establish savings groups. Improved capacity and knowledge of how to empower beneficiaries, through methods such as JoL and to take a pro-active and self-sufficient approach to their economic well￾being were also reported. The following quotes refer: “The GROW programme and Journey of Life helped us to probe and support communities to come up with – and deal with – their issues from their own point of view.”(IP3 staff) 58 “We share our knowledge with them…issues of economic sustenance…and they know that they cannot just sit idle; they must make something of themselves.” (IP4 Volunteer FGD) Having new knowledge on how to facilitate income generating and economic empowerment initiatives has led to heightened feelings of confidence for many of the IP respondents. This, in turn, appears to have improved the quality of service provision as well as relations and interactions with community members. Higher levels of trust of IP staff amongst community members were also noted while, on an implementation level, IP respondents noted that their economic strengthening interventions were far more structured and ‘comprehensive’. However, less positive results in terms of levels of knowledge and confidence to provide Aflateen / YEP65 services were reported. This may be due to the more specific or specialised knowledge required. On a personal level, some implementers noted changes in their own saving and spending behaviour, while a youth coordinator noted that she had been motivated to think about setting up her own business. 6.2OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS 6.2.1 Capacity building support provided by PCI to IPs Much of the focus was on the creation, distribution and roll-out of the IECD training package and the development of the lifeskills toolkit. This seemed to have run behind schedule, but PCI has still been able to utilise drafts when necessary. IECD training was provided to IPs over a period of five days. This covered informal play groups, child development, child care and protection, children’s rights, sanitation, HIV, psycho-social support, hygiene and nutrition. Parents were also invited to this. Lifeskills training was also provided over a period of 5 days and covered issues such as teenage pregnancy and HIV among other topics. 6.2.2 Outcomes for IPs The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? The graph above shows that IP staff feel that they have gained much skills in IECD as they rated their skills as “poor” before the TK Project, and now rate it as “good”. There has been some increase in HIV knowledge, although not to the same extent as the IECD skills. This is probably due to organisations having extensive knowledge in the HIV sector prior to their engagement with PCI. The biggest shift for IPs in relation to IECD is that staff now understand the importance of providing integrated IECD services. IPs mention the importance of dealing with children within the context of their families, health, nutrition, as well as how they are socialised into gender roles. This has led to the provision of more holistic services. “We were able to carry out our work more effectively: understanding the importance of gender roles and how they affected the child’s development from early childhood. Even the one of IECD, the training has also helped us a lot because we can do our jobs better. This is because if we are now aware that a child's development is affected by issues of protection, HIV and also gender roles.. This 65 One of the IPs combined Aflateen and YEP. 59 helps us in IECD because we can then teach them these issues and show that the roles are for all of them.” (IP_Staff) IPs do not mention any significant changes in their knowledge or skills on HIV or lifeskills. 6.3OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN 6.3.1 Capacity building support provided by PCI to IPs Implementing partners’ have all participated in training interventions provided by PCI and received on-site mentoring and support after the training. PCI support to implementing partners and programme staff focused on orientation to Gender Concepts and ways to integrate gender in HIV/AIDS programming. All CSO partners developed and submitted gender action plans to PCI for review. PCI worked with the District Child Protection Committees to prevent and address issues of child abuse, neglect and exploitation in the period of June 2013-September 2014; and provided technical assistance to CSO partners on how to establish and strengthen relationships with the police gender based violence committees, traditional leadership, GeAD, District Gender Committees and other stakeholders to address gender issues through inter-referrals. The table in annexure 2 provides a summary of the gender-related technical training and support provided by PCI to a range of stakeholders throughout the programme life cycle. 6.3.2 Outcomes for IPs The key evaluation question is: How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? The findings reveal notable increases in gender-related skills which is probably because this was a new concept for most of the organisations upon entering the TK Project. The majority of volunteers and programme staff acknowledged that they had limited knowledge about both gender and child abuse, neglect and exploitation beforehand and that the TK Project has led to improved knowledge on these topics. BBM volunteers now have knowledge of the community structures responsible for the coordination of services for vulnerable women and children and are participating actively in these structures. HPP staff mentioned that their knowledge on child related legislation and policies has increased and HWW staff indicated that they now understand gender norms and how this influences gender based violence against women. Even though KWS has been working in the GBV sector for many years, staff members claim that they have learnt a great deal from the training, particularly around gender norms. Similarly, CRS reports increases in knowledge as a result of the training. “Initially I had very little knowledge on gender issues and after PCI and GeAD trainings I became a gender activist and achieved a prize for being rated the top ten among trainees on women and health and gender issues globally”, (IP_staff) 60 The respondents from all organisations indicate that there has been a shift in their attitudes around gender as a result of gender related training interventions. Evidence of this is provided in the examples below. BBM volunteers relate that they now understand the importance of gender roles and the effect they have on a child’s development from early on. This shift in attitude has enabled them to work towards shifting the attitudes of communities through community dialogue methodology: “It happens. In the cultural dialogue, we discuss gender norms in the context of traditional ceremonies. These would be habits that have become second nature to us all. But when we interrogate them using gender norms, we make people come to realize that certain aspects of cultures encourage Gender Based Violence”, (IP_Volunteers) The Gender Committee Chair noted changes in attitude to work as a result of the training and support from PCI and HPP: “As I indicated earlier, I was familiar with HIV&AIDS issues before, but now that I’m involved in community work related to HIV&AIDS, GBV, Child Abuse, etc, I have grown personally and the quality of service that I provide to the community is higher now because of HPP/PCI support”, (IP_Gender Committee Chair). Finally, KWS has shifted their thinking around HIV and gender and learnt the importance of integrating them into their programme activities: “We used to look at GBV only before TK, and not at HIV, but TK opened our eyes to the fact that these issues can only be addressed together. It also gave us a different perspective on GBV”, (IP staff) In general, all of the implementing partners report integration of gender into their programme activities as a result of the TK Project. This is particularly the case for awareness raising activities as this HWW staff member relates: “Not including this type of discussion before in our work – not specifically as gender anyway – now we have more specific messaging that we give and issues that we discuss – their rights, that they are equal partners with their husbands, that they are not supposed to just sit in the home – but can do things for themselves and for their families”, (IP_staff) HWW reports that they are now working more with parents and educating them about the types of child abuse. BBM, HPP and KWS all report that they have strengthened their ability to identify and respond to cases of child abuse and raise awareness of this issue in the community: “In the beginning, with child abuse cases, if I saw a child being abused, I used to think, if I go into that house, I would be told to mind my own business! But now, I walk into that home, I don’t fear anything. I stand up for the rights of the child. I can even quote the act for a person and tell them they aren’t doing the right thing”, (IP staff) As this quote suggests, with knowledge of child rights and legislation, staff members feel empowered to take action and respond to cases of abuse. KWS reports that the TK Project has enabled them to provide more holistic and better quality services as stated below: 61 “You can think – in the past – it was very difficult to do a holistic or integrated provision of services – but through TK all of the possibilities were opened up for us – our programming on gender has tremendously improved”, (IP staff) This improvement in quality of services is attributed to a more integrated HIV and GBV model of intervention; increase in resources and the employment of more and better qualified staff members; increase in training activities, particularly for outreach staff; and the expansion of more comprehensive outreach services. KWS is now operating in six areas whereas previously they were only operating in three, and so they have broadened their reach as a result of PCI support. 6.4OTHER SUPPORT PROVIDED TO IPS Training in Monitoring and Evaluation was provided to IP staff Training over a period of 5-7 days. This was followed by ongoing support through technical advisory visits. 6.5STRENGTHS AND CHALLENGES OF THE SUPPORT PROVIDED BY PCI In this section we discuss the strengths and challenges of the capacity building and mentoring provided by PCI as part of the TK Project. 6.5.1 PCI capacity development and support structure In terms of capacity development, PCI adopted a cascade method; that is, capacity building was provided by PCI or PCI-contracted training service providers to IP staff (generally programme or project officers and key management personnel). These trainees were then responsible for capacitating their colleagues as well as IP-recruited volunteers, local government stakeholders and local structures, such as the VDCs. Volunteers and field staff would then ‘cascade’ the training further in that they would be responsible for building capacity and knowledge amongst community members such as teachers, home-based carers, Dikgosi and other community leaders while implementing programme activities. In addition to this, PCI facilitated opportunities for the larger, more experienced and well-established CSOs to provide technical support and capacity building for smaller IPs, specifically when capacity building in a particular area, such as gender norms, or in the use of specific tools, was required. The following quote outlines this with regard to training provided by KWS: “Also, we have been playing a role of bringing in our technical expertise and training the other partners…we have been training [them] on some of the tools like the In her Shoes…and also another one which we call cultural dialogues. We have played a very big part in terms of training.” (IP_CEO) Technical capacity building was generally conducted over a period of 3 to 5 days and covered a wide variety of skills areas. These include the following: • Methodologies such as JoL; CCE-CC; GROW, Aflateen and YEP models. • Input on gender norms, GBV, HIV/AIDS and IECD. • Resource mobilisation, including networking, proposal writing, and the establishment of partnerships with local businesses as well as other relevant service providers. • Organisational development in areas such as financial management; programme planning, monitoring, evaluation and report writing; human resource system development; and general management skills. In addition to this, input on ‘soft’ skills was also offered to IPs; for example, on leadership and communication skills. 62 Following capacity development, refresher training sessions were offered. These sessions would often combine a number of different skills areas; for example, refresher training may be offered on gender norms and M&E tool completion. In this way, follow-up support could be aligned to IP needs and capacity gaps. In addition to refresher training sessions – and throughout the programme cycle – PCI offered mentoring and support services to the IPs. A crucial component of this support was the use of PCI Technical Advisors (TAs) who served as focal points for a number of allocated IPs. This meant that TAs were available to address any IP queries or requests for assistance. TAs were also responsible for conducting weekly follow ups via email or telephone calls, as well as site visits at each of their designated organisations on a quarterly - or, failing that - biannual basis. During the site visits, IP implementation plans would be reviewed to monitor progress, while the quality of activity roll-out would be checked by means of observations and feedback. This process is further explained in the following quote: “In mentoring, you observe and then provide feedback and someone demonstrates. Then you go back the following day and observe again. For example, you go with them (IP staff / volunteers) to the homes to see how they engage – then you go back to the office to do a role play to demonstrate.” (PCI) The TAs would also follow up with district and community stakeholders during site visits to find out about levels of cooperation and collaboration with the IP. In the event that any challenges were discerned during the site visit, a corrective action plan would be compiled by IP staff, with assistance from the TA. Included in the action plan were areas requiring further technical assistance and input from PCI. In addition to TA support, PCI ran regular management meetings, where the managers from all of the IPs were brought together to review programmes and discuss strengths and challenges, as well as learning exchanges between IPs, where staff were encouraged to share implementation achievements and challenges – as well as any key lessons learned. 6.5.2 PCI approach A key strength of PCIs approach to capacity development and support – noted by a high number of respondents during primary data collection – is its highly collaborative nature. Respondents felt that the approach allowed them to be viewed and engaged with as equal programme partners. The inclusion of the various follow up and mentoring systems also led to the perception that PCIs support was consistent and relevant to their needs. Particularly appreciated was PCI’s allocation of TAs to specific organisations, as well as the TAs ‘open door’ policy. Respondents noted that this had ensured that they felt both encouraged and motivated as well as supported, knowing that someone was their specific point of contact and was available to deal with their queries. Challenges noted in connection with PCIs approach was TA capacity to provide feedback as and when requested. The high level of demand on TAs time, which had to include mentoring duties as well as undertaking various aspects of the TK Project, undoubtedly contributed to this. Other challenges relate to the use of the ‘cascaded’ approach. Selecting and training a few key members of staff from each of the IPs was a sound decision in terms of budget and time constraints. It also enabled a level of interaction and sharing between the different IPs. However, some of the respondents felt that offering training in this manner had not been as effective as that provided by PCI to all IP members. The following quote refers: 63 “Generally only 1 to 2 staff at a time attended with staff from other IPs for joint training. Idea was that those attending would return to the organisation and share their learnings with other staff. This they did, but staff noted that it was often not as thorough as if they had received the content directly from PCI – the horse’s mouth. While attending staff made an effort to pass the information along, much of the content was new to them so they battled to convey it accurately and effectively to their colleagues. Where all staff attended…felt that their capacity development was far more effective.” (IP5 Implementing staff_FGD) Similar reservations were noted where manuals and training materials were only available in English: “We do translate but we feel that by doing so we are perhaps watering down the weight of what we are trying to convey…not so sure of how to translate specific terms and concepts.” (IP2 Implementing staff_FGD) 6.5.3 Content covered during capacity building sessions All of the respondents noted that the content covered during capacity building sessions was relevant to their needs in that it had introduced them to the necessary models, methods and concepts. Furthermore, content was described as comprehensive and interesting. However, many of the respondents felt that the content had been extensive, of a high or ‘advanced’ level and therefore often ‘too much’ to cover within the allocated period of time. This was noted particularly by the volunteers and field staff and is elaborated on in the following quote: “I think that everything was fine, just that the time was too short….there is too much knowledge but little time to absorb. So if you are a slow learner then it means that maybe it is difficult for you. When you have certain questions but you are limited because there is no time.” (IP4 Volunteers) In contrast to this, some of the IP management staff felt that certain aspects of the capacity building had not been useful to them as they already had high levels of knowledge and expertise within those specific areas. 6.5.4 Facilitation of capacity building sessions Facilitators responsible for PCI capacity building sessions were described by the majority of respondents as being knowledgeable66, well-prepared, patient and approachable. A number of respondents also noted that the facilitators were skilled at ‘breaking down’ often complex and new concepts to make them easy to understand. However, when capacity building was cascaded down to those operating at ground level, this was often not enough to make the new content accessible and understandable – and respondents often felt that extra time and input would have been helpful in terms of assisting them to master the content. This was exacerbated by the fact that most of the training materials were only available in English, although the facilitators did conduct training sessions in both English and Setswana. Feedback regarding facilitation methods indicates that they were participatory, practical and engaging; and included a lot of group work and role plays, which enhanced the learning process. 66 This is supported by the input obtained that PCI recruited and allocated specialists for each programme component, such as gender or economic strengthening. 64 6.5.5 Provision of mentoring and support Once again, feedback regarding PCIs provision of mentoring and support is generally very positive. Both PCI and IP staff offered follow-ups to ensure that those capacitated understood and were able to utilise their new capacities. A key challenge, however, were the capacity constraints experienced by staff of both groups in terms of offering timeous responses and follow-up to all queries. It was also felt that, particularly with regard to ‘new’ models such as Aflateen and YEP, capacity constraints on the part of those cascading the training hindered implementers’ understanding or grasp of the content - and hence roll-out of these programmes. 6.5.6 Logistics and planning A number of challenges were noted in regard to the logistics and planning of the capacity building and support. These include the following: • Some capacity building was rolled out following programme commencement. It was felt that this had compromised implementers’ ability to undertake certain interventions. • Similarly, some of the respondents noted that mentoring was offered half way through the programme. Again, it was felt that more timeous monitoring and addressing of challenges and gaps would have enhanced levels of effectiveness. • Teaching and learning materials were often not available, delivered late or were insufficient in terms of the number of training participants. This meant that materials often had to be shared amongst staff or even between organisations, which hampered learning and memorisation of models, tools and concepts. • Invitations to training workshops were sometimes received late. As a result, relevant personnel could not attend these sessions and substitutes had to be sent in their place. Respondents felt that this too had hampered the sharing of knowledge within the organisation and hence, their level of programme effectiveness. This is further explained below: “Sometimes you don’t send the relevant people because sometimes you are given short notice when the relevant person has been assigned to do other things….so you just send someone even though that might not be the right person to be at that training.” (Centre coordinator) 6.6KEY LESSONS LEARNT FOR CAPACITY DEVELOPMENT AND SUPPORT The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to the capacity building support provided by PCI or other grantees:  While a cascade approach to capacity building is a cost- and time-effective measure, adequate capacity development must be ensured at each level to facilitate effective roll-out at subsequent ones.  A ‘one-size-fits-all’ approach to capacity building should be avoided as it leaves some participants feeling disempowered while ignoring the existing capacities and experiences of others. 65  Using larger, more well-established and experienced CSOs to capacitate and support smaller CSOs is an innovative means of addressing capacity constraints in the oversight organisation, whilst simultaneously encouraging knowledge-exchange and possible partner organisation collaboration.  The allocation of a specific mentor to each implementing organisation works well in terms of enabling sound relationship building, which in turn impacts positively on implementer levels of motivation and engagement.  Post-training follow up, refresher courses and mentoring are vital to ensure sound and sustainable capacity development.  Using a variety of support methods, which are aligned with the needs and capacity levels of all of the different implementing organisations and their staff members, is equally essential for sound and sustainable capacity development. 7. FINDINGS: SUSTAINABILITY In this section we assess whether the TK Project is sustainable beyond the USAID funding. 7.1EFFORTS TO IMPROVE SUSTAINABILITY To ensure sustainability IPs needed to improve their skills in resource mobilisation and diversify funding. PCI provided capacity building on resource mobilization, with a particular focus on public-private partnerships (PPP). This has encouraged IPs to become a lot more purposeful in their resource mobilisation, for example HPP has set up a Resource Mobilisation Unit focussed on fundraising. Organisations have strengthened their PPPs. Partnerships with private companies are able to provide organizations with the necessary resources (e.g. funding, donations) and support (e.g. capacity building, pro bono services) they require. Barcley’s Bank, as mentioned under the Economic Strengthening objective, has been one of the success stories in this regard, as they support various adult and youth economic strengthening activities. According to the Mid-term review report, implementing partners have been mobilised around US$ 1,198,92567. Furthermore three of the five organisations sampled seem to have made a concerted effort to apply for further funding from other development agencies such as Global Fund, the UN Trust and the Government of Botswana, to name a few. Some funding has already been secured, while others are still awaiting the outcome of their application. 7.2PERCEPTIONS OF SUSTAINABILITY Half of the IPs interviewed indicate that they believe that their services will continue after funding by USAID ends. In some of the cases, services will be scaled down in terms of reach or number of services offered. This may be a challenge for the sustainability of the CFC approach. However, certain interventions 67 USAID & Oakwood and Associates and Consultants. (2015) Mid-term review of the Tsela Kgopo Project. 66 are well designed to be self-sustaining, particularly GROW and the mobile Stimulation and Play Groups, as they are low cost and community driven. 8. FINDINGS: PROJECT MANAGEMENT The section examines the extent to which organisational factors either enabled or hindered implementation. 8.1HUMAN RESOURCE CAPACITY OF THE TK TEAM The Tsela Kgopo OVC and Gender Project was managed and implemented by ten staff, which included: the PCI Director, Tsela Kgopo Chief of Party and five Technical advisors. The PCI Director and COP are responsible for leadership, co-ordination and supporting advocacy efforts at national level. Technical advisors were responsible for: • Developing tools and content for their capacity development processes • Providing technical guidance to other programme advisors on how integrate services (particularly for gender) • Providing training and ongoing mentoring support to IPs • Assisting with training of the community • Engaging in the development of national policies, frameworks, etc. The extensive knowledge and expertise of the PCI team was acknowledged among interviewees. Team members were appropriately qualified, highly experienced, and brought in with them valuable networks, especially within the Ministries. The expertise of advisors was complemented by PCIs partnership approach which the majority of IPs described as being open, supportive, and respectful. The following are reflections from three IPs : “The spirit of their support is nice. You are free to be honest with what is happening. We can tell them our problem. They do not mind repeating till you understand.” (IP_SSI) The biggest criticism across stakeholders interviewed was the inadequate human resource support for the implementation of the programme. Although Technical Advisors are highly capable, the range of tasks required placed a huge demand on their time and skills. Advisors also did not have sufficient project administration/co-ordination support (for example with organising workshops). Limited funds did not allow for the expansion of the team. As a result of this, PCI staff feel overworked. Likewise, volunteers working at the level of IPs have also been overstretched, as they have to travel across vast geographical distances. The use of CFC approach also placed huge demands on the skills of volunteers who very often have low formal qualifications. Volunteers are emotionally strained in dealing with their community challenges and low volunteer salaries (between P 0 -1000 per month) which prevents the programme from attracting and retaining adequately skilled/qualified volunteers. As a result, volunteer turnover is high in some instances. 8.2PLANNING AND CO-ORDINATION 67 8.2.1 Planning process The planning of the TK Project was preceded by a needs assessment which explored the gaps and challenges faced by government. Project objectives were developed by USAID, and PCI then had the opportunity to design the programme to meet these. Two of the IPs indicate having been part of the conceptualisation of the TK Project; these two IPs also show the most buy-in to the CFC approach. There was no further design after contract. Only one major shift was made to the agreed Objective 2. Positive Innovation for the Next Generation was dropped as an IP, as recommended by USAID. In terms of the partnership approach, another shift was also that PCI would work directly with all ten implementing partners, as opposed to the three key ones as originally conceptualised. Annual programme planning is conducted between PCI and IPs. A Project Management Team comprised of NGOs and PCI are responsible for planning processes. 8.2.2 Documenting the Programme Logic Model The TK OVC and Gender Project Logic Model and Theory of Change is not well documented. While the Co-operative agreement outlines the key intentions of the proect, and annual work plans outline the deliverables for each year, the two documents do not always correlate. There is furthermore no single Logic Model summarising the entire programme implementation (key outputs in terms of services but also capacity building) and how this links to the intended outcomes (short, medium and long term) for both IP and beneficiaries. This is a huge challenge for monitoring and evaluation. Furthermore there are key gaps in services represented in the current objectives and key results areas. While HIV interventions is captured under Objective 2 (IECD and Youth Developmental Interventions), the integration of HIV across all objectives for all target groups, is lost as the objectives limit HIV to Objective 2. Outputs and outcomes for IPs, in terms of their technical capacity, are also underrepresented in current objectives and KRAs. A key aspect of the Theory of Change, as we understand it, is that IPs need to be strengthened to provide and sustain services. Without change at this level, the service delivery results will not be achieved and so outcomes achieved here should not be undervalued. 8.2.3 Internal co-ordination and integration Integrated planning between Technical Advisors seemed to have been a challenge for staff; while there are individual programme strategies, a combined plan does not exist (as pointed out above). A number of factors have hindered integration: • there is no deliberate structure/process to ensure integration across technical areas • even where efforts were made to improve integration, Technical Advisors reverted to what they were comfortable with during implementation. • some Advisors struggled to see any overlap between technical areas; particularly those that were longer standing, as they had an established way of working. 68 The challenges around co-ordination and integration was felt by IPs who mention that there would be disagreement between trainers, and that training would sometimes also overlap. Technical Advisors have recently starting working more collaboratively, as they see the advantages of such an approach. Having TA backstopping organisations around all technical areas has helped to ensure shared ownership of other technical areas. Advisors explained that they could have maximised on project reach and outcomes had an integrated strategy been taken earlier. There has been some internal capacity development, although this was not seen as sufficient for TAs. The TK strategy could have benefited from maximising on target group outcomes through ensuring that integration happens. 8.3MONITORING, EVALUATION, REPORTING AND LEARNING The introduction of new PEPFAR indicators three times within the five-year project implementation period was a major challenge for PCI and its IPS. The change meant that implementers needed to be retrained to collect data on indicators, and familiarise themselves with a new set of indicators and data collection tools which was already a challenge, especially for volunteers, to comprehend, and changes needed to be made to the data management system. This resulted in incorrect reporting, i.e. under or over-reporting, on indicators due to lack of understanding of the parameters of indicators. For some it also meant that IPs had to change their strategy of working with community members. For example, the definition of “active beneficiary” changed, and so unless you supported a client once per quarter, you would not be able to include them in your data on beneficiaries reached. This created pressure for IPs to reach numbers, which they felt at times compromised the quality of their work. It also prevented IPs from using the last couple of years to consolidate their work. Definitions of indicators were also sometimes seen as inappropriate for the TK Project and targets were also reportedly unrealistic given the sparsely populated areas that IPs are working in: “So you have a target of reaching 2140 people per quarter, but it takes you an hour to walk from one homestead to another. The programme is not relevant in terms of implementation and realities on the ground” (IP_Staff) There seems to be little alignment between the various elements of the monitoring system. Indicators are generally not clearly aligned to the programme plans (i.e. objectives, KRAs, or work plans), but operate as a separate entity. Quarterly reports by PCI also do not always cover what is in the work plan. It therefore becomes difficult to judge successful performance on an objective or KRA. There were mixed feelings about the M&E data management system. There was some resistance to the system at first and initial challenges with the setting up of the system was a further barrier for organisations. The ongoing support provided by PCI through site visits and training, although unplanned, was instrumental in the institutionalisation of the M&E system and enabling IPs to report against PEPFAR indicators, as they needed a lot of handholding. Some organisation feel that the M&E journey with PCI and USAID have led them to strengthened M&E systems and practices that will be sustained beyond the programme, while others still lack sufficient buy in and confidence to use the system: 69 “It’s great because right now when we need information we don’t need to go to the files and start counting… the information in in the database. So it’s really been of benefit. When it comes to M&E there was a time we could go to them and say “can you train all of us?” If we did not have the partnership we would not be where we are in terms of reporting…. That was the highlight for me! ...Beyond TK we have the skill, we have the technical knowledge, we have the database so we will continue to collect data. We will continue to analyse data. We will continue to use the data for planning” (IP_Director) Data quality assurance was provided by PCI. USAID conducted data quality audits, SIMS database, and support around M&E. PCI submitted quarterly reports which were followed up with meetings with the PCI Director. Some of the better performing organisations have used the data generated through monitoring for informing the planning of their interventions. Some indicate discussing findings with PCI, and others not. “This is a great job because it give us regular feedback on our performance and help us improve as well as set new plans for execution” (IP_Programme Officer) The Programme Management Team, comprised of the Director or Programme Managers of IPs and PCI met quarterly to discuss progress on project and their improvement plans. PCI conducted quarterly sessions for a review of progress on activities. While this allowed for a good check on progress, it was not necessarily a space for the discussion of strategic issues. 8.4PARTNERSHIP BETWEEN PCI AND USAID Key evaluation question: To what extent has USAID fulfilled its role in the activities by providing adequate funding, responsive managerial oversight and technical support, and assistance in solving problems in implementation as they arise? The total project budget for the five-year period was $ 16 million, which was regarded by PCI as being inadequate given the scope of work. The IECD programme, for example, could not cover what was required to provide basic integrated ECD (a meal and stimulation toys) although this was proposed as part of the minimum package of services for children. Transport was also not adequately budgeted and volunteers at one organisation complain about having to finance projects (e.g. buy seeds for gardens). Although the proposal formed the basis of the co-operative agreement with USAID, USAID indicated that there was space to flag challenges, and motivate for further funding. Challenges with funding releases by USAID were experienced a couple of times during implementation, with the longest delay being around seven months. This had a huge impact on implementation as lower cost activities (which excluded training) then had to take priority. Budget cuts also meant downsizing IP staff which added to the pressure already felt by staff. Both USAID and PCI agree that their partnership allowed for open and frank discussions around the project. PCI appreciated this and the effort that was made to understand the TK Project and provide strategic/technical support where possible, even though this was not considered to be the 70 role of USAID. Interaction with USAID happened mostly at the level of the COP and PCI Director. Technical Advisors indicate that more technical support (advisory and capacity development) from USAID would have been appreciated and would have contribute positively to their job satisfaction, as they would have felt invested in. PCI felt pressurised by USAID to deliver on Objective four (development, implementation and co￾ordination of national framework on women and children). However, PCI firmly believed that for these objectives to be owned, sustained and implemented by government, the process needed to work at their pace. Thus, much work had to go into nurturing a relationship where PCI could be trusted by government. This took more time than anticipated. The programme design was perhaps also overly ambitious about expecting deliverables in the specified timeframes. 8.5CO-ORDINATION WITH GOVERNMENT STAKEHOLDERS The complex problems of GBV and OVC require a variety of interventions spread across multiple Ministries and sectors – government, NGOs, international NGOs and private sector. The TK programme is a multi-level and multi sectoral programme which requires coordination at both policy and programme levels. The extent of coordination with government stakeholders was assessed at national, district and village level. 8.5.1 National PCI attends the Project Advisory Committee at national level on a quarterly basis which includes USAID, FHI and the Directors from DCD, DSP, GeAD, MOE and NACA. At this structure the partners present their reports and discuss any challenges with the programme. Although PCI coordinates with various departments at national level (see below), there is no central coordinating body within government responsible for leading and coordinating the TK Project. As such, there is no lead Ministry or department tasked with the strategic direction of the project, ensuring that the right infrastructure is in place to lead coordinated activity at national, district and ward level. This is mirrored at district and village level, where implementing partners are sitting on various structures which fall under different departments – this is discussed further below. At national level PCI has coordinated with DCD and DSP (Ministry of Local Government); GEAD (Ministry of Labour and Home Affairs) and Ministry of Education and Skills Development. However, it does not appear that there was much collaboration with NACA and there is no coordination with the Ministry of Health. This is a critical gap, particularly since many of the partners working downstream coordinate with health officials, for example, BBM works closely with the community home based care programme (CHBC) which is the responsibility of the MOH. PCI has an MoU with DCD; they meet formally on a quarterly basis; and there appears to be good coordination between the two. The coordination with GeAD is also good and is facilitated by joint planning and TK activities are integrated into the department’s workplans. Although joint workplans are developed with DSP, PCI’s coordination with this department is not as strong. A challenge here is that the department commits to workplans but does not follow through. The level of coordination with the Ministry of Education and Skills Development appears to be limited to ‘joint participation on various projects’. 71 PCI’s partnership approach; MoUs which formalise the relationship; joint work plans with shared objectives; and quarterly meetings all contribute to strengthened partnerships. “They (PCI) involve us all the way. At the beginning of the year we start our planning cycle and we look at what is in their plan and our plan and align our plans”, (National government_GeAD) 8.5.2 District The DMSAC and DHMT were mentioned as the key coordinating structures for creating linkages between implementing partners. Both HWW and BBM work closely with and report regularly to the DMSAC. Whilst PCI and partners’ coordination activities have been focused on both upstream and downstream levels (discussed further in section 6.4.1), there is a gap in coordination at ‘midstream’ or district level. A common theme emerging amongst interviewees is that coordination with government at district level is poor. The main reason given is that not enough effort was put into building linkages with this level: “The problem is that PCI only deal with the organisations and not government. PCI should have come to the DAC office to get a better level of understanding of what is happening on the ground….the government would have engaged politicians and more traditional leaders”, (District government_DAC) In addition to this there was limited capacity building of district government officials to support, monitor and direct the TK Project. The result is that coordination is often dependent on relationships with individual officials rather than being institutionalised and this is exacerbated by the regular relocation of government staff. District government officials also indicated that the TK Project puts strain on already limited resources as the following quote reveals: “I came across TK in 2014. I went for training with them. Its implementation was difficult in our office because we found that we had tasks that exceeded the resources that we had”, (District Senior Welfare Officer) One exception is HWW which has strong links with government at district and local level. HWW reports to both the DMSAC and DHMT and has a good working relationship with the DACs. They regularly request PCI to target the relevant stakeholder at national level to strengthen vertical coordination. This strategy also strengthens the oversight and accountability of district structures by national. Efforts are being made by KSS to improve coordination by establishing a ‘stakeholder forum’ of ‘focal service providers’ in the various districts who are involved in prevention and response to GBV. It is hoped that this will assist in provision of a holistic package of services to clients. 8.5.3 Village/Ward The key coordinating structures at village/ward level are the VDCs, VCPCs and Community Gender Committees and the TK Project has played a key role in establishing or strengthening the latter two structures. 72 There is evidence of joint working together with government stakeholders at this level. This happens through referrals for services; sharing of information and resources such as transport; and attendance at various meetings and training workshops facilitated by implementing partners. HWW is working together with SC&D to identify OVCs; and KWS, through its community partnership model works closely with Community Gender Committees and VCPCs (where functional) to identify vulnerable families. BBM continues to work with home based care under the HMT and with the social worker under S&CD. Whilst CRS works with the District Department of Education and the S&CD, most of their work at local level is with community leaders, VDC and Councillors. Feedback from government officials reveals that they are highly appreciative of the support provided by the partners as it complements their work and assists with ‘filling in the gaps’ where government cannot reach. Although there is evidence of working together, respondents gave mixed responses when asked to assess the level of coordination between implementing partners and government stakeholders at village level. The main reason is that the village coordination structures are not always in place or functional. It was also highlighted that partners are not working closely enough with the VDCs to prevent duplication of services: “The VDC… Linking with that particular structure can help them (partners) better their programs as the VDC includes the local councillor, area MP, the Kgosi, government employees, and is therefore at the forefront of developments as the “parliament of the village”, (CDO S&CD) Other barriers to effective coordination with government at village level includes the change in membership of coordinating structures resulting in loss of institutional memory about the programme; and the time and effort it takes to carry out coordination activities and sustain relationships which is made worse by competing priorities of government officials. 8.6LESSONS LEARNT FOR PROJECT MANAGEMENT The key evaluation question is: What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The following key lessons have been learnt and can inform future improvements to project management:  The PCI (and IPs to some degree) organisational structure did not adequately support the implementation of this programme. In light of this, the achievements are commendable.  Opportunities for combined programme planning, reflecting on integrated approach, and accountability for this method of working could have improved collaboration between TAs.  It is important that M&E processes do not overtake implementation processes and that the data collected is localised and useful for IPs.  The programme design needs to include an initial preparatory phase to allow sufficient time for building government buy-in and to generate evidence to inform programme interventions. 73  Without a centralised coordinating structure, national programmes such as TK will have limited strategic direction and coordination from the side of government.  Strengthening systems at the district level (midstream) should be included as a component of the programme design as districts are responsible for oversight and implementation of national strategies and programmes. It should include building the capacity of district government officials to plan, monitor and evaluate GBV and OVC programmes. 9. CONCLUSION In evaluating the strategic approach and theory of change, the findings show that the TK Project, with its comprehensive family care and ecological systems approach, is well conceptualised to identify and meet the needs of OVCs and their families68. The project is perceived to provide services that are complementary to government, either geographically or in terms of the scope of services. Challenges in the design lie in that particular sub-groups of OVCs (e.g. children on farms and streets) are not purposefully targeted through the project. Furthermore the establishment of good referral systems by IPs has also been a challenge. The CFC approach is perceived to place much demand on the capacity of IPs, who very often relies on community based workers to provide a range of specialised services. In terms of the strategic approach and theory of change, addressing these challenges will strengthen the effectiveness of the project. A cascading model of capacity building was used to strengthen IPs to deliver services. This strategic approach worked well in terms of budget and timeframes. Feedback on capacity building suggests that the expertise and capacity building approach of the PCI team was supportive and well suited to the needs of its IPs. The cascading model was however challenged by the lack of capacity of TAs to provide sufficient mentoring support and the insufficient cascading of skills by trained IPs to the level of implementing staff. The project was generally overly ambitious, and more grantee organisational capacity would have had multiple benefits for the implementation of the theory of change, one of which would have been better mentoring support for IPs. Despite these capacity building challenges, the TK Project has definitely provided organisations with a platform to grow in terms of reach, and their technical and organisational capacity. IPs indicate that their skills in economic strengthening (particularly in relation to the GROW model), integrated ECD and gender have improved the most. This has improved IPs level of confidence to work within these sectors, and has enabled the strengthening of IPs’ existing services or enabled them to venture into new services. IPs, through the TK project, have thus improved the quantity and quality of their services, suggesting that they are in a fairly good position to achieve the higher level outcomes for OVCs and their families. The sustainability of these services is however questionable as only half of IPs feel confident that they will continue to provide current services post USAID funding. The evaluation questioned whether the project has been implemented according to design, and whether it is likely that that the project will achieve their desired end 68 There is no formal theory of change or logic model, and so envisaged changes refers to those represented in Figure 2: TK OVC and Gender Project Theory of Change. 74 results. The latter evaluation question was challenging to answer as there was no results framework. However, the evaluation assessed the emerging outcomes for each objective. Generally, performance against activities set out in work plans and the co-operative agreement was varied. This is also hindered by reporting that is not aligned with work plans. However, in terms of achievement against Key Result Areas, results are positive. Objective 1: Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women, improved. Most of the Objective 1 activities captured in the cooperative agreement and work plans were implemented as planned according to activity design. A key strength of this objective area is the design and use of community engagement tools used as part of the CFC approach (i.e. CCE and JoL) which is adaptable and encourages community ownership and sustainability. The GROW model is reportedly more popular than the youth economic strengthening activities. A number of IP staff felt that the GROW model is relevant, easy to implement, and provides a good platform for engaging participants on other issues such as gender. The youth economic strengthening initiatives in comparison were challenged by implementing staff who did not feel sufficiently skilled to provide mentoring support to youth. Public and private partnerships were formed to facilitate the provision of skills development and mentoring services to beneficiaries. The GROW, YEP and Aflateen models are all challenged by high attrition rates and insufficient contextualisation in terms of market needs, environmental threats and alignment with public and private economic development opportunities. A number of positive outcomes were identified for economic strengthening which are reportedly as a result of the TK interventions. Women and youth have improved knowledge and; as well as improved communication and leadership skills. A substantial number of women and youth beneficiaries noted changes in their understanding of saving, budgeting and business development; self-confidence; self-esteem and communication skills. With the few examples of sustainable IGAs, the economic strengthening interventions have not yet made a fundamental difference to income levels or livelihoods. Furthermore, participation in GROW and youth groups has enabled a sense of belonging, security and community cohesion. Objective 2: Developmental interventions for vulnerable children and adolescents improved. Comparing planned activities in the Annual Work Plans with activities reported in the Annual Progress Reports appears to demonstrate a fair achievement of planned activities. Some of the successes of this objective area is the reach of IECD services, improved perceptions of parents about the importance of ECD, the use of creative models (such as the mobile IECD model) for engaging in IECD stimulation and play groups. Challenges have mostly been in relation to the finalisation of the IECD and life skills guidelines developed and lack of sufficient resources to support IECD initiatives with a nutritional snacks and stimulation materials required for good quality IECD. The stringent structural requirements for ECD centres have furthermore seen the closure of some ECD centres. Positive outcomes for IECD have been felt in terms of more holistic service delivery by IPs as well as parents being more conscious of the holistic needs of their children. This has led to better access to IECD services for children and an enthusiasm for learning among children. Youth targeted with lifeskills interventions have reportedly improved their self-confidence, communication and decision making. HIV/AIDS and health interventions have contributed to related positive changes in awareness, knowledge and practice, including HIV testing. 75 Objective 3: Risk of neglect, exploitation and abuse of vulnerable children, adolescents and women reduced. IPs and PCI have undertaken a range of activities in order to reduce the risk of neglect, exploitation and abuse of vulnerable children, adolescents and women. These included the strengthening of community structures; inclusion or expansion of gender and child protection services offered by IPs; and capacity development of duty bearers (IPs, teachers, social workers) in the area of gender and child protection. Key strengths for this objective include the multi-sectoral partnership approach; establishment of local structures to coordinate prevention and response services; the good quality outreach, counselling and shelter services provided particularly by KWS; and the use of available and existing platforms, commemorative days and community dialogue methodology to raise awareness around GBV and child protection. The main challenges with implementation include the delays in achieving some of the planned activities (e.g. training for partners and GBV referral system) and poor oversight of the VCPCs due to poorly functioning DCPCs. Some emerging outcomes were identified at community level. Women report an increase in GBV￾related knowledge and improved attitudes which have, in some instances, reportedly influenced their ability to negotiate safer sex. Intensive interventions like the ones offered by KWS for men show positive results in terms of changing the abusive behaviour of men sampled. Although there are changes in in relation to child abuse, neglect and exploitation, these were not as significant as those mentioned for gender. Objective 4: Development, implementation and co-ordination of national frameworks and policies that address the needs of children and women improved. Many of the activities under objective 4 have been achieved. The main cause of deviations or delays in achievement was due to lengthy duration of government processes and the limited staff capacity at PCI. PCI has contributed to the development of 16 national documents, including ECD, Lifeskills, Gender and OVC frameworks, policies and plans. PCIs work at national level has therefore made an important contribution to the development, implementation and coordination of national frameworks and policies that address the needs of children and women. This has been enabled by PCIs partnership approach, their understanding of government systems and the funding provided for meetings and conferences for coordination activities and processes. However, not having a lead department or coordinating body within government at national or district level is problematic in terms of the co-ordination and sustainability of the TK project. In terms of the likelihood of the project achieving their desired end results, while there are signs of positive outcomes, across all objective areas, there is insufficient evidence to conclude on whether this will in fact lead to improved quality of life of OVCs. This would require an impact evaluation. In terms of the support provided by USAID for the implementation of activities, there was generally a positive and enabling relationship between USAID and PCI. Budgetary constraints, funding delays, and shifting monitoring and evaluation requirements by USAID however challenged project implementation. The TK project would have benefited from more structured and combined programme planning (i.e. within PCI and between PCI and USAID) which focussed on implementation and results. 76 10. RECOMMENDATIONS Key evaluation questions: • What modifications, if any, to the strategic approach and Theory of Change have potential to substantially increase program effectiveness? • What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? The recommendations below have been structured according to the various sections in the report. Within each section, recommendations are divided into those that we suggest be pursued within the remaining project period (i.e. short term recommendations), and those that are relevant for future programming (i.e. longer term recommendations). These recommendations provide guidance for future funding priorities for USAID. Furthermore, recommendations require further interrogation and detailed planning to clarify roles, priorities and implementation details. 10.1 PROGRAMME DESIGN AND APPROACH Short term • More support is required for strengthening referral networks of IPs and this should be facilitated by PCI. This should include better facilitation of partnerships at District and Village levels. Training should also be conducted in referrals, case management and softer skills on stakeholder relations. Long term • There needs to be a purposeful effort by future grantees, IPs and the GoB to target children with disabilities, children on farms, children on streets and children who have dropped out of school in future OVC programmes. This should include the capacity development of service providers to provide support to these groups. • Programme strategies for including men need to be implemented or strengthened by future grantees, IPs and the GoB where they do exist. Monitoring and evaluating the effectiveness and outcomes of such programmes will be crucial for documenting these models and learning from implementation. • IPs need assistance with developing a strategy for implementing the CFC approach. Grantees working with IPs should provide such support which should include: guidance on how to strengthen existing services (before introducing new ones); a menu of options (including home-based, group-based, referral, and mobile outreach services) of how to deliver the basket of comprehensive services; a phased approach to the delivery of comprehensive services which should make provision for the capacity development of staff and implementers, conducting a community mapping of services available, and the implementation of services. • Access to transport for volunteers need to be facilitated to enable better reach in terms of numbers and geographical location. The provision of bicycles for volunteers should be considered by USAID for future programmes. 10.2 OBJECTIVE 1: LIVELIHOODS 77 Short term • As the programme nears its final phases, PCI should focus on strengthening IP capacity to regularly and effectively monitor the compilation and implementation of community action plans. Long term • The GROW model demonstrates a lot of potential. However, addressing the following areas may enhance its effectiveness: o Grantees should consolidate linkages with existing government economic and livelihood programmes; for example, Iphelegeng, prior to future programme roll-out. This will allow for areas of complementarity to be identified and incorporated into CSO interventions. o Grantees, in partnership with public and private partners, should facilitate participatory decision-making processes regarding appropriate IGAs. All relevant stakeholders should be offered the opportunity to provide input regarding sound business strategies and pertinent market needs / gaps. This may also encourage public-private-CSO collaboration. o IP capacity to facilitate IGA decision-making processes and mentor economic strengthening initiative beneficiaries should be supplemented by grantee support and through private partnerships with training and affiliated business service providers. For example, capacity building by institutions such as banks should be provided to IP staff first – then to beneficiaries – to ensure that adequate IP capacity is set in place to sustain beneficiary skills development. • The Aflateen programme must be adapted by grantees to ensure that the content is relevant to the needs and interests of the youth of Botswana. This might include a higher level of youth-orientated content, as well as more practical or experiential teaching and learning methods, to ensure that the programme offers capacity building in a more “youth friendly” manner. • For programmes such as the YEP, grantees should ensure that linkages with appropriate government and private sector youth empowerment interventions are in place to further benefit programme participants - and ensure sustainability of youth development and entrepreneurial outcomes. This includes the identification of government youth programme entry / access requirements so that capacity building - in line with these specific requirements - can be incorporated into future youth economic empowerment programmes. • Grantee organisations must oversee the translation of Aflateen and YEP training and learning materials into Setswana to ensure better facilitation and learning processes. In general: • Grantees must negotiate partnerships with private and public stakeholders upfront / prior to programme roll-out. Partnerships should be formalised through MoUs and clear agreements to ensure that adequate support structures are in place for entrepreneurs and IGAs prior to intervention roll-out. • The provision of specialist business advice (e.g. business planning) needs to be strengthened for both youth and adult interventions. Therefore, grantees should recruit facilitators and / 78 or mentors from amongst professionals as part of local business CSR programmes, to ensure that the necessary level of specialised knowledge and insight is available. • Interventions that are male-specific / male-targeted need to be included in future economic strengthening initiatives. These could be developed collaboratively by grantees, IPs and local men’s organisations, such as Men’s Sector. Furthermore, grantees should ensure that interventions aimed at male community members are rolled out simultaneously to women￾centred initiatives – not only when issues arise. This will help to ensure that women receive adequate support from their partners and male family members, plus it will encourage men to participate in economic strengthening initiatives. • The JoL methodology has proven to be a highly successful community mobilisation method for all community-based initiatives and, as such, should be up scaled and continued into subsequent programmes by grantees. 10.3 OBJECTIVE 2: DEVELOPMENTAL INTERVENTIONS FOR CHILDREN AND ADOLESCENTS Long term: • Parents and caregivers should be engaged through formal structures and informally to sensitize them on the importance of ECD. • Facilitators and volunteers of IPs require a deeper understanding of IECD so that they can implement this during home visits. • Dealing with youth, particularly around HIV, SRH and lifeskills, requires specific skills. Capacity building around youth friendly facilitation methods is required in this regard and should be facilitated by future grantees. • Mobile IECDs or ECD services linked to existing community structures should be upscaled (by GoB and CSOs) as this allows for accessing of hard to reach areas, and facilitates the provision of IECD within the structural constraints of the ECD legislation. • Having better alignment with government programmes such as Ipelegeng would bring opportunities to provide IECD services under the Poverty Alleviation Programme. Such opportunities should be explored with the Presidency and Department of Local Government. • IECD services implemented by grantees through IPs need to make provision for some nutrition and stimulation materials. This is relevant due to economic context within which these IPs are operating, and also due to the fact that the communities already, through facilitating these groups, have shown commitment to its sustainability. 10.4 OBJECTIVE 3: NEGLECT, EXPLOITATION AND ABUSE OF VULNERABLE CHILDREN, ADOLESCENTS AND WOMEN Short term: • For the remaining project period PCI in partnership with the Department of Community Development should develop a strategy for strengthening the role of DCPCs to play an oversight role of VCPCs. 79 • PCI should finalize the guidelines for DCPCs and provide continued technical support to DSP and DCD in the implementation of these guidelines including the development of a strategy for disseminating and building capacity of district officials on the guidelines. • The work done by Measure Evaluation on the GBV referral system will go a far way to improving access to an integrated package of services. This should be prioritized for the remaining project period and should include a formal referral mechanism together with a clear set of guidelines for referral between the key child protection and GBV stakeholders be developed and piloted. This should be accompanied by a capacity building strategy targeting all government and non-government actors involved in direct service provision at district and local levels. Legal service providers (pro-bono) should be included in this referral chain to improve legal support to survivors of GBV. Long term: • Grantees in partnership with implementing partners and the Department of Community development should make efforts to support DCPCs and VCPCs in the future. This should include as far as possible: strengthening DCPCs to fulfill their role in the monitoring and oversight of VCPCs; developing ToRs or MoUs for all the members of the coordinating structure which includes clearly defined roles and responsibilities for all agencies; clear lines of accountability between the local coordinating structures and the district and national levels; and capacity building of members to undertake their roles. (PCI, implementing partners and relevant government departments) • There is still a critical need for community based safe spaces for GBV survivors particularly in rural, outlying areas, therefore the activities initiated under the TK Project to pilot and roll out these structures should be sustained. This process should be led by GeAD in partnership with future grantees, KWS and WAR. • Grantees, IPs and relevant government departments should develop a strategy for public information campaigns on gender norms, GBV and child protection. These should be followed up with awareness raising activities at community level through community dialogues. Efforts should be made to implement these dialogues in hard to reach, outlying areas to ensure that key messages on the rights of vulnerable children, adolescents and women reach most vulnerable members of society. These campaigns and dialogues should be comprehensive and well-coordinated with clear objectives and ongoing monitoring and evaluation; and CBO transport to these areas should be funded. Target groups for these campaigns should include the public and specific key stakeholders such as parliamentarians, religious leaders, traditional leaders (Dikgosi) and the media. • DSP with technical support from future grantees should develop a strategy for training and skills development around GBV and child protection should be developed and rolled-out which should begin with a review of the minimal skills set and existing training programmes of the child welfare workforce around preventing and handling abuse; and working with the DSS to review and improve the national psycho-social support programme and integrate lay counselling programmes and the Social Work training programme at the University of Botswana.69 69 These activities were planned in the original TK Project concept document. 80 • Grantees and implementing partners working in the gender sector, together with relevant government departments, should focus more interventions specifically on boys and men to teach them how to self-assess and to identify triggers that may lead to violence including how to deal with them. However, a level of sensitivity is required when working with men, particularly around appropriate messaging. For example, emphasis should be placed on the fact that men are not always the perpetrators of GBV. • Grantees and implementing partners should include linkages with women’s economic empowerment initiatives in programme interventions to address, as far as possible, what is one of the underlying causes and contributory factors to GBV. • The TK programme has laid the foundation for an integrated GBV and HIV programme which are critical to prevent and respond to the spread of HIV and this model should be replicated and scaled up by USAID, possibly into the PEPFAR 3. 10.5 OBJECTIVE 4: NATIONAL FRAMEWORKS AND POLICIES Long term: • The Gender and Development Policy has just been passed and technical support for implementation of the policy by GeAD should be continued including its dissemination and communication strategy. PCI with its technical expertise, experience in government processes and already existing relationship with GeAD is well-positioned to provide this support. • USAID should provide financial support to grantees to provide technical and logistical support to DSP and GeAD for the implementation of the National Curriculum on Gender Analysis and Mainstreaming into PEPFAR 3. • USAID should fund the strengthening of the GeAD M&E system which should include ongoing capacity building of the department officials in M&E. • Improved national management of data for OVC and gender services could be a critical step to continuously inform evidence-based policy and programme decision making. It is therefore recommended that TK’s initial plans and activities to undertake this should be continued by future grantees into PEPFAR 3. • Donors such as USAID should consider including an initial preparatory phase prior to the five year programme cycle to allow sufficient time for building government buy-in at all levels and multi-sectoral consensus including clarity on roles, responsibilities and commitments that are built into specific activities; and secondly, to generate evidence to inform programme interventions before implementation begins. • It is recommended for PEPFAR 3 that a lead Ministry or department take responsibility for the strategic direction and coordination of the programme on policy level and programme level. • Nationally formulated plans are implemented at district and village/ward level. It is therefore recommended that, when replicating programmes of this nature, it is important to work midstream at district level. The following should be undertaken prior to implementation to ensure proper coordination and oversight of the programme at district level: 81 • Future grantees should get buy-in and support from district level stakeholders. They should also be involved in joint planning of the programme. • Grantees together with district level government and civil society stakeholders should use a participatory approach, map coordination structures, services and intervention sites relevant to OVC and GBV prevention and respond at district and ward level to identify gaps and inform programme planning. This will allow for planning to be based on evidence and will identify opportunities for strengthening of already existing institutional mechanisms focused on coordinating programmes and services. • Grantees should also build the capacity of district government officials to plan, monitor and evaluate OVC and gender programmes. 10.6 CAPACITY DEVELOPMENT SUPPORT PROVIDED BY PCI Long term: • Future grantees should ensure that capacity development and mentoring systems are well￾structured and form part of the initial programme planning process. This will go a long way towards ensuring that support does not become ad hoc or ignored due to activity implementation deadlines and targets. It will also ensure that sufficient capacity is created at each level of the ‘cascade’ prior to further roll-out. • The systems should be designed in a collaborative and participatory manner following an initial capacity assessment of IPs conducted by the grantee, which should be conducted with each of the programme partners during the initial phases of programme roll-out. This will allow for relevant capacity development and support initiatives to be set in place for each of the IPs as opposed to a more generic approach. • The capacity of allocated mentors to provide comprehensive and effective support must be a key consideration when setting up support and mentoring systems. • Where capacity limitations are present, larger CSOs – with sufficient experience and expertise – could be contracted to provide the necessary support and mentoring (grantees to oversee in consultation with IPs). • Periodic capacity assessments and reviews of IPs should be conducted by grantees to ensure that capacity development and support are still aligned to the needs of each of organisations. Depending on programme timeframes, such assessments could be undertaken on an annual basis. 10.7 PROJECT MANAGEMENT Long term: • There needs to be a proper assessment conducted by grantees and USAID of the HR capacity required to roll-out a programme before implementation. • With projects of this scope, it is important that USAID have an inception period where the programme plan is consolidated, HR capacity is assessed and supplemented if needed and partnerships are set up. (USAID) 82 • There needs to be better combined programme planning and reflection between USAID and the grantee. Programme plans should include: activities, outputs, outcomes and impact. Indicators should be aligned to these different levels. Work plans should be aligned to programme plans, and reports should be aligned to work plans. • USAID should consult grantees and IPs should on the appropriateness and feasibility of M&E requirements before the final selection of indicators and targets. • Grantees should translate data collection tools into Setswana. 83 RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 2 of 43 SECTION B – SUPPLIES OR SERVICES AND PRICE B.1 PURPOSE To provide support to the United States Agency for International Development’s Mission in Southern Africa (USAID/Southern Africa) to conduct a performance evaluation of two USAID activities that are both using a community-based approach to involve and mobilize communities in their response to HIV/AIDS. The two activities to be evaluated are the Maatla Civil Society Strengthening activity and the Tsela Kgopo Orphans and Vulnerable Children and Gender activity. In this performance evaluation, USAID seeks to understand if the activities have been implemented well so far, and whether any mid-course adjustments need to be made. USAID will use the results of this performance evaluation to determine whether an impact evaluation of the same activities is warranted in the near future. B.2 CONTRACT TYPE This is a Firm, Fixed-Price Task Order. For the consideration set forth in the contract, the Contractor shall provide the deliverables or outputs described in Section F and comply with all contract requirements. B.3 FIRM FIXED PRICE The firm, fixed-price is $________________. B.4 PAYMENT The paying office is set forth in section G.4. B.5 OTHER RFTOP INFORMATION The final statement of work for the task order that will result from this RFTOP will be incorporated at the time of award and shall be based on the proposal by the successful Offeror. B.6 PAYMENT SCHEDULE Payment of the fixed price shall be made on the schedule specified below: Deliverable (Percent (%) of Firm Fixed Price) 1. Work Plan approved by USAID _____40%_______ 2. Debriefings to USAID and stakeholders completed _____30%_______ 3. Final evaluation report approved by USAID _____30%_______ TOTAL _____100%______ Payments of the amounts specified above are subject to approval by the Contracting Officer’s Representative (COR) and acceptance of the milestone event or deliverable by the COR. The paying office is USAID/SA as indicated in section G.4; payment processing shall be in accordance with the payment schedule in section G.4 of this RFTOP. END OF SECTION B RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 3 of 43 SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK PROJECTS TO BE EVALUATED: 1. Project name: Maatla (Botswana Civil Society Strengthening Program) Cooperative Agreement No.: 674-A-00-11-00049-00 Project Dates: May 17, 2011 through May 16, 2016 Agreement Value Range: $15 - $20 million Implementing Organizations: Family Health International 360 (FHI360, Prime). Sub-contract: Inveneo. Sub￾agreements: Botswana Network of AIDS Service Organizations (BONASO); Botswana Christian AIDS Intervention Program (BOCAIP); Botswana Network of People Living with HIV/AIDs (BONEPWA); Botswana Council of Non-Governmental Organizations (BOCONGO); Botswana Confederation of Commerce, Industry and Manpower (BOCCIM); National Alliance of State and Territorial AIDS Directors (NASTAD); Ngamiland Council of Non￾governmental Organizations (NCONGO); and World University Service of Canada (WUSC). 2. Tsela Kgopo (Botswana Orphans & Vulnerable Children and Gender Project) Cooperative Agreement No.: 674-A-00-11-00048-00 Project Dates: June 13, 2011 through June 12, 2016 Project Funding: $15 - $20 million Implementing Organizations: Project Concern International (PCI) (prime). Sub-awards: Hope Worldwide (HWW); Catholic Relief Services (CRS); Humana People to People (HPP); Marang Child Care Trust (MCCNT); Kagisano Society Women’s Shelter Project (KWSP); Kgetsi Ya Tsie (KYT); Bakgatla Bolokang Matshelo (BBM): Mother’s Union (MU); Botswana Association of Positive Rehabilitation (BAPR); and House of Hope (HOH). 1) Objective USAID/Botswana’s PEPFAR (President’s Emergency Plan for AIDS Relief) Program seeks to strengthen local health-service-provider organizations in Botswana to improve health outcomes among beneficiaries and communities. The aim of this solicitation is to conduct a performance evaluation1 of two USAID activities that are both using a community-based approach to involve and mobilize communities in their response to HIV/AIDS. The two activities to be evaluated are the Maatla Civil Society Strengthening activity and the Tsela Kgopo Orphans and Vulnerable Children and Gender activity. In this performance evaluation, USAID seeks to understand if the activities have been implemented well so far, and whether any mid-course adjustments need to be made. USAID will use the results of this performance evaluation to determine whether an impact evaluation of the same activities is warranted in the near future. This evaluation will include immediate collection of information on the quality of implementation so far, to allow for mid-term corrections in programming, and on the organizational and technical capacity of sub-partners. Routine and internally collected data on the institutional and technical capacity of the local organizations being supported by the activities should be used by evaluators to monitor trends in their 1 The previously published evaluation solicitation requested an impact evaluation, but USAID has decided to have evaluators conduct a performance evaluation first. If results of the performance evaluation suggest the activities have been well implemented and are likely to be resulting in a change in outcomes among beneficiaries, USAID will consider soliciting for an endline impact evaluation of one or both activities at some time in the future. According to USAID’s evaluation policy (http://www.usaid.gov/evaluation/policy), USAID defines evaluations as either being “impact” evaluations or “performance” evaluations. Impact evaluations are those that use a rigorously defined counterfactual (experimental or quasi-experimentally derived) to attribute changes in outcomes to an intervention. Performance evaluations are everything else. RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 4 of 43 organizational capacity. This internal data should be supplemented by additional data collected by the evaluators to assess changes in organizational outcomes, without attribution to the prime partners. This evaluation should inform USAID of the quality of activity implementation so far and whether a later impact evaluation is warranted. This evaluation will shape the nature of potential future USAID activities. 2) Purpose and Use The evaluation’s purpose is to assess: (1) how well the activities are being implemented so far, whether according to plan; (2) what prioritized, immediate changes should be made in activity implementation to improve performance; (3) how follow-on activities might be informed from the experience of these activities; and (4) whether the activities are being implemented well enough to warrant an impact evaluation before the activities end. Evaluation results will be used as formative data for the activities so that they can better tailor their interventions to beneficiary needs during the final years of implementation. The evaluation will also document lessons learned and best practices from the significant investments made in the activities. The primary audience of the evaluation will be USAID/Botswana and its implementing partners. A secondary audience of the evaluation is the Government of Botswana and other stakeholders, such as other USG agencies, bilateral and multilateral donors, other NGOs, etc. 3) Background of activities2 The Maatla Civil Society Strengthening activity and the Tsela Kgopo Orphans and Vulnerable Children and Gender activity were designed in tandem and share complementary mandates to work with local non￾governmental organizations (NGOs). The two activities are intended to contribute toward the achievement of USAID Botswana’s Results Framework (see Appendix 1), which includes three objectives: (1) Strengthen community agency to seek, support and provide HIV/AIDS related services; (2) Enhance NGO capacity to complement HIV/AIDS response in Botswana; and (3) Strengthen GOB capacity to lead HIV/AIDS response in Botswana. Additionally, Botswana’s activities also contribute to the Regional Development Corporation Strategy (RDCS) Development Objective 2, “Reduced impact of HIV/AIDS in the region,” via its Intermediate Result 2.1, “Mitigate the spread and impact of HIV/AIDS in non-presence countries.” 3.1. Maatla (Botswana Civil Society Strengthening Program) Maatla is a five-year activity managed by FHI 360 and funded by USAID/Botswana with a start date of May 17, 2011. The goal is to significantly and sustainably strengthen the capacity of the civil society sector in Botswana to support HIV and AIDS and related health service delivery. This is meant to be achieved through collaboration with national non-governmental organization (NGO) networks, district￾level coalitions and government structures, providing both support for organizational development and technical capacity building. Maatla works in close consultation with USAID/Botswana, the Ministry of Local Government and Rural Development (MLGRD), the Ministry of Health (MoH), Ministry of Labor and Home Affairs (MLHA) and the National AIDS Coordinating Agency (NACA). 2 In a previously published version of this solicitation, the implementing partner activities were referred to as “projects.” Efforts have been made to remove such references, in accordance with USAID’s current definitions of “project” and “activity”. RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 5 of 43 In order for Maatla to achieve its goal of significantly and sustainably strengthening the capacity of civil society in Botswana in support of HIV/AIDS service delivery, it was designed to achieve the following results: ● Result 1: Strengthened technical competence and organizational capacity of national NGOs and networks to support field offices, affiliates, and member organizations to expand and strengthen delivery of strategic and high quality HIV/AIDS services. ○ Select indicators: • % CSO meeting minimum standards for leadership, governance, management of finances, human resources, inter-organizational linkages, infrastructure, etc. • Mean improvement in Organizational Capacity Assesement (OCA) scores per domain (see Appendix 5) • % CSOs implementing HIV/AIDS prevention, care, and treatment programs according to minimum standards. • Mean improvement in Technical Capacity Assessment (TCA) scores per domain (see Appendix 5). ● Result 2: Model(s) developed and implemented to strengthen district-level systems (CSOs and local government) to provide and coordinate HIV/AIDS services in hard-to-reach areas. ○ Select indicators: • # / % of people successfully referred for targeted clinical services (e.g., PMTCT, SMC, HTC, ART, TB/HIV) • # of # of targeted populations reached with individual and /or small group level preventative interventions (adults, youth, PLHIV) • # of eligible adults and children provided with a minimum of one care service • District reporting rate for CSOs, and mean improvement in reporting rate • Mean improvement in district coordination and collaboration between government and CSOs (see Appendix 5 for this tool) • # of community health workers trained in HIV-related service delivery • # of patients receiving community DOTs • # of individuals reached by activities which promote the reduction of gender based violence • # CSOs reached within the model districts • # of accredited training/capacity building materials developed by NCONGO • # of member organizations receiving trained or mentored by NCONGO • Organizational and technical capacity indicators as above, but for targeted districts and district-level CSOs ● Result 3: Strengthened institution(s) able to provide sustainable capacity building to civil society. ○ Select indicators: • # of successful funding proposals by BONASO and BOCONGO • # of accredited training/capacity building materials developed by BONASO or BOCONGO • # of member organizations or affiliates trained or mentored by BONASO or BOCONGO • % increase in non-USG funding for BOCONGO and BONASO To achieve Result 1, Maatla focuses national-level capacity building on four national NGOs with mandates as leaders of civil society and its response to HIV/AIDS: BONASO, BONEPWA+, BOCAIP and BOCONGO. According to the GOB National Operational Plan (NOP), BONASO, BONEPWA+ and BOCAIP are expected to play lead roles in delivering the CSO technical response to HIV through their members and affiliates; yet, they require significant capacity upgrading to play these roles. BOCONGO RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 6 of 43 was elected as the fourth NGO for capacity building because of its mandate of CSO organizational development and its multi-sectoral CSO membership. Together, the four organizations provide maximum reach into Botswana‘s civil society sector so that Maatla’s efforts should strengthen the technical and organizational capacity of district-level and local CSOs. For Result 2, Maatla aims to strengthen district level capacity by: (1) expanding the participation of local CSOs, the technical quality of their interventions, and their basic management and financial systems to facilitate effective operations through the building of strong multi-sectoral coalitions; and (2) measurably enhancing the ability of government and CSOs to work together to accomplish shared responsibilities for district-level design, implementation, reporting, advocacy and community and private sector mobilization. Maatla will work to develop models in Northwest Ngamiland, Okavango, Ghanzi and Chobe districts in a phased approach. To achieve Result 3, the Maatla targets support to two organizations, BONASO and BOCONGO, to sustain capacity building of civil society over the long term. As the lead program coordinator of AIDS service organizations in Botswana, BONASO is expected to monitor and respond to technical capacity needs, particularly as the epidemic evolves and new evidence-based best practices emerge to address them. BOCONGO is the key national coordinator for institutionalizing NGO capacity in Botswana. Maatla aims to address both organizational capacity building as well as increasing the technical competency in a wide range of HIV/AIDS-related services in the following areas: Sexual prevention, HIV Testing and Counseling (HTC), community TB/HIV care, adult and pediatric care and support, Gender Based Violence (GBV) and support for orphans and vulnerable children (OVC). Maatla aims to advance Botswana‘s ability to achieve its national goals for HIV/AIDS service delivery in the areas of treatment, care, and prevention through implementation of capacity-building strategies to move civil society partners closer to independence and program sustainability. Maatla should also develop models and strategies that will help guide Botswana’s response to HIV/AIDS and inform the National Strategic Framework on HIV/AIDS (NSF) II and other strategic documents. The theory of change assumes that by building the capacity of CSOs and local government coordination structures, the activity will leave structures more capable of supporting the HIV/AIDS response and broader health needs in Botswana. This program focuses on capacity building for the delivery of a wide range of HIV/AIDS related services and provides support to civil society organizations, particularly to those working in the areas of HIV prevention, emphasizing behavioral, biomedical, and structural relevance; HIV testing and counseling; care and support, including OVC; community TB/HIV; and adult and pediatric palliative care. The capacity building for CSOs covers both organizational and technical domains. The organizational aspect includes the following domains: Strategic Leadership, Organizational Structures (Governance), Human Resource Management, Financial Management, Infrastructure, Program and Service Management, Process Management and Inter-organizational linkages. The technical aspect includes program areas of prevention and care and support for adults, youth and OVC. With organizational capacity support for the national-level NGOs, the activity at year four intends to have groomed one or more organizations to be able to build capacity of its affiliates/members. The activity is currently supporting BONASO and BOCONGO to assume this role. The activity aims to strengthen district-level systems to enable them to better provide and coordinate HIV/AIDS services in hard-to-reach and underserved areas. This support should result in local government bodies being able to lead and coordinate HIV/AIDS activities at the district level and it RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 7 of 43 should result in more effective coordination among all partners working in the district. The activity should also result in strengthened organizational leadership and service-delivery capacity. Maatla Activities BOCAIP and BONEPWA+ Sites • Gumare Counseling Center in Okavango. The center has volunteers placed in several villages, including Gumare, Etsha1, Etsha 6 and Etsha 13, Ngarange, Nokaneng, Tsau, Sepopa, Shakawe and Seronga. • Maun Counseling Centre in Ngami. The center has volunteers operating in Maun and several neighboring villages, including Makalamabedi, Phuduhudu, Toteng, Shorobe and Sehitlwa. BOCAIP has plans to expand its sites to Chobe with the current work plan. BONEPWA+ has sites in Okavango, Ngamiland, Chobe and Ghanzi districts. The main site in Ngami is the BONEPWA+ Center of Excellence in Maun, which has volunteers operating in Komana, Toteng and Sehitlhwa. • Okavango/Ngami: BONEPWA+ has volunteers operating in Etsha 1, Etsha 6, Etsha 13, Shakawe, Maun,Gumare, Nokaneng, and Sehitwa • Chobe: The main site is BONEPWA Center of Excellence in Kasane, which has volunteers placed in Pandamatenga, Parakarungu, Kachikau and Kazuungula. • Ghanzi: The main site is the BONEPWA+ Center of Excellence in Ghanzi Township, which has volunteers covering D’kar, Charles Hill, New Xade, and Ghanzi Township. Table 1 below shows the Maatla implementing partners’ original indicators and 5-year targets3 . Appendix 2 gives a map of Maatla activity sites. Table 1. Maatla indicators and life-of-activity targets for number of beneficiaries served PEPFAR Indicators Totals Maatla 5-year targets, by district and partner Ngami Okavango Ghanzi Chobe BOCAIP BONEPWA BOCAIP BONEPWA BONEPWA BONEPWA # of targeted populations reached with individual or small group level preventative interventions that are based on evidence or meet the minimum standards required 26,600 6,605 3,472 8,489 1,205 5,186 1,642 # of targeted populations reached /with individual or small group level preventative interventions that are primarily focused on abstinence or being faithful, and are based on evidence or meet the minimum 18,850 5,669 1,372 6,487 1,760 2,370 1,192 3 Indicators and targets may have changed slightly since the project has started. We provide the originals to give bidders a sense of what activities are taking place and where, to help prepare their proposals. Final indicators and targets will be shared with the winning bidder. RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 8 of 43 PEPFAR Indicators Totals Maatla 5-year targets, by district and partner standards required # of people reached by individual, small group or community level intervention or service that explicitly addresses gender based violence & coercion related to HIV/AIDS 9,350 2,312 1,671 2,419 749 1,497 702 # of people who received Testing and Counseling services for HIV and received their test results 15,340 8,557 6,783 # of People Living w/ HIV/AIDS (PLHIV) reached with a minimum package of package of Prevention with PLHIV (PwP) interventions 3,800 217 2,057 185 1,342 # of eligible adults and children provided with a minimum of one care service 11,400 1,973 2,359 1,946 2,956 1,340 826 N.B Please note that partner targets may change over the life of the activity due to partner performance, funding, and level of effort. Some of the indicators in the above table have also been retired or modified during the course of the activity. Changes to Maatla since Implementation Started Maatla is now in year four of implementation. Changes to the activity since implementation began include the following. a) Inclusion of service delivery The Maatla Activity started as a Health Systems Strengthening (HSS) capacity-building activity; however, in year two, FHI 360 was advised to include support to service delivery in the four model districts. Service delivery was started in the last quarter of year two. b) Two partners in crisis In the first two years of the activity, two partners were in a crisis situation that was not anticipated. BONASO had lost all secretariat staff in year one, while BONEPWA+ lost its three key personnel (Executive Director, Programs Manager and Finance Manager) in year two. Therefore, for the first two years, BONASO and BONEPWA+ were being resuscitated. This limited the type and intensity of capacity building services for these organizations. c) Technical support to the government The Maatla Activity was asked in year two to include technical support to the National AIDS Coordinating Agency (NACA) to develop national combination prevention strategies. This included: RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 9 of 43 • National HIV Prevention Pitso, a strategic activity for stakeholders and community members working on HIV prevention to reflect on past activities and ongoing activities, interventions, and future plans so as to re-focus prevention interventions. • Designing, printing, and disseminating the HIV Prevention Pitso report. • Training of 30 combination-prevention champions (program officers from various stakeholders). • Consultative meetings for the development and dissemination of the CATCH (Communities Acting Together to Control HIV) model. The goal of CATCH is to establish a new comprehensive service-delivery model at the community level that would integrate HIV prevention, treatment, and care to improve the realization of zero new HIV infections in Botswana. The above was additional, unanticipated work for the activity personnel. d) Support to GeAD In year three, Maatla was asked to support the Ministry of Labor and Home Affairs (MLHA), Gender Affairs Department (GeAD) to implement a GBV referral pilot in two pilot districts (Kgatleng and Ngami). This also added additional work onto the limited number of FHI 360 activity personnel, more so that FHI 360 was advised by USAID against recruiting additional staff from year 3 onwards. e) Exit of BOCCIM from Maatla BOCCIM was initially spearheading the private-sector mentorship component, but it has exited from the activity. The process of exit affected the private-sector mentorship deliverables and FHI 360’s ability to meet cost-share targets. FHI 360 is now implementing the Private Sector-CSO Mentorship program directly, although without additional staff to make up for the two previously employed through BOCCIM. This also has implications on FHI 360’s outputs regarding technical assistance and mentorship from the private sector. f) Budget cuts In year four, the Maatla activity will experience challenges with the planned Organizational Development (OD) capacity building interventions due to cuts to the HSS budget by approximately one-third in order to meet the PEPFAR-mandated care and treatment earmark. WUSC and partner NGOs and networks had to scale back or drop several of their planned OD activities, which might constrain the activity’s ability to achieve desirable outcomes in OD capacity building. g) Changing of indicators and introduction of SIMS The activity has had to respond to substantial changes in PEPFAR indicators, together with new tools, such as PEPFAR’s Site Improvement through Monitoring System (SIMS). Changes such as these have led to additional time and resources being spent on updating activity training manuals, job aids, and M&E tools. RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 10 of 43 3.2 Tsela Kgopo (Botswana Orphans & Vulnerable Children and Gender Project) Tsela Kgopo is a five-year activity managed by Project Concern International (PCI), with funding from USAID/BW and a start date of June 13, 2011. The goal is to improve the quality of life of orphans, vulnerable children, and their caregivers and to improve gender-based responses and programming in HIV/AIDS activities in Botswana. With a family/household perspective, the activity intends to achieve this goal through implementing activities that achieve the following results: ● Result 1: Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women, improved, through achievement of the following results: (1.1) Community structures to identify and support vulnerable households and linkages to other relevant services strengthened; (1.2) Participation of vulnerable households, including men, in economic strengthening activities increased; and (1.3) Livelihoods opportunities for vulnerable youth increased. ○ Select indicators: • Mean (or median) increase in income • % employed (by age, sex, etc.) • % enrolled in vocational training • % vocational-tech graduates who are currently working (including self￾employed) • % of communities with community action plans implemented • % of households able to access money to meet important family needs ● Result 2: Developmental interventions for vulnerable children and adolescents improved, through achievement of the following results: (2.1) Access of most vulnerable children to quality early childhood development services increased; (2.2) Strengthened collaboration between parent/s caregivers and Integrated Early Childhood Development (IECD) centers for improved child development; and (2.3) Access of adolescents to mentoring, life skills, sexual/reproductive health and HIV prevention programs increased. ○ Select indicators: • % enrolled in IECD • % increase in enrollment in IECD programs • % change in enrolment, grades, retention and attendance in primary school • % youth with correct knowledge of HIV/AIDS transmission and prevention • % with safer sexual behaviors • % youth with stigmatizing attitudes toward PLHIV ● Result 3: Risk of neglect, exploitation and abuse of vulnerable children, adolescents and women reduced, through achievement of the following results: (3.1) Strengthened systems (referrals networks and community structures) to prevent abuse and meet the needs of those neglected, exploited and abused; (3.2) Services that cater for and protect abused children and women increased; (3.3) Child welfare work force skills to recognize signs of and reduce potential abuse and exploitation improved; (3.4) Involvement of men as peer educators, mentors, care-givers and role models increased; and (3.5) Awareness of rights and ability to seek help by vulnerable children, adolescents and women increased. ○ Select indicators: • % women, youth, and children suffering from physical, emotional, or sexual abuse • % of men expressing agreement with or practicing abusive beliefs or practices • % with knowledge of rights and services available • % accessing child- or woman-protection services • % with protection case resolved or acted on RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 11 of 43 ● Result 4: Development, implementation and coordination of national frameworks and policies that address the needs of children and women improved, through achievement of the following results: (4.1) Support to implementation of OVC and gender acts improved; (4.2) Support to development and implementation of OVC policy and draft national policy on gender and development; (4.3) National management of data for OVC and gender services strengthened; (4.4) National curriculum on gender analysis and gender mainstreaming developed and implemented; (4.5) Participation of children and women in policy planning and implementation increased; and (4.6) Logistical support to Department of Social Protection (DSP) and Gender Affairs Department (GAD) annual meetings. ○ Select indicators: • # of District Child Protection Committees (CPCs) and Village CPCs formed or strengthened in activity target areas. • # of districts where Gender and development policy and domestic violence act has been disseminated and implemented. • # of districts where the Children’s Act has been disseminated and implemented. To achieve activity objectives, PCI will partner with Catholic Relief Services (CRS), Hope World Wide Botswana (HWWB), Humana People to People (HPP), Marang Child Care Network Trust (MCCNT); the Government of Botswana (GOB); and other community and private sector organizations. The activity operates in Central, Kweneng, Southern, Southeast, Chobe, Francistown, Northwest, Ghanzi, and Kgalagadi Districts, while helping to catalyze enhancements in other districts through national initiatives. The Tsela Kgopo OVC and Gender activity uses Journey of Life (JOL) and Community Capacity Enhancement – Community Conversations methodologies to sensitize and mobilize communities for effective action. These methodologies bring a diverse range of community actors together to analyze problems that touch on issues of neglect, abuse, and gender, and to then develop and commit to solutions. The use of these methodologies has an almost immediate effect of strengthening linkages and coordination among families, school, health centers, social services, faith institutions, village development committees, and other community members leading to increased sense of community ownership and responsibility. The program will serve 18,922 (as per the CA) people directly with services over 5 years. Table 2 below shows the numbers that each partner will contribute towards the overall life-of-activity target of 19,600, and Appendix 3 gives a map of Tsela Kgopo sites. Table 2. Targeted Tsela Kgopo beneficiaries by partner and site. Partner & Site Number of beneficiaries Kgetsi Ya Tsie, Lerala 1,200 Kagisano Women’s Shelter Activity sites 1,000 • Gaborone & Molepolole Humana People to People sites 3,600 • Ghanzi 600 • Selibe Phikwe 1,200 • Charleshill • Mabutsane 1,800 Hope World Wide Botswana sites 4,100 • Molepolole, Mmakanke, Mosokotso, Mmanoko, Mmatseta, Kopong, Mogonono Catholic Relief Services sites 4,400 • Francistown 600 • Kasane 700 • Gumare 900 • Maun 800 • Sebina 700 • Palapye 700 House of Hope, Palapye 2,000 Mother’s Union, Mahalapye 1,500 Bakgatla Bolokang Matshelo, Mochudi 1,200 Botswana Association of Psychosocial Rehabilitation, Lobatse 600 N.B Please note that partner targets may change over the life of the activity due to partner performance, funding, and level of effort. Changes to Tsela Kgopo Since Implementation Started Tsela Kgopo is now in year three of implementation. There have been no major changes to the activity other than the decision not to award a sub-grant to Positive Innovation for the Next Generation (PING). 4) Existing Data Sources Existing data for the two activities is summarized in Table 3 below. Data collection tools used by both implementing partners can be found in Appendix 5. Data from these tools will be made available to the evaluators. Table 3. Summary of existing data sources for potential use in the evaluation Data available Brief description of data Organisation RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 13 of 43 Organizational capacity assessment results Results from standards-based assessments of partner organizations’ institutional capacity Maatla Technical capacity assessment results Results from standards-based assessments of partner organizations’ technical capacity Maatla District OCA & TOCA reports Results from standards based assessments of District Maatla Maatla & Tsela Kgopo progress reports Program data on people and organizations reached by the activities USAID, Tsela Kgopo & Maatla Child Status Index Longitudinal, client-level data on status of the child monitoring the needs, services provided, and progress of child wellbeing Tsela Kgopo District profiles HIV prevalence data, names and number of service providers per district Tsela Kgopo & Maatla 2011 Population & Housing Census; BAIS I-IV reports; other GOB survey reports (e.g., Family Planning, Income & Expenditure) - Government-produces reports on numbers of people in Botswana, by geography and demographics - Other reports are topic specific and present frequencies of traits of interest, based on sample surveys Central Statistics Office Botswana HIV prevention program data Numbers of people reached by activity partners with prevention services Tsela Kgopo & Maatla Care and Support Numbers of people reached by activity partners with care and support services Tsela Kgopo & Maatla Community TB/HIV data Numbers of people reached by activity partners with TB services Maatla GIS data Coordinates of partner and other CSO/GOB services in target districts Maatla CSO HTC data Results from BOCAIP HIV Testing and Counseling activities Maatla Training data/TraiNet Results from all activity training activities Tsela Kgopo & Maatla SIMS service assessments Standard PEPFAR tool for assessing the quality of services at facility and community sites, and at above-site assessment points. Tsela Kgopo & Maatla 5) Additional Data Potentially to Be Collected via this Evaluation 5.1 Maatla • Mid-term organizational and technical capacity assessments of sub-partners and districts • Qualitative interviews with program staff, beneficiaries, and other stakeholders 5.2 Tsela Kgopo • Mid-term technical capacity assessments of sub-partners • Qualitative interviews with program staff, beneficiaries, and other stakeholders RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 14 of 43 • Qualitative interviews with program staff, beneficiaries, and other stakeholders 6) Evaluation Questions The evaluation questions to be answered are: 1. How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? (Three results for Maatla, as above, in Section 3.1; four results for Tsela Kgopo, as above, in Section 2.) 2. What modifications, if any, to the strategic approach and theory of change have potential to substantially increase program effectiveness? 3. To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? 4. To what extent has USAID fulfilled its role in the activities by providing adequate funding, responsive managerial oversight and technical support, and assistance in solving problems in implementation as they arise? 5. What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? 6. Has implementation so far been successful enough to warrant an impact4 evaluation of each activity before they close? If not, why not? 7) Evaluation Design and Methodology USAID is currently interested in a performance evaluation of these two activities. USAID expects that both quantitative and qualitative methods may be used. USAID expects the evaluators to make extensive use of existing, routinely collected data by the implementing partners, as listed in Table 3 above. Offerors should propose the most rigorous but cost-effective methods they think feasible to assess performance so far, given budget limitations. As part of their proposal, ALL OFFERORS must complete the table in Appendix 4 and give their ideas for methods to be used in answering each evaluation question. Offerors should also consider whether secondary data from national surveys and other studies may be useful in assessing performance. See list of potentially relevant household surveys at the Botswana Central Statistics Office website5 . Such surveys may include the Botswana Core Welfare Indicator Survey, the Botswana AIDS Impact Survey, the Botswana Demographic Survey, and the Botswana Family Health Survey. To answer questions on the quality of implementation, USAID expects the methods to include thorough, rigorous reviews of activity documents (activity awards, M&E plans, annual workplans, progress reports, OCA assessments, and any special studies undertaken by the 4 According to USAID’s evaluation policy (http://www.usaid.gov/evaluation/policy), USAID defines evaluations as either being “impact” evaluations or “performance” evaluations. Impact evaluations are those that use a rigorously defined counterfactual (experimental or quasi￾experimentally derived) to attribute changes in outcomes to an intervention. Performance evaluations are everything else. 5 http://www.cso.gov.bw/index.php?option=com_content1&parent_id=370&id=430&nid=370 RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 15 of 43 activities), key informant interviews and focus groups, online surveys (if appropriate), visits to service-delivery sites, and extensive use of routinely collected program data to analyze changes in the reach, efficiency, and service quality of the service-delivery CSOs whose capacity is being built by the prime partners (FHI360 and PCI). The evaluators will have access to routine program data, collected by FHI 360 and PCI, listed above, but will be expected to collect complementary primary data nonetheless in order to get the most objective evaluation possible. Although there is routine data from the Child Status Index on some beneficiaries (orphans and vulnerable children), the reliability of this data for evaluation purposes has been questioned by studies in other countries where the CSI is in use. Upon award, but before fieldwork is conducted, the contractor will submit a detailed evaluation design, methodology and implementation plan for review and approval by USAID. Offerors shall develop innovative approaches to conduct this evaluation, using the attached program description for the two activities, the USAID Evaluation Policy6 and any other relevant information. The methodology will be comprised of a mix of tools appropriate to the evaluation’s research questions. It is expected that the offeror will discuss the relative strengths and limitations of the methodology proposed within the proposal. The methodology should take into account – and perhaps independently assess, where possible – the quality of the routine monitoring data being collected by the activities. Additionally, the offeror should discuss data disaggregation (by gender and other categories) and gender considerations in the evaluation. Before data collection, the contractor in coordination with USAID will finalize the data analysis methods as part of the methodology plan. Potential Limitations to Evaluation Methods There are several important limitations to answering the evaluation questions posed in this solicitation. For assessing the change in institutional capacity that occurred among sub-partners during the activity, evaluators are expected to use a simple pre-post analysis, or propose whatever they believe is most likely to be most effective, even if the change cannot be attributed to the activity in a rigorous way. Although evaluators may want to collect their own data on organizations’ capacity, they will have access to internally collected data on organizational capacity, but this data may be subject to bias because it was collected by the implementing partners. Although evaluators will have access to beneficiary outcome data in the form of Child Status Index data (beneficiary cohort data on nutrition, psychosocial, educational, and other outcomes), collected by program implementation staff, this data has proven unreliable for evaluation purposes in other settings, so use of it in this evaluation should be done with caution and the appropriate caveats. 6 http://www.usaid.gov/evaluation/policy RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 16 of 43 Offerors are encouraged to suggest ways to mitigate the weaknesses mentioned above, noting that weaknesses in the design and data must be stated explicitly in the final evaluation report in alignment with USAID policy END OF SECTION C RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 17 of 43 SECTION D – PACKAGING AND MARKING D.1 TASK ORDER PACKAGING AND MARKING Task order packaging and marking shall be performed in accordance with the Evaluation Services IQC, Section D. END OF SECTION D RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 18 of 43 SECTION E - INSPECTION AND ACCEPTANCE E.1 TASK ORDER INSPECTION AND ACCEPTANCE Task order inspection and acceptance shall be performed in accordance with the Evaluation Services IQC, Section E.2. END OF SECTION E RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 19 of 43 SECTION F – DELIVERIES OR PERFORMANCE F.1 PERIOD OF PERFORMANCE This projected Task Order is expected to begin on or about February 18, 2015 and be completed no later than June 8, 2015. This would include preparation days, in-country work, and report writing and finalization of all deliverables F.2. DELIVERABLES The offeror should propose a timeframe for each deliverable, not exceeding the final date of performance. 1. Team-planning meeting At least one team planning meeting will be conducted in advance of any international travel. This can be done over the phone. This meeting will allow USAID to present the evaluation team with the purpose, expectations, and agenda of the assignment as well as provide any feedback on the evaluation design and methodology submitted as part of the proposal. 2. Work plan Before international travel is authorized; the team will prepare a detailed work plan that will include the methodologies to be used in the evaluations. The work plan will be submitted to the COR at USAID for approval after the activity is awarded. 3. Methodology plan & study protocol with local IRB approval Written methodology plans (e.g., evaluation design, key questions, methods, tools, data collection instruments, and data analysis plan, operational work plan) will be prepared by the evaluation team and approved by USAID prior to international travel. The written design of the evaluations will be shared with country-level stakeholders before being finalized. After USAID approval, the study protocol must be submitted to, and approved by, a local Institutional Review Board (IRB) in Botswana (see Appendix 6 for the schedule of IRB meetings). 4. In-country team planning meeting On day one of fieldwork, the evaluation team will hold a meeting at USAID/BW to discuss the methodology plans with in-country stakeholders, including the USAID and its implementing partners, the GOB, and other relevant partners. USAID or other stakeholders may join the evaluation team. 5. Interim briefings, including status reports The team leader will provide weekly status reports to USAID on work plan implementation. The evaluation team will also conduct at least two interim briefings with USAID while in Botswana to review the progress and methodology of the evaluations. 6. Debriefing with USAID The team will present the major findings of the evaluations to USAID after submission of each of the draft reports and before the team‘s departure from Botswana. 7. Debriefings with other stakeholders The team will independently present the major findings of the evaluations to the USAID partners (as appropriate and as defined by USAID) and to GOB through PowerPoint presentations prior to the team’s departure from Botswana. The debriefing will include a discussion of achievements, activities, and recommendations. The evaluation team will consider partner comments and revise the draft reports accordingly, as appropriate. 8. Draft evaluation report Draft reports of the findings and recommendations should be submitted to the USAID Task Order Contracting Officers Representative (TO COR). Proposals should specify when the partner anticipates this RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 20 of 43 to happen. The written report should describe the mid-term data collected, and any findings, conclusions, or recommendations. The report will be shared with the two activity partners. 9. Final evaluation report USAID has 14 working days to review the draft report and provide written comments to the contractor on the draft report. The contractor will have 14 working days to incorporate these comments into a final report. Contractor must provide both an electronic version and 5 hard copies of the final report to USAID. The reporting format is listed in Section 6. The report will be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov), which the partner will need to submit to USAID for upload, after the TO COR provides formal written approval. 10. Data sets Upon activity completion, the contract will submit to USAID clean qualitative and quantitative anonymous data sets with code books and interview guides and questionnaires. F.3. REPORTING FORMAT AND CRITERIA Two separate reports will be produced for USAID, one for each activity. a) Format The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins 1 inch top/bottom and left/right. The final evaluation report should not exceed 35 pages, excluding references and annexes. Final Evaluation Reports Each final evaluation report produced by the successful offeror shall: (1) Use rigorous evidence to answer the evaluation questions listed above; (2) Identify and articulate succinctly best practices and lessons learned; (3) Make clear, prioritized recommendations for future community-based health implementation and the design of forthcoming interventions. Additionally, the final reports should include: 1. Mid-term data on organizational and technical capacity of the sub-partners being strengthened by Maatla and Tsela Kgopo 2. Qualitative and/or quantitative interview data with program staff and other stakeholders on the quality of activity implementation, what things are working, what needs to be changed, etc. 3. Review and summary of internally collected program data to assess progress toward meeting the activity results/objectives. The evaluation reports (one for each activity) should include the following sections: Each Final Evaluation Report 1. Executive Summary: Summarizes activity purpose and background, key evaluation questions, methods, findings, and recommendations. (3-5 pgs.); 2. Table of Contents (1 pg.); 3. Introduction and Background: Purpose, audience, and synopsis of task, brief overview of Maatla / Tsela Kgopo activities in Botswana, USAID program strategy and activities implemented in response to the problem, brief description of implementing partners (1-2 pages); 4. Evaluation Design and Data Collection Methodology: Describes evaluation methods, including constraints and gaps (1-2 pg.); 5. Findings/Conclusions/Recommendations: For each evaluation question; also include data quality and reporting system that should present verification of spot checks, issues, and RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 21 of 43 outcome (17–20 pgs.); 6. Issues: Provide a list of key technical and/or administrative, if any (1–2 pgs.); 7. Future Directions: To inform the design of any further activities (2-3 pgs.); 8. Dissemination plan: The evaluation report should include a dissemination plan for how the findings of the evaluation will be disseminated to relevant stakeholders (e.g. reports, presentations, publications, agency websites, annual reports, policy briefs). 9. Use of findings plan: The evaluation report includes a stated plan for how the evaluation findings will be used for decision-making and program improvement (e.g. mid-course corrections, new procurements, resource allocation, and intervention uptake) and timeframe, if appropriate. 10. Total cost: The evaluation report should include the total cost of implementing the evaluation. 11. References (including bibliographical documentation, meetings, interviews and focus group discussions); 12. Annexes: Annexes must include the evaluation tools, schedules, interview lists, tables, all sources of information, the evaluation statement of work, statements of differences, weakness of the data, and a copy of USAID Criteria to Ensure the Quality of the Evaluation Report7 , all of which must be met. b) Final Report Criteria • The evaluation report should represent a thoughtful, well-researched, and well organized effort to objectively evaluate what is working in the activity, what is not, and why. • Evaluation reports shall address all evaluation questions included in this Scope of Work. • The evaluation report should include the scope of work (as written in the signed contract) as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the Contracting Officer’s Representatives. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, reporting bias, etc.). • As much as possible, evaluation findings should be presented as analyzed facts, evidence, and data, rather than anecdote or hearsay. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. F.4. TECHNICAL DIRECTION AND DESIGNATION OF RESPONSIBLE USAID OFFICIALS TASK ORDER CONTRACTING OFFICER: Adam Walsh Task Order Contracting Officer 7 USAID’s Criteria to Ensure the Quality of the Evaluation Report is available as Annex 1, pg. 11, of USAID’s Evaluation Policy, online here: http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 22 of 43 U.S. Agency for International Development 100 Totius Ave Groenkloof 0027 Pretoria, South Africa Telephone: +27 12-452-2230 / Fax: +27 12-460-3177 Email: awalsh@usaid.gov The Task Order Contracting Officer Representative (TO COR): Dinah Zeltser Task Order Contracting Officer Representative U.S. Agency for International Development 100 Totius Ave Groenkloof 0027 Pretoria, South Africa Telephone: +27 12-452-2062 Email: dzeltser@usaid.gov F.5. PLACE OF PERFORMANCE The place of performance under this Task Order is Gaborone, Botswana, as specified in the Scope of Work. END OF SECTION F RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 23 of 43 SECTION G – TASK ORDER ADMINISTRATION DATA G.1 CONTRACTING OFFICE The Contracting Officer with authority to administer the order is Adam Walsh at: USAID/Southern Africa 100 Totius Street P. O. Box 43 Groenkloof 0027 Pretoria South Africa The Contracting Officer is the only person authorized to make or approve any changes in the requirements of this task order and notwithstanding any provisions contained elsewhere in this task order, the said authority remains solely in the Contracting Officer. In the event the Contractor makes any changes at the direction of any person other than the Contracting Officer, the change shall be considered to have been made without authority and no adjustment shall be made in the contract terms and conditions, including price. G.2 TECHNICAL DIRECTION The Contracting Officer will formally appoint a Contracting Officer’s Representative (COR) and an Alternate Contracting Officer’s Representative in a formal letter to the contractor. The Contracting Officer and the COR are the only official representatives of USAID for this contract and are the only ones authorized to provide technical direction to the contractor throughout the evaluation. The contractor is expected to work together with the COR to implement the scope of work. USAID will provide overall direction to the contracted evaluation team, identify key documents, and assist in facilitating a work plan. The evaluation team will be responsible for arranging meetings with key stakeholders and will be required to advise USAID prior to each of those meetings. The evaluation team is also responsible for arranging transportation in and around Botswana. The evaluation team will be responsible for procuring its own office space, computers, Internet access, printing, and photocopying. Evaluation team members will be required to make their own payments. USAID personnel will be made available to the team for consultations regarding sources and technical issues, before and during the evaluation process. USAID will send letters of introduction informing key GOB staff and other high-level partners of the nature, timing, and scope of the evaluation and of the evaluation team members. G.3 ACCEPTANCE AND APPROVAL In order to receive payment, all deliverables must be accepted and approved by the TO COR. G.4 INVOICES One original of each invoice shall be submitted on an SF-1034 “Public Voucher for Purchases and Services Other Than Personal” to the Financial Management Office at USAID/Southern Africa. One copy of the voucher and the invoice shall also be submitted TO COR. Electronic submission of invoices is encouraged. The SF-1034 must be signed, and it must be submitted along with the invoice and any other documentation in Adobe PDF to the Regional Financial Management Office to this address: invoice@usaid.gov RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 24 of 43 Paper invoices may be sent to the following address: USAID/Southern Africa, P.O. Box 43, Groenkloof, Pretoria 0027 or via courier to 100 Totius Street, Groenkloof, Pretoria If submitting invoices electronically, please do not send a paper copy. G.5 ACCOUNTING AND APPROPRIATION DATA (To be completed with included with the final task order): Budget Fiscal: Operating Unit: Strategic Objective: Team/Division: Benefiting Geo Area: Object Class: Amount Obligated: $ END OF SECTION G RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 25 of 43 SECTION H – SPECIAL TASK ORDER REQUIREMENTS H.1 AUTHORIZED GEOGRAPHIC CODE The authorized geographic code for the purchase of goods and services under this task order is 937. H.2 LANGUAGE REQUIREMENTS All deliverables shall be produced in English. H.3 GOVERNMENT FURNISHED FACILITIES OR PROPERTY The Contractor and any employee or consultant of the Contractor is prohibited from using U.S. Government facilities (such as office space or equipment) or U.S. Government clerical or technical personnel in the performance of the services specified in the Task Order unless the use of Government facilities or personnel is specifically authorized in the Task Order or is authorized in advance, in writing, by the TO COR. H.4 CONFIDENTIALITY AND OWNERSHIP OF INTELLECTUAL PROPERTY All reports generated and data collected during this project shall be considered the property of USAID and shall not be reproduced, disseminated or discussed in open forum, other than for the purposes of completing the tasks described in this document, without the express written approval of a duly-authorized representative of USAID. All findings, conclusions and recommendations shall be considered confidential and proprietary. H.5 CONTRACTOR’S STAFF SUPPORT, AND ADMINISTRATIVE AND LOGISTICS ARRANGEMENTS The Contractor shall be responsible for all administrative support and logistics required to fulfill the requirements of this task order. These shall include all travel arrangements, appointment scheduling, secretarial services, report preparations services, printing, and duplicating. H.6 ENVIRONMENTAL COMPLIANCE The Foreign Assistance Act of 1961, as amended, Section 117 requires that the impact of USAID’s activities on the environment be considered and that USAID include environmental sustainability as a central consideration in designing and carrying out its development programs. This mandate is codified in Federal Regulations (22 CFR 216) and in USAID’s Automated Directives System (ADS) Parts 201.5.10g and 204 (http://www.usaid.gov/policy/ads/200/), which, in part, require that the potential environmental impacts of USAID-financed activities are identified prior to a final decision to proceed and that appropriate environmental safeguards are adopted for all activities. [Offeror] environmental compliance obligations under these regulations and procedures are specified in the following paragraphs of this [RFTOP]. In addition, the Contractor/recipient must comply with host country environmental regulations unless otherwise directed in writing by USAID. In case of conflict between host country and USAID regulations, the latter shall govern. No activity funded under this task order will be implemented unless an environmental threshold determination, as defined by 22 CFR 216, has been reached for that activity, as documented in a Request for Categorical Exclusion (RCE), Initial Environmental Examination (IEE), or Environmental Assessment (EA) duly signed by the Bureau Environmental Officer (BEO). (Hereinafter, such documents are described as “approved Regulation 216 environmental documentation.”) An Initial Environmental Examination has been approved for projects funding this RFTOP/Task Order/Contract. The IEE covers activities expected to be implemented under this Contract. USAID has determined that a Categorical Exclusion applies to the proposed activities. This indicates that if these RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 26 of 43 activities are implemented subject to the specified conditions, they are expected to have no significant adverse effect on the environment. As part of its initial Work Plan, and all Annual Work Plans thereafter, the Contractor, in collaboration with the USAID COR and Mission Environmental Officer or Bureau Environmental Officer, as appropriate, shall review all ongoing and planned activities under this contract to determine if they are within the scope of the approved Regulation 216 environmental documentation. If the Contractor plans any new activities outside the scope of the approved Regulation 216 environmental documentation, it shall prepare an amendment to the documentation for USAID review and approval. No such new activities shall be undertaken prior to receiving written USAID approval of environmental documentation amendments. END OF SECTION H RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 27 of 43 SECTION I – CONTRACT CLAUSES All applicable clauses as outlined in the basic Evaluation Services Indefinite Quantity Contract (IQC) and its subsequent amendments (if any) are incorporated by reference. In addition, the following clauses are added: 52.232-39 Unenforceability of Unauthorized Obligations. (JUN 2013) 52.244-6 Subcontracts for Commercial Items. (JUL 2013) END OF SECTION I RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 28 of 43 SECTION J – LIST OF DOCUMENTS EXHIBITS AND OTHER ATTACHMENTS Appendix 1 – Results Framework for USAID/Botswana Appendix 2 - Map of Maatla’s districts Appendix 3 – Map of Tsela Kgopo sites Appendix 4 – Matrix of proposed methods by evaluation question Appendix 5- Maatla’s Organizational and Technical Capacity Assessment Tools Appendix 6 – Botswana Institutional Review Board Schedule END OF SECTION J RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 29 of 43 SECTION K – REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS Not required. END OF SECTION K RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 30 of 43 SECTION L - INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS L.1 GENERAL The Government anticipates the award of one (1) task order as a result of this RFTOP. L.2 ACQUISITION SCHEDULE RFTOP issued 12/11/2014 Offeror Questions due 12/23/2014 Proposals due 01/26/2015 Performance begins 02/18/2015 L.3 GENERAL INSTRUCTIONS TO OFFERORS a) RFTOP Instructions: If an Offeror does not follow the instructions set forth herein, the Offeror’s proposal may be eliminated from further consideration or the proposal may be downgraded and not receive full or partial credit under the applicable evaluation criteria. b) Accurate and Complete Information: Offerors must set forth full, accurate and complete information as required by this RFTOP. The penalty for making false statements to the Government is prescribed in 18 U.S.C. 1001. c) Offer Acceptability: The Government may determine an offer to be unacceptable if the offer does not comply with all of the terms and conditions of the RFTOP. d) Proposal Preparation Costs: The U.S. Government will not pay for any proposal preparation costs. L.4 DELIVERY INSTRUCTIONS (a) Submission, Marking and Copies Proposals shall be submitted in two separate parts (a) technical proposal and (b) cost proposal in accordance with the structural format set forth below. Do not include these two parts in the same document. The Offeror must submit the proposal electronically - internet email with up to 5MB per email compatible with Microsoft Office (MS WORD or Excel) in a MS Windows environment. Multiple emails may be sent to accommodate the proposal size and content, but each must contain very clear identification of the attachment and instructions for assembling the proposal. Offerors may also send an Adobe Acrobat portable document format (.pdf) for electronic submission; however, zipped files attachments are not allowed. Any budget spreadsheets should be sent in unlocked Excel format (formula shown). The subject line for every such email must include the following “SOL-674-15-000007– BOTSWANA PROJECTS IMPACT EVALUATION” (b) Closing Date and Time All proposals in response to this RFTOP shall be due at the following address, no later than the Time and Date indicated on the cover page to this RFTOP. (c) Address Offerors must submit their proposals electronically via email to Lauda Mthembu at lmthembu@usaid.gov and proposals@usaid.gov RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 31 of 43 L.5 INSTRUCTIONS FOR THE PREPARATION OF THE TECHNICAL PROPOSAL A. Technical Proposal The technical proposal must be concise, and complete. The proposal will demonstrate the Offeror’s expertise with respect to achieving the evaluation program results. (1) General The Technical Proposal in response to this solicitation should address the technical evaluation factors listed in Section M. The technical proposal shall consist of the technical proposal itself and an annex. These together will constitute the Offeror’s technical proposal. The proposal must be well organized, complete, clear, and succinctly presented. (2) Page Limitation The technical proposal itself shall not exceed twenty-five (25) pages (page limit does not include, cover page, dividers, table of contents, or annex). Any additional pages will not be evaluated. Proposals shall be written in English and typed on standard 8 1/2" x 11" paper (210mm by 297mm paper) or A4 paper, single spaced, 12 characters per inch with each page numbered consecutively. Font size of less than 12 characters per inch may be used only in tables, charts and footnotes, and abuse of such exceptions may be a basis for a reduction in the evaluation score of any such Offeror, at the Contracting Officer’s discretion. Margin should normally be 1”. (3) Organization The technical proposal should be organized into the following sections as follows: Table of Contents Cover Page Executive Summary Technical Proposal Body 1. Technical Approach 2. Personnel Annex (4) Introductory Materials As stated above, a cover page and executive summary are required at the beginning of the technical proposals. The Offeror may also want to include other introductory materials such as a table of contents at the beginning of the technical proposal. The Cover Page should be the first page of the proposal and include the names of the organization proposed as the prime partner involved in the proposed evaluation. Major sub-contractors (those with over 20% of the total estimated amount) if included should also be listed separately. The Cover Page should include the name of the proposed Task Order Program Director (or other principal point of contact as determined by Offeror) and the name of a contact person (if different from the Program Director) for the prime Offeror and her/his contact information. The Executive Summary should summarize the key elements of the Offeror’s technical approach, personnel plan, implementation plan, expected results. The Executive Summary should be no more than two (2) pages long. (5) Technical Proposal Body (a) Technical Approach: Technical Approach part of Offerors’ proposal must include the following: 1. Specific methodology and techniques used to gather information and answer the evaluation questions. RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 32 of 43 2. A level of effort (LOE) matrix and timeline that illustrate key tasks and milestones for the evaluations. The timeline should show a realistic schedule that lists specific activities that will occur each week that the evaluation research is conducted. It should also specify when the final evaluation report will be submitted to USAID. It is anticipated that the Task will be awarded on February 05, 2014. The contract start date should be no later than February 16, 2015. Offerors should use these dates when developing a timeline. Timelines should include an initial in-briefing with USAID and an oral presentation of the final evaluation (delivered prior to the submission of the written evaluation). The latest possible date for receiving proposals is November 10, 2014. Proposals will be evaluated on the basis of the evaluation criteria in Section M. (b) Personnel: The contractor will identify at minimum a core evaluation team composed of a team leader, three experts, and a logistics/administrative consultant, most of who should be local Batswana hires. Please review Section M for a detailed list of requirements for each evaluation team member. USAID strongly recommends that the evaluation team include the part-time services of a qualified statistician. All attempts will be made for the team to be comprised of a representative number of male and female members. Core team members should include: 1. Team Leader/Senior Evaluation Specialist. The Team Leader will take specific responsibility for assessing and analyzing the organization‘s progress towards targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. S/he will also suggest ways of improving the present performance, if any. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/SA team and other major partners. 2. Senior Technical Advisor for Organizational Capacity Qualifications: Senior Technical Advisor for Organizational Capacity should have a post-graduate degree in communications, public health or an applicable social science. S/he should have several years’ experience with organizational capacity building in Sub-Saharan Africa. S/he should be knowledgeable in program assessment and evaluation methodologies. S/he should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting evaluations/assessments of activities that aim to build organizational capacity. Roles and responsibilities: The Senior Technical Advisor for Organizational Capacity will assist the Team Leader in developing the evaluation methodology, creating the data collection tools, conducting interviews, analyzing the data, and writing the report. S/he will pay special attention to the organizational capacity aspects of the evaluation, and will take the lead in creating or identifying tools for the evaluators’ own assessment of the local organizations’ capacity. S/he will take part in meetings with USAID and other stakeholders, both in the planning phase, but also in the dissemination of findings. As a lesson learned from previous evaluations, the evaluation needs to be carried out in a participatory fashion, forming a team that, in various places and times, includes a range of USAID staff, GOB staff, and other stakeholders, as feasible. As a lesson learned from previous evaluations, the evaluation needs to be carried out in a participatory RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 33 of 43 fashion, forming a team that, in various places and times, includes a range of USAID staff, Government of Botswana staff, and other stakeholders. Additionally, Offeror’s proposed evaluation team members should provide a signed statement attesting to a lack of conflict of interest or describing an existing or potential conflict of interest relative to the program being evaluated that could lead reasonable third parties to conclude that the evaluator or evaluation team member is not able to maintain independence and, thus, is not capable of exercising objective and impartial judgment on all issues associated with conducting and reporting the work. Real or potential conflicts of interest may include, but are not limited to: 1. Immediate family or close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant/ material though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant/material though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. (6) Technical Proposal Annex The technical proposal annex shall contain position descriptions, CVs/resumes, letters of commitment from personnel or partners, letters of support, experience/performance references and any other supporting documentation requested by this RFTOP. Each CV/resume must not exceed 3 pages in length and must address specific Personnel requirements listed in Section M. B. Cost Proposal (1) Firm, Fixed Price Offerors must complete Section B.3 of this RFTOP, entitled “Firm, Fixed Price” and submit (as a separate section) with its Technical Proposal. A firm-fixed-price contract provides for a price that is not subject to any adjustment on the basis of the Contractor’s cost experience in performing the contract. The Firm, Fixed Price includes all Contractor’s costs to perform the work and includes Contractor’s fee (profit). This contract type places upon the Contractor maximum risk and full responsibility for all costs and resulting profit or loss. It provides maximum incentive for the Contractor to control costs and perform effectively and imposes a minimum administrative burden upon the contracting parties. (2) Costs Offerors must also submit a cost proposal that includes the following information that demonstrates how costs were calculated to arrive at the Firm, Fixed Price. Failure to include all information, or to organize the proposal in the manner prescribed, may result in rejection of the proposal as being unacceptable. The following guidance is provided with respect to the organization of the cost proposal. Line Item Organization: In order to undertake a meaningful analysis of price, Offerors shall use the following standard line items and cost elements organized as presented below. 1. Direct Costs RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 34 of 43 Labor Costs 1. Offeror’s individual labor category (i.e.Team Leader, Senior Technical Advisor, etc.) times the Level of Effort to arrive at individual labor category’s cost. Such costs shall be all inclusive and include salary and wages, fringe benefits and labor overhead. 2. Consultants (if any) - Consultant’s title and his/her fixed daily rate times the Level of Effort to arrive individual consultant’s cost. Travel & Allowances Travel, Transportation, Per Diem and Miscellaneous (Visas, Inoculations, travel allowances, etc. as one line item), Other Direct Costs (if any) 2. Fixed Fee (Profit) 3. Total Firm, Fixed Price END OF SECTION L RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 35 of 43 SECTION M – EVALUATION FACTORS FOR AWARD M.1 GENERAL INFORMATION (a) The Government may award a contract without discussions with Offerors in accordance with FAR 52.215-1. (b) The Government intends to evaluate Offerors in accordance with Section M of this RFTOP and make contract award to the responsible Offeror whose proposal represents the best value to the U.S. Government. (c) The submitted technical information will be evaluated by a technical evaluation committee using the technical criteria shown below. The evaluation committee may include industry experts who are not employees of the Federal Government. When evaluating the competing Offerors, the Government will consider the written qualifications/capability information provided by the Offerors, and any other information obtained by the Government through its own research. (d) For overall evaluation purposes, all evaluation factors other than cost or price, when combined, are significantly more important than cost or price. M.2 TECHNICAL PROPOSAL EVALUATION CRITERIA The following criteria will be used by the technical evaluation committee (TEC) to evaluate the proposals. The TEC members will assign values totalling 100 points to score the various components of the proposal set forth below. The specific evaluation criteria are as follows: Technical Evaluation Criteria Weight A. Technical Approach 50 points B. Personnel 50 points Total 100 points Technical Approach (50 Points): Evaluation of how the proposed technical approach is consistent with and supports the project objectives and evaluation questions as outlined in this RFTOP. The evaluation will include an assessment of the following: 1) Methodologies and Techniques (35 points): • Understanding of local capacity building measurement and evaluation methods and challenges/limitations to these methods • Proposed design/methods to answer evaluation questions is sound and feasible and likely to produce robust evidence for answering the evaluation questions Balance between efficiency and rigor 2) LOE matrix and timeline which illustrate key tasks and milestones for the evaluations (15 points) Personnel (50 Points): The personnel factor evaluates the extent to which the qualifications, skills and experience of proposed personnel meet or exceed those required below. Evaluation includes the overall staffing plan and approach, as well as the capabilities of specific key personnel. The following considerations will be evaluated under this factor. These considerations will not be scored separately as individual elements. They are included to RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 36 of 43 provide potential Offerors with additional information regarding this evaluation criterion and to help Offerors prepare their proposals: • Diversity of experience amongst team members • Relevant educational credentials of the team “Personnel” must include two key personnel: 1) Team Leader/Senior Evaluation Specialist , and 2) Senior Technical Advisor for Organizational Capacity. Required (“must”/”will”) and recommended (“should”) qualifications, skills, and experience of proposed personnel are as follows: Team Leader/Senior Evaluation Specialist (20 points): Team Leader/Senior Evaluation Specialist should have a post-graduate degree in public health or social sciences. S/he should have extensive experience in conducting mixed methods (combining quantitative and qualitative) evaluations in Sub-Saharan Africa and strong familiarity with process/performance evaluation methods and HIV/AIDS activities. Excellent oral and written skills in English are required. The Team Leader should also have experience in leading evaluation teams and preparing high quality documents. Roles and responsibilities: The Team Leader will take specific responsibility for assessing and analyzing the organizations’ progress towards targets, factors underlying their performance, benefits of the strategies, and compare with other possible options. S/he will also suggest ways of improving the present performance, as needed. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Botswana team and other major partners. Senior Technical Advisor for Organizational Capacity (10 points): Qualifications: Senior Technical Advisor for Organizational Capacity should have a post-graduate degree in communications, public health or an applicable social science. S/he should have several years’ experience with organizational capacity building in Sub-Saharan Africa. S/he should be knowledgeable in program assessment and evaluation methodologies. S/he should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting evaluations/assessments of activities that aim to build organizational capacity. Roles and responsibilities: The Senior Technical Advisor for Organizational Capacity will assist the Team Leader in developing the evaluation methodology, creating the data collection tools, conducting interviews, analyzing the data, and writing the report. S/he will pay special attention to the organizational capacity aspects of the evaluation, and will take the lead in creating or identifying tools for the evaluators’ own assessment of the local organizations’ capacity. S/he will take part in meetings with USAID and other stakeholders, both in the planning phase, but also in the dissemination of findings. Additional Personnel (20 points): Additional personnel will be evaluated based on the following: • Documented appropriateness of proposed staff and organization chart for technical approach proposed RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 37 of 43 • Qualifications and experience of the proposed key personnel vis-à-vis the positions for which the Offeror is proposing them. Demonstrate how these qualifications fit the function of their position particularly with regard to your technical approach • Established lines of authority and communications internal and external to the project M.3 PRICE/COST EVALUATION Price/Cost will not be scored but will be used for best value determination with Technical score being significantly more important than cost. USAID will evaluate the Offeror’s proposed price for completeness, reasonableness, and realism. USAID will evaluate to determine whether the estimated proposed costs elements are realistic for the work to be performed, reflect a clear understanding of the requirements, and are consistent with the unique methods of performance described in the Offeror’s technical proposal. M.4 DETERMINATION OF CONTRACT AWARD In accordance with FAR 52.215-1(f), the Government intends to award a Contract from this solicitation to the responsible Offeror whose proposal represents the best value after evaluation in accordance with the factors and sub-factors as set forth in this solicitation. This procurement also utilizes the trade-off process set forth in FAR 15.101-1. If the Contracting Officer determines that competing technical proposals are essentially equal, price may become the determining factor in source selection. Conversely, if the Contracting Officer determines that competing price proposals are essentially equal, technical factors may become the determining factor in source selection. Further, the Contracting Officer may award to a higher priced Offeror if a determination is made that the higher technical evaluation of that Offeror merits the additional cost/price. END OF SECTION M Appendix 1. Results Framework for USAID/Botswana DO: Strengthened Response to HIV/AIDS and other Health Services in Botswana IR 2.1 Increased management and technical capacity of the Batswana health workforce Intermediate Results Critical Assumptions or Risks to Program:  Sufficient yearly funding to support and implement Botswana Health HIV/AIDS and other health care services  GOB will be able to absorb and fund results  GOB/NGO policies will support efforts. llustrative Activity Types • Journey of Life • Go Girls! Community mobilization approach • Child Protection Committees/DMSAC • Community Conversations • Livelihood opportunities • Community Health Care workers supported • Comprehensive program for MARP  Transformation of the Central Medical Stores;  Strengthening DQAs and Data Flow of M&E systems  Mapping of PEPFAR activities and results by District;  Improve coordination between DAC, DMSACs, DHMT and NGOs at district level  Increase resources at district level through PPPs;  Better use of ICT ;  Costing to increase efficiency  Gender analysis training curriculum development and implementation;  Expansion of GBV services;  Increased GBV awareness;  Logistical Support to DSS and Women’s Affairs Department; Sub-Intermediate Results Objective 1 Mobilized Communities to support & provide HIV/AIDS, OVC & Health related services & practices Objective 2 Strengthened GOB and civil society capacity to support HIV/AIDS response in Botswana Objective 3 Strengthened GOB Policies and programming especially for Women and Children IR 1.2 Strengthened ref. networks to increase uptake of Prevention, Care and Treatment services IR 1.3 Expanded resources to support community efforts IR 2.2 Strengthened Botswana HIV/AIDS M&E Systems IR 2.3 Improved MOH Supply Chain Management System Cross cutting strategies: Gender –Focused Programs; Country ownership and Investment in Country –Led Plans; Policy-Informing Research; Strategic Coordination and Integration; Synergies between IRs, Partners, and Implementers IR 3.1 Improved implementat ion of the Children’s Act IR 3.2 Increased mainstreaming of gender in programs IR 3.4 Improved implementati on of the Domestic Violence Act Country Context:  Botswana is a sparsely populated country of 2.1 million people.  HIV adult prevalence rate of 25%.  In 2009 an estimated 350,000 Batswana were living with HIV with nearly 19,000 new infections.  The TB/HIV co-infection rate is 68% and an estimated 40% of deaths among persons living with HIV are due to TB. IR 2.4 Strengthened business systems for NGOs and National Networks IR 1.1 Community structure strengthened /leaders motivated RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 39 of 43 Appendix 2. Rough Map of Maatla’s districts RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 40 of 43 Appendix 3. Map of Tsela Kgopo sites RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 41 of 43 Appendix 4. Matrix of proposed methods by evaluation question Evaluation Questions Illustrative indicators Data Source/ Collection Methods Sampling / Selection Criteria Data Analysis Method How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? (Three results for Maatla; four results for Tsela Kgopo.) What modifications, if any, to the strategic approach and theory of change have potential to substantially increase program effectiveness? To what extent has each activity been implemented according to activity design? Where there have been deviations or shortfalls, what are the causes? To what extent has USAID fulfilled its role in the activities by providing adequate funding, responsive managerial oversight and technical support, and assistance in solving problems in implementation as they arise? What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? Has implementation so far been successful enough to warrant an impact evaluation of each activity? If not, why not? RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 42 of 43 Appendix 5. Maatla’s Organizational and Technical Capacity Assessment Tools Maatla tools for assessing organizational capacity, technical capacity, and district capacity OCA - 2013 clean copy!.xlsx Technical Assessment Tool TAT FINAL DISTRICT ASSESSMENT TOOL 2 Tsela Kgopo tools for technical visit assessments TA Short Visit Monitoring Tool A.doc TA Short Visit Monitoring Tool A.doc TA assessment and findings Presentation SIMS tool and SOP SIMS Community Site Visit SOP_for use_9.2 SIMS Community Tool_for use_9.25.do RFTOP No. SOL-674-15-000007 Botswana Performance Evaluation Page 43 of 43 Appendix 6. Botswana Institutional Review Board Schedule HRDC SCHEDULE FOR 2014-2015 DATE MONTH & YEAR TIME & VENUE 15 May 2014 0800 hrs 17 July 2014 0800 hrs 04 September 2014 0800 hrs 14 November 2014 0800 hrs 22 January 2015 0800 hrs 12 March 2015 0800 hrs Notes: • All submissions for new applications should be made at least five (5) weeks before the scheduled meetings. • Continue review application should be made at least three (3) months before the permit expiry date. • Amendments and close-out applications can be made anytime during the course of the study. • The Principal Investigators should make sure that all documents are complete before submissions and all other required documents are attached. Appendix 2: Evaluation methodology Contents Appendix 2: Evaluation methodology.................................................................................................................................................................. 1 1 Evaluation matrix ................................................................................................................................................................................................... 1 2 Data analysis.......................................................................................................................................................................................................... 3 1.1.1 Qualitative data analysis ....................................................................................................................................................................... 3 3 Southern Hemisphere’s model .............................................................................................................................................................................. 4 3.1 An overview of the Southern Hemisphere model ......................................................................................................................................... 4 4 Evaluation model assumptions.............................................................................................................................................................................. 6 5 Southern Hemisphere’s quality assurance mechanisms ....................................................................................................................................... 7 Annex 2: Methodology for evaluation of Maatla Project, for USAID 1 1 Evaluation matrix The matrix below provides an overview of the data collection and analysis methods against the evaluation question. However, limitations are that 1) activity/output level indicators are not provided and so it is challenging to align indicators to evaluation questions; 2) in some instances indicators are not necessarily relevant as evaluation question speak more to the overall design/theory of change of the programme. Evaluation question Illustrative indicators Data source / Collection methods Sampling/ Selection Criteria Data analysis method How likely is each activity to achieve their desired end results, as defined in each activity’s results framework? (Three results for Maatla; four results for Tsela Kgopo) Output level indicators for Tsela Kgopo are not provided; until these are available, tentative indicators could be: Reflection of programme implementers and beneficiaries on the effectiveness of activities (relevant for both projects) Semi-structured Interviews with: • USAID Head Office (national) • PCI (national) Focus groups and semi￾structured interviews with IPs, GoB and beneficiaries. • Representatives were purposively selected from these organisations, e.g. implementing staff • Purposive sampling, addressing the programmes’ relevant target groups (adults, youth, ECD centre staff, etc) • Qualitative analysis using thematic analysis – identifying the salient themes from these interviews • What modifications, if any, to the strategic approach and theory of change have potential to substantially increase program effectiveness? Indicators are not necessarily relevant here. Rather the design of the programme will be reviewed for its alignment to priority issues in the local and international context. Semi-structured Interviews with: • USAID Head Office • PCI • Government stakeholders Document review of internal programme records. Focus groups with: • CBO site staff • Beneficiaries Representatives will be purposively selected from these organisations, such as Programme Managers and M&E representatives Thematic analysis – identifying the salient themes from these interviews To what extent has each activity been implemented according to activity design? Where there have been Defined programme outputs and their relevant indicators level indicators for Tsela Kgopo are not provided. Focus groups and semi￾structured interviews with IPs Representatives were be purposively selected Thematic analysis – identifying the salient themes from these interviews Annex 2: Methodology for evaluation of Maatla Project, for USAID 1 deviations or shortfalls, what are the causes? Generally we will assess: • Whether key activities/outputs have been delivered within time and with sufficient quality • Strengths and challenges in implementation. These may be related to the context, implementation or design of the programme. Document review of internal programme records. To what extent has USAID fulfilled its role in the activities by providing adequate funding, responsive managerial oversight and technical support, and assistance in solving problems in implementation as they arise? Defined programme indicators have not been provided, however some general questions for exploration may include: • Strengths and challenges in support provided by USAID • Semi-structured Interviews with: • USAID Head Office (national) • PCI (national) Focus groups and semi￾structured interviews with IPs Representatives will be purposively selected from these organisations – e.g. Programme managers Thematic analysis – identifying the salient themes from these interviews What actionable, prioritized lessons have been learned for the design of potential follow-on activities? I.e., what should continue and what should change from the current activities in a potential future activity? Defined programme indicators not relevant here. Focus groups and semi￾structured interviews with: • All evaluation participants. Purposive sampling, addressing the programmes’ relevant target groups (adults, youth, ECD centre staff, etc) Thematic analysis – identifying the salient themes from these interviews Annex 2: Methodology for evaluation of Maatla Project, for USAID 2 2 Data analysis 1.1.1 Qualitative data analysis In order to ensure rigor in the analysis, NVIV0 10 software will be used to analyse the qualitative data. NVIV0 10 is data analysis software which assists handling very rich information. A summary of key themes, lessons learned and recommendations that arise from the study will be presented in the generated coding reports. The software removes many of the manual tasks associated with analysis like classifying, sorting and arranging information, so the researcher has more time to explore trends, build and test theories and ultimately arrive at answers to questions. The diagram below summarises the process that will be used in the thematic analysis of qualitative data. Codes and code book developed Data analysis team briefed to ensure inter-coder reliability Qualitative data categorised according to each relevant code Coding reports generated to summarise data for each indicator/code Data is analysed: description; comparison with quantitative & documents; inferences about the outcomes and impact of program. Report (qualitative and quantitative data) presented through narrative report and graphics Annex 2: Methodology for evaluation of Maatla Project, for USAID 3 3 Southern Hemisphere’s model 3.1 An overview of the Southern Hemisphere model The Southern Hemisphere model for undertaking applied research and outcome / impact evaluations might be described as a naturalistic-pragmatic model:- naturalistic, in that the consultancy focuses upon grounded inquiry or study within the ‘real-world’ as opposed to experimental or manipulated settings, whilst acknowledging that such a reality is comprised of multiple perspectives, which are both divergent and inter-related; and pragmatic in that the evaluation aims and objectives – plus resources available for the study – will influence methodological decisions. The model is also described as being participatory; that is, it is underpinned by broad consultation and engagement with a variety of stakeholders affiliated to the programme or project under review. This approach is based upon three key considerations, endorsed by all four of the consultancy’s partners; namely: a) The ethical imperative of the evaluator to ensure that all relevant parties are offered the opportunity to give their input; that is, to ensure equality of representation amongst all relevant stakeholders, including programme management, programme implementers, target groups, and beneficiaries. b) Maximum involvement of the client organisation to ensure that its evaluation requirements, constraints and information needs, are clarified and addressed; and c) Empowerment of the client organisation, which is offered insight into evaluation processes, methodologies and benefits, as well as the skills to act upon such knowledge in future evaluations. This is in keeping with the consultancy’s belief that programme evaluations offer a valuable means of facilitating reflection, learning and organisational development and, as a result, the client organisation is encouraged to view the evaluation process not as an accountability measure only, but as a means of improving upon its work and effectiveness1 . SH’s evaluation model is underpinned by an established process, which is divided into a number of stages or activities, which collectively form a coherent, linear process or ‘mode of operation’. These are outlined in the graphic below. 1 Such an approach is referred to by Bawden and Packham (cited in Patton 2002: 179) as ‘systemic praxis’. Annex 2: Methodology for evaluation of Maatla Project, for USAID 4 Diagram 1: An outline of the Southern Hemisphere Evaluation Model Step One: Planning workshop hosted with the client organisation/s and other relevant stakeholders, invited at client organisation's discretion. Following this workshop, an inception report is produced, to confirm the way forward in the evaluation process. Step Two: Following a comprehensive document review, research instruments are compiled and sent to client organisation/s for members' input and approval. Step Three: Fieldwork is conducted. Step Four: Preparation and dissemination of a draft report, outlining research findings, conclusions and recommendations. Client organisation offered opportunity to review and comment. Step Five: Feedback and recommendations workshop hosted to present findings and recommendations. Client organisation - and other attending stakeholders' - feedback incorporated into final report. Dissemination of final report at client organisation's discretion. Annex 2: Methodology for evaluation of Maatla Project, for USAID 5 4 Evaluation model assumptions The assumptions underpinning the SH evaluation model may be summarised as follows: Diagram 2: A summary of the assumptions underpinning the SH evaluation model •Reality is mediated through human constructions, which may be inter-related, shared or divergent. SH's work demonstrates a commitment to recording and explaining phenomena from the perspective of those being studied. •Hence, I would argue that SH's approach and model falls within the constructivist 'camp' of evaluation research. Ontological assumptions •The SH evaluation model is fallibilistic, in that the production of knowledge is attempted in as accurate or ‘truthful’ a manner as possible, whilst acknowledging that all findings are provisional and may be amended in the course of a study as new data comes to light. •A subjectivist position is also adopted, in that the world and the researcher are perceived as inter-dependent. Whilst neutrality is a guiding ideal, it is acknowledged that research can never be entirely ‘value-free’; that is, the subjective and interpretive nature of all research is acknowledged. Epistemological assumptions •The consultancy follows a pragmatic approach, in that methods are selected on the basis of their level of appropriateness to the evaluation’s aims, objectives and available resources. Methods are often mixed and are utilised flexibly. • Whilst there is no methodological hierarchy, the consultancy firmly supports the use of qualitative methods for the collection of rich and detailed data (for provision of the how and why input of a study). • A mainly inductive - as opposed to deductive - analytical process is used, coupled with an emergent or grounded theory approach - as opposed to the use of a priori theories. Methodological and Analytical assumptions Annex 2: Methodology for evaluation of Maatla Project, for USAID 6 5 Southern Hemisphere’s quality assurance mechanisms The following table outlines SH’s quality assurance mechanisms. Annex 2: Methodology for evaluation of Maatla Project, for USAID 7 Key Criteria Definition Quality Indicators Credibility (this corresponds with the positivist criterion of internal validity) Evaluation evidence is perceived by stakeholders both internal and external to the study to be trustworthy, plausible, useful, and relevant to the aims and objectives of the evaluation. 1.1 Evidence of member checks / reflections on research design, process, findings and recommendations (planning workshop, submission of draft report, feedback and recommendations workshop). 1.2 Evidence of investigator and/or data source and/or methodological triangulation. 1.3 Evidence of external review of the research process and findings. 1.4 Rationale and description of sample composition and selection (for example, characteristics of sample, basis for inclusions and exclusions, sample size and how sample allowed comparative data and / or negative case analyses to be undertaken). 1.5 Client feedback indicates that the SH outcome and / or impact evaluation outputs are perceived as being trustworthy, credible and believable. 1.6 Client feedback indicates that the SH outcome and / or impact evaluation outputs are perceived as being relevant and useful, incorporating supportive evidence; that is, examples of client utilisation of evaluation findings and recommendations. Transferability (this corresponds with the positivist criterion of generalisability external validity) The applicability or ‘fittingness’ and value of evaluation findings, conclusions and recommendations to other contexts or programmes of a similar nature. 2.1 Evidence of ‘thick’, detailed descriptions of research context to enable readers’ assessment of transferability (background section of report). 2.2 Evidence that client organisations regard SH evaluation outputs to be of value in / transferable to other contexts or programmes of a similar nature. Dependability / Auditability (this corresponds with the positivist criterion of reliability) The consistency, coherence, logic and stability of the study process and data analysis over time and across researchers and methods. 1.1 Dependability audit: 3.1.1. Inclusion of detailed audit trail of data collection process and procedures. 3.1.2. Inclusion of detailed audit trail of data analysis process and procedures. 3.2. Documentation of any changes to evaluation design, including reasons and any implications thereof. 3.3. Documentation of any study limitations and their possible impact upon the evaluation. 3.4. Evidence that client organisations regard the SH evaluation outputs to be stable and replicable across researchers and methods. Confirmability (this corresponds with the positivist Findings are drawn from available data and are independent of the perceptions 4.1 Confirmability audit: 4.1.1 Evidence of use of multiple fieldworkers /interviewers. 4.1.2 Indication that fieldworker training was conducted. 4.1.3 Evidence of the use of appropriate and unbiased questioning techniques in data collection Annex 2: Methodology for evaluation of Maatla Project, for USAID 8 criterion of objectivity) and possible biases of the evaluators; that is, they are deemed to be value-neutral and representative of all stakeholders’ input. instruments. 4.1.4 Evidence that data collection instruments were piloted. 4.1.5 Indication that quality checks were conducted on submitted transcripts. 4.1.6 Indication of the use of multiple coders for inter-coder cross-checks. 4.1.7 Use of peer review of coding and data analysis. 4.2 Use of a set of pre-defined and clearly stipulated criteria against which the programme /intervention will be assessed. 4.3 Evidence that client organisations regard the evaluation process and outputs as being sufficiently neutral and inclusive of all stakeholders’ input. Annex 2: Methodology for evaluation of Maatla Project, for USAID 9 Appendix 3: List of documents for Tsela Kgopo Folder housed in in Tsela Kgopo folder in Drop box Name on Doc Name on System Government Documents Republic of Botswana, Ministry of Labour and Home Affairs, National Youth Policy, 1996. 1996 National Youth Policy pdf. Botswana Children’s Act 2009 pdf. Botswana Partnership Framework for HIV & AIDS, December 2010 Chapter 28:05 Domestic Violence Domestic Violence Act 2008 pdf Government of Botswana: The Second Botswana National Strategic framework for HIV & AIDS 2010-2016. Final Botswana NSF 11 2010-2016 doc. Draft National Policy on Gender and Development, Republic of Botswana, Gaborone, 2012 GAD Policy Draft Final Version 10 2012 National Gender & Development Policy, Brief Version National GAD Policy Brief 01 06 2011. (PS) 1. ppt Monitoring & Evaluation documents￾monitoring tools, OCA, TCA Tsela Kgopo Partner Summary Data Flow Copy of TK data flow summary 1 10 13 April 2014 Copy of Tsela Kgopo OVC & Gender Annual Report Indicator Template 2014 April 2014 Copy of Tselakgopo OVC & Gender semi Annual Progress Report- Data by site￾April PCI- USAID December 2011 Final M & E Plan document approved by USAID Feb 11 1 12 update- Clean TK Tools Miscellaneous Implementation organisation documents Program Description for OVC& Gender Program Botswana Association of Psychosocial Rehabilitation (BAPR) BAPR Programme Description OVC￾Gender Programme 18 Nov 2011 final Program Description Bakgatla Bolokang Matshelo- BBM BBM Programme Description 18 April 2012 final PROGRAM DESCRIPTION Catholic Relief Services (CRS) Botswana with the Vicariate of Francistown (VoF) CRS Attachment 2- Programme Description Updated April 2012 Program Description for OVC& Gender Program HOUSE OF HOPE TRUST HOH Programme Description OVC & Gender 18 Nov 2011 final Humana People to People Botswana Program Description OVCG Project HPP- SOW final submission 13-9-2011 PROGRAM DESCRIPTION HOPE Worldwide Botswana HWWB attachment 2 –Programme Description submitted to USAID 18 April, 2012 PCI celebrating 50 years of positive community impact Kagisano OVC-Gender sub grant Template 18 April 2012 final Program Description for OVC& Gender Kgetsi Ya Tsie (KYT) KYT Programme Description 18 April 2012 final November 21, 2011 PCI celebrating 50 years of positive community impact Mother’s Union OVC- Gender sub grant Template 18 Nov 2011 15 September. 2011 PCI celebrating 50 years of positive community impact PCI OVC-Gender Sub grant agreement final – Marang 27 September 2011 Log frames, proposals and strategic plans Presentation – USAID evaluation final 2 – TK.pptx Tsela Kgopo OVC and Gender Programme Description submitted – June 2011.pdf CharlesHill Profile Final 13052013.docx Programme Implementation documents Francis Profile Final 13052013.docx Francis Profile Final 13052013.docx Ghanzi Profile Final 13052013.docx Ghanzi Profile Final 1305013. docx Chobe District Profile Final Kasane Profile Final 1305013.docx Kgatleng Profile Final Kgatleng Profile Final 13052013 Kweneng East District Profile Final Kweneng Profile Final.docx Lobatse district Profile Final Lobatse Profile Final 13052013.docx Mabutsane Sub-district Profile Final Mabutsane Profile Final 13052013.docx Mahalapye District Profile Final Mahalapye Profile Final 13052013.docx Ngami District Profile Final Ngami TK Profile Final 13052013.docx Palapye District – Profile Final Palapye Profile Final 13052013.docx Selebi Phikwe District Profile Final SelebiPhikwe Profile Final 13052013.docx 1 DRAFT PCI Botswana – OVC & Gender Program Child Protection 2012-2014 Community mobilization Strategy April 2012 v 2 (2) GENDER AND DEVELOPMENT “STRIVING FOR JUSTICE IN AN UNEQUAL WORLD” Core Curriculum Core clean all modules final 30 01 2015 1 Economic Development Strategic Plan, 2012-2014OVC & G Project DRAFT_April 19, 2012 Economic Development Strategic Plan 18-04-2012 1 INTEGRATEDEARLYCHILDHOODDEVELOPMENT(IECD):STRATEGYDOCUMENT IECD Main Strategy document Botswana Tsela Kgopo OVC Gender Programme 30 August, 2012, Gender Equity Strategy Tsela Kgopo OVC & Gender Programme Strategy Final ppt. TK Reports President’s Emergency Plan for AIDS Relief- Annual National Report PCI OVC Program June 2011-Sept 2011 OVC & Gender Program USAID Cooperative Agreement No. 674-A-00-11-00048-00 Program Start Date: 06/13/2011 Program End Date: 06/12/2016 PCI-SAPR (Oct 2011-March 2012) Report Tsela Kgopo OVC & Gender Program USAID Cooperative Agreement No.674-A-00-11-00048-00 Program Start Date: 06/13/2011 Program End Date: 06/12/2016 PCI Annual Report (October 11-Sept 12) Final OVC & Gender Program USAID Cooperative Agreement No.674-A-00-11-00048-00 Program Start Date: 06/13/2011 Program End Date: 06/12/2016 PCI OVC & G Quarter 1 2012 Report.doc Tsela Kgopo Program USAID Cooperative Agreement No.674-A-00-11-00048-00 Program Start Date: 06/13/2011 Program End Date: 06/12/2016 PCI Tsela Kgopo Quarter 3 April-June Tsela Kgopo OVC & Gender Program USAID Cooperative Agreement No. 674-A-00-11-00048-00 gram Start Date: 06/13/2011 gram End Date: 06/12/2016 TK Annual Progress report FY 2013.docx Tsela Kgopo OVC & Gender Program TK Q 1 FY 13.docx Tsela Kgopo OVC & Gender Program TK Q 3 Report 2013.docx Tsela Kgopo OVC & Gender Program TK SAPR FY 13 Final Report.docx Appendix 5: List of indicators – Targets vs actual Summary indicators Indicator Cumulative target Cumulative total Total % of target reached Number of eligible adults and children provided with a minimum of one care service 317831 308342 97% <18 Males 9605 8287 86.3% <18 Females 10106 9604 95% 18+ Males 4513 3418 75.7% 18+ Females 7569 9525 125.84% Number of HIV positive adults and children receiving a minimum of one clinical service3 (PEPFAR C2.1.D) 2931 5056 172.5% <15 Males 212 210 99% <15 Females 184 339 184% 15+ Males 996 1600 160% 15+ Females 1593 2907 182.5% Objective 1: Livelihoods Indicator Cumulative target Cumulative total Total % of target reached Number of eligible children provided with education and/or vocational training. (PEPFAR C5.4D) 11179 8976 80.2% Males 1609 1502 93% Females 1744 1539 88% <18 Males 3692 2861 77.49% 1 Male and female to not equal total target 2 The data is exclusive of HPP Mabutsane which had not submitted by the reporting deadline for the 1st year 3 Not reported on in all Annual Reports <18 Females 4134 3074 74.35% Number of eligible adults and children provided with economic strengthening activities. (PEPFAR C5.7D) 3958 4054 102% <18 Males 433 95 21.9% <18 Females 531 170 32% 18+ Males 913 450 49% 18+ Females 2081 3339 160% Number of persons trained in a community mobilization methodology (Tsela Kgopo 1.1.4) 1265 1025 81% <18 Males 339 227 66.96% <18 Females 631 522 82.7% 18+ Males 113 57 50% 18+ Female 122 219 180% Number of people reached by an individual, small group, or community-level intervention or service that explicitly aims to increase access to income and productive resources of women and girls impacted by HIV/AIDS4 (PEPFAR P12.4.D) 3411 5553 162.79% Males 521 234 45% Females 1366 1832 134% <18 Males 234 702 300% <18 Females 544 2298 422% 18+ Males 87 55 63% 18+ Females 195 432 222% Objective 2: Developmental interventions for vulnerable adolescents and children Indicator Cumulative target Cumulative total Total % of target reached Number of children accessing IECD services (Tsela Kgopo 2.1.2) 2809 3000 106.8% Males 1293 1501 116% Females 1516 1499 98.9% Number of children ages 0-3 accessing structured play stimulation activities through community groups.5 (Tsela Kgopo 2.1.3) 912 304 33% Males 416 136 32.6% Females 496 168 33.8% Number of adolescents participating in life skills activities. (Tsela Kgopo 2.3.2) 8090 5921 73% Males 3527 2485 70% Females 4563 3436 75% 4 Only reported on in two Annual reports 5 No target set in 2012 Annual Report Number of eligible children provided with health care referral6 (PEPFAR C5.3D) 850 1119 131% Males 349 570 163% Females 501 549 109% Number of targeted population reached with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet the minimum standards required7 (PEPFARP8.1.D) 18355 18557 101% Males 7076 7304 103% Females 11279 11253 99.7% Number of active beneficiaries served by PEPFAR OVC programs for children and families affected by HIV/AIDS.8 (PEPFAR OVC_SERV) 13268 8750 66% <1 Males 95 7 7% <1 Females 115 3 3% 1-4 Males 631 379 60% 1-4 Females 597 441 73% 5-9 Males 788 632 79% 5-9 Females 1,068 631 58% Number of HIV positive adults and children receiving care and support outside of the health facility9 (PEPFAR Care Comm) 2,567 1,208 47% <1 Males 19 0 0% <1 Females 23 0 0% 1-4 Males 57 0 0% 1-4 Females 70 2 3% 6 No target set in 2012 Annual Report 7 Only reported in 2014-2015 Annual Report 8 Only reported in 2014-2015 Annual Report 9 Only reported in 2014-2015 Annual Report 5-9 Males 97 8 8% 5-9 Females 125 7 6% 10-14 Males 92 21 23% 10-14 Females 112 17 15% 15-19 Males 135 30 22% 15-19 Females 201 34 17% 20-24 Males 232 18 8% 20-24 Females 291 33 12% 25- 49 Males 336 123 37% 25- 49 Females 591 576 97% 50+ Males 86 101 117% 50+ Females 100 238 238% Number of active beneficiaries receiving support from PEPFAR OVC programs to access HIV services.10 (PEPFAR OVC_ACC) 3200 86 3% <1 Males 58 0 <1 Females 1-4 Males 220 1 1-4 Females 2 5-9 Males 379 4 5-9 Females 1 10-14 Males 479 4 10-14 Females 3 15-17 Males 441 5 15-17 Females 5 18+ Males 1623 11 10 Only reported in 2014-2015 Annual Report 18+ Females 50 Objective 3: Abuse, neglect and exploitation of women and children Indicator Cumulative target Cumulative total Total % of target reached Number of eligible clients who received food and/or other nutrition services (PEPFAR C5.1D) 15090 18182 120.5% <18 Males 4592 6234 135.75% <18 Females 5158 6666 129.23% 18+ Males 2134 1541 72.11% 18+ Females 2732 3741 136.92% Number of eligible children provided with shelter and care11(PEPFAR C5.2D) 3103 3086 99.45% Males 1419 1557 109% Females 1664 1529 91.88% Number of eligible adults and children provided with protection and legal aid services (PEPFAR C5.5D) 6750 6992 103.58% <18 Males 1964 5636 286% <18 Females 2438 2003 82% 18+ Males 857 1023 119% 18+ Females 1491 2362 158% Number of eligible adults and children provided with psychological, social or spiritual support. (PEPFAR C5.6D) 19379 24095 124% <18 Males 5636 6118 108% <18 Females 6571 6625 100.8% 18+ Males 2753 2862 103% 18+ Females 4437 8490 191% Number of people reached by an individual, small group, or community-level intervention or service that explicitly addresses norms about masculinity related to HIV/AIDS (PEPFAR P12.1.D) 6034 8537 141% Males 1500 1053 70% Females 3000 2696 90% 11 No target set in second report <18 Males 367 1354 369% <18 Females 783 2903 370% 18+ Males 199 200 101% 18+ Females 185 331 179% Number of people reached by an individual, small group or community-level intervention or service that explicitly addresses gender-based violence and coercion related to HIV/AIDS12 (PEPFAR P12.2D) 16285 15926 97.8% Males 5327 5607 105% Females 10958 9988 91.14% <18 Males - 0 <18 Females - 0 18+ Males 0 131 18+ Females 0 200 Number of people reached by an individual small group, or community-level intervention or service that explicitly addresses the legal rights and protection of women and girls impacted by HIV/AIDS13 (PEPFAR P12.3.D) 9565 8942 93.48% Males 3207 3160 98.5% Females 6359 5426 85% <18 Males - 0 <18 Females - 0 18+ Males - 119 18+ Females - 237 Number of community health and para – social workers who successfully completed a pre-service training program within the reporting period (PEPFAR H2.2.D) 335 907 270% Males Not indicated14 250 Not indicated Females Not indicated 657 Not indicated Number of Health care workers who successfully completed an in-service training program within the reporting period15 (PEPFAR H2.3.D) 1620 683 42% Males Not indicated 149 Not indicated Females Not indicated 543 Not indicated Number of people receiving post GBV care16 822 306 37% < 10 Males 9 15 166% < 10 Females 15 10 66% 10-14 Males 17 4 23% 10-14 Females 48 7 15% 15-17 Males 55 5 9% 15-17 Females 76 7 9% 12 No target set in 2012 Annual Report 13 No target set in 2012 Annual Report 14 Indicates that progress reports did not consistently report on numbers and so cumulative totals are not possible. 15 Includes para-social workers in 2012 Annual Report (2nd) 16 Only reported in 2014-2015 Annual Report 18-24 Males 73 5 7% 18-24 Females 132 14 10% 25+ Males 124 52 42% 25+ Females 273 187 68% Number of people completing an intervention pertaining to gender norms17 (PEPFAR GEN_NORM) 1,900 653 34% <0-9 Males 149 29 50% <0-9 Females 45 10-14 Males 355 67 39% 10-14 Females 71 15-19 Males 359 35 34% 15-19 Females 88 20-24 Males 366 41 20% 20-24 Females 32 25+ Males 671 79 36% 17 Only reported in 2014-2015 Annual Report Appendix 6: Planned versus actual Objective 1: Livelihoods of households for vulnerable adolescents, especially girls and vulnerable women improved. Key result area 1.1 Community structures to identify and support vulnerable households and linkages to other relevant services strengthened The table below presents the planned activities for KRA 1.1 based on annual work plans and activities achieved as per Annual Progress Reports. Reporting period June 2011 - September 2012 Planned activities Actual activities Modify Journey of Life (JoL) to incorporate stronger gender aspect, integrate with Community Capacity Enhancement (CCE) methodology, and integrate vulnerability assessment and targeting elements REPSSI revised JoL tools to ensure that methodology could be appropriately contextualized, according to communities’ needs. In preparation for training in quarter three, PCI staff examined the Community Capacity Enhancement (CCE) methodology. Provide JoL/CCE training In October 2011, PCI and selected IPs conducted an information-gathering exercise in all of the TK Program targeted districts1. During this exercise a number of community-level structures were identified for JoL capacity building / strengthening. These included: • Kgosi (traditional leaders) • Village Development Committees (VDCs) • Child Welfare Committees and • Village Extension Multi-Sectoral HIV/AIDS Committees (VMSAC) PCI provided technical support to DSS to orient previously trained JoL facilitators, including CSO staff, government social workers, DACs, nurses, and teachers (36). PCI, in partnership with DSS and Marang, conducted JoL training with CSO staff, social workers, health and police officers, students and teachers, DACs and staff from Ngamiland Coalition of NGOs (NCONGO) (209). PCI conducted JoL TOT training, including government social workers, brigades, DACs, teachers and CSO staff. (21) Assist DCPCs and CSO centres to implement JoL No reporting on this activity Assist DCPCs and CSO centres to help communities achieve their JoL /CCE community action plans No reporting on this activity Capacity building of DCPCs and VCPCs to further strengthen identification of those in need and to ensure that the necessary referrals take place PCI, together with Marang Child Care Network Trust (MCCNT) and DSS, worked on re-constituting the District Child Protection Committees (DCPCs) for identification and support of the most vulnerable households. 1 Tsela Kgopo Program Annual Report (October 2011 – September 2012) Reporting period October 2012 - September 2013 No work plan available for this reporting period TK Annual Program Report (October 2012 – September 2013: 13) notes that “…intensive community and resource mobilisation activities to identify and support vulnerable households…” were conducted during FY 2013. Despite delays with DCPC establishment guidelines, PCI trained potential DCPC and VCPC members including social workers, community leaders, CSO staff, teachers, caregivers and youth in the programme’s community mobilisation methods - and supported them during implementation. By end of reporting period, six DCPCs had been established through collaboration between MCCNT and DSP in TK districts of operation. A total of 749 people were trained in community mobilisation methods for this reporting period, with MCCNT reported as making significant progress in rolling out JoL methodology. Reporting period October 2013 - September 2014 Continue to work with Selebe Phikwe Secondary School to do Journey of Life (JoL) in school camps for OVC to enhance children's coping mechanism and address issues of trauma, in partnership with MCCNT, DSP and selected schools No reporting on this activity Provide support to DSP and MCCNT to strengthen DCPCs in the six PCI focus sites (Ghanzi, Mabutsane, Maun, Gumare, Francistown, Lobatse) MCCNT was tasked to play a leadership role in establishing and strengthening DCPCs. Seven DCPCs have been formed in the TK Program districts of Francistown, Lobatse, Ghantsi, Ngamiland, Okavango, Chobe, and Kgatleng. Functionality of the DCPCs varies across districts. Work with DSP and Marang to reconstitute DCPCs in 4 PCI focus sites (Kasane, Kweneng, Phikwe and Mahalapye) No reporting on this activity Assist DCPCs and CSO centres to implement JoL within their districts No reporting on this activity Provide JoL awareness training to Francistown S&CD Social Workers No reporting on this activity Provide JoL action training to 50 participants (teachers, CSOs and government officials) No reporting on this activity Assist DCPCs and CSO centres to help communities achieve their JoL community action plans No reporting on this activity Provide refresher TOT training to 25 JoL facilitators No reporting on this activity Support DSP to train caregivers in Ikago Centres and at Mpule Kwelagobe in JoL PCI worked with DSP and Department of Community Development (DCD) to develop a joint work plan which was completed and signed off by PCI, DSP and DCD. JoL workshops conducted for 21 staff (17 females and 4 males) from Mpule Kwelagobe and 15 staff (9 females and 6 males) from Ikago Rehabilitation Centres. Work with REPSSI to conduct PCI and REPSSI developed JoL and Weaving of Hope for training of selected teachers, social workers and CSO partners (already trained in JoL) on Weaving Hope for Children children’s manuals. In addition, most of the partner staff members received training in basic counselling and communication skills. PCI, in partnership with REPSSI, commenced planning of evaluation of JoL methodology effectiveness. Aim was to document lessons learned to inform programmers both locally and internationally. Reporting period October 2014 - March 2015 Support PCI partners to follow up on community action plans TK Program Semi-Annual Progress Report notes that PCI offered continuous on-site support to IPs to improve implementation of community action plans. No specific input on what this support included. PCI continued to track and report upon progress in this area, plus documented lessons learnt. Work with REPSSI to finalize the documentation of JoL impact assessment PCI and REPSSI developed Terms of Reference for the evaluation of the JoL methodology. To source service providers in Quarter Three of reporting period. Key result area 1.2 Participation of vulnerable households, including men, in economic strengthening activities increased The table below presents the planned activities for KRA 1.2 together with the actual activities achieved for each reporting period. Reporting period June 2011 - September 2012 Planned activities Actual activities Develop methods and materials for financial literacy/small business management development programme PCI held discussions with the Botswana National Youth Council (BNYC) to develop and launch the Youth Entrepreneurship Development Academy (YEDA). PCI facilitated a meeting with the Director of Botho College/e￾Skills Foundation to explore opportunities for assistance in curriculum development for YEDA. Pre-screening questionnaire - to identify YEDA participants – and selection committee short list drafted, together with program fact sheet. PCI held discussions with Junior Achievement Botswana (JAB) as possible alternative to BNYC partnership. Establish partnerships with local businesses/resource organisations to assist with implementing financial literacy/small business management skills training and to provide ongoing mentoring support to beneficiaries in livelihoods activities PCI approached Barclays Bank in Palapye regarding collaboration with Tumasera GROW groups Implement financial literacy/small business management skills development program through CSO centres See below Provide training and technical assistance to help CSO centres In April 2012, PCI Regional GROW Technical Advisor visited Botswana to initiate implementation of GROW model. Training and clients begin savings and loan groups (GROW) was provided for PCI Botswana and KyT GROW implementation team. PCI Botswana awarded funding to expand GROW in Tswapong Hills area – this enabled KyT to hire seven more GROW facilitators. PCI also assisted KyT with the identification and selection of the GROW Coordinator and GROW Community Facilitator, two key partner personnel responsible for driving GROW implementation. PCI held meetings with district officials to obtain insight into programmes and services available to clients to assist them with IGAs and micro businesses. PCI designed and distributed (via IPs) a ‘business assessment form’ to obtain basic business information from all clients who have a self-started IGA. Information gathered will be used by PCI to determine the level of support required by IPs in assisting their clients to develop economically viable and locally relevant business activities. Provide training and technical assistance to help CSO centres and clients establish Food for Life/Honey for Life Schemes No reporting on this activity Assist IPs to incorporate the GROW model to help women using their services to become more economically self-reliant. Eight KyT GROW groups established with membership of 161 women and over BWP 10 000 saved. Five GROW groups established in Kweneng by HWW with membership of 100 and over BWP 700 saved. Other IPs, including HPP and CRS, continue to work with existing women’s groups to strengthen their various IGAs, including beading, poultry farming, baking and food gardens. PCI held consultative meetings with both organisations educating them about the model, its requirements and expectations. Reporting period October 2012 - September 2013 No work plan available for this reporting period TK Annual Program Report (October 2012 – September 2013: 18) notes that HWW and KyT formed 21 GROW groups in this reporting period with a total of 419 members (406 women and 13 men). Total savings of BWP 55 037 and 54 loans allocated to value of BWP 11 835. PCI hired a GROW Officer to spearhead the GROW component of the TK Program. PCI provided training for BBM GROW Coordinator and seven BBM GROW facilitators. BBM formed 11 GROW groups in Kgatleng District with 129 members (all women). PCI-Barclays partnership launched officially in fourth quarter. Allocation of BWP 884 444 allocated by Barclays Bank of Botswana for GROW expansion in Molepolole and Kasane as result of PCI’s public private partnership (PPP) efforts. IPs continued to support self-started economic strengthening initiatives. Those commenced prior to TK Programme have started generating an income. Reporting period October 2013 - September 2014 Train additional facilitators and coordinators as part of GROW expansion through WE acceleration funding No reporting on this activity However, the Annual Progress Report for this period notes that in 2015, PCI will continue to build the skills and capacity of GROW coordinators and facilitators to actively support GROW groups. Provide on-going monitoring and support to partners implementing and expanding GROW GROW implemented by three TK partners: HWW, BBM and HPP. PCI implementing GROW directly in Chobe District. Total number of groups established: 150, with total number of members: 2016 (1809 female and 207 male). Total savings: BWP 235 181; number of distributed loans: 485 with total amount loaned: BWP 118 399. PCI hosted GROW Learning Forum in August 2014. This was attended by 37 staff from HWW, HBB, HPP and PCI-Chobe, including project coordinators and site leaders. Explore potential GROW adaptation for GOB Youth Empowerment Scheme (YES)- Barclays-PCI partnership No reporting on this activity Work with selected partners to develop and implement a plan to strengthen existing IGAs based on gaps identified following the assessment conducted in Year 2 No reporting on this activity Mobilise resource partners to provide on-going support to implement the plan No reporting on this activity TK Program Annual Progress Report (October 2013 – September 2014) notes, however, that IPs are providing support to group and individual IGAs through mentorship and linkages with government and private sector partners for training and funding. IPs also assisted individuals and groups to explore marketing opportunities for their products. Provide on-going TA for implementation of the plan No reporting on this activity Strengthen the "Food for Life/Honey for Life" model with HoH, MU and BBM (In FY 14 the ES team will assess whether the program is being implemented and, if so, how worthwhile the program is) No reporting on this activity Not in work plan PCI produced a video on GROW model to provide insight into the methodology and its impact on beneficiaries and communities. Reporting period October 2014 - March 2015 Assist implementing partners in strengthening and monitoring GROW groups GROW implemented by three TK partners: HWW, BBM and HPP. PCI implementing GROW directly in Chobe District. Following an audit at BBM, GROW figures were revised as follows: Total number of groups established to date: 146, with total number of members: 1768 (1598 female and 170 male). Total savings: BWP 262 847; number of distributed loans: 558 with total amount loaned: BWP 111 875. Two group IGAs in Chobe were assessed and await procurement of equipment and materials by PCI. PCI allocated BWP 150 000 of Barclays Bank funding to support an additional 5 GROW groups in Chobe. Applications are being reviewed by PCI. PCI economic strengthening and M&E teams conducted a four day refresher training on GROW tools for 29 coordinators, site managers and facilitators of the GROW model. PCI and CRS Kasane trained 15 GROW group representatives and CRS volunteers (11 female and 4 male) on GBV and gender norms. Organise learning forum for coordinators and facilitators from GROW implementing partners No reporting on this activity. It appears to have been conducted in August 2014 (see above). However, PCI conducted discussions with 4 Chobe youth GROW groups to out how the model might be improved to appeal more to youth beneficiaries/clients and adopt a more youth-centred approach. Based on input obtained, PCI has set up a Facebook group account – Chobe Youth GROW Group – which has 23 members; plus has held discussions regarding hosting of a youth festival to raise funds for the Chobe youth GROW groups in third quarter. Provide start-up capital from Barclay grants to Kweneng and Chobe groups for individual/group IGA activities PCI and HWW disbursed grants from Barclays Bank funding to 3 individuals and 3 groups for IGA support. Implement the mentoring program to support GROW groups and individuals with IGA PCI contracted BizKid – a business mentoring organisation - to provide mentoring services to HWW individual and group IGAs funded by Barclays Bank. BizKid conducted a situation analysis of the business and action items were identified. Submit a proposal for a follow￾on funding to Barclays Bank Botswana PCI held a meeting with Barclays Bank Botswana’s management following submission of a concept note for additional funding for strengthening and expansion of GROW implementation. Proposal to be submitted in third quarter. Train and mentor GROW coordinators and facilitators in basic financial and business management skills to improve IGA support to groups and members No reporting on this activity Provide assistance to TK partners supporting clients with IGA activities and create sustainable market linkages No reporting on this activity Collaborate with partners to identify community resources that can provide long-term mentoring to clients with IGAs No reporting on this activity Key result area 1.3 Livelihoods opportunities for vulnerable youth increased The table below presents the planned activities for KRA 1.3, based on annual work plans, together with the actual activities achieved for each reporting period, which are based on Annual Progress Reports. Reporting period June 2011 - May 2012 Planned activities Actual activities Develop job readiness skills building methods and materials PCI engaged World University Services of Canada (WUSC) to identify, recruit and select an experienced volunteer to collaborate with PCI in development of a job skills/job readiness programme. Not noted in work plan PCI, in partnership with Stepping Stones International (SSI) trained 10 staff from seven PCI partners in the Aflateen Programme. Following training, IP staff were to share learning with rest of their respective organisations, recruit youth to participate, conduct a baseline study with intake group plus 10 programme sessions. PCI and SSI provided ongoing support to IPs through technical assistance visits to observe sessions and provide feedback to facilitators. Partners recruited 70 youth participants and two partners completed required number of sessions (Mothers Union and FECH). SSI organized and hosted a five-day camp for selected Aflateen youth in August 2012 to enhance learning process and financial and social skills development. Establish partnerships with local financial/other businesses to assist with job readiness skills development activities (training, mentoring, financial support) PCI held a preliminary meeting with the Construction Industry Trade Fund (CITF) to explore vocational opportunities for youth in targeted programme areas. CITF presented to IPs during implementation planning retreat in Palapye in August 2012. Based on this, IPs included activities in their respective annual implementation plans to identify and link up with vocational training institutions and to link OVC to available jog opportunities. PCI established relationship with Barclays Bank to enable support of financial skills development and economic empowerment activities, plus with Orange Botswana to facilitate linkages between the business and PCI’s CS partners. Collaborate with business resource organisations, such as LEA, to provide business skills and small business management mentoring to vulnerable youth. No reporting on this activity Mapping of vocational training programs in priority districts and create linkages for enrolment of NGO partner clients. No reporting on this activity Conduct a value chain assessment of entry-level employment sectors to identify HR core business needs. No reporting on this activity Select potential employers and vocational schools for piloting activities to strengthen the link between training provided and companies’ needs. Preliminary meeting held with Construction Industry Trade Fund (CITF) to explore vocational opportunities for youth. Reporting period October 2012 - September 2013 No work plan available for this reporting period Only HWW implementing the Aflateen Program. Total of 112 youth (56 males and 56 females) from villages of Borakalalo, Mosokotso and Kopong participating. PCI and WUSC recruited a volunteer to spearhead the development of the newly formulated Youth Employment Program (YEP). IPs selected for initial YEP roll out (BBM, HPP and HWW). IPs continued to support youth enrolled for vocational training at CITF and Zwenshambe Brigade (total of 15 youth). PCI Economic Strengthening team developed and facilitated a business and entrepreneurship skills programme with BBM in Mochudi. Thirty candidates were selected, of which 21 successfully completed the three-month programme. PCI investigating possible partnerships with organisations that can provide training in business and entrepreneurship skills to facilitate access to such capacity building by youth located in other TK districts. Reporting period October 2013 - September 2014 Support BBM to monitor implementation of business plans developed by the youth who participated in the Business and Entrepreneurship Skills programme in previous reporting period No reporting on this activity Explore potential resources that can provide business skills and entrepreneurship training to youth with other organisation HWW engaged BiZkid, a company specialising in providing business and entrepreneurship support, to conduct a five-day entrepreneurship programme with 15 Aflateen youth in Borakalalo. HWW invited Barclays Bank staff to conduct a session on entrepreneurship with 12 Aflateen youth (under Barclays Bank’s Unlocking Youth Potential initiative). Topics covered included challenges associated with entrepreneurship, establishing a small business, marketing, and how to conduct a SWOT Analysis. Train 4 additional TK partners in Aflateen Not reported on; but Annual Progress Report notes that four IPs are implementing Aflateen; namely, HWW, HOH, BBM and MU. Monitor and support partners implementing Aflateen program Not reported on; but Annual Progress Report notes PCI plans to establish an Aflateen consortium to allow IPs to meet, exchange ideas and lessons learned, and coordinate their Aflateen Programme activities across all sites. Explore potential for partnership with Youth Empowerment Scheme (YES) program for implementation of Aflateen No reporting on this activity Review and refine YEP to better meet needs of employers, training institutions and beneficiary youth No reporting on this activity Monitor, review and expand relationships with key stakeholders PCI, through YEP, strengthened partnerships with vocational training institutions. Twenty – out of 40 – youth were enrolled in vocational training programmes (16 male and 4 female). Support and monitor partners implementing YEP Through PCI oversight and guidance over the past 12 months, partners are now beginning to own the rigorous programme processes Assess progress and refine program based on lessons learned No reporting on this activity Explore additional partnerships with the private sector/training institutions to expand the The Annual Progress Report notes that “Building partnerships with the private sector and vocational training institutions is an on-going endeavor.” program Botswana-Baylor Children’s Clinical Centre of Excellence (COE) joined the consortium and is noted as being scheduled to conduct first round of job readiness training (JRT) with approximately 15 youth from its Young Adults Group in October 2014. Support partners to establish linkages with vocational institutions for enrolment of their youth No reporting on this activity Annual Progress Report notes, however, each of the three IPs have conducted a number of job readiness trainings, resulting in 47 youth (29 females and 18 males) being placed in positions, while 20 youth (14 females and 6 males) have been enrolled for vocational training, out of a total of 140 participants. Conduct support visits to partners implementing economic strengthening activities No reporting on this activity Reporting period October 2014 – March 2015 Integrate Aflateen into BAPR youth programs No reporting on this activity Organise Aflateen camp for select youth participants and partners No reporting on this activity Conduct Aflateen Learning Forum No reporting on this activity Develop marketing materials to strengthen Aflateen implementation No reporting on this activity Conduct Aflateen training for TK partners and affiliate schools Teachers from two schools in Mahalapye were trained in October 2014. Went on to establish Aflateen clubs with combined membership of 51 youth (35 females and 16 males). SSI trained two teachers and three youth from Linchwe Primary School in March 2015. PCI trained 8 Botswana Family Welfare Association (BOFWA) and Youth in Business Botswana (YIBBO) members (4 females and 4 males) in Aflateen through the Youth Employment and Education Project (YEEP). This training was then cascaded to 687 members and 18 Tweende groups. Also launched Money Matters campaign, which reached 793 out-of-school, unemployed youth (292 males and 492 females). Not included in work plan HWW facilitated Aflateen sessions at Kopong Junior Secondary School, using Aflateen manual plus Go Girl manual to address issues relating to gender norms. Total number of youth reached: 56 (32 females and 24 males) HOH implementing Aflateen at two sites: Khurumela and Mmaphula Secondary Schools with total number of youth: 35 (25 females and 10 males) Review and finalize YEP program materials No reporting on this activity Organise a youth employment marketing event for resource partners and private sector No reporting on this activity TK Program Semi-Annual Progress Report for October 2014 – March 2015 notes that PCI strengthened partnerships with vocational training institutions including those involved in construction, carpentry, electrical work, and wood work. Provide technical assistance to YEP implementing partners PCI Advisors provided on-going technical oversight during implementation. Conduct YEP Consortium No reporting on this activity meetings to monitor program progress PCI signed an MOU with Botswana Baylors Children Center of Excellence (COE) to be part of the YEP implementation consortium. Collaborate with partners to identify community resources that can provide long-term mentoring to YEP graduates HPP Mabutsane assisted 8 YEP graduates to enrol in the National Service Program, while HPP Ghanzi invited Ghanzi Brigade to conduct a career session with 35 YEP youth. Twenty four youth were interviewed and five were selected to commence training in January 2015. Other activities noted in terms of resource mobilisation include: BBM submitted a second proposal to Barclays Bank for further support of its YEP (target: to reach 100 unemployed and vulnerable youth) HWW, in collaboration with their Peace Volunteer, secured financial assistance from the US Embassy to implement a photography project with Aflateen / YEP youth from HWW and other IPs. Collaborate with HOH to launch ICT Resource Centre No reporting on this activity However, semi-annual progress reports notes that the HoH IECD Resource Centre (IECDRC) came into operation in second quarter. Little progress with connecting phone lines for internet services; however, children are learning basic computer skills such as how to use a keyboard and mouse. Support HOH with RM for the ICT centre No reporting on this activity Attend ES-related conferences or seminars No reporting on this activity Develop a mentorship program for youth HWW providing life skills and mentorship to individual members of the Borakalalo youth group due to low levels of attendance Objective 2: IECD and Youth Developmental Interventions Key result area 2.1.1 Capacity strengthening of CSOs and key government departments on IECD Reporting period June 2011- May 2012 Objective 2:Developmental interventions for vulnerable children and adolescents improved 2.1.1 Capacity strengthening of CSOs and key government departments on IECD Planned Activities Actual Activities Work with partners to map out IECD centers in project districts and determine training needs IECD centers in areas of operation mapped until March 2012 Provide training to CSO Centers in Say and Play methodology Say and Play methodology not mentioned Work with UB to develop basic IECD training package for CSO Centers Continued work on development of IECD resource pack Provide TA to UB and BOCODOL to assist them with developing standardized IECD training packages for key categories/level of early childhood workers, parents/caregivers PCI continued work on the IECD resource pack Reporting period June 2013- September 2014 2.1.1.Capacity Strengthening of CSOs and key government departments on IECD Support CSO partners to form/strengthen Informal Play/Stimulation groups (centers, wards, community structures) Partners collaborating with other groups to provide IECD and life skills education to children Undertake validation and pilot of IECD resource PCI collaborated with the DCD to train 22 pre￾school teachers from Remote Area Settles and 6 TK partners using the draft resource pack. validation not mentioned in report Conduct TOT on IECD resource pack TOT not mentioned in report Finalize, print, and distribute the IECD Resource Pack for CSO partners and other stakeholders Planned for Quarter 1 in FY 15 Work with the DCD to develop the operational standards/guidelines for service providers for informal play groups Revised draft to be ready for review in Quarter 1 FY 15 Develop a guidance document to assist CSO partners to identify and acquire the necessary IECD learning materials Revised draft to be ready for review in Quarter 1 FY 15 Create a partnership with Bokamaso Pre-school teacher training program to provide TA to have the IECD resource pack to be BOTA accredited and collaborate with them to provide institutionalized Training using the IECD resource pack for CSO partners Not mentioned in report Provide TA to MOESD in the review of ECCE Policy and piloting of the 0-3 curriculum The National Policy on Early Childhood Care and Education is still being reviewed Reporting period October 2014- March 2015 2.1.1.Capacity Strengthening of CSOs and key government departments on IECD Support CSO Partners to form/strengthen Informal Play and Stimulation groups With PCI Support, TK partners BAPR and BBM are partnering with local public libraries following a training provided by PCI Work with DCD to disseminate the IECD Informal Play/Stimulation Guidelines Final copy of guidelines submitted to PCI and undergoing editing and dissemination to different stakeholders for final input Support and mentor TK CSO partners, DCD Final copy of guidelines submitted to PCI and and other government stakeholders to operationalize informal play/stimulation guidelines undergoing editing and dissemination to different stakeholders for final input Work with DCD to conduct TOT on IECD Resource pack While awaiting accreditation, PCI with DCD will conduct TOTs mainly with ECD supervisors in Quarter 3 Finalize, print and distribute the IECD Resource Pack to the TK partners and other selected CSO and Government partners Printing and distribution planned for Quarter 3 Support DCD to roll-out IECD Resource Pack Printing and distribution planned for Quarter 3 Support implementation of IECD by TK CSOs, DCD and other stakeholders to ensure quality 7 TK partners provided education services using various IECD model. Partnerships with parents and caregivers an on-going process Support partners to acquire/procure IECD learning materials/ equipment With PCI Support, TK partners BAPR and BBM are partnering with local public libraries following a training provided by PCI Support Bokamoso Pre-School teacher training program to provide TA for accreditation of IECD Resource Pack and institutionalized training to identified stakeholders Finalization of accreditation of IECD Resource Pack aimed for Quarter 3 Reporting period June 2013- September 2014 Objective 2:Developmental interventions for vulnerable children and adolescents improved 2.1.2 Resource Mobilization for IECD Program Organize and conduct IECD Pitso for resource mobilization and public private partnership opportunities Pitso not mentioned in report Identify and meet with private sector entities to lobby for IECD funding CRS/VoF mobilized donations for play equipment, blankets, mats and crayons for play groups. Also received donation from catholic Church in Francistown Reporting period October 2014- March 2015 Organize and conduct IECD Pitso for resource mobilization and public private partnership opportunities Pitso not mentioned in report Mobilize private sector to fund IECD initiative BBM received $10 000 in funding from PCI private funders in the U.S. TK partners (BAPR, MU, CRS, BBM and HoH) receive food donations from supermarkets and individuals 2.2 Strengthen collaboration between parent/caregivers and IECD centers for improved child development Reporting period June 2013-Sepetmber 2014 Objective 2:Developmental interventions for vulnerable children and adolescents improved 2.2 Strengthen collaboration between parent/caregivers and IECD centers for improved child development Provide technical support to CSO and government partners to develop/strengthen PTA structures and related activities PCI supported TK partners to strengthen or establish PTAs using MoE guidelines. MU and HoH have strengthened their PTAs and they meet regularly Reporting period October 2014- September 2015 Provide technical support to CSO and government partners to develop/strengthen PTA structures and related activities in selected districts MU and HoH have established their PTAs and meet regularly at scheduled times to discuss issues pertaining to child development 2.2.1 Pediatric ART adherence Conduct training for CSO partners of pediatric ART adherence Training on ART adherence not mentioned Provide TA to TK implementing partners to increase reach to children and youth living with HIV PCI Advisors provided on-going technical oversight during implementation 2.3 Access of Adolescents to mentoring, life skills and HIV prevention programs increased Reporting period June 2011- May 2012 Objective 2:Developmental interventions for vulnerable children and adolescents improved 2.3 Access of Adolescents to mentoring, life skills and HIV prevention programs increased Planned Activities Actual Activities Review SRH materials and methods for use in CSO centers Materials not mentioned in report Consultations with Lovelife South Africa Lovelife not mentioned in report Planning for start-up of Young Africa Live Platform PCI continuing discussions with PING on designing Young Africa Live (YAL) platform. Anticipated that partner will be awarded in quarter 4 to begin implementation Provide support for the formation and optimal use of the HIV+ Teen Clubs attached to CSO centers to promote healthy lifestyles and coping successfully with HIV infection Teen Clubs not mentioned in report Provide technical assistance to the MoE to help ensure the effectiveness of the planned new life skills curriculum for adolescents PCI will also collaborate with and provide TA to MoE pre-school division Reporting period June 2013- September 2014 2.3.1 Strengthen existing life skills programs among CSOs Using the National life skills framework and PCI developed tool-kit, work with key stakeholders to conduct TOTs Tool-kit draft reviewed and refined. TOT not mentioned in report Support other stakeholders to roll out life skills trainings to CSOs Trained social workers and CSO staff in life skills Provide TA to CSO partners to implement effective life skills programs Not mentioned in report Develop a tool-kit for adolescent life skills training in line with the national skills framework (incl, integration of SRH and prevention of HIV) Tool-kit draft reviewed and refined. Provide TA to DCD to implement life skills based programs in remote area dwellers hostels (MoH is now included) PCI and DCD conducted needs assessments in hostels in Mabutsane, Shadi-Shadi and Mantshwabisi Reporting period October 2014- September 2015 Work with MoESD to finalize life-skills toolkit Aim to complete in Quarter 3 Work with MoESD to train TOTs Not mentioned in report Provide TA and mentoring to TK CSO partners to roll-out life skills based education and implement effective life skills programs PCI working with BNSC to implement a life skills training program targeting past, present and future sports achievers Finalize, print and distribute the Adolescent Lifeskills Toolkit to partners and other selected CSO and government partners Tool to be finalized in Quarter 3 Provide mentoring support to DCD to implement Lifeskills based programs in remote area dwellers hostels Hostels not mentioned in report Identify, adapt and document promising life-skills methods Not mentioned in report 2.3.2 Strengthen HIV+ Teen Clubs Reporting period June 2013- September 2014 Work with UNESCO, Baylor and MoESD to strengthen HIV+ Teen Clubs for selected CSO partners MU and HPP Mabutsane and Phikwe sites through partnership with Baylor Children’s Clinics and local clinics established and manage HIV+ Teens Clubs Assisted selected CSO partners to form and run HIV+ teen clubs depending on the needs identified HwwB identified adolescents living with HIV among their kids clubs who will be assisted to join an HIV+ teen club previously run by Baylor Rollout the HIV/AIDS Integration Strategy Rollout not mentioned specifically but HIV integrated into various approaches Reporting period October 2014- September 2015 Work with DHMTs, Baylor and MoESD to strengthen HIV+ Teen Clubs for selected CSO partners PCI will work with partners to promote testing and counselling among children and continue to strengthen Teen Club activities for adolescents Assist selected CSO partners to establish HIV+ teen clubs depending on needs identified Not mentioned in report Objective 3: Neglect, abuse, exploitation of children, adolescents and women Key result area 3.1 Strengthened systems to prevent abuse and meet the needs of those neglected, exploited and abused The table below presents the planned activities for key result area 3.1 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Work with WAD District Officers, the DCPCs and focal CSO centers to develop and implement strategies for strengthening systems to prevent and address abuse at the district and community level. KWSP together with WAD, DSS and PCI identified community structures that could be strengthened in order to identify and provide support to the most vulnerable households Reporting period October 2012-September 2013 No workplan available for this reporting period HWWB, established two VCPCs at Mogonono and Maitseta, KYT established one in Lerala, while CRS Sebina formed a child protection committee which will be formalized into a VCPC once the DCPC is established and functional. KWSP established Lekgwapheng ward gender committee which works closely with Kweneng District Gender Committee. With membership of 20 (9 males; 11 females), the Lekgwapheng committee aims to enhance community capacity to respond to gender based violence at community level. In collaboration with DSP, Marang spearheaded the formation of DCPCs in six of the TK districts. Reporting period June 2013-September 2014 Conduct GBV TOT for social workers KWSP trained social workers and CSO staff in life skills and GBV. It was reported that PCI and implementing partners worked with DSP and GeAD to strengthen the capacity of existing district and community structures to address children and gender related issues Provide technical assistance on the GBV training roll out at district level for social workers No reporting on this activity Assist GAD to conduct national validation workshop on the draft national GBV referral system No reporting on this activity Support GAD to develop national 365 days GBV campaign strategy and action plan No reporting on this activity Work with CRS to advocate and ensure mass birth registration of children without birth certificates in the Ngami and Okavango areas. (We will let you know the date) Major activities planned in Quarter One FY 15 include organizing and conducting mass birth registration campaigns Reporting period October 2014-September 2015 Work with Childline and lifeline to strengthen existing OVC services PCI, in partnership with Childline Botswana conducted training on recognizing and reporting abuse targeting both children and teachers at MU and HoH pre-schools. Work with government stakeholders to conduct mass campaigns on birth registration in TK districts. PCI initiated discussions with the Department of Civil and National Registration (CNR) and Masiela Trust to advocate for extension of the waiver for birth registration penalty fees announced last year by MLHA and to also launch a birth registration campaign targeting vulnerable and hard to reach population. Work with Marang to strengthen DCPCs TK partners worked with available community structures to sensitize them about child protection, care and support as well as gender related issues to enable them to identify and support neglected, exploited and abused children and women. Such structures include schools, community leadership structures, existing groups as well as DCPCs and VCPCs. Marang and other TK partners continued to form and strengthen the lower level coordinating body – VCPCs. Marang also supported various DCPCs; Charleshill Tsabong and Ghanzi with different forms of assistance. Support TK partners to engage community leaders in actions towards ending violence against women and girls No direct reporting on this activity TK partners have established networks with traditional leadership such as Dikgosi, VDC members and community members who make referrals. PCI and its partners at national and district level organized and participated in a number of activities to commemorate the 16 Days of Activism against GBV, International Men’s Day, World AIDS Day and International Human Rights Day, all of which raised awareness among communities on child abuse and GBV related issues. Such activities included community dialogue, targeted group discussions, drama, workshops, organized matches, training and campaigns. Provide technical support to GeAD to develop 20/20 National ActionPlan towards ending GBV PCI worked with GeAD and other stakeholders to develop the National Gender Based Violence Strategy aimed at ending GBV by 2020. Build capacity of BBM, WAR, KSWSP and CRS for piloting GBV Referral System Literature review to inform development of national GBV service providers directory was completed. The piloting of the GBV referral system was not implemented but expected to start soon. Through community conversation, home visits, school programmes, GROW and other group activities TK partners identify, refer and follow up clients that have been abused and neglected. Conduct rapid district GBV response assessment in Gantsi, Mabutsane and Selibe Phikwe No reporting on this activity Conduct media and social media communication training on reporting GBV targeting commentators, artists, Radio and TV presenters and sports icons No reporting on this activity Support GeAD to coordinate the implementation of the GBV referral system The piloting of the GBV referral system was not implemented but expected to start soon. Key results area 3.2: Services that cater for and protect abused children and women increased. The table below presents the planned activities for key result area 3.2 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Explore with government and civil society stakeholders options for creating low-cost safe havens in implementation districts No reporting on this activity Assist Kagisano and KYT to incorporate the GROW model to help women using their services to become more economically self-reliant. PCI held consultative meetings with both organizations educating them about the model, the requirements and expectations. PCI has assisted KYT with the identification and selection of the GROW Coordinator and GROW Community Facilitator, two of the key partner personnel that will drive GROW implementation Reporting period October 2012-September 2013 No workplan available for this reporting period Kagisano Women’s Shelter continued to provide services to abused and is working with HWWB in Molepolole to provide counseling services to HWWB clients. The Women’s Shelter conducts outreaches to communities to raise awareness on gender based violence and how they can support and refer survivors to relevant services. Reporting period June 2013-September 2014 Support GAD to develop a national tool to assess GBV prevalence at the community level PCI in partnership with GeAD is finalizing procurement for recruitment of a consultant to develop national GBV community assessment tools Based on assessment, identify potential safe-haven opportunities and support start-up PCI, Women Against Rape (WAR), KSWSP and GeAD will be embarking on a partnership to create community safe spaces for GBV survivors. The model will be initially piloted in Mochudi and Maun. PCI, in partnership with Women Against Rape (WAR), KSWSP and GeAD, developed a concept note in August 2014 on Community Safe Spaces (CSS) for GBV survivors. Reporting period October 2014-September 2015 Continue to support GAD to develop a national community level GBV prevalence assessment tool No reporting on this activity Develop National Guideline on Community Safe-Havens No reporting on this activity Conduct training on the use of the National Guidelines on Community Safe-Havens No reporting on this activity Based on the Safe-Havens Guidelines, identify potential safe-haven opportunities and support start-up with selected communities No reporting on this activity KWS continued to provide services to individuals experiencing physical, emotional and sexual abuse and the shelter was expanded to other districts. Document Community Safe Havens best Practice and Lessons learned No reporting on this activity Key results area 3.3: Child welfare workforce skills development. The table below presents the planned activities for the key result area 3.3 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Work with government and CSO stakeholders to review the minimal skill set needed for the child welfare workforce to improve their performance in preventing and handling abuse, and to review existing training programs to identify gaps and strengths that can be built upon. Planned for next Q: Hold consultations with DSS, Childline, WAD and other stakeholders to review the minimum skills set needed for child welfare workforce to improve the skills in recognizing, preventing and handling abuse. Reporting period October 2012-September 2013 No workplan available for this reporting period One module in IECD Resource Pack (ready for use in 2014) is on child protection and will be used to train the child welfare workforce including teachers, social workers, CSO staff and police. Reporting period June 2013-September 2014 Support DSP to conduct TOT for selected Child Care Workforce on skills to recognize abuse among children (DSS is replaced with DSP) During the IECD pilot training for 22 pre-school teachers and selected TK partners, using the resource pack, sessions were facilitated on “Recognizing signs and symptoms of child abuse” Conduct training to CSOs and other Child Welfare Workforce to recognize abuse among children See above Provide technical assistance to partners on recognition of abuse among children In FY 15, PCI will conduct training for selected child care workforce on skills to recognize and abuse among children. The cascading of the initial training is planned to be provided with Marang and other major partners such as Child Line, the University of Botswana and others. Reporting period October 2014-September 2015 Conduct training for selected Child Care Workforce on skills to recognize abuse among children PCI in partnership with Childline Botswana conducted targeted training with children and teachers at Mothers Union and House of Hope pre-schools on recognizing and reporting abuse. 46 children and 19 adults participated (teachers and social workers) from two districts in three separate sessions. HwwB conducted a half day awareness activity with 9 individuals from Bonewamang school management on child protection and how to identify children who need help. DSP and Ministry of Education tasked Marang who collaborated with REPSSI to train 353 teachers (including guidance and counselling teachers), social workers and civil society from various districts in psycho-social support. Sensitize DCPCs and DGC in TK districts on GBV referral system and recognition of abuse among children GBV referral system to be piloted in future Key results area 3.4: Involvement of men as peer educators, mentors and caregivers and role models increased. The table below presents the planned activities for the key result area 3.4 based on the annual work plans and the actual activities based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Hold consultation meetings with MSYC & BNSC to build on their Sport Role Modeling/Heroes program to change male attitudes/behavior and engage them as peer educators, mentors, caregivers and role models. No reporting on this activity. PCI, working with WAD have established strategic alliances with the Men’s Sector and Gender Links and agreed to develop a concept note on the 365 Days’ Campaign on GBV against women and children. PCI CSO partners identified the Men’s sector as a strategic partner in addressing gender issues at the district level. Partners such as HPP and Hww have implemented activities targeting men to increase their role in prevention of GBV. Discussions have also started between PCI OVC & Gender and DHAPP programs on how to collaborate to address gender issues within the BDF- as a membership to the men’s sector. In addition, PCI CSO partners have also identified the Men’s sector as a strategic partner in addressing gender issues at the district level. Partners such as Hope World Wide and Mothers Union have activities targeting men to increase their role in prevention of GBV Reporting period October 2012-September 2013 No workplan available for this reporting period HPP organized a workshop for 12 men to discuss GBV and In her Shoes toolkit to raise awareness on domestic violence. BBM (in partnership with Stepping Stones and Mencare) works with expectant fathers preparing them for the role of caring for children. BBM conducted training sessions to taxi drivers. Community mobiliser in Lerala district targeted 3 football teams around gender related issues. Commemorative days are used to reach out to men around GBV and PCI collaborated with Men’s Sector to conduct a men’s dialogue on 16 Days of Activism on VAWC. KWS partnered with BDF to commemorate Father’s Day and included gender messages. Reporting period June 2013-September 2014 Conduct training for TK implementing partners on male involvement Plan for next quarter: organize and conduct training in gender for PEPFAR implementing partners and Managers Provide technical assistance on the implementation of male involvement by TK CSO IPs In FY 15, PCI will work with Men Engage to implement activities geared to strengthening involvement of men in addressing GBV and gender inequalities. Activities will include training on “One Man Can,” “Men Care” and also to mark commemorations of International Men’s Day. Finalize a concept note on men as role models /mentors in addressing GBV No reporting on this activity This concept note is to guide men’s inclusion as a role model against GBV. Right now the draft has been completed through a collaborative effort of Men's Sector, GAD and PCI. Develop a strategy and implementation plan on male role modeling and mentoring No reporting on this activity The strategy is to help increase men's broader involvement in addressing GBV. We actually utilize the MoH training manual in our trainings. Provide technical assistance to partners on male involvements In FY 15, PCI will work with Men Engage to implement activities geared to strengthening involvement of men in addressing GBV and gender inequalities. Activities will include training on “One Man Can,” “Men Care” and also to mark commemorations of International Men’s Day. Collaborate with DHAPP to conduct training on GBV for service providers within BDF No reporting on this activity This activity is in an effort to synchronize PCI's TK and DHAPP programs to benefit from each other in a cost effective way. We thought we can take advantage of the fertile ground at BDF to mainstream gender effortlessly. We have not planned to expand the TK program to prisons and police. Reporting period October 2014-September 2015 In Collaboration with Men and Boys for Gender Equality (MBGE) conduct training media and social media communication targeting social commentators, artists, Radio and TV presenters and sports icons No reporting on this activity In collaboration with MBGE and GeAD develop street GBV harassment campaign in commemoration of the IMD 2014 PCI and partners worked with stakeholders including GeAD, Men’s Sector, Men Engage, Social Workers and Health Workers to organize activities on International Men’s Day – media briefings and community conversations. In collaboration with GeAD and MBGE conduct CEO’s male Dialogue on the role of men in ending GBV against women and children No reporting on this activity PCI and its partners at national and district level organized and participated in a number of activities to commemorate the 16 Days of Activism against GBV, International Men’s Day, World AIDS Day and International Human Rights Day, all of which raised awareness among communities on child abuse and GBV related issues. Such activities included community dialogues. HPP conducted sessions with 3 football teams (65 men) on gender norms and gender related issues. Rollout the implementation of the men as role models/mentors strategy in ending violence against women and children CRS Francistown in collaboration with True Men identified and trained 21 men as peer educators; and 27 male adolescents as peer educators. Key results area 3.5: Awareness of the rights and ability to seek help by vulnerable children, adolescents and women increased The table below presents the planned activities for key result area 3.5 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Hold consultations with DSS to develop a simple user-friendly guide on the major laws affecting children and how to get help when rights are violated. PCI OVC Advisor met with the Head of Child Protection Unit and discussed ways to finalize the DCPC’s guidelines and also inaugurate the DCPCs (concerning OVC) Hold consultations with WAD to develop a simple user￾friendly guide on the major laws affecting women and how to get help when rights are violated To start later in programme Reporting period October 2012-September 2013 No workplan available for this reporting period Working with the Department of Social Protection (DSP), Gender Affairs Department (GAD), Men’s Sector and other NGOs at the district level, the program disseminates both the National Children and Gender related Acts to the existing structures. Implementing partners use a powerpoint presentation developed by PCI on women and child related acts in Botswana. Marang, BAPR, Mothers Union, House of Hope and HPP conduct seminars to disseminate Acts to different audiences. Reporting period June 2013-September 2014 Provide technical assistance to TK partners to disseminate gender and children’s laws and acts targeting traditional leaders TK partners used all available platforms to disseminate children and gender related acts. The PCI Gender Advisor also presented on the marital power act. K partners deliberately organized dissemination meetings targeting community gate keepers to include traditional and religious leaders. (For further activities in dissemination of acts please see activities 4.1.1, 4.1.3 and 4.1.5.) Reporting period October 2014-September 2015 Provide technical assistance to TK partners to disseminate gender and children’s laws and acts targeting traditional leaders. TK partners use available platforms to disseminate both the children’s and gender related acts (GROW group meetings, kids and teen clubs etc). 09/03/2015 Session on Children’s Act and Marital Power Acts for Mohembo and Shakawe caregivers and traditional leaders PCI implementing partners continued to disseminate Gender and Children related Acts to communities; and are implementing gender and OVC supportive activities related to the Acts. (For further activities in dissemination of acts please see activities 4.1.1, 4.1.3 and 4.1.5.) Objective 4: National frameworks and policies Results area 4.1: Support to implementation of Children’s and Domestic Violence Acts/Plans. The table below presents the planned activities for key result area 4.1 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Work with DCPC and other community structures to determine how they can address existing service gaps in order to be fully compliant with both OVC Acts/plans. PCI assisted with workplans The PCI Gender Advisor held consultative meetings with the Chief Gender Officer (who is the OVC & Gender Focal Person) on the workplan process which will guide joint implementation of the OVC & Gender Program by PCI and WAD Training in Gender Concepts/Awareness and Integration of gender in HIV Programming for Maatla project Training workshop for 35 Maatla CSO staff Support to DSS to develop Annual Work Plan Slow progress, contracts were ending and meeting planned for Q4 Support to WAD to develop Annual Work Plan 2 year WAD/PCI joint work plan finalized in Q3 MOU drafted, to be signed off in Q4 Provide technical assistance to strengthen the functioning of DCPCs, and potentially to expand their mandate to focus on vulnerable families including gender mainstreaming PCI OVC Advisor met with the Head of Child Protection Unit and discussed ways to finalize the DCPC’s guidelines and also inaugurate the DCPCs Reporting period October 2012-September 2013 No workplan available for this reporting period PCI developed standardized presentation and facilitations guidelines on Children and Gender Acts and shared with CSO partners to disseminate Reporting period June 2013-September 2014 Provide continued support for the implementation of the guidelines During the reporting period, PCI worked with DSP and Department of Community Development (DCD) to develop a joint work plan which was completed and signed off by PCI, DSP and DCD Provide technical assistance to develop the Probation and After Care guidelines In April 2014, PCI provided financial and technical support to DSP to conduct a two-day stakeholders meeting to review the Probation and After Care Guidelines. DSP is yet to organize validation workshops to finalize these guidelines Support GAD to develop a dissemination plan on National Policy on Gender PCI provided support for consultative meetings for validation of the policy and will support dissemination/implementation once the Policy is approved. The delay in the approval of the Gender and Development Policy hampered the plan for PCI to support GeAD with dissemination and implementation of the policy during FY14. Develop a popular version of gender and Children related acts and laws (communities, NGOs and the GOV partners) In October 2013, the PCI OVC Advisor participated in a two-day meeting with DSP to review and finalize the Child Protection Protocol. During the meeting, the team also reviewed and finalized the implementation framework of the Children’s Act. Translate the popular version acts/laws above- mentioned into Setswana No reporting on this activity Reporting period October 2014-September 2015 Provide support to DSP to conduct annual meetings No reporting on this activity Finalize and translate into Setswana popular version of gender and Children related acts and laws. During the reporting period, PCI and BONELA agreed to work together to finalize the already existing simplified children’s Act and translate it into Setswana Results area 4.2: Support to development and implementation of OVC policy and draft national policy in Gender and development improved. The table below presents the planned activities for key result area 4.2 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Provide support to WAD to complete the National Gender and Development policy and the communication strategy. Only documents reviewed Provide technical assistance to WAD to help with dissemination and implementation of the Gender & Development policy once adopted. Only documents reviewed Finalization of Women’s Affairs Department Strategy and Annual Plan PCI Gender Advisor participated in finalisation of WAD strategic plan Reporting period October 2012-September 2013 No workplan available for this reporting period PCI worked with GAD and other development partners on: • Gender and Development Policy • The GAD Policy Strategy • GAD Policy National Operational Plan PCI provided technical and financial assistance to development of National Gender Analysis and Mainstreaming curriculum. Documented Kweneng District Gender Committee as best practice guide – Step by Step guide to be used by District Gender Committees and HIV/AIDS committees to establish these structures. Supported DSP in development of guidelines on the destitute housing programme – the guidelines will support the shelter and care needs of vulnerable families Reporting period June 2013-September 2014 Report refers to detail in activity 3.5.1. Reporting period October 2014-September 2015 Provide technical support to DSP to finalize the Child protection protocol PCI developed CFC assessment tools and the government (Social Protection) assessment tools and recommended harmonization of the two. PCI and DSP had an initial meeting and shared the tools The project will hold further discussions with DSP in Quarter Three and depending on outcome, develop a harmonization plan. Support finalization and rollout of the Children's Act dissemination plan 21/01/2015 Dissemination of Children’s Act for caregivers at Mabele Support GAD to develop a dissemination plan on National Policy on Gender and Development (once approved) In 2014 the Cabinet reviewed the Botswana draft national policy on Gender and Development. Comments from the Cabinet were incorporated by GeAD and stakeholders and re-submitted to cabinet which finally approved it in March 2015. Policy has now been submitted for the next parliament sitting for adoption Results area 4.3: National management of data for OVC and gender services strengthened. The table below presents the planned activities for key result area 4.3 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Review the status of existing data on OVC and OVC-related services and determine national priorities for additional research to inform policy and action. OVC data not reviewed but the PCI Gender Advisor was sponsored by UN to attend a Gender Statistics meeting in Tunisia Explore the feasibility of incorporating a GIS component for use of spatial data that would include mapping of OVC services, client mapping, and other features based on need. No reporting on this activity Provide technical assistance to WAD to develop and implement a national gender database Based on database developed under Building Bridges, PCI completed database in Q3 Reporting period October 2012-September 2013 No workplan available for this reporting period PCI supported TK partners with database and plans to key partners access to database to track information and beneficiaries reached under the TK programme. PCI will work with DSP and GAD to develop relevant databases. Reporting period June 2013-September 2014 No activities contained in workplan Annual report for 2013-2014 notes that PCI will continue to support DSP to develop a database stating that “It is true that Database for DSP is proposed earlier. We will continue to highlight this fact and work with DSP, however It is also important to have a discussion and agree on the way forward with DSP on this particular issue” Reporting period October 2014-September 2015 Provide technical assistance to GeAD to develop M&E system. PCI conducted a full assessment of the GeAD M&E system from October 16-23, 2014 and compiled a report. The report had gaps due to inadequate information provided by GeAD, who has since been provided the service of a Peace Corps Volunteer who is a specialist and conducted a similar assessment. The two parties met in Q2 Plans are underway to complete the report and jointly develop a capacity building plan Provide technical assistance to DSP for the development database by participating in the technical working group. PCI Country Director met with the Director of DSP to discuss the way forward on the OVC database. The DSP Director confirmed that this matter is still pending before MLGRD. TWG not mentioned Results area 4.4: National curriculum on Gender Analysis and Mainstreaming developed and implemented. The table below presents the planned activities for key result area 4.4 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Assist WAD to form a National Reference Group on gender analysis & mainstreaming PCI and WAD established two structures: The reference group and the Technical Working Group Work with WAD to develop a national gender analysis and mainstreaming curriculum TWG to design and draft curriculum Work with WAD to train Master Trainers No reporting on this activity Reporting period October 2012-September 2013 No workplan available for this reporting period Gender analysis and mainstreaming training curriculum completed and technical review pilot training conducted in August 2013. Completed facilitators and participant guides and TOT workshop materials. Purpose is to provide a standard package for practitioners to delivery comprehensive training on gender and development in line with National Gender Policy on Gender Development. Reporting period June 2013-September 2014 Support GAD to finalize the national gender analysis and mainstreaming training curriculum The curriculum is now under final review in preparation for printing Develop curriculum implementation roll out plan with GAD No reporting on this activity Collaborate with GAD to conduct Master TOTs workshop on gender analysis and mainstreaming Training of PEPFAR managers and implementers in gender using the National Gender Analysis and Mainstreaming Curriculum planned for 2015 Q1 Reporting period October 2014-September 2015 Support GeAD to package and print the national Gender Analysis and Mainstreaming Curriculum No reporting on this activity Work with GeAD to develop curriculum rollout plan Working with GeAD, PCI finalized the Gender Analysis and Mainstreaming Curriculum and developed a rollout strategy Collabrate with GeAD to conduct Master trainers on Gender Analysis and Mainstreaming Curriculum Training of Trainers (TOT) will be done in Quarter Three Print and distribute the Gender Analysis and Mainstreaming Curriculum A draft type set curriculum is available and undergoing internal review Train TK Partners on Gender Analysis and Mainstreaming In collaboration with GeAD and USAID, PCI conducted the Gender Analysis and Mainstreaming training workshop for PEPFAR implementers Mainstream Gender and GBV issues into the police training curricula. No reporting on this activity Results area 4.5: Logistical support to DSS and Women’s Affairs Department Annual meetings. The table below presents the planned activities for key result area 4.5 based on the annual work plans and the actual activities achieved based on the Annual Progress Reports. Reporting period June 2011-May 2012 Planned activities Actual activities Provide support to WAD to help with effective organization and conduct of their annual meetings Details were provided in the joint PCI and WAD workplan Reporting period October 2012-September 2013 No workplan available for this reporting period Financial and technical support for documentation and finalisatiaon of proceedings of National Child Care Forum in February 2013 to review DCPC Guidelines – finalized and awaiting official sign off. Reporting period June 2013-September 2014 Work with DSS to organize and conduct national events on children and annual NGO forum Annual NGO forum not mentioned in annual report. In next Quarter, support GeAD and TK partners to plan and implement activities/events to commemorate 16 Days of Activism, International Men’s Day, World AIDS Day and International Human Right Day. Support GAD, DSP and NACA on commemoration of the international, and national Gender and HIV/AIDS related events. The PCI Gender Advisor worked closely with GeAD to organize and conduct commemorative events during the reporting period including IMD, 16 Days of Activism against GBV; World AIDS Day, International Human Rights Day and International Women’s Day CI funded some activities and participated in press briefs and panel discussions Reporting period October 2014-September 2015 Support GeAD to conduct annual strategic planning No reporting on this activity Support GAD, DSP and NACA on the commemoration of the international and national Gender, OVC and HIV/AIDS related events. PCI supported GeAD to organize annual gender related commemorations. PCI worked with DSP, Marang and Regional Psychosocial Support Initiative (REPSSI) to organize and conduct the first ever Psychosocial Support (PSS) Conference Capacity development activities for IPs Table 1 Summary of technical support provided by PCI relevant to Objective 1 Organisation Support provided (i.e. specify which training; what type of technical support etc.) Staff targeted Duration of Training / support (and dates/year) Key content covered After-training support provided Training: Journey of Life (JOL) 363 First 3 quarters Community mobilisation methodology PCI will work with partners to support implementation of action plans developed as a result of the training Training: Aflateen 10 participants from seven PCI partners Between Oct 2011 and March 2012 (five days) The trained partners are expected to recruit youth they regularly work with, to enrol them in the programme and lead them through the financial skills building and social development curriculum PCI staff One-on-one support Tier One partners Quarter Two Assist partners to reflect on their strengths and prioritising activities accordingly. A phased approach was adopted starting with designing activities and approaches that will assist Helped PCI to identify capacity gaps which will inform capacity building plans that are tailored to the individual needs of the organisations. partners to perfect their technical areas of strength and building onto that to introduce other components of the programme, such as gender into their work PCI GROW KyT PCI assisted KYT with the identification and selection of a GROW Coordinator and GROW Community Facilitator, to drive GROW implementation. PCI expects at least two GROW groups to be formed by the end of May, 2012. JOL Awareness Workshops CRS, Humana People to People, House of Hope, Kgetsi Ya Tsie and BAPR. DACs, Social Workers and Health Officers from Mabutsane, Moshupa, Good Hope, Tlokweng Sub district, Ramotswa, Francistown, Lobatse Council, Southern and one psychologist from Sabrana mental hospital attended. All the individuals trained will form either part of the DCPCs or are members of other community structures that PCI will be Trained how to implement the JOL Awareness Workshops with communities. Trainees are expected to form district JOL teams in their respective districts or to join existing JOL teams within their districts, and start implementation. working with for identification of vulnerable households in the programme districts. (Other individuals trained outside the program districts were brought on board by MCCNT and DSS who will be directly responsible for mentoring and monitoring implementation). The training was also attended by officers from Otse Home Based Care and by primary and secondary school teachers from Lobatse and surrounding villages. A total of 59 individuals (43 females and 16 males) were trained. PCI with MCCNT JOL Four groups previously trained in how to implement the JOL Awareness Workshops during the Building Bridges Programme, to teach them how to implement the JOL Action Workshops. The four groups included Ngamiland Coalition of NGOs (NCONGO) members together June 25 – 30 2012 in Francistown Participants presented their district reports and shared challenges and successes. It was encouraging to hear from participants who have used JOL Picture Codes to generate community discussion and how that led to community members raising key issues affecting them The JOL district partners will follow up with communities in addressing issues they identified from the discussions. PCI will use platforms that bring partners together as an opportunity to share and learn from others regarding some of the upcoming promising practices. with government social workers in the NCONGO districts, Kgalagadi and Ghantsi Social Workers, NGOs, Police, DACs and health staff from Tonota, Mmadinare, and Tutume sub￾districts. A total of 65 participants attended the training. PCI Regional GROW Technical Advisor GROW PCI Botswana team and KyT GROW implementation team. April 2012 Topics covered include structure of GROW groups; introduction to wealth ranking; roles and responsibilities within GROW groups; record keeping; decisions on loans and interest. Following the training, the first group with a membership of 20 women was formed in Tumasera. SSI Aflateen financial skills and social development curriculum 10 staff from seven PCI partners – BAPR), HWW, Kagisano KWSP, MU, Vicariate of Francistown (Maun), FECH and Hope Mission who are MCCNT members February 2012 After successfully completing the programme, partner staff were given until the first week of April to communicate the benefits of the programme to the rest of staff in their organisations, sensitise the community, recruit youth ages 14-18+ to participate, administer the first baseline survey to students and hold the first session. Aflateen Regional Master Training (RMT) course in Kampala, Uganda The PCI Economic Strengthening Project Officer, a staff member from Stepping Stones International (SSI) and a representative from the Curriculum Development Department, Ministry of Education attended the training The three trainees from Botswana envisage developing a consortium relationship that will coordinate all Aflateen activities across Botswana. During Quarter One of FY 15, PCI is taking the lead to organise an Aflateen training workshop that will include facilitation support from SSI and the MoE, serving as a key first activity to formalize the tripartite collaboration. TK implementing partners will also have opportunities to share their Aflateen experiences and lessons learned. TK partners Training Village Child Protection Committees (VCPCs) Their mandate and functions An estimated 109 VCPCs have been formed in the TK areas of operation JOL workshops 21 staff (4 males; 17 females) from Mpule Kwelagobe and 15 staff (6 males; 9 females) from Ikago Rehabilitation Centres PCI GROW Learning Forum 37 staff from BBM, HPP (Phikwe, Mabutsane and Ghanzi), HwwB, and PCI-Chobe August 2014 The interactive, participant-driven forum provided an opportunity for each site to present its progress to date, challenges and the solutions that can help build strong groups and learn innovative problem resolving approaches to take back into their own communities. To capture the learning ideas, each site developed a 4- month plan, outlining the key initiatives and activities they will pursue through the end of calendar year 2014. PCI is following-up with each Coordinator on implementation and providing guidance as necessary. Following assessments conducted by PCI staff on all GROW groups in Q1, which revealed errors in documentation using the GROW books provided to capture savings, PCI Economic Strengthening and M&E team conducted a four day refresher training on GROW tools including GROW books , Life Changing Index, Management Information System and the Comprehensive Family Care Approach for all 29 Coordinators, Facilitators and Site Managers implementing GROW in Chobe, Ghanzi, Kgatleng, Kweneng, Southern and Phikwe Districts PCI Economic Strengthening (ES) and M&E team Refresher training: GROW tools All 29 Coordinators, Facilitators and Site Managers implementing GROW in Chobe, Ghanzi, Kgatleng, Kweneng, Southern and Phikwe Districts Four days GROW books, Life Changing Index, Management Information System and the Comprehensive Family Care Approach. The training was participatory, with data exercises and examples using actual data from their GROW groups provided to help the Coordinators and Facilitators understand the common errors found in the MIS forms At the end of the training, all sites developed action plans to share learnings with all GROW book writers and representatives and this task has been completed. Table 2 Summary of gender related technical support provided by PCI Organisation Support provided Staff targeted Duration Content covered After￾training support PCI program team •to orient partners on the OVC & Gender program and the technical areas •to re-orient partners to M&E basics •to introduce partners to the OVC & Gender program indicators and •to discuss the importance of data, data collection tools and processes CSO staff from all implementation sites. A total of 45 individuals (20 males and 25 females) attended the training G1: 5- 9March 2012 G2: 19-23 March 2012 During the training, the PCI Technical Advisors continued to work with partners to refine their implementation plans and assisted them to align their activities to the program indicators. During the Through the administration of a pre and post - test during the training, the results indicated that Monitoring and Evaluation was a new concept to many. More than 50% of the participants had never been trained in M&E while the rest training, different organizations shared their data collection tools. Most of the time the data collection tools were similar and the team proposed to adapt elements from each other’s tools to come up with more comprehensive and acceptable consolidated tool for all to use had been trained at different times by IDM, PACT, AED, SAT and PCI and needed a re￾fresher. PCI will continue to provide one￾on-one technical support to partners to improve their general understanding of M&E and application of skills learned during the training. PCI Gender Advisor Technical support Women Affairs Department (WAD) Quarter One Preparation of the inaugural launch of the National Gender Based Violence Pitso (campaign) which was held in Maun 27-28 October 2011 Launch of the campaign was an important event for the OVC & Gender program as it laid a foundation for the work that PCI and partners will be undertaking to achieve the program’s Gender objectives in Maun and other areas. PCI Gender Advisor technical support WAD Preparation of and attended the commemoration of the International Women’s Day (IWD) 2012 held in Serule with the theme “Connecting Girls-Inspiring the future”. PCI Gender Advisor technical support WAD Quarter two Develop the concept paper for the Gender Based Violence (GBV) roundtable discussions held in Kasane from March 20-21, 2012 PCI OVC Advisor technical support DSS Quarter 1 Develop their annual workplan and ensure that activities in the OVC & Gender Program under their mandate are included PCI Gender Advisor (on request from the Maatla project management The objectives of the workshop were as follows: To understand gender dynamics at personal level in order to be in a position to interrogate them. Define and introduce gender concepts and review international, regional and national commitments on gender equality. Introduce participants to Gender analysis and Gender Mainstreaming. Inform participants on how to integrate/mainstream gender issues into 35 Maatla CSO staff at Oasis Motel Tlokweng May 30 to June 01, 2012 Develop gender awareness among Maatla Project CSOs to enhance their gender advocacy in policy development, programming and integrating gender into the national planning processes Following the training, the Maatla program and World University Services of Canada agreed to do the following to assist their implementing partners in carrying out activities related to lessons learned from the training; Develop a follow up action plan and establish a Mentorship Group Provide the CSOs with data their work. Provide information to the participants on monitoring and evaluation of gender issues. sheet on up-to￾date gender issues/facts in Botswana Mobilize resources to support the implementation of the follow up action plan activities Maatla committed to offering on￾going support and document best practices and lessons learned PCI Training TK partners Village Child Protection Committees (VCPCs) VCPCs mandate and functions an estimated 109 VCPCs have been formed in the TK areas of operation PCI worked with DSP and Department of Community Development (DCD) to develop a joint work plan. Activities implemented in the work plan include training Training Social Workers and CSO staff life skills and GBV PCI Gender Advisor Training TK partners and PCI staff: attended by 31 participants (13 male; 18 female) February 17-20 Following the development of the gender norms facilitation guide to assist partners to conduct focused TK partners developed action plans which stipulated how they were going to roll out the training and quality dialogue on gender norms, training was held to their fellow staff and also identified community groups that they will be targeting. PCI, in partnership with Childline Botswana Training targeting both children and teachers at Mothers Union and House of Hope pre￾schools. 46 children (20 Boys, 26 girls) participated in addition to 19 adult participants (2 Males, 17 females), including teachers and social workers from the two districts in three separate sessions. recognizing and reporting abuse At the end of the training, PCI and Childline Botswana agreed to continue to implement more collaborative activities PCI and TK partners worked with stakeholders including GeAD, Men Sector, Men Engage, Social Workers and Health Workers Organized activities that included media briefings, community conversations different audiences including taxi drivers and soccer teams, seminars, and training Discussions included both men and women and generated issues around the role of men in HIV prevention including condom negotiation, multiple concurrent partners and male circumcision as well as issues around child care and GBV. training gender norms February partners 2015 discussed strategies to involve men to ensure facilitators are able to bring out and discuss issues pertaining male involvement comfortably. In collaboration with GeAD and USAID, PCI Training workshop PEPFAR implementers: 53 PEPFAR Program Managers and Implementing Partners attended the workshop. Represented agencies included: USAID, CDC Botswana, UB, BOCAIP, BORNUS, Peace Corps, FHI 360, AIHA, Red Cross, Intra￾Health, BONEPWA, Botswana Baylor, Botswana Upenn Partnership, DOD DHAPP, PEPFAR, Tebelopele, NASTAD, MSH and Botswana Harvard AIDS Institute. October 28 – 30 2014 Gender Analysis and Mainstreaming: introduce the Gender Mainstreaming curriculum as a standard training package for development practitioners PCI and DSp, Marang, REPSSI Psychosocial Support (PSS) Conference 136 individuals (38 males; 98 females) child care practitioners ranging from The theme of the two days’ conference was “No child protection without PSS; No government workers, private practitioners, Civil Society Organization (CSO) researchers, the Academia, traditional leaders and regional and international development partners/donors. PSS without Child Protection.”