USAID/South Africa Umbrella Grants Management Project End of Project Partner Evaluation PROJECT CONCERN INTERNATIONAL MARCH 2013 This publication was produced for review by the United States Agency for International Development. It was prepared by Creative Consulting & Development Works. USAID/South Africa Umbrella Grants Management Project End of Project Partner Evaluation PROJECT CONCERN INTERNATIONAL This report is made possible by the generous support of the American people through the U. S. Agency for International Development (USAID), under the terms of APS 674-07-011, through United States Government’s President’s Emergency Plan for AIDS Relief, operated by FHI 360. The contents and opinions expressed herein do not necessarily reflect the views of USAID or the United States Government. Project Concern International (PCI) UGM End of Project Evaluation i ACKNOWLEDGEMENTS Many people have contributed to the conduct of this evaluation assignment and report preparation. Our acknowledgement is due to USAID for the financial support that made this evaluation possible. Creative Consulting and Development Works additionally would like to acknowledge the FHI 360 Umbrella Grants Management Project, Project Concern International, the Western Cape Network on Violence Against Women, the KZN Network on Violence Against Women and their staff whose efforts have been instrumental in making this evaluation process possible. We greatly acknowledge and thank the Networks, CIs, CEs, PAGs, PACs, sector and non-sector partners who gave of their time and insights during their participation in the evaluation. Our evaluation team consisted of Lindy Briginshaw (Director), Caroline Wills (Lead Evaluator), Shamillah Wilson (Technical Expert), Dee Smythe (Technical Expert), Susannah Clarke (Research Co￾ordinator), Stefanie Schulein (Researcher), Candice Mitchell (Researcher), Karin Nandoo (Researcher), Sanni Kujala (Researcher), Paul Dube (Researcher), Davina Jonathon (Office Manager), Nomonde Ngalo (Researcher) and Thabisile Nkosi (Researcher). The team was responsible for the evaluation’s administrative and logistical need, the collection of data through interviews, focus group discussions, the review of project-related documents and the synthesis and analysis of the data. We thank our team for their dedicated efforts during this assignment. ii Project Concern International (PCI) UGM End of Project Evaluation TABLE OF CONTENTS ACKNOWLEDGEMENTS ....................................................................................................................... i ACRONYM LIST ...................................................................................................................................... iii EXECUTIVE SUMMARY ............................................................................................ V I. INTRODUCTION .................................................................................................. 1 II. BACKGROUND ..................................................................................................... 3 III. METHODS .............................................................................................................. 4 IV. FINDINGS AND DISCUSSION ............................................................................ 7 KEY EVALUATION QUESTION 1: ........................................................................................ 7 KEY EVALUATION QUESTION 2: ...................................................................................... 21 KEY EVALUATION QUESTION 3: ...................................................................................... 27 KEY EVALUATION QUESTION 4: ...................................................................................... 30 V. CONCLUSIONS .................................................................................................. 33 VI. RECOMMENDATIONS ...................................................................................... 35 APPENDICES ............................................................................................................. 38 APPENDIX 1: COMPOSITION OF EVALUATION TEAM ................................................................ 38 APPENDIX 2: SAMPLING FRAMEWORK AND LIST OF SITES VISITED...................................... 43 APPENDIX 3: DATA COLLECTION TOOLS ....................................................................................... 46 APPENDIX 4: SCOPE OF WORK............................................................................................................. 59 APPENDIX 5: MER PIA DOCUMENTS ................................................................................................... 62 APPENDIX 6: PCI PIA PROGRAM TIMELINE ....................................................................................... 63 PERSONS CONTACTED ............................................................................................................................ 65 REFERENCES ................................................................................................................................................... 66 Project Concern International (PCI) UGM End of Project Evaluation iii ACRONYM LIST AIDS Acquired Immune Deficiency Syndrome APS Annual Program Statement BFL Brothers For Life CADRE Centre for AIDS Development, Research and Evaluation CBO Community-based Organization CC&DW Creative Consulting and Development Works CE Community Engager CE/O Community Engager Organizer CI Community Influencer DoJ Department of Justice FBO Faith-based Organization FY Fiscal Year GBV Gender-based Violence GCF Gender Challenge Fund HIV Human Immunodeficiency Virus HSRC Human Sciences Research Council KZN KwaZulu-Natal LGBT Lesbian, Gay, Bisexual and Transgendered M&E Monitoring and Evaluation MER Monitoring, Evaluation and Research MOU Memorandum of Understanding NGO Non-Governmental Organization NPO Nonprofit Organization PAC Prevention in Action Committee PAG Prevention in Action Group PCI Project Concern International PEPFAR President’s Emergency Plan for AIDS Relief PIA Prevention In Action PVO Private Voluntary Organization SAPS South African Police Service TA Technical Assistance UCT University of Cape Town UGM Umbrella Grants Management Project iv Project Concern International (PCI) UGM End of Project Evaluation UNAIDS United Nations Joint Program on AIDS UNICEF United Nations Children’s Fund USAID United States Agency for International Development VAW Violence Against Women VCA Visible Community Action VFZ Violence Free Zone WC Western Cape WHO World Health Organization Project Concern International (PCI) UGM End of Project Evaluation v EXECUTIVE SUMMARY The Umbrella Grants Management Project (UGM) is a five-year grants management program administered by FHI 360 with funding from the U.S. Agency for International Development (USAID). Through UGM, FHI 360 provides funding and technical assistance (TA) to USAID-selected non￾governmental organization (NGO) partners who provide HIV and AIDS services at local, provincial and national levels in South Africa. FHI 360-UGM seeks to promote high quality service delivery in alignment with the priorities and goals of the South African government’s HIV and AIDS framework. FHI 360-UGM provides specialized capacity-building and support services to build partners’ skills and competencies in program management, governance, human resource development, budgeting and finance, and monitoring, evaluation and reporting (MER). The FHI 360-UGM project objectives are to: 1. Provide grants to USAID/President’s Emergency Plan for AIDS Relief (PEPFAR) partners that ensure an adequate resource flow to foster scale-up of activities, 2. Implement effective monitoring, evaluation and reporting systems to assess and document activities and to; 3. Provide ongoing capacity-building to support and enhance scale-up of activities and sustainability of activities and partners. Since 2007, FHI 360-UGM has supported thirteen South African NGOs including Project Concern International (PCI). PCI received a grant of approximately US $15 million under this program from October 1, 2009 to September 30, 2012. PCI, in partnership with the Western Cape Network on Violence Against Women and the KwaZulu￾Natal Network on Violence Against Women, implemented a Prevention in Action (PIA) program in South Africa. PCI is an international private voluntary organization that is committed to preventing disease, improving community health and promoting sustainable development worldwide. The PIA program was implemented in the Khayelitsha sub-district in the Western Cape (WC) and the eThekwini District in KwaZulu-Natal (KZN), providing an opportunity for communities to prevent violence against women (VAW) and thereby reduce the transmission of HIV. Both of these communities experienced a high prevalence of violence and HIV. The PIA program provided individuals in the community with the opportunity to access training that provided them with knowledge and skills to implement actions that prevent violence against women. An evaluation commissioned by USAID was conducted from December 2012 through March 2013. This evaluation focused on the extent to which the PIA program has successfully addressed the prevention of VAW in the two targeted communities, coordinated efforts and built the capacity of all partners and produced sustainable results. The PIA program’s objective was to reduce the prevalence of physical and sexual violence against women who were most vulnerable to HIV infection in KZN and the WC. PCI followed a five-step social mobilization approach that informed the design and implementation of the program to enable it to reach its three program results, which were to: 1. mobilize civil society and public sector partners to include prevention of VAW in their activities; 2. facilitate community mobilization activities to build a community-led sustainable PIA movement; and 3. amplify community actions to prevent VAW using innovative communication channels, including mass and social media to foster an enabling environment for more actions that would eventually change the social norms that perpetuate VAW. vi Project Concern International (PCI) UGM End of Project Evaluation During the course of the four-year project, there were changes in its strategic direction. In 2009, a baseline study was undertaken. Findings from the baseline found that respondents knew that VAW was wrong. Despite this, VAW was found to be pervasive; therefore, the project decided to focus on changing another norm—inaction—and try to encourage communities to act to prevent VAW. Another strategic shift saw a move away from mass communication strategies to more targeted, intensive communication in smaller geographical areas. Creative Consulting & Development Works (CC&DW) evaluated the PIA program using a predominantly qualitative evaluation design, using secondary data for quantitative analysis. The purpose of this evaluation was to determine whether the objectives of the PCI program were achieved and to evaluate the key program outcomes and impacts on the beneficiaries. The following evaluation questions were addressed: 1. Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? 2. How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? 3. What is the sustained benefit of this project to the Networks and their community partners? 4. What were most significant changes brought about by PCI and their partners with regard to behavior or actions in preventing VAW? The evaluation included a desktop review and site visits to the head offices of PCI, the two Networks, their sector partners and other sector players to conduct key informant interviews with project staff. Site visits to the two targeted communities were arranged and focus groups were conducted with Community Engagers (CEs), Community Influencers (CIs), Prevention in Action Groups (PAGs) and Prevention in Action Committees (PACs); and key informant interviews were held with program beneficiaries. A total of 35 focus groups and 25 key informant interviews informed the evaluation findings. In summary, the PIA program engaged civil society and public sector partners, known as sector partners, to roll out social mobilization activities. Essentially, the PIA program encouraged action by the community to prevent VAW. This involved training over 100 individuals who were recruited from the 11 sector partners1. These trained individuals were known as CEs and their task was to:  Recruit opinion leaders in their communities, known as CIs.  Facilitate five small group training sessions with the CIs.  Provide the CIs with program materials and continuous support enabling them to form PAGs with five to ten of their friends. The aim of these groups was to implement actions that intervened to stop VAW in their communities2.  Establish Violence Free Zones (VFZs), which are geographical areas known to the community to be crime hotspots, that were branded and monitored by a PAC to ensure they became safer community spaces. 1 In December 2012 when the evaluation was conducted, the following sector partners formed part of the program: KZN: Wentworth Victim Friendly Centre, SAPS Hillcrest, SAPS KwaMakutha, Durban Coastal and Mental Health, Department of Justice, World Council for Religion and Peace, Wentworth Organization of Women. WC: DKTSA, JCC, Rape Crisis, Nonceba. 2 Project Concern International. Quarterly Report: July to September 2011. October 2011. Project Concern International (PCI) UGM End of Project Evaluation vii The training program consisted of five 90-minute training sessions and was focused on enabling CEs and CIs to activate the community to prevent VAW. The key findings for each of the four evaluation questions are summarized as follows. Evaluation Question 1: Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? Project partners were able to deliver a social mobilization strategy that addressed VAW in the targeted communities and reached the project targets set by PEPFAR. An important feature of a social mobilization strategy is for a program to have a clear agenda. In this program, the agenda was to prevent VAW. The evaluation found that the agenda became diluted as the project unfolded, to include gender-based violence (GBV) and then violence. Branding, through the use of a logo and slogan, initially reflected the program’s agenda but with the introduction of the VFZs near the end of the project, the slogan “working together to prevent violence against women” was removed. The community largely drove the move to a shifting focus. Men expressed that they wanted the agenda to recognize GBV, not just VAW. Whilst the involvement of men was a positive shift, the project could have done more to recognize other forms of GBV, for example targeting same-sex couples. When the focus shifted towards creating Violence Free Zones, the primary agenda became even more diluted. VFZs were well received by communities and were seen as a valuable asset to the community. In the future, they could provide a good entry point into communities; however, project partners need to ensure strong links are maintained to the primary agenda of preventing VAW. The PIA program delivered training to more than 100 CEs. Training session 1 and 2 was attended by 12,783 and 6,203 CIs went on to complete training session 3, 4 and 5. The training provided skills and knowledge to enable and encourage action by the community to prevent VAW. More than 3,000 CIs implemented small but effective actions to prevent VAW. Larger actions, known as visible community actions (VCAs), were also undertaken. VCAs included marches, rallies and the establishment of VFZs. During the life of the project 72 VCAs were implemented—VFZs emerged as a project innovation. Whilst the training was well received by CEs and CIs, the findings show that greater focus could have been placed on ensuring CEs, CIs and members of PAGs and PACS are focused on the root causes of VAW and are able to clearly articulate the link between HIV, VAW and gender inequality. PCI invested significant resources in strong MER systems throughout the life of the PIA program, and there is evidence to show that MER findings were used to monitor, guide, influence, and shape the social mobilization strategy. Whilst the project benefitted from the social mobilization approach and its process-driven nature, this approach affected the project management of the PIA program. Activities were mostly implemented in the latter part of the life of the project with very little time spent on properly closing out the project with stakeholders—his impacted on the sustainability of the project. The evaluation found that both Khayelitsha and eThekwini were good sites for the PIA program. Evaluation participants unanimously expressed a strong need for the PIA program beyond the geographical areas targeted and felt that many communities were in need of such an initiative. However, there were some practical disadvantages in targeting Khayelitsha. Firstly, the Western Cape Network on VAW did not have existing strong partnerships with organizations in Khayelitsha; secondly, the residents of Khayelitsha are generally quite experienced with research/pilot projects wanting to initiate new projects, which meant that they had expectations, based on previous experiences, with regards to receiving stipends and other allowances. These demands were made to project partners and provision was made. This contributed towards a dependency relationship where CEs and CIs looked to the viii Project Concern International (PCI) UGM End of Project Evaluation project partners for materials, stipends, meals, travel money, etc. and detracted from ensuring an independent and sustainable continuation of the PIA program. The evaluation found that greater time and attention could have been given to more strictly selecting CIs, ensuring that those chosen had recognizable influence in the community. This could have prevented a drop in the number of CIs that finally completed the program. Furthermore, opportunities, particularly with adolescents, were highlighted where the PIA program could be adapted for use in secondary schools. Evaluation Question 2: How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? The social mobilization approach comprised five phases: Analysis, Engagement, Alignment, Implementation and Evaluation. During each phase a set of activities were undertaken to test ideas and agree on ideas and activities before the next phase could be entered. Whilst the social mobilization approach followed by the PIA program was very thorough, the evaluation team found that there could have been much greater investment, particularly in NGO sector partners, to enable their continued implementation the PIA program independently. This would have ensured that there was PIA program infrastructure (office infrastructure with access to phones, internet and PIA branding) in the community, as well as project management and social mobilization skills and knowledge in the community that could encourage CEs and CIs to continue to promote a ‘prevention of VAW’ agenda. Other groups, such as public sector partners, were also important sector partners. Involvement of other partners, such as the business community and media could have contributed towards the success of the PIA program. Communication was a core strategy that was integral to this initiative. Initially this was accomplished through a mass communications approach; however, it shifted more appropriately to focus on the development of toolkits and DVDs, which were deemed to be more appropriate and relevant to supporting social mobilization. The five training sessions increased the communities’ awareness to VAW and promoted action, as evidenced by the more than 3,000 active CIs. However, focus groups did not reflect that the training had assisted CEs and CIs to implement activities facilitating a deeper process for communities to construct alternatives to VAW as a result of their increased consciousness, nor did it encourage them to practice these new behaviors over time and in a supportive environment with friends, colleagues and institutions supporting and reinforcing these efforts. While more than 3,000 CIs reported actions, it appears that the activism was not even, with some communities being more active than others. The main reason for this imbalance is that the training was offered to CEs and CIs in different parts of the community and resulted in activism in those respective areas. Ideally, these actions would have together contributed towards creating a critical mass and ‘tipping point’ that would have ultimately sustained change. In the future, should a strong link continue to be maintained between VFZs and VAW, the VFZs could possibly be a mechanism to facilitate sustained action; it was too early for the evaluation to assess this. Finally, the evaluation found that more could have been done to address formal processes that could prevent VAW, such as engaging in secondary and tertiary prevention efforts by engaging with partners who work on relevant legislation and policies. Project Concern International (PCI) UGM End of Project Evaluation ix Evaluation Question 3: What is the sustained benefit of this project to the Networks and their community partners? Evidence shows that the PIA program raised public awareness of the issue, promoted a culture of taking some form of action to counter VAW, increased knowledge among women of their rights and recourse mechanisms, made known the availability and location of services, and created safer public spaces through VFZs. PCI and the Networks were provided with the opportunity to pilot the PIA program in a context where HIV prevalence is high and violence is rife. At the outset of the PIA program, both Networks were extremely under-resourced and quite inexperienced in implementing large-scale activities. As a result of the PIA program, they gained program management experience and invested in developing their internal organizational systems. PCI provided strong technical, operational and management expertise. Sector partners supported the focus of the PIA program and the investment made in their staff via CE training. Public sector partners appeared to benefit more from the PIA program when compared to the other sector partners who were community-based organizations (CBOs), NGOs and faith-based organizations (FBOs). This was because the public sector partners interviewed had a very clear mandate to prevent violence and VAW in their communities; the PIA program offered them a vehicle to achieve this mandate, in a way that they would not have achieved independently. Finally, the targeted communities benefitted from the program; however, not enough was done to ensure the PIA program was sufficiently sustained. In Khayelitsha, where the program has ended, ongoing activities are ad hoc and people involved in the project expressed the need for ongoing support. Evaluation Question 4: What were most significant changes brought about by PCI and their partners with regard to behavior/actions in preventing VAW? At the individual level, all those consulted in this evaluation reported that the project contributed towards positive change within them. Accounts included an increase in knowledge, self-esteem, self￾worth and independence. At the community level, there was heightened awareness and visibility of VAW and of recourse and actions that could be taken. Community members reported a change in their behavior; specifically, they felt more empowered to take action to prevent VAW. The PIA program gave them confidence to act appropriately against VAW. Some CIs reported that they had been perpetrators of VAW—the PIA program gave them insight into their behavior and supported them to change. PAGs and PACs were able to engage the community on VAW and also reported that they learned skills in how to arrange community activities. As a result of the project’s discontinuation in several locations, however, it seems that those particular shifts will be difficult to maintain. The involvement of a wider network of sector partners also impacted positively on organizations and groups in the community, as they felt that there was a wider network of actors working together to address the issue. An unintended consequence was that community members felt more empowered to tackle everyday issues that were affecting the community, beyond just the issue of VAW. In conclusion, the critique of the PIA program rests on two levels: a project management level and a theoretical level. x Project Concern International (PCI) UGM End of Project Evaluation From a project management perspective, future initiatives need to offer a balance between innovation and implementation, to allow more time for project implementation, and also to activate communities to ensure sustained change. The evaluation team believes that the PIA program has less features of a social mobilization model and offers more project-based features. It is therefore worth considering what entity is best equipped to implement activities relating to the PIA program and whether it could be managed by an NGO that has project management experience, is familiar with the norms, customs and practices of their community, and has technical GBV/HIV knowledge. In future, clear partner roles, reporting processes, project implementation plans and modest project budgets will assist in the smooth implementation of the PIA program. Greater investment should be made in CEs and CIs through mentorship, refresher training and debriefing in order to ensure their skills are retained in the PIA program. On a theoretical level, greater focus should be given to GBV, which is not limited to the involvement of men. In reviewing the overall social mobilization strategy to reduce VAW, the evaluators also concluded that:  There was an absence of strategies to reduce HIV infections in the long-term.  The current strategy’s initial entry point was to look at physical and sexual violence, which could have been the ideal link to the intersection between VAW and HIV; however, restricting the focus to physical and sexual violence excludes verbal, mental and emotional violence, sexism and hate speech.  The PIA model does not look at high-level advocacy, accountability and legislation as a critical entry point for creating more sustainable change. The most significant change identified was a shift in awareness of VAW. Notable changes in behavior were self-confidence in CEs and CIs and, for those who had been perpetrators of violence, adopting positive behaviors where they avoided alcohol and drugs and realized that VAW was wrong. At the community level, the issue of VAW has become more visible and actors and actions to address it have been promoted. Some sector partners, notably public sector partners, indicated an expansion of focus and an increase in reach and capacity of their staff. Evidence is needed about the impact and effect of the community communication initiatives. A greater balance was needed between project innovation and project implementation to ensure that sufficient time is given to activate communities to ensure sustained change in behavior to prevent VAW. Based on the findings and conclusions drawn, we recommend that baseline research and community assessments continue to be firmly rooted in the cultural context in order to gain a clear understanding of the norms and cultures with regards to gender and explore new norms with communities to create alternative, positive realities. Community assessments should identify existing services in the community and assess the viability of successfully implementing the PIA program in order that sustainability infrastructure is identified from the outset. We further recommend that the PIA program adopts an agenda that is concrete, clear and transferrable over time and that strong links to the prevention on VAW are maintained, even as the constituency and reach of the program expands. Greater consciousness-raising opportunities should be used to ensure that communities share a deep and common understanding of VAW and its negative consequences. Project Concern International (PCI) UGM End of Project Evaluation xi The PIA program should expand its stakeholder base and reach out to include other actors and leaders in communities. Clear engagement strategies and focus as to how these stakeholders can contribute to the broader agenda and outcomes are necessary. Greater consideration should be given towards secondary and tertiary prevention measures in order to implement a more well-rounded prevention strategy. Finally, we recommend that elements of the program design are reconsidered to ensure the PIA program is able to be implemented effectively and efficiently. These elements include the following:  Who is best placed to implement the project?  How can CEs and CIs be retained? How can they be supported in their work to ensure high quality, sustainable results are achieved?  How can the PIA program be expanded to target other groups, such as adolescents?  How can the PIA program include minority and potentially vulnerable groups, such as people of different sexual orientations?  What budget is realistic to sustain the PIA program to a point where the program has ‘taken root’ in the community?  Is the brand sufficiently inclusive and do people correctly understand its meaning? Project Concern International (PCI) UGM End of Project Evaluation 1 I. INTRODUCTION PURPOSE OF EVALUATION AND GUIDING QUESTIONS The U.S. Agency for International Development (USAID) called for an external evaluation of Project Concern International (PCI)’s ‘Social Mobilization to End Gender-based Violence’ Program, funded under the FHI 360 Umbrella Grants Management Project (UGM). The program was implemented in partnership with the KwaZulu-Natal (KZN) and Western Cape (WC) Networks on Violence Against Women (VAW) over a four-year period, from October 2008 to September 2012. USAID awarded approximately US $15 million3 towards the full implementation of the program. The purpose of this evaluation was to determine whether the objectives of the PCI program were achieved and to evaluate the key program outcomes and impacts on the beneficiaries. The overall objectives of the program evaluation were as follows: 1. To determine whether the program objectives were achieved 2. To evaluate the extent to which the key program outcomes have been achieved (both intended and unintended outcomes) 3. To determine the key pragmatic enablers and barriers to the success of the program in reducing the prevalence of physical and sexual VAW who are most vulnerable to HIV infection in the two communities studied 4. To determine the sustainability and potential for continuity of the work done by networks under the program The following evaluation questions were addressed: 1. Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? 2. How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? 3. What is the sustained benefit of this project to the Networks and their community partners? 4. What were most significant changes brought about by PCI and their partners with regard to behavior/actions in preventing VAW? KEY AUDIENCES The primary audiences for this evaluation are USAID/President’s Emergency Plan for AIDS Relief (PEPFAR) and FHI 360-UGM. The secondary audiences are: PCI (for whom recommendations may inform future programs), their two network partners, provincial sector partners, and all of those involved in implementing the Prevention in Action (PIA) program. 3 PCI received the following annual budgets: FY09: US$ 4.5 million; FY10: US$ 3 million.1; FY11: US$ 3.1 million and FY12: US$4 million (Project Concern International, Final Report of Project Implemented with USAID/PEPFAR Funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012, 2012). 2 Project Concern International (PCI) UGM End of Project Evaluation KEY COMPONENTS OF THE REPORT The report is divided into six sections. The Introduction section introduces the report, and outlines the purpose, evaluation objectives, evaluation questions and the audience of the evaluation report. The Background section provides an overview of the FHI 360-UGM project and PCI. It provides a context for the program and provides greater insight into the communities targeted. The PIA program is then introduced to provide an overview of the activities undertaken and process followed to implement the social mobilization model. This section also explains the shifts in the program during the life of the project and concludes with the life of project indicators. This is followed by an outline of the Methodology used in the evaluation, and indicates how the evaluation was implemented, its limitations and ethical considerations. The Findings and Discussion are then presented, according to the four evaluation questions, and Conclusions are drawn. Finally, the Recommendations are presented. Project Concern International (PCI) UGM End of Project Evaluation 3 II. BACKGROUND THE UGM PROJECT UGM is a five-year grants management program administered by FHI 360 with funding from USAID/PEPFAR. Through the UGM in South Africa, FHI 360 provides funding and technical assistance to USAID-selected NGO partners who provide HIV and AIDS services at local, provincial and national levels in South Africa. FHI 360-UGM seeks to promote high quality service delivery in alignment with the priorities and goals of the South African government’s HIV and AIDS strategic plan. PCI PCI was founded in the United States in 1961 and works in vulnerable communities to improve health and create long-term change by helping people help themselves4. PCI’s primary interventions are in the areas of maternal and child health; disease prevention and mitigation, with an emphasis on HIV and AIDS prevention, treatment, care and support; food aid and food security, including livelihoods and agribusiness; humanitarian assistance; water and sanitation; and capacity building of local organizations and networks. PCI is a USAID-registered private voluntary organization (PVO) with decades of health and development programming experience in over 20 countries5. Vision PCI envisions a world where abundant resources are shared, communities are able to provide for the health and well-being of their members, and children and families can achieve lives of hope, good health and self-sufficiency. Mission PCI's mission is to prevent disease, improve community health and promote sustainable development worldwide. PCI South Africa This was the first program to be implemented by PCI in South Africa. The office was based in Cape Town and staffed by a small senior staff component of five people responsible for directing the program and its operations. The team received support from the PCI Africa Regional Office and International Office. SCOPE OF THE SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE PROGRAM Context South Africa has one of the highest rates of intimate partner violence in the world.6 Closely linked to 4 Accessed from http://www.pciglobal.org/en/pci/pcis-mission-disease-prevention-community-health-sustainable-development, January 2013. 5 Project Concern International South Africa. APS 674-08-003 Technical Narrative. May 2008. 6 Thaler, Kai. “Norms about intimate partner violence among urban South Africans: a quantitative and qualitative vignette analysis,” CSSR Working Paper No. 302, January 2012. 4 Project Concern International (PCI) UGM End of Project Evaluation the high levels of VAW, South Africa has a very high HIV prevalence. In response, both the UNAIDS Agenda for Women, Girls, Gender Equality and HIV, as well as the UNAIDS Strategy 2011-2015 recognize the importance of addressing VAW and girls as an integral part of an effective HIV response.7 Gender-based violence (GBV) is violence that is directed against a person on the basis of gender or sexual identities and is a violation of human rights. It includes acts that inflict physical, mental or sexual harm and suffering. Men, women, boys and girls can be victims, but it is primarily women and girls who are victims, hence the need to focus on VAW. VAW is defined as physical, sexual or mental harm occurring in public or in private life and includes sexual, physical or emotional abuse by an intimate partner; physical or sexual abuse by family members or others; and sexual harassment and abuse by authority figures such as teachers, police officers or employers. The links between VAW and HIV are undeniable. VAW arises from and perpetuates gender inequality within societies. It increases women’s risk of HIV and can also be a result of her being HIV-positive. HIV prevention programs must therefore address the interrelated problems of gender inequality and VAW in order to be effective—not only at preventing heterosexual transmission of HIV, but also at interrupting all interpersonal HIV transmission routes. Despite the advancements in recent years towards empowering women, gender equality and GBV (and in particular VAW) remains a huge challenge in South Africa. There is no doubt that there is a critical need for responses that bring together all actors at all levels in an integrated, multi-sectoral approach requiring greater coordination amongst the various stakeholders; civil society organizations, government departments, researchers and communities. In terms of responses to VAW in South Africa, limited, sustained, active networking or sharing of experiences and lessons between organizations around the country characterizes the sector. Many projects and initiatives remain undocumented and rigorous evaluations of the impact of projects is rarely conducted. In addition, fragmentation on the basis of race, geography, education, sexual orientation and political belief (to name but a few), as well as competition for scarce resources, hamper organizations’ ability to work together8. Strategies to address VAW have also varied over time. Traditionally, they have focused on treatment, care and supporting the needs of women, with prevention primarily being in the area of advocacy to create an enabling environment for the cessation of all forms of VAW. Over the last ten years in South Africa, prevention has also included working with men to end VAW, resulting in greater visibility of a ‘men’s movement to end VAW.’ The focus on prevention is premised on the need to prevent violence before it starts, including primary prevention, that is, changing social norms and behaviors that promote VAW. Communities Targeted by the Social Mobilization to End Gender-based Violence Program Within this context, PCI and its two partners in South Africa, the WC and KZN Networks on VAW, implemented a four-year program, Social Mobilization to End Gender-based Violence: An Essential HIV Prevention Strategy for South Africa. 7 UNAIDS. Agenda for accelerated country action for women, girls, gender equality and HIV. Geneva, 2010. 8 Vetten, Lisa. Addressing domestic violence in South Africa: Reflections on strategy and practice. Paper presented at the Division for the Advancement of Women, Vienna. May 2005. Accessed from http://www.un.org/womenwatch/daw/egm/vaw-gp￾2005 /docs/experts/vetten.vaw.pdf, January 2013. Project Concern International (PCI) UGM End of Project Evaluation 5 PCI’s program focused on the district of eThekwini in KZN and the sub-district of Khayelitsha in the WC. This evaluation specifically focused on Khayelitsha, as well as two of the communities where the program was implemented in eThekwini, namely Wentworth and Inanda. eThekwini9 is a high HIV prevalence district with 38% of pregnant women reported to be HIV-positive in 201110. Wentworth is a predominantly coloured11 suburb in the south of Durban. The area comprises families forcibly resettled under apartheid and has a population of 30,000. The community faces high rates of alcohol abuse, domestic violence, teenage pregnancy and HIV and AIDS12. More than 30% of men between the ages of 15 and 65 are unemployed13. Inanda is an informal settlement area demonstrating similar patterns of GBV. The area is reported as having very high incidences of crimes against women and children14, with high levels of sexual abuse, particularly in schools15. Khayelitsha is a large township with approximately 500,000 inhabitants. Located on the outskirts of Cape Town, it is a sub-district of the city of Cape Town. A vast majority (96.8%) of the population are Africans of Xhosa descent. Unemployment stands at 51%, with 80% of those employed earning less than R1,600 a month. The majority (57.7%) of the population lives in shacks in informal settlements. Khayelitsha has one of the highest burdens of HIV in South Africa, with an estimated HIV prevalence of 16%16, and it is a priority sub-district of the National Department of Health17. Khayelitsha is notorious for its high rate of violent crimes (especially murder), which stands at over two and half times the South African average18. All three communities, Wentworth, Inanda and Khayelitsha show skewed patterns of development that are reflected in the high rates of poverty, unemployment and violent crime. Overview of the Network Partners Crucial to the implementation of PCI’s Social Mobilization to End Gender-based Violence program was the partnership between PCI and the KZN and WC Networks on VAW. The KZN Network on VAW was founded in 1996 and is registered as a nonprofit organization (NPO). The Network aims to advocate for the prevention and eradication of all forms of violence against women and promote gender equality through lobbying, advocacy, capacity-building and raising public awareness. It also aims to address the issue of HIV and AIDS through its efforts to prevent sexual 9 eThekwini Metropolitan Municipality is a metropolitan municipality created in 2000 that includes the city of Durban, South Africa and surrounding towns. Accessed from http://en.wikipedia.org/wiki/EThekwini, February 2012. 10 National Department of Health. The National Antenatal Sentinel HIV and Syphilis Prevalence Survey, 2011. Pretoria, 2012. 11 “Coloured” is one of the four main racial classifications adopted in South Africa under apartheid. A legacy of apartheid is the perpetuation of racialized suburbs more than two decades after spatial segregation was legally dismantled. 12 Anderson, Bronwynne. Coloured boys in trouble: an ethnographic investigation into the construction of Coloured working￾class masculinities in a high school in Wentworth. Thesis (PhD), University of KwaZulu-Natal, Durban. 2009. 13 Project Concern International. Monitoring, evaluation and reporting plan - Social mobilization to end gender-based violence: An essential HIV prevention strategy for South Africa, October 1, 2008 - September 30, 2012. March 2012. 14 Injobo ne Bandla, CSIR, IPT and IDASA. Kwa Mashu and Inanda crime program prevention program. Report of first phase. (Undated). 15 Nkani, Francis N. Factors that explain gender based violence amongst secondary school learners in the Inanda area. Thesis (Masters), University of KwaZulu Natal, Durban. 2006. 16 Garone, D.B., Hilderbrand K., Boulle A., Coetzee D., Goemaere E., Van Cutsem, G., and Besada, D. Khayelitsha 2001-2011: 10 years of primary health care HIV and TB program. Southern African Journal of HIV Medicine, 12 (4). 17 Khayelitsha sub-district within the Cape Town metropole was identified in the Western Cape, due to high levels of deprivation. Accessed from http://www.childrencount.ci.org.za/nsp-priority_districts.php, January 2013. 18 Nleya, Ndodana and Thompson, Lisa. Survey methodology in Violence prone Khayelitsha, IDS Bulletin 40 (3). 6 Project Concern International (PCI) UGM End of Project Evaluation violence. The Network represents a wide array of members, from urban, rural and traditional communities and encourages the utilization of local resources to achieve its aims19. The WC Network on VAW is a coalition of organizations and individuals working in the VAW sector. The Network is primarily active in the Western Cape province but facilitates participation and coordination of programs from provincial through to national level. The Network currently has over 500 members, consisting of counselors, trainers, researchers, legal advocates, shelter workers, government officials, parliamentarians and concerned women and men from the local to international level20. Social Mobilization to End Gender-based Violence Program The Social Mobilization to End Gender-based Violence program was initially envisaged with the goal of reducing HIV transmission by changing social norms related to sexual violence and other forms of GBV against women. The objective was to reduce the prevalence of physical and sexual violence against women in target areas21. This was to be achieved through a large-scale social mobilization program that would: mobilize public and private sector partners to combat GBV; create an enabling environment to changing social norms related to GBV; and develop and support the implementation of multi-sectoral activities, which will achieve and maintain significant reductions in GBV22. Activities were to include mass media communications to address social norms perpetuating GBV, as well as harnessing the province-wide reach of the KZN and WC Networks on VAW. A range of sector partners was also to be engaged to support the achievement of project goals. The role of PCI was to manage sub-awards, provide capacity-building and technical support to the two Networks, and conduct a communications campaign to amplify the concept of VAW prevention23. Table 1 below depicts the results and social mobilization approach that the project aimed to achieve as per the 2008 Annual Program Statement (APS). 19 Accessed from http://www.womensnet.org.za/kwazulu-natal-network-violence-against-women, January 2013. 20 Accessed from http://www.wcnetwork.org.za/, January 2013. 21 Project Concern International South Africa. APS 674-08-003 Technical Narrative. May 2008. In PCIs 2010 Strategic Directions document, the program objective is stated as follows: To reduce the prevalence of physical and sexual violence against women most vulnerable to HIV infection in KwaZulu-Natal (KZN) and the Western Cape (WC) Provinces. 22 Project Concern International South Africa. APS 674-08-003 Technical Narrative. May 2008. 23 Project Concern International. Final Report of Project Implemented with USAID/PEPFAR Funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012. 2012. Project Concern International (PCI) UGM End of Project Evaluation 7 Table 1: Results and Social Mobilization Approach Results and Social Mobilization Approach RESULT 1: Key public and private sector partners mobilized to combat sexual and other gender-based violence against women RESULT 2: Improved enabling environment for changing social norms related to sexual and other gender-based violence against women. RESULT 3: Program to reduce sexual and other gender-based violence against women implemented and sustained. Analysis Engagement Alignment Implementation Evaluation Activities Facilitated meeting with core group of stakeholders Facilitated meetings and planning with each sector Facilitated inter￾sectoral meetings Amplification communications Sector-level mobilization Outcomes Action Plan: a. Confirm target social norm. b. Consensus re: Social Mobilization Objectives c. Agreement on Priority Sectors d. Clarity on next steps e. Form steering group Sector Level Action Plans: a. Expression of commitment b. Activity plan and budget c. Sector support plan Inter-sectoral Alignment: a. Joint statement of mission b. Joint activity plans c. Accountabil ity structures Social Change: a. Healthier actions and behaviors b. New public perception of norm Insights: a. What has changed? b. What has worked? c. What did not work? Programmatic Shifts During the first year of the program, the focus was on developing the social mobilization strategy through intensive consultation with stakeholders and a qualitative research study with community members. Findings from a baseline study conducted on norms related to VAW led to an awareness￾building campaign to provoke debate around the question “What is keeping VAW alive?” The program also formed a partnership with the National Stop Gender Violence Helpline, promoting its use, funding weekend hours and providing a regularly updated database of service providers for referrals in KZN/WC24. 24 Project Concern International. Quarterly Report: July to September 2011. October 2011. 8 Project Concern International (PCI) UGM End of Project Evaluation Reduction in available funding early on in the life of the project resulted in a shift away from the initial mass media component of the program25. In order to ensure a more targeted geographical reach, the focus also shifted from the entire KZN and WC provinces to more focused interventions in eThekwini (KZN) and Khayelitsha (WC). In addition to being particularly well suited to the program’s interventions due to high HIV and VAW statistics, the fact that these regions are densely populated also facilitated the process of reaching PEPFAR targets within the set timeframes. During Year 2, the program strategy was also revised due to findings from the baseline quantitative research study conducted in Year 1. These findings revealed that people know that the norms legitimizing VAW are wrong, yet do not know how to act to prevent VAW. This shifted the focus to fostering the normalization of positive actions and behaviors to address VAW. Using a participative approach called Action Media, the program developed a slogan for the movement called “Prevention in Action (PIA) – Working Together to Prevent Violence Against Women.” In Year 2 the program goal therefore shifted to changing the norm of ‘inaction in response to VAW’ to one of ‘action in response to VAW’ through:26  mobilizing civil society and public sector partners to include prevention of VAW in their activities;  facilitating community mobilization activities to build a community led sustainable Prevention In Action (PIA) movement; and  amplifying community actions to prevent VAW using innovative communication channels, including mass and social media to foster an enabling environment for more actions that would eventually change the social norms that perpetuate VAW. In Year 3 and Year 4, the program engaged civil society and public sector partners, known as sector partners to roll out social mobilization activities. Essentially, the PIA project encouraged action by the community to prevent VAW. This involved training over 100 individuals who were recruited from the 11 sector partners27. These trained individuals were known as Community Engagers (CEs) and their task was to:  Recruit opinion leaders in their communities, known as Community Influencers (CIs).  Facilitate five small group training sessions with the CIs.  Provide the CIs with program materials and continuous support enabling them to form PIA Groups (PAGs) with five to ten of their friends. The aim of these groups was to implement actions that intervened to stop VAW in their communities28. The training program consisted of five 90-minute training sessions as follows29:  Session 1: Orientation to PIA Movement and VAW Objectives: - To increase CIs knowledge and understanding about VAW. 25 In addition, mass media activities became increasingly discouraged by USAID, as these activities did not necessarily lead to behavior change. 26 Project Concern International. Monitoring, evaluation and reporting plan - Social mobilization to end gender-based violence: An essential HIV prevention strategy for South Africa, October 1, 2008 - September 30, 2012. March 2012 27 In December 2012 when the evaluation was conducted, the following sector partners formed part of the program: KZN: Wentworth Victim Friendly Centre, South African Police Service (SAPS) Hillcrest, SAPS KwaMakutha, Durban Coastal and Mental Health, Department of Justice (DoJ), World Council for Religion and Peace, Wentworth Organization of Women WC: DKTSA, JCC, Rape Crisis, Nonceba 28 Project Concern International. Quarterly Report: July to September 2011. October 2011 29 Guide for CE to use during 90-minute session with Community Influencer Groups (Sessions 1-5). Project Concern International (PCI) UGM End of Project Evaluation 9 - To enable the CIs to begin discussion of VAW with people within their circle of influence. - To enable the CIs to begin to identify a pool of people in their community, who will be recruited after Session 2 to join the PIA movement that acts to stop VAW.  Session 2: Taking Action to Prevent VAW Objectives: - To discuss feedback from the CIs attempts to form PAGs as requested at the end of Session 1. - To distribute PIA toolkits to CIs and ensure they understand how to use them. - To distribute PIA Stories booklets to the CIs for them to review it and watch the DVD together and brainstorm how to use it to effectively help their PAGs act to stop VAW. - To ensure the CIs understand how to report the formation of PAGs and actions to stop VAW to the network and work out how to keep connected to each other and the PIA movement.  Session 3: Taking Group Action to Prevent VAW Objectives: - To reconnect with the CIs and discuss the challenges they are facing in forming PAGs and implementing actions to stop VAW in their communities. - To watch the DVD of action stories and distribute PIA Stories, Volume 2 booklets to the CIs and brainstorm on how to use it to effectively help their PAGs act to stop VAW. - To ensure the CIs understand how to report the formation of PAGs and actions to stop VAW to the network and are able to keep connected to each other and the PIA movement.  Session 4: Empowering the CI to Form Prevention in Action Groups (PAGs) Objectives: - To increase the CIs ability and motivation to form PAGs in their community. - To increase the CIs understanding of what constitutes an action to stop VAW. - To ensure the CIs understand how to report having formed a PAG or actions to stop VAW and increase their motivation to do so. - To ensure the CIs understand the PIA movement manifesto.  Session 5: Sustaining the Prevention in Action Movement Objectives: - To continue forming/working with their PAG and implementing actions to stop VAW and for the CIs to report this to their CE or the Network. - To join the Prevention in Action Committee (PAC) formed by their CE in their community and to participate in the PAC actions to create a violence free zone (VFZ). The social mobilization model evolved throughout the life of the project as lessons were learned. Following the first year of implementation of the revised strategy, the program conducted operational research with movement members to gain an understanding of the types of actions the CIs and PAGs were taking. The action stories gathered in 2011 became the centerpiece of the movement and booklets 10 Project Concern International (PCI) UGM End of Project Evaluation with DVDs were developed to distribute to CIs30. Another programmatic shift occurred at the beginning of 2012. To strengthen community ownership of the PIA movement, CEs became Community Engager Organizers and formed PACs. These committees were made up of CIs and other community stakeholders and became responsible for creating and maintaining VFZs. VFZs were geographical areas identified and demarcated to raise community awareness at crime hotspots. Through VFZs, the project gained strong support from those living in the neighborhood. As outlined above, the strategic direction of the project shifted throughout its lifetime. See Table 2 below for key stages of the project and how the strategic direction changed over time31. Table 2: Overview of Programmatic Shifts Overview of Programmatic Shifts Timeframe Phase April-June 2009 Analysis Phase: Conducting of qualitative research and stakeholder meetings to develop strategy July-Sept. 2009 Engagement Phase: Launching of What's Keeping Violence Against Women Alive? campaign during Women's Month (August) Oct.-Dec. 2009 Implementation Phase: Changing Faces Campaign conducted by Ogilvy Jan.-March 2010 2010 Strategic Direction: Due to findings from quantitative survey, shifting to ‘fostering the normalization of positive actions and behaviors to address VAW’ and PIA campaign launched Jan.-March 2011 2011 Strategic Direction: Shifting focus to ‘Actions’ Jan.-March 2012 2012 Strategic Direction: Focusing on CE/Os forming PACs and setting up VFZs The PIA model which evolved during the process of project implementation is outlined in Figure 1 below. The model depicts a structure that is managed by an implementation team. The model has four key pillars that reflect essential elements and activities needed to successfully implement the project namely: (1) sector partnerships; (2) training and capacity-building/development; (3) communication activities and resources; and (4) research, monitoring and evaluation. As reflected in the model, all of these elements need to be in place to establish a PAC that will support a VFZ. 30 Project Concern International. Final Report of Project Implemented with USAID/PEPFAR Funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012. 2012. 31 Project Concern International. PIA Program Timeline. October 2012. Project Concern International (PCI) UGM End of Project Evaluation 1 Figure 1: Prevention in Action Intervention Model32 32 Project Concern International. Final Report of Project Implemented with USAID/PEPFAR Funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012. 2012. Project Concern International (PCI) UGM End of Project Evaluation 1 Figure 2 outlines the PIA program implementation structure which was in place when this end-of-project evaluation was conducted in November and December 2012. Figure 2: 2012 - Prevention in Action Implementation Structure In 2012, the KZN Network on VAW received additional financial support from the Gender Challenge Fund (GCF) to the amount of US $300,000—his fund period overlapped with the PIA program implementation period. The GCF supported a more intense intervention of the project in the community of Wentworth through the creation of a Brothers For Life (BFL) chapter and the execution of a ‘Taxi-Talk Campaign’. The GCF also supported the launch of the VFZs in Wentworth and other areas of the eThekwini district and an inter-sectoral training workshop on VAW in eThekwini conducted jointly with the Department of Justice (DoJ)33. Activities by the KZN Network on VAW continue under a separate grant from the UGM for 2013. 33 Project Concern International. Quarterly Report: July to September 2012. October 2012. Project Concern International KZN Network on Violence Against Women eThekwini Sector Partners Wentworth Victim Friendly Centre, SAPS Hillcrest, SAPS KwaMakutha, Durban Coastal and Mental Health, DoJ, World Council for Religion and Peace, Wentworth Organization of Women Community Engagers (CE) Community Influencers (CI) Prevention in Action Groups (PAG) /Prevention in Action Committees (PAC) WC Network on Violence Against Women Khayelitsha Sector Partners DKTSA, JCC, Rape Crisis, Nonceba Community Engagers (CE) Community Influencers (CI) Prevention in Action Groups (PAG) /Prevention in Action Committees (PAC) 2 Project Concern International (PCI) UGM End of Project Evaluation Life of Project Indicators A number of quantitative targets were set for the PIA Program by PEPFAR and PCI. The PEPFAR gender/prevention indicator that the program reported against was: Individuals reached with interventions focused on gender-based violence and coercion. The result to which the indicator responds is the number of adults and children reached by an intervention or service that explicitly addresses gender-based violence and coercion related to HIV and AIDS34. In this program, the indicator ‘number of people reached’ is in real terms the ‘number of people trained.’ As indicated above, CIs were trained by CEs over five training sessions. These figures formed the basis for the PEPFAR indicator. By fiscal year (FY) 1135, the PIA program needed to reach 12,000 CIs with training Sessions 1 and 2. In FY 1236 the training was continued to include training Sessions 3, 4 and 5; therefore, a target was set at 6,000 CIs (which is a subset of the 12,000 CIs) to be reached by training Sessions 3, 4 and 5. In line with the social mobilization approach, PCI aimed to deepen the response to VAW prevention at the individual, family and community level. The emphasis was placed on increasing and promoting actions to stop VAW based on the premise that a sufficient number of actions at the community level would lead to a ‘tipping point’ where VAW would be consistently addressed and where the prevalence of VAW declined over time as primary prevention became the norm37. Activities therefore focused on ensuring that 50% of those CIs who completed Session 5 were active to meet one of the key internal project targets. An active CI has either reported forming a PAG, reported an action to stop VAW in their community, or who has joined a PAC38. Another internal target set by the PIA program in FY 12 was to achieve a total of 60 visible community actions (VCAs). A VCA involved the setting up of VFZs and organizing community marches and other events. All of the above indicators were measured using Salesforce39 and verified to ensure they were legitimate. As mentioned earlier in this report, an early component of the PIA program was a large mass communications program. These program activities were also reported under the ‘number of individuals reached’ indicator. In FY 0940, the individuals reached were participants at Women’s Month events in August 2009 and in FY 1041 it was individuals reached through mass media. These specifically related to 16-34 year-olds who heard a radio campaign in KZN/WC from October–December 2009. With regard to monitoring, Salesforce42—a cloud-based monitoring system—was identified, adopted and adapted by PCI for use by the program to allow project partners to track the ongoing activities of the program. Network staff were trained to input data into Salesforce including action narratives from CEs 34 Project Concern International. Monitoring, evaluation and reporting plan - Social mobilization to end gender-based violence: An essential HIV prevention strategy for South Africa, October 1, 2008 - September 30, 2012. March 2012. In this case no. of individuals ‘reached’ equated to number of individuals ‘trained.’ 35 FY 11refers to October 2010 – September 2011. 36 FY 12 refers to October 2011 – September 2012. 37 Project Concern International. Monitoring, evaluation and reporting plan - Social mobilization to end gender-based violence: An essential HIV prevention strategy for South Africa, October 1, 2008 - September 30, 2012. March 2012. 38 Project Concern International, Quarterly Report: July to September 2012. October 2012. 39 Salesforce is PCI South Africa’s cloud based monitoring and evaluation (M&E) platform. 40 FY 09 refers to October 2008 – September 2009. 41 FY 10 refers to October 2009 – September 2010. 42 For more information on Salesforce, see www.salesforce.com. Project Concern International (PCI) UGM End of Project Evaluation 3 and CIs. Initially, Salesforce was just regarded as a management information system database, but as staff became more proficient in using it, they realized that the system could add more value. A PCI interviewee reported that they found Salesforce to be “innovative, progressive and allowed them to be creative” with regards to the data it could collect. In Sessions 3 and 4, CEs and CIs were taught that when they called to report an action, that the action would be recorded in Salesforce. Strict criteria about what constitutes the basis of each indicator were in place and monitored, to ensure that reported activities (training, actions and VCAs) by project partners and their beneficiaries were legitimate. This system was particularly crucial towards tracking the project’s quantitative indicators. Salesforce also enabled PCI to track progress towards targets and identify potential problems relating to the implementation of activities early on, in order to rectify them. 4 Project Concern International (PCI) UGM End of Project Evaluation III. METHODS EVALUATION DESIGN AND METHODOLOGY Creative Consulting & Development Works (CC&DW) evaluated the PIA program using a predominantly qualitative evaluation design, using secondary data for quantitative analysis tailored to address the following evaluation questions: 1. Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? 2. How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? 3. What is the sustained benefit of this project to the Networks and their community partners? 4. What were most significant changes brought about by PCI and their partners with regard to behavior/actions in preventing VAW? The evaluation began with a desktop review of documents relating to the existing state of the project, the communities the project served, and contextual information on VAW in South Africa. This assisted in tool development and provided a context for the subsequent analysis phase. The desktop review focused on project documents with reference to findings from the baseline study; objectives of the project, specifically looking at project indicators emerging after the commencement of the grant funding; project data from Salesforce; project strategic plans and reports; and key contextual information on issues relevant to the geographical locations where the evaluation was conducted. The evaluation specifically focused on activities in Wentworth and Inanda (eThekwini, KZN), as well as Khayelitsha (WC). Fieldwork was conducted in two provinces, namely KZN and the WC. Data was collected in Inanda and Wentworth (eThekwini District in KZN) as well as in the Khayelitsha sub￾district (Cape Town Metro in WC) where the Social Mobilization to End Gender-based Violence project was implemented by the KZN and WC Network partners, respectively. Initially, the evaluation team had only proposed to select one evaluation site for KZN, so that there was one site selected in WC and one site in KZN. Inanda was identified as a suitable site, because it had not been used as a primary site for PCI-led operational research. Wentworth was also suggested as a potential site; given that the evaluators only wanted one site, the evaluation team avoided Wentworth, which had recently been used as the site for the PCI-led internal evaluation. However, FHI 360 and PCI felt that visiting both Inanda and Wentworth would be useful in order for comparisons to be drawn as well as to develop a better understanding of less intensive (Inanda) versus more intensive (Wentworth) intervention areas in KZN. As a result, both sites were selected. Khayelitsha was chosen because it was the only community where the PIA program was implemented in the WC. The research team conducted site visits at the head offices of the Networks, as well as their sector partners. Key informant interviews were conducted with PCI staff, sector partners and other sector players, as well as selected program beneficiaries at the individual level. Focus groups were conducted with CEs, CIs, PAGs and PACs. Three fieldwork teams consisting of two researchers each collected data in KZN and the WC. Researchers were selected in accordance with relevant experience and spoken languages. Fieldwork took place over a period of seven days in each province (see Appendix 4 for a list of sites visited and dates of visits). Project Concern International (PCI) UGM End of Project Evaluation 5 SAMPLE SIZE AND STRATEGY The sampling framework was selected with the aim of maximizing the utility of data collected within the timeframe allocated for fieldwork. Purposive sampling was used to ensure a wide range of perspectives was captured in the evaluation (see Appendix 2 for the detailed sampling framework). Table 3: Sampling Framework Sampling Framework Western Cape KwaZulu-Natal Sector Partners - 3 sector partners (2 NGOs, 1 FBO) - 6 sector partners (3 NGOs, 1 FBO, 2 public sector) Other Sector Players (not part of network) - 1 other sector player in Khayelitsha - 2 other sector players in Wentworth - 2 other sector players in Inanda CEs - Total of 25 CEs in Khayelitsha (WC) - 4 focus groups - 64% of CEs reached - Total of 50 CEs in KZN - 25% of CEs in KZN reached - 2 focus groups in Wentworth - 1 focus group in Inanda CIs - Total of 2, 010 CIs in Khayelitsha - 4 focus groups  2 in VFZs  2 in non-VFZs - Total of 871 CIs in Wentworth - 4 focus groups  2 in VFZs  2 in non-VFZs - Total of 367 CIs in Inanda - 4 focus groups  2 in VFZs  2 in non-VFZs PAGs - 4 focus groups in Khayelitsha  2 in VFZs  2 in non-VFZs - 4 focus groups in Wentworth  2 in VFZs  2 in non-VFZs - 4 focus groups in Inanda  2 in VFZs  2 in non-VFZs PACs - 1 focus group in Khayelitsha - 1 focus group in Wentworth - 1 focus group in Inanda In summary, a total of 35 focus groups and 25 key informant interviews informed the evaluation findings. DATA COLLECTION METHODS AND PROCESSES CC&DW obtained ethical approval for this evaluation through the Faculty of Law at the University of Cape Town’s (UCT) Ethics Review Committee. A common approach that was used in the range of interviews and focus groups was the most significant change technique. This technique involves the generation of significant change stories by various stakeholders involved in implementing the intervention, as well as benefitting from the intervention. All key informants and focus group participants were required to provide their written consent for their participation. Participation was voluntary and information shared remained anonymous. The interviews were recorded on a dictaphone for record-keeping purposes and were only shared amongst the 6 Project Concern International (PCI) UGM End of Project Evaluation research team. Semi-structured interview guides were used to guide the discussion (see Appendix 3 for the data collection tools used). LIMITATIONS OF THE EVALUATION An evaluation constraint was the timing of the fieldwork, which took place from November 19-23, 2012. This was just prior to the 16 Days of Activism against Gender Violence, which takes place annually between November 25 (International Day of No Violence against Women) and December 10 (International Human Rights Day). In order to prevent any bias, the research did not coincide directly with the 16 Days, although this added pressure to the fieldwork process and condensed the time in the field. Data collection could not be delayed because most NGOs closed shortly thereafter for the holidays. The time period during which the data collection took place was also limited (five days), which restricted the amount of data that could be collected and imposed certain methodological constraints— it only allowed for a once-off snapshot view of people’s attitudes towards the PIA program. Therefore, an in-depth impact assessment could not be conducted, which would have provided greater insight into the long-term changes which this program has brought about, as well as the levels of sustained behavior change amongst individuals in response to program interventions. A further constraint was that the program had ended in the WC and many sector partners in KZN were also no longer receiving funding. In many cases, it was a challenge to find evaluation participants. Bias may have also been introduced as those that did decide to participate may have come forward because they had a particular affinity or dislike for the program. Additionally, there was a certain amount of participant fatigue because those involved in the PIA program had been part of ongoing operational research conducted by PCI. A precedent had also been set that payment would be provided for participation in PCI-led research. This reimbursement was approved by the Human Sciences Research Council (HSRC) Ethics Committee, and these payments were given as reimbursement for transportation and as a token of recognition and gratitude for time taken. CC&DW were unaware of this precedent prior to entering the field and did not have ethical approval to provide for this. Whilst CC&DW research teams offered refreshments at focus groups and transport reimbursement, this precedent proved to be problematic for the research teams, as many individuals refused to participate because the team offered no direct financial reimbursement. Project Concern International (PCI) UGM End of Project Evaluation 7 IV. FINDINGS AND DISCUSSION This section offers a set of findings organized according to the four evaluation questions: 1. Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? 2. How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? 3. What is the sustained benefit of this project to the Networks and their community partners? 4. What were most significant changes brought about by PCI and their partners with regard to behavior/actions in preventing VAW? The findings are accompanied by some discussion and interpretation of the findings. KEY EVALUATION QUESTION 1: Have the project partners (PCI and the two Networks) ensured the delivery of a social mobilization strategy that addressed VAW in the targeted communities? PCI articulated their social mobilization approach as mobilizing public and private sector partners to combat GBV, creating an enabling environment for changing social norms related to GBV, and developing and supporting the implementation of multi-sectoral activities. PCI’s model, as illustrated in Table 1, drew heavily on following a strong internal process of analysis, engagement, alignment, implementation and evaluation. After reviewing the substantial documentation associated with the project, it is clear that PCI invested tremendous time and resources43 that resulted in the design and development of the PIA program. The documentation shows an impressive track record evidenced by annual plans and reports illustrating the achievement of targets (described later in this section) set during the course of the program. In addition, the resulting social mobilization model, as shown by Figure 2, affirms that PCI did deliver a social mobilization strategy in the targeted communities. Notwithstanding the achievement of targets and the resulting model, it is important to interrogate the real value of the PIA program to the targeted communities. Understanding Social Mobilization In reflecting on the value of the PIA program, CC&DW reviewed literature that exists on community mobilization, a term often interchangeably used with social mobilization and/or movement building to effect social change. Social mobilization, as defined by UNICEF, is “a broad scale movement to engage people's participation in achieving a specific development goal through self-reliant efforts.” These definitions are affirmed in other literature that underpins the importance of building community capacity to identify their own priorities, needs and solutions in such a way as to promote sustainable change44 43 This evaluation does not include an assessment of financial resource efficiency for the PIA program. 44 Batliwala, Srilatha. Changing their worlds, Concepts and Practices of Women’s Movements. Association for Women’s Rights in Development. 2009. 8 Project Concern International (PCI) UGM End of Project Evaluation Unlike narrow project-based approaches, community mobilization provides a viable comprehensive alternative that adds up individual interventions, sequences them into a logical progression, strives to build on what is achieved and has an overview on how various activities will slowly come together to change the social climate. It is responsive, participatory and based on a holistic analysis of the root cause of VAW45. Instead of delivering messages that are top-down and implementing short-term activities, comprehensive community mobilization seeds activism by sparking critical thinking, working with whole communities and encouraging individuals to change and uses multiple strategies that build a critical mass to support women’s rights. Finally, this approach supports people to face the fact that violence is not something from which they are far removed; rather, it is something we all grapple with in our relationships, and it inspires and creates multi-faceted activism among a cross-section of community members. Reflections on Social Mobilization as per the PIA Program i. Analysis as a Basis for Shaping the Agenda As part of the original APS, the Centre for AIDS Development, Research and Evaluation (CADRE) was appointed to take responsibility for the evaluation and research aspects of the project. Following the award of the project to PCI, Dr. Warren Parker, originally a Director of CADRE, was appointed as the projects’ research consultant and was tasked with the responsibility of guiding, conceptualizing and implementing the evaluation and research elements of the project. PCI invested significant resources in conducting research throughout the life of the project. A baseline study, ongoing research and a final internal evaluation all contributed towards providing PCI with high quality information upon which to plan and review their approach and influence the social mobilization strategy. At least 15 different reports have been compiled by PCI that speak to the evaluation and research processes (see Appendix 5 for a list of these documents). The baseline study46 conducted in 2009 aimed to explore a range of issues including experiences of general violence and GBV, attitudes and beliefs related to VAW and perceptions of VAW at community level. The survey questions included a series of ‘norm statements’ that potentially underpinned and fostered VAW. Surprisingly, an overwhelming majority of respondents rejected the ‘social norms’ statements with most respondents recognizing that such attitudes and beliefs were inappropriate. For example, only 1% of respondents agreed that a husband could hit his wife when they have a disagreement; the notion that if a husband hit his wife, it was a sign of love, was also not widely held (4%). Similar low levels of support were found for other ‘social norms’ statements, and there was also a general agreement that people should not be silent in response to VAW. Despite these findings, there remained a widely held perception that VAW was pervasive in communities. For example:  50% agreed that “men in this community often hit their girlfriends.”  38% agreed that “girls and women in this community are often raped.”  37% reported that “men in this community do not respect women.” 45 Michau, Lori. Approaching old problems in new ways: community mobilization as a primary prevention strategy to combat violence against women. Gender & Development, 15(1). 46 PCI. 2013. Final Report of Project Implemented with USAID/ PEPFER Funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012. Project Concern International (PCI) UGM End of Project Evaluation 9  53% reported that “violent crime is a problem in this community.” While the general findings of perceived high levels of VAW in communities in the two provinces illustrated the importance of addressing VAW through the program, it was not expected that the ‘social norms’ statements would be understood as inappropriate and be rejected by the vast majority of participants. In the original program proposal, it was assumed that widely held social norms were failing to view GBV as a problem, and it was assumed that such norms contributed to the acceptance of VAW. This in turn was believed to underpin incidence of VAW while also discouraging reporting and prosecution of VAW. The unexpected and finding prompted a deeper review of the program assumptions regarding the links between social norms and VAW. The findings of the baseline study provided the justification for the shift from transforming social norms, to a focus on enabling people to take action. Yet, a point raised by a project partner was whether the correct questions were asked in the baseline study. According to this partner, the questions asked were not necessarily rooted in the cultural context and thus did not illuminate the fact that the cultural norms and beliefs relating to VAW are still very strong—these norms are what shape attitudes and behavior and maintain current realities. They believed that whilst action was an important response to preventing VAW, greater attention should have been placed on addressing cultural and structural issues contributing to VAW. The findings were discussed by the partners, leading to the conclusion that a ‘missing link’ was the lack of action in response to VAW. A commitment was thus made to developing a ‘social brand’ that validated and motivated acts and s that addressed and prevented VAW. The focus of the program thus became transforming a norm of “inaction in response to VAW” to “action in response to VAW,” with the goal being to increase the “number of women and men advocating or acting against VAW” (PCI, 2010). With sector partners, CEs, CIs, PAGs and PACs, the evaluation team explored the following questions: 1. Why do you think it is important to address VAW in your community? 2. What is the extent of the problem here? 3. What are some of the factors that contribute towards the problem of VAW in your community? The respondents spoke about pervasive cultural norms that are difficult to resist or overcome. For example, if a man has paid lobola for his wife, he “owns” her. Another example frequently cited was “if he beats you, then he loves you.” All respondents identified structural factors—very high rates of unemployment and substance abuse (drugs and alcohol)—as contributing factors to VAW. Based on the findings from the baseline research, the PIA program shifted their focus from ‘norms’ that may somehow maintain the high levels of VAW to instead address the norm of inaction. This shift in strategy did ensure that the PIA program addressed the attitudes and values that justified VAW and it highlighted the wrongfulness of VAW and the intolerability of VAW perpetration. Notwithstanding, where the program fell short was that it did not interrogate and confront the deep-seated cultural beliefs and customs that drive VAW. This cannot be conducted through simple actions and simply highlighting the wrongfulness of VAW, but it requires a more focused intervention that enables communities to ultimately create new norms and create alternative realities. ii. What was the Agenda? 10 Project Concern International (PCI) UGM End of Project Evaluation One of the distinguishing features of successful and sustainable movements for social change is that they have a clear and consistent agenda focused on transforming power relations47. While the PIA program was initiated to reduce VAW and consequently reduce HIV infection, it became apparent from reviewing the program documentation and also during the fieldwork that the agenda was diluted during the course of implementing the program at the community level. With regard to HIV, few respondents reported that the PIA program overtly addressed HIV. According to PCI, this was their intention—their approach was premised on the knowledge that the underlying links between HIV prevention and VAW prevention were well established. The program’s focus took into account that while there were many HIV programs being conducted in the two provinces, the gap lay in the need for a program with an explicit, singular focus on VAW. Making a link to HIV would have been counterproductive as a result of issues such as ‘AIDS fatigue,’ which undermines interest in community level HIV programming. A mixed VAW/HIV prevention program thus has the potential to be undermined by a general lack of interest among community members in HIV programs. With respect to the outcomes of the program in relation to HIV, the links between VAW and HIV are multifaceted and include: 1. Forced sex may directly increase women’s risk for HIV through physical trauma. 2. Violence and threats of violence, may limit women’s ability to negotiate safer sexual behavior. 3. Sexual abuse as a child may lead to increased sexual risk taking as an adolescent/adult. 4. Women who test for HIV and share test results with partners may be at increased risk for violence. PCI asserts that the documented actions and reflections of program participants demonstrate that the program extensively addressed HIV prevention links 1 and 2 through its focus on VAW without an overt focus on HIV. Furthermore, while the program was contractually bound to the over-18 age range, a number of actions involved responding to sexual abuse of girls younger than 18 years of age, thus addressing link 3. Determining the HIV status of women who experienced VAW was not encouraged for reasons of privacy/confidentiality. However, conducting the program in high HIV prevalence areas can be expected to have addressed link 4. While an indirect approach has merit, many of the evaluation participants—trained as CEs and CIs—did not make the link between HIV and VAW for themselves. Therefore, this academic understanding of the links between HIV and VAW, as outlined above, seemed to rest amongst the PIA program partners and sector partners. Whilst the evaluators support the focus remaining on VAW, there does appear to be a missed opportunity in the training to make an explicit link to HIV that will empower CEs and CIs with information that could help them in their work. For example, when CEs and CIs are undertaking PIA actions, they would be alert to the risk and vulnerability of community members to HIV. When providing counseling and support, where appropriate, they can then provide the client with appropriate HIV prevention and care information and refer them to relevant HIV services in the community. In 2010, extensive time and resources were invested into designing, testing and adopting the PIA brand48. Different design options and slogans were reviewed, and their meanings were explored. This resulted in the final logo, which depicted different colored hands interwoven, extending upwards. This was accompanied by a slogan that said “Working Together to Prevent Violence Against Women.” 47 Batliwala S. Changing their Worlds, Concepts and Practices of Women’s Movements. Association for Women’s Rights in Development. 2009. 48 PCI. 2010. Report on theDevelopment of Central Icon/Symbol and Slogan for Social Mobilization Campaign. Project Concern International (PCI) UGM End of Project Evaluation 11 At this point, the PIA program was still focused on VAW; however, once implementation of the PIA program started, project partners received feedback from both men and women who complained that the focus of the PIA program was solely on women and felt men’s needs needed to be acknowledged by the program. This started to shift the agenda more broadly towards focusing on GBV. The rationale to include men in a social mobilization strategy is clear and the resulting focus on GBV is important. According to the Manual on Gender Based Violence Affecting Young People, by Gender Matters: Gender-based violence is an umbrella term for any harm that is perpetrated against a person’s will; that has a negative impact on the physical or psychological health, development and identity of the person; and that is the result of gendered power inequities that exploit distinctions between males and females, among males and among females. Although not exclusive to women and girls, GBV principally affects them across all cultures. Violence may be physical, sexual, psychological, economic, or sociocultural. Categories of perpetrators may include family members, community members and those acting on behalf of or in proportion to the disregard of cultural, religious, state, or intra-state institutions. In the case of women in general, gender-based violence is a way of assuring women’s inferior position in society. Violence against women and the threat of it is a form of gender-based violence that deprives women of their rights socially before the law becomes involved. This is one of the reasons why long-standing laws on equality of the sexes, or general legal sanctions for most forms of violence against women, have not been able to end or even significantly limit the inequality of women and men by themselves. In the case of LGBT (Lesbian, Gay, Bisexual and Transgendered) people and men who do not act according to dominant masculine gender roles, gender-based violence has the function of correction by example. The severity of the ‘punishment’ for men who do not act according to the demands of male gender roles (whether gay, bisexual or heterosexual) may be related to the perceived danger that their difference presents to normalized and dominant assumptions about gender. Their very lives might collide and appear to contradict the idea that there are natural forms of behavior and social roles in general for men and women.49 As can be seen by the definition of GBV, VAW is certainly a part of GBV. GBV does not only refer to the inclusion of men, but to violence resulting from a range of gender and sexual identities. One sector partner in KZN felt strongly that there was an opportunity to do more to address GBV. Working in Khayelitsha, the targeted community of the WC provided an opportunity to address another form of GBV in South Africa, which is corrective rape, defined as “a hate crime in which a person is raped because of their perceived sexual or gender orientation. The common intended consequence of the rape, as seen by the perpetrator, is to ‘correct’ their orientation, to turn them heterosexual, or to make them ‘act’ more in conformity with gender stereotypes.”50 These factors show that with the shift to GBV, the PIA program had the opportunity and potential to address a broader definition of violence as it relates to gender and sexual identities underpinned by a power analysis. In 2012, the agenda was extended to focus on violence, VFZs emerged and the slogan was removed from the logo. PCI and its partners place emphasis on the fact that the overwhelming focus of the work was still VAW and that the zones could not have been called ‘violence against women free zones’ because it would have been too cumbersome. 49 Gender Matters. Undated. Manual on Gender Based Violence Affecting Young People. Accessed from: http://eycb.coe.int/gendermatters/chapter_2/1.html on 7 March 2013. 50 Bartle, EE. Lesbians And Hate Crimes. Journal of Poverty 4 (4): 23–44. 2000. 12 Project Concern International (PCI) UGM End of Project Evaluation Data from the focus groups and key informant interviews showed strong support of the VFZ. They felt it was a visible and valuable initiative and that VFZs enhanced community safety and community awareness of crime hotspots. Respondents highlighted that VFZs also improved women’s safety. For example, members of PACs would cut back trees and bushes that improved visibility and created fewer spaces where muggers could hide. It is important that the future custodians of the PIA program maintain a clear link to the primary objective of the project otherwise individuals and organizations can potentially hijack the agenda and use the VFZ to focus only on issues that they think are important. For example, a CI from Khayelitsha said ‘in the near future we wish gangsterism to be the main focus of the PIA program.” It should be noted that the VFZs were a new initiative introduced very late in the life of the PIA program. In the communities in KZN, VFZs were gaining momentum due to support from other sources of funding; in contrast in WC, the program had already closed out and participants said the activities were not continuing as before. It was therefore difficult to get a clear picture of the exact value of the VFZ in reducing VAW and GBV. Finally, the call for the agenda to be framed in gender neutral terms (not seen to be favoring only one gender), is often characteristic of mixed gender spaces. In the case of the PIA program, this call from members of the community can be attributed to the fact that the training component of the initiative did not have a consciousness-raising approach that would enable members of the community to see the rationale for focusing on transforming gendered power relations as a means to reduce VAW. This is by no means unique to the PIA program, as is confirmed by the literature51, but it is something that needs to be taken into account for future initiatives that confront and challenge power relations that maintain not only VAW, but also high rates of HIV infection among women. iii. Training The role of training in a social mobilization strategy is of critical importance to ensure that there is a critical mass of individuals and groups who no longer tolerate VAW. The training program employed by the PIA program consisted of five 90-minute training sessions as outlined in the earlier introduction. As depicted in Figure 3, in FY 10 and FY 1152, the PIA program trained 12,783 CIs in Session 1 and 2 of the training. By province, the KZN Network on VAW reached 8,507 and the WC Network on VAW reached 4,27653. In FY 12, 6,203 CIs were reached by Session 3, 4 and 5 (thereby completing all five training sessions) and 3,089 were defined as active. In both instances the PIA program met the PEPFAR target. 51 Meer, S. Struggles for gender Eqquality: Reflections on the place of men and men’s organisations. OSISA. 2012. 52 In FY 10 and FY 11 an annual target of 6,000 was set. In FY 10, 4,726 CIs were trained and in FY 11, 8,057 CIs were trained which made up a total of 12,783, as highlighted above. 53 Project Concern International. PIA Program Timeline. October 2012. Project Concern International (PCI) UGM End of Project Evaluation 13 Figure 3: Number of CIs Reached by PIA Training As shown in Figure 4, it is important to note that the CIs trained in FY 12 are a subset of the CIs trained in FY 10 and FY 11. Figure 4: CIs Trained with Session 3, 4 and 5, as a Subset of CIs Trained with Session 1 and 2 The target for FY 12 was not to reach any more new CIs but to deepen the ownership and actions from the current CIs54. Activities during the final quarter of the four-year project therefore focused on the completion of Session 5 with the CIs to meet the PEPFAR target. Almost all training respondents (CEs and CIs) who participated in the evaluation focus groups reported training Sessions 1 and 2 to be of particular value. They stated that they gained knowledge and a good understanding about VAW, as well as the different types of abuse that can occur. Many said they were not aware of the meaning of emotional abuse prior to the training. A CE in Khayelitsha said that the training gave her confidence. She reported “we received the skills to attend the clients by ourselves. I attended to a couple with problem. I fixed it now and they are happily married and I feel more confident.” The fact that the PIA program aimed to enable action is reflected in the content of the training. As reflected in the documentation provided by PCI, the training and related materials made the link between HIV, VAW and gender inequality, however, in the focus group discussions with those who had undergone the training, this particular link was not made and required considerable probing to elicit 54 Project Concern International. Quarterly Report: July to September 2012. October 2012. FY10/11 FY12 Target 12000 6000 Achieved 12783 6203 0 2000 4000 6000 8000 10000 12000 14000 Target Achieved 14 Project Concern International (PCI) UGM End of Project Evaluation responses. With those who had not received such training (members of PAGs and PACs), the link was not made at all—this once again raises questions as to the efficacy of the training and awareness raising components of the program. While the evaluators acknowledge that the task of challenging an entrenched value system is complex and in efforts to make it manageable, a “do what you can” strategy is often adopted, with the underlying assumption being that doing something is better than doing nothing. The PIA program highlights that in the implementation of their program, they explored VAW and the underlying aspects of VAW, such as attitudes, values, norms and justifications and that these elements gave rise to spaces where violence was no longer tolerated. The analysis draws on the narratives of CEs, CIs, PACs and PAGs who themselves stated that they gained more information on VAW, the different forms of violence and actions that could be taken to prevent VAW. Actions were mostly small, immediate interventions; while these actions benefit the community in the short-term, the PIA program could be strengthened to ensure a greater focus on longer-term actions that contribute towards sustained change. Unfortunately, the PIA program was implemented within a short time period. Many CEs and CIs were enthusiastic about the project when they were recruited; however, when the PIA program closed out, as evidenced in Khayelitsha, the CEs and CIs felt demoralized. Lori Michau warns against short-term interventions, saying that: “when it comes to prevention of violence against women, without synthesis of values underpinning individual activities, without a longer-term plan for managing the process, without pragmatic sequencing of interventions aimed at challenging normalized acceptance of VAW and without shrewd strategizing, interventions are likely to have little long-term impact.”55 iv. Enabling Action Transforming the norm of inaction to the norm of action is the trigger for the social mobilization model employed by the PIA program. The PIA program aimed to ensure that 50% of those CIs who complete Session 5 are active to meet one of the key internal project targets. An active CI is one who has been certified at Session 5 and has either formed a PAG, reported an action or joined a PAC56. The FY 12 target was to achieve 3,000 active CIs; as shown in Figure 5, 3,089 CIs were active. CI actions reported per month peaked between April 2012 and June 2012, just prior to the completion of the project, with WC peaking at 200 actions in April 2012 and KZN peaking at 500 actions in June 2012. 55 Michau, Lori. Approaching old problems in new ways: community mobilization as a primary prevention strategy to combat violence against women. Gender and Development, 15(1). March 2007. 56 Project Concern International. Quarterly Report: July to September 2012. October 2012. Project Concern International (PCI) UGM End of Project Evaluation 15 Figure 5: Salesforce Data Showing the Number of Active CIs In summary, the quantitative data reflects that 12,783 CIs were trained in Sessions 1 and 2. A total of 6,203 CIs completed training Sessions 2, 3 and 5 and 3,089 reported actions. Another internal target set by the PIA program in FY 12 was to achieve a total of 60 VCAs, by setting up VFZs and holding community marches and rallies. This target was exceeded, with a total of 72 VCAs being recorded (see Figure 6). A total of 37 VFZs were established—28 in KZN and nine in Khayelitsha. Figure 6: Sum of Verified Visible Community Actions in FY 12 Most of the actions were small actions—diverting attention when a couple is fighting, speaking to a perpetrator, speaking/supporting a survivor of abuse. Some of the actions that were taken as reported by the focus groups include: “We hosted rallies to make people aware about violence in the community and also how we could address violence in the community.” (CI focus group, VFZ, Inanda) “We do information sharing and call community meetings to address issues that concern the community as a whole.” (CI focus group, Non-VFZ, Inanda) “Taxi’s association transported us to court to stand with the victims’ families.” (PAC, Wentworth) Other activities reported include door-to-door campaigns, poster development, talking to the police and the dissemination of materials (stickers, pamphlets, etc.). In addition, other activities in 2012 included the launch of the VFZs as well as a Taxi Talk campaign. This campaign, limited to the KZN Network on VAW, involved 70 co-branded PIA (and Brothers For Life) taxis with the slogan ‘Action Alert: Do the right thing.’ A series of ten audio skits and songs promoting 16 Project Concern International (PCI) UGM End of Project Evaluation taking action to prevent GBV were played to commuters as they traveled on the taxis from June 2012 onwards. When locating a prevention response to VAW in a community, it is important that the response addresses the root causes of VAW and influences the value systems and environments that tolerate and allow VAW to occur. This approach engages the entire community in a process of examining core issues and builds a common understanding of VAW as a manifestation of women’s low status and their relative lack of power in relationships. This then means that the ultimate agenda would aim to influence the context of women’s lives, rather than respond to specific forms of violence57. PCI maintain that the PIA program was extensively committed to the exploration of causal factors underpinning VAW through the organic and community-led nature of the actions that aimed to ensure that responses to VAW were directly connected to the context and lived experiences of community members. The evaluation team acknowledges their important contribution in terms of the research and their activities; however, the fieldwork failed to provide strong evidence for this. The emergence of VFZs can be considered both an innovation and an unintended outcome. The project worked with CE/Os and their PACs to launch VFZs and to develop sustainability plans for the long-term maintenance of the zones. As described earlier, VFZs were geographical areas identified and demarcated to raise community awareness of crime hotspots. This program gained strong support from those living in the neighborhood. An important observation of this project was the timing of the implementation of activities. Whilst the process driven nature of the social mobilization approach is acknowledged, better management of the four-year project was needed to ensure that there the time necessary for wrapping up the program (under this grant)was included. As seen in Appendix 658 a substantial amount of activities, core to this program, were implemented in the final year of the program. Furthermore, VFZs were introduced for the first time in the last six months of the grant. Better management of these program iterations needed to be in place so as to allow the project partners to gradually withdraw while ensuring that sector partners, CEs and CIs are continuing with the work of the PIA program. The VFZs could be seen as a missed opportunity if PCI had decided against implementation; however, they could have been explored through a closely managed pilot project, with recommendations made for future use. This way, the organization could have avoided a situation in Khayelitsha, where the PIA program had no post-PIA program funds. As a result, there was no organizational support available to PACs and their VFZ, which impacted on the sustainability of the initiative59. A further constraint reported by PACs in Khayelitsha was the lack of power that the committees could leverage to get public sector partners to join and actively participate in the PACs; PACs did not have a strong organization that they were affiliated to and were therefore not taken very seriously. Project Constraints and Enabling Factors Targeted Communities In the Western Cape, the WC Network on VAW originally targeted a larger geographical area servicing their existing network partners who were located across the province. This approach yielded a broad geographical reach into both rural and urban areas. However, the number of people reached was low and significantly lagged behind the PEPFAR targets set for the project. While this was in line with the 57 Michau, Lori. Approaching old problems in new ways: community mobilization as a primary prevention strategy to combat violence against women. Gender and Development, 15(1). March 2007. 58 Adapted from PCI PIA Project Timeline. 59 Ibid. Project Concern International (PCI) UGM End of Project Evaluation 17 initial program strategy, it appeared that the program was thinly spread and would have struggled to amplify the actions. This factor, along with the reduction of anticipated funds in Year 2, meant that PCI adapted their approach to implement the PIA program in a more concentrated geographical area. Thus, they decided to target Khayelitsha, a sub-district of the City of Cape Town Metropole. Despite a good public health rationale for working in Khayelitsha, this decision was a challenge for the WC Network on VAW for following three reasons: 1. They did not have strong, existing sector partners based in Khayelitsha and therefore the Network had to establish new partnerships with organizations in the community and as a result, only four sector partners were mobilized. 2. Existing members of the WC Network on VAW anticipated benefiting from the capacity development opportunities offered by the PIA Program. As a result of the decision to significantly narrow the geographical reach of the project, member organizations of the Network did not benefit from the project. 3. Khayelitsha has a long history of being targeted by organizations doing research and implementing pilot projects, which meant that it is a saturated research environment. In Khayelitsha, four sector partners, three NGOs and one FBO, were recruited: Rape Crisis, Joy Christian Church, Nonceba and DKTSA. Each organization had a strong presence in the community, with staff being active in the community. In eThekwini, where the KZN Network on VAW was located, many PIA activities focused on the urban communities of Wentworth and Inanda. Sector partners in eThekwini were well-established organizations representing NGOs, FBOs and the public sector and they included; Wentworth Victim Friendly Centre, South African Police Service (SAPS) Hillcrest, SAPS KwaMakutha, Durban Coastal and Mental Health, DoJ, World Council for Religion and Peace and the Wentworth Organization of Women. When participants of focus groups were asked the question ‘Do you think other communities in Cape Town/Durban need this initiative? Why?’ all replied that other communities would benefit from the project. There was a very strong recognition of the need for the PIA program in other communities, because VAW and the factors associated with it were seen to be just as prevalent. CE and CI Selection The criterion for selecting a CE was their affiliation to a sector partner as a staff member or volunteer. They were generally active in the community and had consented to become involved in the PIA program. The criterion for a CE selecting a CI was that they had to choose someone who had influence through their position in the community. Following Session 2, CEs and CIs were expected to start recruiting CIs or initiate a PAG. CEs and CIs reported that they felt very pressured at this point to meet the expectations of project stakeholders. Reasons for this feeling of pressure may have been because they had not yet developed much confidence in taking what they had learnt in the training and convincing others to join the initiative. Some CEs and CIs admitted to the evaluators that they recruited members via the path of least resistance by targeting family and friends. “The challenge was to recruit – trying to reach the target of numbers. It was a big deal and I had to make empty promises of photographers and t-shirts.” (CE, Khayelitsha) From the earlier quantitative data, reflecting CIs trained (Sessions1and 2 – 12,873) and CIs active (3,089), less than a quarter of CIs were retained and active in the PIA program. This dropout rate could 18 Project Concern International (PCI) UGM End of Project Evaluation be attributed to the poor adherence by CEs to the criteria for selecting CIs. Other reasons for attrition, noted by PCI, were shifts in the strategy in the WC to focus activities in Khayelitsha, as well as people moving away or not remaining active for an extended period60. Many CEs and CIs reported that they themselves have been victims of intimate partner violence, and in some cases, other CIs reported they were perpetrators of abuse. One CI from a FBO in Khayelitsha reported “I am the beneficiary of the project because I was a heavy drinker, but now I learned a lot. I know how to approach people and how to communicate. I was abusing people emotionally and physically when I am drunk, but I am changed now.” In the future, the approach followed to recruit CIs could be improved to ensure that greater time, support and strategy is given to CEs and CIs to carefully consider community recruitment strategies and select influential people. The project may also consider making counseling and debriefing services available to CEs and CIs who are survivors of and perpetrators of abuse. PIA Target Audience The PIA program targeted people over the age of 18. This was appropriately targete, because of the nature of the subject matter and the actions required; however, the emergence of the VFZ created an inconsistency. PACs decided that local secondary schools should be targeted as VFZs because in many of the evaluation focus groups, participants cited gangsterism in schools as a major challenge that was associated with drugs, violence and potentially VAW. Whilst secondary schools were branded as VFZs, learners themselves were too young to be CIs which meant that the PIA program could not be properly implemented—his was a missed opportunity. In an interview with an NGO, Mosaic, in Khayelitsha, it was reported that between 48-52% of their clients seeking support for rape were under 18 years of age. In the future, the PIA program may wish to consider a PIA approach that is relevant to the needs of a younger audience. Project Partner Roles Whilst the social mobilization approach was informed by strong research and consultation, it appears that much of the intellectual and academic understanding about social mobilization was held by PCI. PCI appeared to manage and implement the process of social mobilization and also had experience in process driven methodology. It is important to understand that the two Networks were separate entities (with very different identities) and came with their own strengths and challenges. Evaluation question 3 expands on this further to understand the different dynamics. The two Network partners and sector partners reported that, at times, especially during the first half of the program, they struggled to articulate the nature and activities of the PIA program because activities were developed in a process driven way. Therefore, strong relationships and collaboration were necessary but not always possible. It appears that project partner roles shifted during the lifetime of the PIA program. Initially PCI was only going to be responsible for directing the program and its operations. Based on the Networks’ performance, PCI played a lesser or greater role in directing, managing and implementing project activities. At times the shifts in roles created tensions amongst the project partners; with some staff reporting that at times it undermined their authority. 60 Extracted from PCI. 2012. Monitoring, evaluation and Reporting Plan. Social Mobilization to End Gender-based Violence: An Essential HIV Prevention Strategy for South Africa – October 1, 2008 to September 30, 2012. Version 4.0 (page 4 and 32). Project Concern International (PCI) UGM End of Project Evaluation 19 Based on key informant interviews with the two Networks and PCI, it appears that the two network partners were extremely under-capacitated at the outset of the program, which meant that there was a lack of organizational development, capacity and infrastructure. PCI provided a high standard of technical support locally and from personnel in their international office in the U.S. Technical expertise that added tremendous value focused on monitoring and evaluation(M&E), the conceptual development of the social mobilization strategy, and overall grant management and accountability. PCI described the relationship it had with the two Networks as a partnership. With these findings in mind, it is important to consider the term ‘partnership’ more broadly in order to consider some of the challenges and learning’s in achieving real partnerships. Looking at this term in context, this relationship is a North (PCI) - South (two Networks) NGO partnership, which brought “benefits based on their comparative advantage”61. In simple terms, PCI provided donor funds and technical knowledge and the two Network partners had local knowledge and presence which offers greater opportunities for sustainability and improving poverty reach. Brehm (2011) highlights three limits to partnerships: 1. The role of the Northern NGO as donor: this is a major obstacle to achieving equality. The imbalance in the relationship created by the Northern NGO’s control of over resources skews the power balance. 2. Funding processes and distorted accountability: whilst in theory accountability to local constituencies is important, in practice, the funding processes ‘hijack’ the accountability mechanisms and re-orient them towards Northern donors. Northern NGOs assume a control function, whilst Southern NGOs risk becoming donor-driven and distanced from their grassroots constituencies. 3. Organizational capacity limits: capacity mismatch often occurs between Partners of different sizes; partnership dialogue is more feasible between organizations of a similar size and capacity. Brehm also warns against the term ‘partnership’ being “overused and applied to a whole lot of inter￾organizational relationships.” During her research, Brehm reports that some NGOs prefer the term partner cooperation, as the term partnership is seen as too idealistic. Brehm (2001) recommends, “all parties be clear about the purpose of the partnership and the mutual expectations and responsibilities.” Furthermore, she warns that “given the tendency of funding processes to dominate North-South partnerships, it is critical for Northern NGOs to develop a more systematic approach to ‘good practice partnerships’.” The relationship between PCI and its two Network partners had many positive features. In the future, greater consideration of the above mentioned issues by Brehm should be given to ensure strengthened relationships. Stipends and Allowances Substantial project funding was made available by PEPFAR. This was an enabler to the success of the program. Despite the reduction in anticipated project funds in FY 10, respondents at all levels of the program reflected on the significant financial resources available to them. The budget translated into program staff being very well paid, high quality and large quantities of PIA branded products being developed and disseminated (t-shirts, jackets, badges), as well as expenditure on capital costs such as well-resourced offices and large motor vehicles. Branded items supported the amplification of the PIA program and were also seen as aspirational items by people involved in the program. The community perceived the program to be well-resourced and made demands to the Networks for a range of items such as stipends, meals, transport money, airtime 61 Brehm V.M (2001) NGOs and Partnership. NGO Policy Briefing Paper No.4. April 2011. 20 Project Concern International (PCI) UGM End of Project Evaluation for cell phones and other incentives. On the whole, project partners appeared to have provided these items, saying that in some instances it was reimbursement for a project-related cost, for example making calls to members of a PAC or PAG. Meals, food parcels, stipends and branded clothing items seemed to create the most problems as this created an expectation by the community and in many cases caused problems within the program as people would only participate if they were given or promised something. When participants did not get what they expected, they believed that the PIA program was withholding money or other incentives from them and this undermined the relationships within the program. One organization, not involved in the PIA program, based in Khayelitsha said that the community knows what they can get from different organizations and that it is a challenge to get participation based only on the value of the intervention they will participate in and the certificate on offer at the end of the intervention. This organization reported that they had overcome this challenge by; consistently not offering incentives, having a sustained presence in the community and by the community knowing that the work of the organization is a valuable community asset. Program funds certainly smoothed the way for community cooperation and helped project partners to reach the numbers of people required. PCI staff reported that the stipends given to CEs were a way to ‘prime the pump’ and make CEs accountable for their work with CIs because they were getting something. The assumption was that it would get them initially engaged and after that, once trained, they would be emotionally engaged and continue with the work. PCI reported that providing stipends was a strategy that has worked for PCI in other projects. This dynamic of stipends and allowances is challenging to manage, especially when working in communities with high unemployment rates and high levels of poverty. Ideally project partners should have had stricter measures in place from the outset to manage expectations by communities. One way of achieving this could have been through consultation with sector partners who could have advised the project partners about the best approach and shared their policies on stipends and allowances. Given that they are based in the community, sector partners know what would have been a more appropriate strategy acceptable to all parties. Because the program was heavily dependent on stipends and allowances, it created problems for sustainability. At the end of the program, when the funds stopped, on the whole, the project stopped. The evaluation team in Khayelitsha entered two months after the program ended and it was a challenge to find willing participants to discuss the program. Evaluation participants, mostly CEs and CIs, asked the evaluation team when the PIA program was going to return to Khayelitsha. It appears that the initial costs associated with starting the PIA program were high—mass communications component, baseline research and ongoing research, development of the brand and logo and the development of the training program. Now that much of the primary work has been completed, this program could leverage from the groundwork and be implemented with modest resources that cover the cost of training, support the implementation and amplification of the PIA activities, monitoring systems (Salesforce), as well as the development of branded items, which could be limited to inexpensive items. Branding Branding is an important feature of social movements and social mobilization activities. It contributes towards fostering a personal association with the brand’s values and goals, through an association with a color, logo or slogan. The PIA logo was seen by all respondents as being important. It gave the project an identity and members of the community associated the logo with the PIA brand, which was powerful. One CE from Project Concern International (PCI) UGM End of Project Evaluation 21 Khayelitsha said “in the shebeens, if people are fighting, people from PIA would show the PIA bangle and say ‘stop, this should not happen’.” The PIA brand has also enabled community members to identify PIA members when they needed help. A few sector partners felt that an unnecessary amount of funds were spent on branding. The distribution of branded items to CIs also links to the issue of incentives as many branded items served a dual purpose. The branded jackets, bags and t-shirts were items that people aspired to own and respondents reported that it was a cause of conflict and unhappiness in cases where some people received items and others did not when the same work had been done. In the future, clear communication about who can access the branded items should be put in place to avoid conflict within the project. KEY EVALUATION QUESTION 2: How comprehensive was the social mobilization strategy in addressing VAW in the targeted communities? There is an extensive body of work on the different elements of a comprehensive approach to reducing VAW. Such an approach would include: 1. putting in place the infrastructure and institutions to address VAW; 2. using the ecological model to achieve changes in individual, interpersonal, community and wider societal behaviors and practices; 3. community education on gender equality, human rights or violence against women, coupled with livelihood support for women and youth; 4. the involvement of key and strategic groups, in particular women, girls, adolescents and men, key community, traditional, cultural and faith-based leaders; and 5. legislation, policies and norms that support various outcomes relating to ending violence against women, since it could support changes at the individual, interpersonal and community levels, as well as within the broader society62. Figure 7 below provides a view of what a comprehensive approach would look like to address VAW in communities. As will be noted, such an approach would put communities at the center but would ultimately locate it in a broader context of their social and economic realities. When reflecting on whether the PIA program employed a comprehensive approach in addressing VAW, we look at it against the common elements identified above. Infrastructure and Institutions in Place to Address VAW Considering that this is a social mobilization model, the infrastructure referred to here would include the partnership between PCI and the two Networks, as well as the sector partners. These were then supported by CEs, CIs, PAGs and PACs. The assumption inherent in working across these levels is that these different actors and structures would be able to take forward the mobilization beyond the life of the program. As described in the previous chapter, PCI appeared to manage and implement the process of social mobilization. The two network partners and sector partners reported that at times, they struggled to 62 Bachelet, Michelle. Good Practices and Remaining Gaps in the Prevention of Violence Against Women. Paper prepared for Human Rights Council Annual Day, panel discussion on Good Practices and Remaining Gaps in the Prevention of Violence against Women. Geneva, June 2011. 22 Project Concern International (PCI) UGM End of Project Evaluation articulate the nature and activities of the project because every activity was only conceptualized as the project unfolded. This meant that the initiative, in most cases, was not seen as wholly owned by the communities, but as a project of PCI and its partners. PCI assert that they did not expect PIA participants to “wholly own” the initiative, but that the program followed a partnership approach involving participatory engagement through the implementation structure depicted in Figure 1. While this implementation structure was useful for the duration of the project, it has shortcomings with regard to program sustainability. Should PCI and its partners have adopted an approach targeted at seeding a movement, CC&DW would have expected to see an approach that included the elements outlined below: Figure 7: Social Mobilization Approach63 Whilst PCI followed a strong process, in hindsight there are some opportunities for improvements with regard to the implementation of the PIA program. Figure 7 offers an opportunity to briefly highlight some missed opportunities. The first phase deals with pre-planning of the program. The initial phase of the program saw the development of a strong APS proposal. However, one weakness in the APS was perhaps an over￾estimation of the capacity of the two Networks at the outset of the program. As indicated earlier, both Networks were under-resourced and did not have significant project implementation experience based on their primary role being that of a network. The networks brought a wealth of knowledge and experience and had relationships with network members, but it may have been more useful to benefit from their expertise via a project reference group or to use them in some kind of advisory capacity. Nonetheless, the two Networks benefitted enormously from the PIA program and from the resources invested in them and the experience they gained. 63 Mercycorps. Community Mobilization Framework. Guide to Community Mobilization, 2009. 2. Assessment and Planning with / by Community 3. Structures and Agreements 4. Leadership and Capacity Building 5. Co-monitoring and Learning 6.Repositioning 1. Pre-Planning 7. Handover    Project Concern International (PCI) UGM End of Project Evaluation 23 The second phase focuses on assessment with/by the community. This was completed with PCI undertaking a baseline study that engaged the community with regards to, amongst other things, their perceptions of VAW. This assessment was done with the community rather than by the community. Phase three saw structures and agreements being put in place. Sub-grants were awarded to the two Networks, and partnerships were entered into with sector partners via Memoranda of Understanding (MOUs). On reflection, it appears that there was an insufficient investment by the project partners in their relationship with sector partners. Due to time constraints, it appeared that in practical terms, the Networks worked directly with CEs, rather than directly with sector partners. As a result, the program relied heavily on PCI and the two Networks, which meant that there was a missed opportunity for PCI and the Networks in capacitating community-based sector partners to ensure that the infrastructure and base for the work that was seeded was solidified for it to continue beyond the presence of PCI and the Networks. Perhaps if greater investment (time, finance and capacity) had been made with sector partners, this could have informed and sustained the implementation of program activities. Phase four focuses on leadership and capacity-building. Much of the investment in training appears to have been with CEs and with CIs. Training of CEs was a very worthwhile investment. However because CEs were mostly employed by sector partners, such training and capacity development should have been closely linked to the development of sector partner capacity and leadership in preventing VAW. CIs were mostly ordinary members of the community; whilst this is a worthy approach, it may have been more strategic to adhere more strictly to the criteria of finding CIs who are influential and who were committed to the PIA program objectives from the outset. In addition, a key focus of this particular step in a holistic approach would be to ensure that all those involved in the program have a deep understanding and buy-in of the values and principles such as the fact that prevention strategies must address the root causes of VAW. Moreover, such efforts will be most effective where they are holistic and connect wherever possible with other key issues such as HIV and AIDS. Phase five looks at co-monitoring and learning, which was a strong feature of the PIA program. Phase six offers an opportunity for repositioning—if the sector partners had greater involvement and investment in the PIA program, they could have stepped up and taken greater ownership of the program, allowing the two network partners and PCI to step back without a very high risk to program sustainability. Program partners could have implemented a handover to the community as depicted in phase seven and from there, the cycle could have started again. Using an Ecological Model to Achieve Change In the previous evaluation question, we reflected considerably on the social mobilization strategy. A key component of the PIA program framework developed was focused on communications. In social mobilization, the importance of communication materials, impromptu discussions and training workshops is that they strive not to give answers and tell community members what to think, but rather to provoke personal reflection and critical thinking. Sustained exposure to arguments promoting women’s rights to live violence-free lives keeps issues alive in the public domain, in people’s minds, in homes and in workplaces. Over time, acceptance of these ideas around women’s rights and VAW becomes so common that they become viable alternatives to the negative normalized attitudes and 24 Project Concern International (PCI) UGM End of Project Evaluation practices. It sets a new tone or climate in the community/one where violence and disrespect of women is no longer tolerated64. In the case of the PIA program, the communications strategy developed provided a detailed analysis of the psychographics65 of the audience, which included the normative nature of violence, historic disempowerment, as well as the impact of drugs and alcohol on communities66. As a result, the focus of communications was articulated as preventing VAW at the broader societal level by shifting social norms and supporting the efforts of the WC and KZN Networks on VAW to provide support to survivors of violence to change their circumstances and encouraging friends and family to intervene67, At a broader societal level, the most significant and visible initiative in PCI’s communications was the Changing Faces campaign, a large-scale mass media campaign intended to reach large audiences to directly affect individual recipients by invoking cognitive or emotional responses68. It could be argued that such an initiative could provoke the critical thinking that would affect decision-making processes at the individual level and ultimately inspire change. In the past, such campaigns have been criticized for the large costs associated with them and the difficulty in assessing their impact. In addition, while mass media campaigns can produce positive changes in behaviors across large populations; this is only possible with the concurrent availability of required services and products, availability of community-based programs and policies that support behavior change69. Whilst there were indeed services, programs and policies available across the sector that focused on the prevention of VAW, it is not clear whether these were aligned with the Changing Faces communications campaign to facilitate the desired outcome. Even though the campaign boasts the achievement that it reached up to 3.6 million people across both provinces and acclaim for communications excellence70, the reduction in funding and strategic shift by USAID away from mass communications work meant a shift away from this particular campaign, thus not allowing for the longevity to achieve adequate population exposure to media messages and the opportunity to measure the impact of such exposure. The shift from mass media amplification to the development of toolkits and support materials was seen as being more directly relevant to supporting community mobilization. The approach employed Action Media workshops to develop symbols and slogans, as well as the dissemination of toolkits containing a branded t-shirt, backpack, cellphone holder, stickers and badges and A4 posters promoting the Stop Gender Violence Helpline. Furthermore, the recording and dissemination of actions to amplify the core message of the PIA program formed part of the strategy. As noted by PCI, this was meant to create broad societal ownership of and participation in the transformation of destructive social norms and to galvanize communities to take action to preventing VAW’71. On the one hand, the use of branded materials certainly achieved the objective of providing visibility to the initiative; on the other hand, it was also not clear whether people participated in the initiative merely to gain incentives. As noted by different actors: “Some do know about the issue of the linkage between HIV and VAW. The brands of the bags and the pictures and colors of (PIA) have meaning to the community at large.” (Sector partner, Khayelitsha) 64 Michau, Lori. Approaching old problems in new ways: community mobilization as a primary prevention strategy to combat violence against women. Gender & Development, 15(1). 65 Psychographics is the study of personality, values, attitudes, interests and lifestyles. 66 Project Concern International. (Social mobilization programme to end violence against women: An essential HIV prevention programme, South Africa – Communications strategy. 2009. 67 Ibid. 68 Wakefield, M., Loken, B., and Hornik R.C. (2010). The use of mass media campaigns to change health behaviours. Lancet 2010; 376: 1261–71. 69 Ibid. 70 PCI South Africa APS 674-08-003 Technical Narrative. 71 PCI South Africa APS 674-08-003 Technical Narrative. Project Concern International (PCI) UGM End of Project Evaluation 25 “People notice those (VFZ) and they know the meaning of colors, paintings, jackets, t-shirts and bags through the trainings they attend.” (PAC) “The bad side was that people thought you are eating their money when the Network stopped. People thought it is an opportunity for getting money. They became less interested when there was no incentive and could not accept the reality that it was voluntary then.” (CI, Khayelitsha) Community Education on Gender Equality, Human Rights, VAW and Livelihood Support for Women and Youth In considering the work of the PIA Program, indeed there is “increased social awareness of what could be done (would not cost money). In the end, the idea was to ignite a movement – and a sense that VAW is everybody’s business and that things can change.” (Network Respondent) In the previous evaluation question, we reflected on the training program employed by the initiative. The training curriculum provided participants with information and focused on inspiring communities to take action. As a result of their increased consciousness, communities found ways to intervene and prevent VAW in a supportive environment with friends, colleagues and institutions, supporting and reinforcing these efforts. The focus on action put enormous pressure on the different actors to move from information to taking action to show that they had changed behavior. While more than 3,000 active CIs were recorded by the PIA program, in reality, the activism was not even, and the interpretation of an action varied. Some communities were more active than others. At an individual level, all those who have been directly linked and exposed to the PIA program have been challenged and have an increased awareness of the need to take action in response to VAW72. The question that remains is whether this exposure would lead to sustained shifts in behavior and action thus entrenching new norms and values in the communities. While all respondents that were interviewed indicated that the movement from inaction to action was indeed a positive shift, the fact that these actions were not rooted in an agenda that aimed at chipping away at those barriers that inhibit equality, dignity and respect results in action for the sake of demonstrating that actions are taking place. While it does have meaning for the community in which it is located, the PIA program actions need to be looked at in the context of their contribution to reducing VAW and HIV incidence. The purpose of increased engagement of any group of people is to contribute towards a critical mass and ‘tipping point’ that would ultimately create sustained change. Whilst the VFZs are possibly a means or mechanism to facilitate the sustainability of community structures to enable continuous action towards change, at this point it is not possible to assess whether the increased engagement is sustainable, because this activity was so recently implemented. The absence of a broader change agenda defined by the community to form the container for such actions also makes it unlikely that this level of activism has the probability of continuing beyond the life of the program, as can be seen in certain instances where project activities have been discontinued. At least two examples were cited of sites where the project had stopped and the rates of crime and violence picked up again. This is reflected in some of the sites in the WC and even those in KZN where the initiative was discontinued. Participants noted: 72 This was affirmed by all respondents who participated in fieldwork as they were all involved in the project in some form or the other. 26 Project Concern International (PCI) UGM End of Project Evaluation “VAW has slowed down during the project, but when the project stopped it started again.”(Wentworth CI focus group) “The timing of the project was too short to me it felt like ‘Hit and run.’ (WC Network, sector partner) “To us this project came for a short period, some of the things were not clarified.” (CE, Khayelitsha) Indeed the actions and the VFZs could point to the fact that community structures and actors did own what was happening in their locality by responding to the social dilemmas that occurred in their environment. However, as noted previously, the depth and strategic nature of actions taken by these structures are uneven and not reflective of a movement that focuses their actions on creating sustained structural change. Involvement of Key and Strategic Groups Beyond the infrastructure that was created for the implementation of the PIA program, the question to ask is what more the program could have done to contribute to an environment that is indeed more sensitized to VAW. Whilst sector partners did engage different groups at the community level and public sector units such as the DoJ and SAPS (in KZN), the program would need to consider what other actors could have been engaged. One issue that emerged during discussions was the need to engage young people, especially secondary school learners. There is an opportunity to reinforce positive behavior and address structural problems, such as substance abuse and the potential for unemployment, before they are well entrenched. The importance of involving strategic groups such as local government and institutions, the business community, media, civil society organizations and communities is based on their potential to contribute to a climate in which VAW is no longer tolerated. Prevention requires achieving both economic and political empowerment of women and a critical mass within any given society or community that rejects VAW. In the current model, this is a gap—collaborations can increase the scope and reach of VAW prevention efforts and support the reach to a cross-section of stakeholders, with diverse strategies through a sustained process of change. All strategies used need to have a clear sense of what they are seeking to change and how to facilitate such change. Potential actors should encompass the community and go beyond women, men, youth and children and extend to include religious and other opinion leaders, local governments, health care providers, police officers, journalists, professional associations, researchers, trade unions and other NGOs. Legislation, Policies and Norms that Support the End of VAW Primary prevention cannot be effective where there is minimal commitment to secondary and tertiary prevention. Where there is lack of political will to (a) implement existing laws or (b) provide the basic services necessary to address the consequences of violence, prevention measures are unlikely to be prioritized and are likely to fail. It is, therefore, critical that a holistic coordinated approach would engage in coordinated efforts to ensuring this in place. The PIA program supported victims of VAW to access police services and helped with protection orders and other aspects of the legal system. They also ensured that people were linked to counseling and support services. However, there was no data to reflect whether PCI addressed broader issues relating to the functionality of some of these systems. For instance, some sector partners highlighted system challenges such as the police failing to properly enforce protection orders Project Concern International (PCI) UGM End of Project Evaluation 27 KEY EVALUATION QUESTION 3: What is the sustained benefit of this project to the Networks and their community partners? Overall it was found that the benefit of the PIA program was that it served to: 1. raise public awareness of the issue of VAW, 2. promote a culture of taking some form of action to counter VAW, 3. increase knowledge among women of their rights and recourse mechanisms and making known the availability and location of services; and 4. create safer public spaces through VFZ. When looking at the sustained benefit of the PIA program, it is useful to review it from different perspectives. Figure 8 identifies four levels of beneficiaries. Figure 8: PIA Program Beneficiaries Project Concern International The greatest benefit of this project to PCI was that they were given an opportunity to design and pilot a social mobilization strategy to reduce VAW in a South African context. Through generous funding from PEPFAR, resources were available to invest in the research and evaluation of the context, as well as the effectiveness of different PIA activities. For PCI, the two networks on VAW offered legitimacy and access to communities in South Africa. Given that PCI were entering South Africa for the first time, access would have been difficult without local partners. From this perspective, both networks offered a good entry point. Community Sector Partners Network on VAW ‐ Western Cape & KwaZulu‐Natal Project Concern International 28 Project Concern International (PCI) UGM End of Project Evaluation As mentioned earlier, there was an over-estimation by PCI of the capacity of both networks at the start of the PIA program. PCI had the constraint of working with two under-resourced and under-capacitated networks that required a large amount of support in order to activate the PIA program. The PIA program, through PCI, provided the two Networks with an opportunity to re-establish themselves and develop good internal systems, recruit staff and gain the experience of implementing a large project. Networks on Violence Against Women Overall, the two networks on VAW benefited greatly from the PIA program. The program has increased their institutional capacity to absorb and manage funds, because they have strong financial systems and were regularly audited. Furthermore, it improved the Networks’ project management systems and knowledge of and experience with monitoring systems, such as Salesforce. Overall, they both now have a strong track record with project implementation. When considering the value of the two network partners, it is important to understand that they are two different entities and they work autonomously. The KZN Network on VAW operates differently to the WC Network on VAW, with characteristics that are more like that of an NGO than a network. The KZN Network on VAW is autonomous and they decide what activities they will implement and who they will partner with. They viewed the PIA program as an opportunity to build their organization, developing internal systems and a strong program resulting in them being very well established by the end of the PIA program. The KZN Network on VAW has been provided with additional short-term funding from PEPFAR because it is located in a province with an extremely high HIV prevalence. This additional PEPFAR funding has enabled them to continue the PIA program. The PIA program is therefore still vibrant, with activities continuing to be implemented and experienced staff in place to manage the program. The KZN Network on VAW has other donors in place to support their work. Overall, this has enabled activities started during the PIA program to continue. The WC Network on VAW is a membership-based organization that gets its mandate from its members, who are NGOs and CBOs working in communities to provide care and support to survivors of violence. At the start of the PIA program, the WC Network on VAW was a small under-capacitated organization and was inexperienced in project implementation of this scale. Their skills and experience were focused on their primary role as a network, focused on providing a platform for sharing information, advocating against VAW and capacity development. The WC Network on VAW was extremely motivated to become involved in the project particularly because of the innovative thinking relating to the prevention of VAW and the seemingly developmental nature of the project. PCI was interested in benefitting from the existing relationships they had with NGOs and CBOs in WC. The Network acknowledged their underperformance at times and reported that they struggled with the project for different reasons:  They were implementing a project that they felt was different to the original concept they had agreed to in terms of the geographical reach.  They were being asked to implement a project that was constantly under development.  Their performance was constantly compared to the KZN Network on VAW.  They felt that PCI’s approach could have been more developmental in nature and that they could have made a real investment in the sector partners and members of the community. Whilst PCI were ultimately closing at the end of the project, the Network still had a reputation to uphold. Project Concern International (PCI) UGM End of Project Evaluation 29 During much of the project implementation phase, Network staff reported that it felt like they were ‘fighting fires’ especially relating to managing the logistical issues around expectations and stipends. There was mixed support by Network staff to the support offered by PCI. Some reported that it had a negative effect on some relationships between personnel in the Networks and PCI, as Network staff felt the skills and management authority were being undermined. Others were receptive to and grateful for the support offered by PCI and felt they were being mentored and supported. The PIA program required the total commitment of all Network staff to ensure that the PIA program met its deliverables. Therefore, the primary activities of the Network were not implemented, which meant that the Network was unable to service their membership base. Unfortunately, in WC where the PIA program closed out at the end of September 2012, much of the capacity development had already been undermined with project staff being retrenched after the project ended. Core skills, especially with regard to monitoring, evaluation and reporting (MER), have already been lost. With support from other donors, such as Irish Aid, the WC Network on VAW has continued to implement the PIA program beyond Khayelitsha, to other geographical areas of the WC. This is occurring on a smaller scale, with less financial resources. In Khayelitsha specifically, sector partners reported that they are not doing anything to continue the project due to a lack of financial resources. Some CEs and CIs continue to implement small scale actions and to maintain the VFZs. The gap in Khayelitsha is that there is no organization coordinating the actions, and the morale of CEs and CIs is low. Sector Partners Sector partners were organizations that were already well-established in the communities prior to the launch of the PIA program. Sector partners recruited by the PIA program were a range of NGOs and CBOS working in the area of care and support of vulnerable people in the community for a variety of reasons. Sector partners also included FBOs and public sector partners, such as SAPS and the DoJ. Sector partners welcomed the PIA program when it was started because it provided an opportunity to prevent VAW, which they felt was a real need. The PIA program signed an MOU with sector partners outlining roles and responsibilities. The MOU made a small provision for funds to the sector partner to pay for some administration costs associated with the PIA program. Sector partners were responsible for identifying individuals from their organizations who were then trained by the PIA program to become CEs. Once they had been trained as CEs, most of the future communication was directly between PIA program staff and CEs, thereby excluding the managers of sector partners. Managers reported that they were often only contacted when the PIA program required something—this resulted in them feeling used. Sector partners reported that they felt the PIA program was beneficial, but that they would have liked opportunities to be consulted about the content of the training and to look at how it could be amended for their own context. Overall, the evaluation found that there was a lack of investment by the PIA program in the sector partners, especially those sector partners that were NGOs, CBOs and FBOs, who participated in the PIA program. It appeared that public sector partners, SAPS and DoJ, benefitted more from the PIA program because it strongly supported one of their mandates which relates to the prevention of VAW in communities. 30 Project Concern International (PCI) UGM End of Project Evaluation As a result of the broader challenging economic environment, especially in the WC, some NGO sector partners have a severe funding shortage. The evaluation team believes that some financial resources available within the PIA program could have been used to provide greater support to NGO and CBO sector partners. Through some financial investment and greater technical investment in the program, there could have been a more meaningful involvement of sector partners in the PIA program, which could have created greater opportunities to sustain the PIA program. Community Whilst the PIA program had a strong brand in Khayelitsha, it did not have a strong organizational presence in the community, with many CEs and CIs requesting that an office be set up in Khayelitsha where they could meet and get administrative support to implement the PIA program. Their requests reflected the fact that they looked to the WC Network on VAW as the support organization for the program. The Network does not have an office in Khayelitsha; therefore, it was seen as inaccessible because it was 25 kilometers from Khayelitsha. Sustained mobilization takes place when communities remain active and empowered after the program ends. As mentioned earlier in this report, project activities were being implemented right up until the end of the project implementation cycle, which did not allow sufficient time for real engagement with regards to sustainability. Also mentioned earlier in the report, the social mobilization process followed did not allow time for repositioning or handover to other entities, which undermined opportunities for sustainability. Broader Sustainability Challenges Whilst the funding towards this project was extremely generous, the four-year time frame over which it was implemented was relatively short. As described earlier, the social mobilization approach, characterized by a five-step process meant that many activities were implemented near the end of the funding cycle, which reduced the ability of the PIA program to ensure that communities were able to independently lead sustainable activities at the end of the program. Despite USAID/PEPFAR and FHI 360-UGM requiring PCI to report on sustainability plans, it was unrealistic that these plans could account for every aspect of the PIA program’s sustainability. Future initiatives may require funders to specifically ask about issues of sustainability and require sustainability plans at the proposal development phase. Furthermore, close management of the tapering of funding as the funding period comes to an end could help programs close out more effectively. KEY EVALUATION QUESTION 4: What were most significant changes brought about by PCI and their partners with regard to behavior/ actions in preventing VAW? Everyone who participated in focus groups identified that the initiative had changed them in some way. Accounts ranged from an increase in knowledge, self-esteem, self-worth and independence. Shifts in Awareness Whilst acknowledging that social change occurs in many ways, the evidence shows that indeed there had been shifts in the awareness amongst CEs, CIs and beneficiaries with regards to them gaining a better knowledge and understanding of VAW. Many individuals expressed that they realized that they were experiencing or witnessing types of abuse for the first time that they never realized existed. Project Concern International (PCI) UGM End of Project Evaluation 31 “It made me understand that what happened to me was wrong and something should have been done.” (Beneficiary, Inanda) At the community level, there seemed to be heightened awareness and visibility of VAW through the PIA brand. PIA training covered issues relating to the types of recourse and actions that could be taken. This occurred during the implementation phase of reducing VAW and crime and created a situation where both men and women were speaking out. “I gained a lot in terms of preventing violence especially gangsterism. We used to wear PIA jackets for identification. We also learned to create love and communication. I learned about the importance of openness and building others self-esteem. We also helped in marriage counseling and help others to join church.” (CE, Khayelitsha) Shifts in Behavior All those consulted in this evaluation reported that the PIA program has contributed towards change within themselves. There was evidence that there were shifts in behavior, with both women and men saying that the training had empowered them and given them confidence to act appropriately against VAW in their communities. Awareness had also extended to giving them insight into the negative consequences of VAW, especially on children witnessing violence. “The program helped me to be able to speak out.” (Beneficiary, Wentworth) “My uncle was abusing my aunt. We used to stop violence with violence by hitting him back with my nieces, but after attending the sessions of PIA we realized it was wrong because we were instigating him indirectly. PIA helped because we did not understand the reason of doing that. It affected us a lot because at school we could not concentrate well and my progress was delayed because I am still doing grade 11. My sister is a drop-out because of the situation. I learnt a lot from this project. I refer a person with similar problem to PIA.” (Beneficiary, Khayelitsha) “The project was effective because I was one of those who changed because of it. I was abusing my sister that I am living with emotionally and verbally. I used to come at home very late at night, drunk and demand the money and food that I did not work for to her if she resists I would swear. In my case I was also a heavy drinker and used to beat my wife. She used to go home and come back, but because of the trainings I attended I am a changed person. I had to lead by example. That is the reason why I changed.” (CI, Khayelitsha) Shifts in Engagement During the PIA program, it was clear that PAGs and PACs engaged the community through small, as well as visible community actions to address VAW. These actions were restricted to communities targeted by the program. “Since my involvement, I have gained knowledge and respect in the community as I am also helping other people in the community who have problems.” (CI, Wentworth) “I learned that when your neighbor needs assistance you need to take that initiative to help them and not wait for someone else from another area to help them.” (CI, Inanda) Some people gained skills in how to go about arranging activities in the community. “I learned the administrative skills like to recruit people from different categories and arrange them according to their categories and organize venues for meetings organize t-shirts for them. I also learn to socialize more with people.” (PAC, Khayelitsha) 32 Project Concern International (PCI) UGM End of Project Evaluation As a result of the program’s discontinuation in several locations, it seems that shifts in engagement, particularly VCAs, are unlikely to be maintained. Aside from the data provided by PCI and its partners on their mobilization efforts and the actions taken by communities, there is no concrete data to show how in the life of the program VAW has been reduced. The only evidence that is available are those shared anecdotally, as well as reports from sector partners showing a reduction in demand of their services as it relates to VAW. Some of the sector partners who shared that indeed during the life of the project they saw a decrease of incidences related to VAW were SAPS, Durban Coastal and Mental Health and Rape Crisis. The involvement of a wider network of sector partners also impacted positively on organizations and groups in the community, as they felt that there was a wider network of actors that were working together to address the issue. In the case of VFZs, the amplification and demarcation of boundaries has also resulted in a clean space, improved safety and increased ties with the community in helping to maintain the VFZs. Possibly the most significant change, which could be considered an unintended outcome, was that community members felt more empowered to tackle everyday issues that tore at the fabric of the community. During the interviews, the evaluation teams heard countless accounts of the community taking on issues such as robbery, violence of children towards their parents or grandparents and any other related issues that affected the social cohesion and well-being of the community. In essence, the most significant change in this regard was the fact that ordinary citizens connected to their sense of community and felt that they could take their issues to community structures. This had a deterrent impact on perpetrators of violence and crime in the areas where the PIA program was most active and reduced VAW. However, as noted in areas where the project discontinued, some of these incidences have started recurring. Sector partners noted that the PIA program had increased the capacity of their staff around the issue of VAW. Project Concern International (PCI) UGM End of Project Evaluation 33 V. CONCLUSIONS PCI undertook a four-year social mobilization approach that resulted in the design, development and implementation of the PIA program that was informed by good quality data. It is clear from the literature and the findings that VAW is not restricted to the communities targeted by the PIA program and therefore there is tremendous scope for the PIA program to go to scale. Before further scale up is considered, a number of issues need to be taken into account in order to improve project performance and assure sustainability. Overall, the critique of the project rests on two levels, a project management level, as well as a theoretical level. PROJECT MANAGEMENT LEVEL Project partners are applauded for investing significantly in process and innovation. However, this was not just a research and design process and needed to be balanced against implementing a sound, sustainable strategy to prevent VAW in communities in need. In the future, careful thought is needed to ensure a balance is struck between innovation and implementation and that sufficient time is taken to activate communities to ensure sustained change in behavior to prevent VAW. Whilst this was a four-year project, the majority of the PIA program was implemented in the last two years. While valid reasons exist, this was a short time period for project implementation and scale up in terms of following a strict social mobilization process, especially with regard to a new project that involved new partners. A constraint of such a process was that project partners were introducing and implementing new and innovative activities right up until the end of the project. The donor understood the progressive nature of social mobilization and accepted the low numbers often reported, but the PIA program ultimately still needed to show reach. From a project management perspective, this is challenging for all parties to implement, particularly for the WC Network on VAW that had no additional funds to support ongoing work beyond the end of this grant. As a result, the project ended very quickly for WC partners and substantially undermined the opportunity for the sustainability of the PIA program. For the KZN Network on VAW, the PIA program has some funding, which will sustain the program in the short-term. Given the findings so far, it appears that the PIA program has less features of a social mobilization model and offers more project-based features. It is therefore worth considering what entity is best equipped to implement activities related to the PIA program and that perhaps it could have been managed by a well￾capacitated NGO that is experienced in project management, familiar with the norms, values, customs and culture of the community they are implementing the PIA program in and has technical knowledge with regard to VAW, GBV and HIV/AIDS. Ongoing partnerships with networks on VAW would be crucial and offer access to organizations that could benefit from the PIA program and also expose the program to changes happening in the sector to ensure they remain relevant to the needs of the sector and the communities they serve. Activities under the PIA program were largely focused on VAW, with the exception of the project focused on VFZ. Should the project continue to implement VFZs, it is recommended that strict measures be put in place to ensure that the link to VAW is maintained and not lost. Furthermore, opportunities to discuss and highlight the relationship between HIV in relation to VAW and gender inequality should be identified. VAW is a social norm that needs continuous education, awareness and action. Therefore, targeting young people with appropriate GBV messaging is important as part of an early intervention to prevent VAW. 34 Project Concern International (PCI) UGM End of Project Evaluation Because VFZs is a relatively new initiative under the PIA program, this activity may require greater management to ensure greater focus and safeguarding of PACs, in order to make sure that they were undertaking community safety awareness, rather than policing areas. The evaluation found the MER systems to be innovative, substantial and useful and set high standards for the PIA program. From a capacity development perspective, it also provided partners with the opportunity to learn new approaches to MER and use data as part of their project management and project implementation cycles. THEORETICAL LEVEL PCI and its partners utilized a theory of change that is premised on transforming the norm of inaction to one of mobilizing communities to take action in response to VAW. The core of their strategy has thus included: (1) ongoing research and development; (2) awareness-raising and mobilization; (3) collecting stories of action; and (4) communication and amplification of actions to address VAW. In reviewing the overall strategy to reduce VAW, the following points are worth noting:  There is an absence of strategies to reduce HIV infections long-term. Despite the fact that the funding for this project is linked to PEPFAR funding, the existing strategies have not made the link between VAW and HIV explicit, nor is there any concrete evidence that this will be included in areas where the project will be continuing. As noted earlier, this was a conscious omission by PCI in the design of the PIA program.  The current strategy’s initial entry point was to look at physical and sexual violence which could have been the ideal link to the intersection between VAW and HIV. However, restricting the focus on physical and sexual violence does not include verbal, mental and emotional violence, sexism and hate speech.  In the literature and discussions, there appears to be a shift from explicitly targeting VAW to moving the focus to be on GBV. However, the existing model does not address sexual orientation and gender identities that may put certain groups within the community at risk of violence—he model therefore does not look at the gendered nature of violence at all. This was also apparent in the focus group discussions where increasingly, forms of violence that were mentioned were more generic forms of violence, in particular inter-generational violence from children to their parents.  The model does not look at high-level advocacy, accountability and legislation as a critical entry point for creating more sustainable change. Overall, the approach to addressing VAW did not include any major strategies to address the structural causes of VAW such as substance abuse or unemployment that were widely cited by evaluation respondents. In terms of most significant change at the individual level, all those interviewed expressed that they have been changed through either increases in knowledge, self-confidence and also awareness. At the community level, the issue of VAW has become more visible and actors and actions to address it have been promoted. Sector partners have indicated an expansion of focus, increase in reach and capacity of their staff. Project Concern International (PCI) UGM End of Project Evaluation 35 VI. RECOMMENDATIONS BASELINE RESEARCH AND COMMUNITY ASSESSMENT For future initiatives, it is critical that the baseline research is rooted in the cultural context and entails a deeper exploration of the cultural context and how those contribute to gender relations, gendered power dynamics and how this results in changes in cultural norms, beliefs and behaviors that maintain VAW. The baseline research should also explore with the community ways to confront and create new norms within their cultural contexts to create alternative realities. Additionally, the community assessment will help identify existing services in the community. Engaging stakeholders working in the area of HIV and/or GBV will provide insight into the opportunities for the PIA program in the community. Furthermore, community organizations could emerge that can contribute towards the success of the project by being a sector partner. DEVELOPMENT OF A CONCRETE AGENDA It is important that the communities are involved in the development of an agenda that will remain concrete, clear and transferable over time with strong links maintained to the prevention of VAW. The development of the agenda should be combined with the development of the community capacity to engage in a deeper analysis and connection to gendered power relations and thus the buy-in of the strategic focus on the reduction of VAW and HIV. VFZs need to be evaluated to understand the real and sustained value they offer to the community. In this evaluation, VFZs were relatively new initiatives and the positive spin offs seemed to be located in the prevention of violence, with the prevention of GBV as one aspect of this. Greater monitoring and support is needed to ensure that people involved in VFZs do not place themselves at undue risk. CONSCIOUSNESS-RAISING The training component of future initiatives to enable action has to include the following components: (i) analysis of power and how it contributes to VAW and HIV; (ii) gender training; and (iii) activism strategies (based on best practices and focused on relevance for the context). The approach should focus on creating processes for the community to develop a common understanding of what VAW is and engaging community members in analyzing the root causes of violence and how violence negatively affects their relationships, families and communities. This would then contribute to the change agenda, which comes after the initial consciousness-raising with key actors (a cross-section) in the community. EXPANDING THE STAKEHOLDER BASE AND REACH To increase the scope and reach of violence prevention efforts, a strategy should be developed that demonstrates how a range of different stakeholders would be engaged and contribute to the broader agenda and its outcomes. REVIEWING PREVENTION OF VAW AND HIV IN A HOLISTIC SYSTEM Primary prevention cannot be effective where there is minimal commitment to secondary and tertiary prevention. This would mean that the model has to take into account what is needed to create the political will to (a) implement existing laws or (b) provide the basic services necessary to address the consequences of violence. In the roll-out of the model, consideration needs to be given to how such 36 Project Concern International (PCI) UGM End of Project Evaluation activities and actions may initially be driven by the coordinating organization and lead stakeholders (with capacity-building in the community)can transition to eventually being championed by community groups themselves. ASSESSING IMPACT In order to understand the true impact of the investment made of resources (financial, technical and human), it is critical that an impact assessment is conducted to understand what has shifted and to what extent (beyond the anecdotal evidence), so as to ascertain whether the outputs are equivalent to the inputs. SUSTAINABILITY INFRASTRUCTURE More attention needs to be given to the sustainability aspects of the model. The model is currently heavily dependent upon the presence of PCI or the Networks and their financial and technical support. If such an initiative is to be implemented again, it is critical that pre-project roll-out research goes into considering what would be the best infrastructure to sustain activities and the investments made. There is considerable research that has been done on different models that have worked. These models could form a basis for initial discussions and through community engagement would refine and shape the ultimate infrastructure that emerges to continue the activities. Program activities would be implemented simultaneously to the capacity-building and strengthening of community infrastructure to ensure that the work and agenda lives on long after actors such as PCI and the Networks may leave the communities. PROGRAM DESIGN  From a strategy perspective, greater consideration needs to be given to who is responsible for implementing the PIA program, at all levels, to ensure the project is well implemented and sustainable. Consideration needs to be given to what type of entity is best placed to be an implementing partner and is responsible for managing the overall implementation of the PIA program. Whilst networks offer tremendous advantages, they could contribute their expertise through other structures, for instance through a project reference group.  There should be greater clarity with regard to partner roles to ensure that responsibility is clearly allocated and processes are in place to support areas of weakness. Clear vertical reporting and communication is critical to support the partnerships and ultimately ensure a high quality project is delivered to communities.  Clear criteria needs to be developed and adhered to when recruiting CEs and CIs in order to ensure that trained CEs and CIs are retained in the PIA program.  Future training of CEs and CIs should capacitate them to work with their communities to deeply consider and respond to issues that contribute to VAW, rather than just focus on the prevention of VAW and also recognize the link between HIV, VAW and GBV.  Opportunities for refresher training need to be made available to CEs and CIs to support them in sustaining appropriate PIA actions. Mentoring of CEs via sector partners will help entrench and expand their skills, also offering time for reflection, learning and debriefing. These processes should be clearly documented to guide organizations providing such support. Project Concern International (PCI) UGM End of Project Evaluation 37  Developing a mentoring program for CIs to be implemented by CEs, will ensure they are provided with support to take what they have learned in the training and implement it according to the PIA program guidelines and standards.  Refresher training, mentoring and debriefing may support the retention strategies to keep CEs and CIs active in the program. Beyond this, strong monitoring systems need to be in place to understand the reasons for high or low performance of CEs and CIs to help inform future recruitment criteria.  The evaluation highlighted the potential to adapt the PIA program to work with adolescents. Adapting the content to ensure its relevance and training a cadre of CEs and CIs could have a positive influence on the school environment, the community and the home. Furthermore, adolescents have the opportunity to learn positive behaviors and explore alternative and peaceful ways to express frustration and negotiate challenging situations.  The PIA program should explore making provision for different target groups based on gender, age, sexual orientation and people living with disabilities.  The PIA program could be implemented at substantially lower cost than that spent in this project by using existing community infrastructure to target sector partners and their staff to integrate and mainstream the PIA program into their existing programs. Although this may be time-consuming it would ensure sustainability. Furthermore, inexpensive branding could ensure the brand is still visible.  From the outset, clear agreements need to be in place with regard to the budget and associated budgetary line items especially in relation to branding and incentives. The provision of incentives need to only reward actions and be offset against the personal and community benefits contributed by the PIA program. Greater investment should be made in sector partners rather than in individuals, to ensure the sustainability of sector partners in furthering the amplification of the PIA program. This will further increase community accountability.  In the pre-planning phase, the program needs to develop a realistic, modest and comprehensive long-term budget that should be secured to ensure the PIA program has time to take root in the community. Financial resources can be secured from multiple sources and partners and can also include non-monetary resources such as venues, equipment, transport, etc.  Consultation needs to be applied to the use of the PIA brand, logo and slogan. Careful thought needs to be given to the use of the current brand to ensure that the brand is sufficiently inclusive, so that when it is used across the different PIA activities, it remains relevant and meaningful. CC&DW recommends that inexpensive branded items are identified to ensure the brand is visible in the community to ensure it is recognized by the community and that people are able to articulate the correct meaning. 38 Project Concern International (PCI) UGM End of Project Evaluation APPENDICES APPENDIX 1: COMPOSITION OF EVALUATION TEAM Director Lindy Briginshaw has a BA degree in Development Administration and International Politics from the University of South Africa and is currently completing a MPhil in Community and Development at the University of Stellenbosch. Lindy has 20 years professional experience, specifically in project planning, coordination, management and evaluation of large complex projects involving multiple beneficiaries and stakeholders. She is highly skilled in managing teams, budgets and timeframes. She has significant experience in development communications and marketing and producing communication products, campaigns and strategies. She has extensive socio-political and government relations experience, having worked at local government level for Greater Durban Metropolitan Negotiating Forum during the first democratic local government elections, prior to being employed in a communications and project management role for five years at the European Union funded Parliamentary Support Program. This was an ambitious legislative capacity-building program, providing support to Parliament and Provincial Legislatures. She was also involved in developing public participation campaigns and communication strategies and materials and overseeing the development of a number of research publications on best practice in the sector. She was also tasked with a National Plain Language Project and initiating the upgrade of the Library of Parliament. Since establishing Creative Consulting & Development Works in 2003 as the founder, she has taken the lead in all projects in a project oversight role. She has also taken on various specialist consultancy assignments during this time. Her areas of expertise include strategic planning and management, organizational development, communications and social entrepreneurship. Lead Evaluator Caroline Wills holds a Bachelor of Social Science degree in Social Work and Master of Public Health degree from the University of Cape Town. In addition to her tertiary education she has a certificate in Advanced Health Management, co-certified by the Foundation for Professional Development and Yale University. From 2008-2009, Caroline received training on Results Based Management (RBM) offered by UNAIDS Regional Support Team for East and Southern Africa and the Technical Support Facility, Southern Africa. Caroline has worked in the South African public health environment for the past 12 years, designing, implementing, managing and evaluating projects. Using her public health post-graduate degree, she has focused her work on strengthening the health policy and program arena, specifically in HIV and AIDS and TB. She has strong technical skills in project management, qualitative research, strategic planning and program evaluations. She has well developed management skills from working for six years on a USAID/PEPFAR funded project, where she managed all aspects of a project including, staff, finances, reporting obligations and importantly, key programs, to ensure that projects reach their objectives. In terms of consulting services, Caroline has spent the last six years working with a range of funders, NGOs, tertiary institutions and public sector partners including; NACOSA, Mindset, Marie Stopes South Africa, Desmond Tutu TB Centre at Stellenbosch University, University of the Western Cape, University of Stellenbosch, Health Policy Initiative, Department of Health and the Department of Social Development. Caroline is also the Chairperson of Khululeka Grief Support, an organization committed to building the resilience of children who have experienced grief and loss. Project Concern International (PCI) UGM End of Project Evaluation 39 Technical Expert Shamillah Wilson has significant experience in strategy development and formulation. She has a background in social studies and has had extensive involvement in community and youth development. Furthermore, Shamillah is an expert in strategy development. One of her core competencies is developing planning skills and her work in this field primarily involves planning and facilitation of strategic planning processes. In addition, Shamillah is well trained in identifying and putting in place monitoring and evaluation systems. Her professional experience is complemented by the Master of Business Administration degree she holds from the University of Liverpool in the UK and her BA degree in Social Research and Religious Studies. Shamillah is experienced with community development skills and training. Her qualifications in social research in combination with her MBA empower her to deliver quality services, particularly in the field of youth development strategizing. For example, she has been responsible for developing mechanisms to engage young women in defining women’s rights and development and contributing to the sustainability of women’s movements internationally. She has had numerous program development and implementation roles in various companies and organizations and has been instrumental in strategy planning, drafting policies and content development. She is also experienced with risk analysis and other diagnostic services. In terms of consulting services, Shamilla has been a Strategic Advisor and Associate for the Movement Building Initiative for Southern Africa. This initiative aimed to build capacity, impact and voice for women infected and affected by HIV and AIDS and to empower them to participate in responses to the pandemic in their countries. As part of the project, Shamillah conducted needs assessments in Malawi and Zambia, which entailed consulting with various national stakeholders (including government, bilateral companies, community service organisations and HIV/AIDS women’s groups). The Assessments also entailed a scan of facilitating policies and frameworks and an examination of economic, social and political issues related to women’s responses to HIV and AIDS. Shamillah is currently an independent consultant and entrepreneur. She is a founding board member of the Youth Against AIDS Network, a regional network of youth leaders in Africa; a founding member and Chairperson of the Sonke Gender Justice Network; a member of the Advisory Council for the Africa Region for the Global Fund for Women; and the Director of the Learners' Network, a group that develops youth leadership to engage them in the struggle against HIV/AIDS. Technical Expert Dr. Dee Smythe is Director of the Law, Race and Gender Research Unit and an Associate Professor in the Department of Public Law. Prior to joining the Faculty of Law in 2009 she was Principal Researcher at the Gender, Health and Justice Research Unit in UCT’s Faculty of Health Sciences. Dee was a Fulbright Fellow at Stanford Law School in 2004. Her research spans a range of areas at the intersection of law, policy and social justice. It is particularly concerned with producing methodologically rigorous empirical data on the operation of the legal system, as a basis for evaluating the effectiveness of and supporting legal and policy interventions. She is an expert on state responses to gender-based violence, with a specific focus on sexual offences. She has also written on HIV/AIDS, crime prevention and police transformation and convened the African Network of Constitutional Lawyers’ focus group on Women, Equality and Constitutionalism. In 2009, Dee was awarded the UCT College of Fellows’ Young Researcher Award for outstanding scholarly work by a young academic. In the same year the work of the Gender, Health and Justice Research Unit was recognized with UCT’s inaugural Social Responsiveness Award. In 2011 the Law, 40 Project Concern International (PCI) UGM End of Project Evaluation Race and Gender Research Unit’s Rural Women’s Action Research Project was recognized with UCT’s Social Responsiveness Award. Dee is on the Board of Trustees of the Rape Crisis Cape Town Trust. Research Coordinator Susannah Clarke has a BA Humanities degree, with majors in Psychology and Sociology and an Honors degree in Sociology, both of which she obtained from the University of Stellenbosch. She is currently completing her Masters in Social Justice at the University of Cape Town. Susannah works for Creative Consulting & Development Works as a full-time social science researcher and coordinator of the company's research team. Susannah provides a sound theoretical background, as well as both qualitative and quantitative research expertise. Susannah has worked intimately on various projects commissioned to Creative Consulting & Development Works. She completed an accredited life skills facilitation program, offered by the Umsobomvu Youth Fund. Susannah has also completed several projects for the Department of Transports and Public Works, Expanded Public Works Program and the Department of Economic Development and Tourism. These projects have involved both report writing and statistical analysis of survey data. She has also managed various public health evaluation studies for clients and has strong monitoring and evaluation skills. While studying at Stellenbosch University Susannah, as a co-leader of two teams, designed and implemented two community development initiatives at two schools within the greater Stellenbosch area. Prior to her appointment with Development Works, Susannah volunteered for People Against Suffering Suppression Oppression and Poverty (PASSOP). Researcher Paul Dube is a qualified Researcher in legal, civic and social arenas. He has an excellent knowledge base in the fields of Law, Politics and Criminal Justice. His specialities lie in legal, civic and social research in areas of Law, Politics, Social Development, Social Policy Analysis, Governance and Security, Crime, Criminal Justice, Criminology, Police and Policing, Prisons, Restorative Justice, Nodal Governance and Environmental Governance. Paul has an extensive academic research background bridging between socio and legal research. Paul has conducted research for independent organizations such as Rape Crisis Cape Town Trust assessing challenges confronting service delivery for male rape survivors. For Extra-Mural Education Project (EMEP), he designed a research project and conducted an audit of security mechanisms employed to protect learners at a high-risk school in the Western Cape. In addition, Paul held the position of Security Governance Observer/Researcher at the Centre of Criminology, Faculty of Law and University of Cape Town. Since joining Creative Consulting and Development Works, Paul has worked on a variety of research projects with the most recent being an evaluation of a five-year EU-funded intervention at 11 select Marie Stopes South Africa sites. Paul holds a Bachelor of Social Science degree in Law and Politics from the University of Cape Town. He also completed his Honours and Master’s degrees in Criminal Justice from the University of Cape Town and is currently obtaining his PhD. He has produced a number of academic research working papers and theses. Project Concern International (PCI) UGM End of Project Evaluation 41 Researcher Candice Mitchell has a Bachelor of Social Science degree from the University of Cape Town. She holds a Masters in Applied Medical Anthropology from Wayne State University in Michigan, where she graduated with top honors. Candice is planning on continuing her studies in the near future with a PhD in International Health and Development. Her main focus of research has been to examine the various social, political and economic contexts of infectious diseases in South Africa, with particular emphasis on treatment. Candice has a strong understanding of the current public health landscape throughout South Africa and continuously attends conferences, seminars and workshops , as well as conducting her own research in order to keep abreast of developments within this sector. Throughout her studies and work, Candice has developed both quantitative and qualitative research skills; as well as strong analytical skills. Since joining Creative Consulting & Development Works, she has been involved in numerous evaluation projects, particularly within the public health and education sectors. She is currently working on developing her monitoring and evaluation skills. Candice has both international and national work experience. She has worked as a clinical trial research assistant for the Michigan Institute for Neurological Disorders (MIND), an assistant lecturer, as well as a research assistant for professors at whilst at university. Candice is also passionate about education and has spent a few years working as a teacher and tutor throughout the world. She steadfastly believes in the empowerment that education brings and tutoring is close to her heart. Candice has worked with tutoring autistic children in reading and writing, as well as life skills and donates her time to tutoring English to underprivileged children. Researcher Stefanie Schulein has a BA in International Studies (Cum Laude) and a master’s degreein Political Management (Cum Laude) from Stellenbosch University. She has worked for the Centre for International and Comparative Politics (Stellenbosch University) as a researcher and coordinator where she was involved in an international project tracking social values and development trends through survey research. She has completed three years towards a Medical Degree at UCT and has an active interest in global health issues. She has had extensive training in monitoring and evaluation methods and is capable in the areas of questionnaire design, quantitative analysis, integration of research inputs and report writing. She conducted an evaluation of cervical cancer screening policy at Nolungile Clinic in Cape Town and presented the findings to health workers in Khayelitsha. She also conducted an audit of post caesarean section pain management at Somerset Hospital and presenting findings at the UCT Anesthetics Department Research Day. She has also presented papers at various conferences and is fluent in English, German and Afrikaans. Fieldworker Thabisile Nkosi holds a BA in Geography and Environmental Management from the University of KwaZulu Natal (UKZN) and is currently in the second year of her Master’s in Town and Regional Planning (UKZN). Between 2011 and 2012 she worked as a fieldworker at the University of Cape Town’s Law, Race and Gender Research Unit. Her key responsibilities included training of volunteers, capacity-building, conducting follow-up interviews and liaising with the Traditional Council secretary and Headmen. Thabisile worked as a NEAS Administrative Clerk at the Department of Environmental Affairs (Gauteng 42 Project Concern International (PCI) UGM End of Project Evaluation region) between October 2011 and December 2011 and interned at the Gauteng Department of Agriculture and Rural Development (2010 to 2011). Here her responsibilities included reviewing and evaluating Environmental Impact Assessments (EIA). Thabisile is a member of Timberwatch NGO Coalition, which is an environmental and development organization. Through it she has been able to apply her environmental studies and evaluation skills. She has a strong interest in social and economic justice issues concerning poor and marginalized communities. Her key areas of interest include environment and planning, public interest research and social and economic analysis. She is fluent in English and Zulu. Research Intern Karin Nandoo is currently a Masters Student in Social Science in Development Studies at the University of the Western Cape (UWC). Her thesis seeks to explore the livelihood activities in female-headed households in Vrygrond, Cape Town. The objective of this study is to identify and analyze livelihood strategies adopted by female-headed households. Karin also holds a BA Psychology Honors degree from the University of the Western Cape. She has worked as an Assistant Conference Coordinator for one of South Africa's leading conference organizing companies in Cape Town. At the Faculty of Arts, University of the Western Cape Karin undertook the role of student assistant, offering mentorship to undergraduate and post-graduate student with the smooth transition into the academic careers. Karin demonstrates a proactive and meticulous approach in the execution of her responsibilities. She has a proven ability to work independently or as a member of a team. She aspires to work in an interactive and cooperative environment where personal growth and development is strongly promoted, with a high degree of respect and cooperation. Karin is highly motivated and able to take initiative and has sound planning, time management and organizational skills. Project Concern International (PCI) UGM End of Project Evaluation 43 APPENDIX 2: SAMPLING FRAMEWORK AND LIST OF SITES VISITED Sampling Framework and List of Sites Visited Method Western Cape (planned) Western Cape (actual) Data collected November 19-23, 2012 KwaZulu-Natal (planned) KwaZulu-Natal (actual) Data collected: November 19-23, 2012 Key Informant Interviews with Network Staff i. WC Network Director ii. Program Manager (also MER Manager) iii. Program Officer All complete i. KZN Network Director ii. Program Manager iii. MER Manager All complete Key Informant Interviews with Sector Partners Four Sector Partners: i. Joy Christian Church ii. Rape Crisis, Khayelitsha iii. Nonceba iv. DKTSA Three of the four complete. Nonceba could not be reached to schedule an interview. Seven Sector Partners: i. World Council for Religion and Peace ii. DoJ iii. Durban Coastal Mental and Health iv. Wentworth Organisation of Women v. Wentworth Victim Friendly Centre vi. South African Police Service (SAPS): Hillcrest vii. SAPS: KwaMakhutha Six of the seven complete. SAPS KwaMakhutha could not be reached to schedule an interview. 44 Project Concern International (PCI) UGM End of Project Evaluation Khayelitsha (planned) Khayelitsha (actual) Inanda (planned) Wentworth (planned) Inanda (actual) Wentworth (actual) Key Informant Interviews with other Sector Players Two Key Informant Interviews Two Key Informant Interviews completed Two Key Informant Interviews Two Key Informant Interviews Two Key Informant Interviews completed Two Key Informant Interviews completed Key Informant Interviews with Individual Program Beneficiaries Five Key Informant Interviews Five Key Informant Interviews completed Five Key Informant Interviews Five Key Informant Interviews Three Key Informant Interviews completed Five Key Informant Interviews completed Focus Groups CEs Four CE Focus Groups Four CE Focus Groups completed OneCE Key Informant Interview completed One CE Focus Group Two CE Focus Groups One CE Focus Groups completed Two CE Focus Groups completed Focus groups with CIs: from non- Violence Free Zones Two CI Focus Groups Two CI Focus Groups completed Two CI Focus Groups Two CI Focus Groups Two CI Focus Groups completed Two CI Focus Groups completed Focus Groups with CIs: from Violence Free Zones Two CI Focus Groups Two CI Focus Groups completed Two CI Focus Groups Two CI Focus Groups Two CI Focus Groups completed Two CI Focus Groups completed Focus Groups with PAGs: from non-Violence Free Zones Two PAG Focus Groups Two PAG Focus Groups completed Two PAG Focus Groups Two PAG Focus Groups Three PAG Focus Groups completed Two PAG Focus Groups completed Focus Groups with PAGs: From Violence Free Zones Two PAG Focus Groups Two PAG Focus Groups completed Two PAG Focus Groups Two PAG Focus Groups One PAG Focus Group completed Two PAG Focus Groups completed Project Concern International (PCI) UGM End of Project Evaluation 45 Prevention in Action Committee (PAC) One PAC Focus Group Three Key Informant Interviews conducted with members of PACs One PAC Focus Group One PAC Focus Group One PAC Focus Group completed One PAC Focus Group completed Method Scheduled Interviews Comment Key informant interviews with PCI Staff PCI SA Country Director: Clayton Davis Completed on December 4, 2012 Technical Consultant: Warren Parker Completed on December 4, 2012 PCI Vice President for Strategic Information and Impact: Clara Eder Completed on November 27, 2012 Finance Manager: Ben van der Linde Completed on December 4, 2012 Communication Lead: Gil Lang Completed on December 4, 2012 46 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE APPENDIX 3: DATA COLLECTION TOOLS 1. Key Informant Interview Guide with PCI Staff About the Research and Development Process 1. Can you describe your role in PCI in relation to this project? 2. When PCI was awarded the grant for this project, what was the process followed to put together an implementation plan (work plan)? 3. Would you define this process as a research and development approach? If so, why? 4. Was this process applied throughout the life of the project? Please provide details of different stages where this approach was applied. 5. Based on this approach, how did you allocate resources to ensure that this approach was a central component of how this project was implemented? 6. Through the life of the project, how did the Research and Development process explore socio-cultural relevance, cultural beliefs, practices and local community norms? 7. How were these then taken into account in the development of the social mobilization strategy? 8. What were the innovations that came about as a result of the Research and Development process? 9. Through the Research and Development process, what evidence has PCI collected that shows that VAW has been reduced? 10. At the end of the project cycle, would PCI recommend an approach that centres on the Research and Development for addressing violence against women? Is there anything that you would change? Implementation of Social Mobilization Strategy 11. What were the different activities implemented as part of the strategy? 12. How did these relate to the work plan submitted to FHI 360-UGM/USAID? 13. Did PCI achieve the project indicator as laid out in the work plan/agreements with FHI 360- UGM/USAID? 14. What were the roles of each of the project partners (PCI and 2 networks)? Probe: How were these roles carried out? 15. What capacity-building, general and supervision support was provided by PCI to ensure that people at different levels of the project were well trained to manage and implement the social mobilization strategy? 16. What were some project constraints in implementation? 17. What were some of the key enablers that supported project implementation? 18. What were the outcomes of the project (intended and unintended)? Addressing Violence Against Women 19. What strategies were implemented to address violence against women? 20. Which settings were strategies implemented in? 21. What was the reason for the choice of these settings, over and above others – i.e. who was targeted? 22. Did the strategies address all the risk factors associated with VAW? 23. What other prevention activities could have been implemented? 24. What other types of activities are also important in providing a holistic response to VAW (care, support, legal aid, SAPS, etc.)? 25. Do you think that the activities that were decided on for this project provide sufficient exposure for communities to effect changing behavior in relation to VAW? 26. What activities were developed and implemented that address the intersection between VAW and HIV and AIDS? Longer-term Sustainability 27. What activities have continued beyond the end of the project (Sept 2012)? 47 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 28. What are the key enablers/barriers to continuing the project? 29. What was the purpose/value of choosing the Networks as the implementing partners of the project? 30. How did the project contribute towards the long-term sustainability of the Networks to address VAW? 31. What social mobilization activities are the Networks continuing to implement (independent of USAID funds)? 32. To what extent has the community been able to maintain some of the activities put in place by the project? Most Significant Change 33. Identify instances where there has been a significant change with regard to behaviour/actions in preventing VAW from PCIs perspective. 34. Does PCI think that the model that was developed (as it is currently) is a valuable model to share with others to address violence against women? 35. What is the most significant change experienced by PCI during the life of this project? 48 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 2. Key Informant Interview Guide with WC and KZN Networks About Research and Development Process 1. Can you describe your role in the Network in relation to this project? 2. What was your involvement in the process followed to put together an implementation plan (work plan) when the grant was awarded? 3. Would you define this process as a research and development approach? If so, why? 4. Was this process applied throughout the life of the project? Please provide details of different stages where this approach was applied. 5. Through the life of the project, how did the R&D process explore socio-cultural relevance, cultural beliefs, practices and local community norms? 6. How were these then taken into account in the development of the social mobilization strategy? 7. What were the innovations that came about as a result of the Research and Development process? 8. Through the Research and Development process, what evidence has the Network collected that shows that VAW has been reduced? 9. At the end of the project cycle, would you recommend an approach that centres on Research and Development for addressing violence against women? Is there anything that you would change? Implementation of Social Mobilization Strategy 10. What were the different activities implemented as part of the strategy? 11. What did you see as the role of the Networks in implementation of this project? 12. How did the Network understand the role of PCI? How did these roles complement each other? 13. What capacity-building, general and supervision support was provided by PCI to ensure that people at different levels of the project were well trained to manage and implement the social mobilization strategy? 14. What were some project constraints in implementation? 15. What were some of the key enablers that supported project implementation? 16. What were the outcomes of the project (intended and unintended)? Addressing Violence Against Women 17. What strategies were implemented to address violence against women? 18. Which settings were strategies implemented in? 19. What was the reason for the choice of these settings, over and above others – i.e. who was targeted? 20. Did the strategies address all the risk factors associated with VAW? 21. What other prevention activities could have been implemented? 22. What other types of activities do you consider as also important in providing a holistic response to VAW (care, support, legal aid, services provided by the South African Police Service etc.)? 23. Do you think that the activities that were decided on for this project provide sufficient exposure for communities to effect enduring changing behavior in relation to VAW? 24. What activities were developed and implemented that address the intersection between VAW and HIV and AIDS? Longer-term Sustainability 25. What activities have continued beyond the end of the project (Sept 2012)? 26. What are the key enablers/barriers to continuing the project? 27. What was the purpose/value of choosing the Networks as the implementing partners of the project? 28. Do you think that the Networks are the most strategic partners for a project like this? 49 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 29. With the support you have received during this initiative, what have you been able to achieve better or differently as a Network in terms of your own strategic objectives? 30. How has the implementation of this project fit in with your wider goals and other activities? E.g.: 31. How easily did it fit in with your other management processes – e.g. around reporting or learning? 32. Has it helped or hindered you to pursue your own priorities or support different kinds of projects or social change? 33. Has it helped or hindered you to reach ‘hard to reach’/different kinds of beneficiaries and organisations? (What challenges do you face here?) 34. Has it caused confusion among staff or members? 35. Has it taken up more or less of your time than other project models? 36. How did the project contribute towards the long-term sustainability of the Networks to address VAW? 37. Has working on this initiative with the other partners helped you to identify and share lessons on how to address violence against women from an organizational perspective? 38. What social mobilization activities is your Network continuing to implement (independent of USAID funds)? Probe: What made you choose these activities? Who made the decision to continue these activities? Most Significant Change 39. Identify instances where there has been a significant change with regard to behaviour/actions in preventing VAW from the Network’s perspective. 40. Does the Network think that the model that was developed (as it is currently) is a valuable model to share with others to address violence against women? 41. What is the most significant change experienced by the Network during the life of this project? 42. What is the most significant change that you as an individual working on this project experienced during the life of the project? 50 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 3. Key Informant Interview Guide for Sector Partners Introduction/Context 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. What is the extent of the problem here? 3. What are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. What is your organization/institution’s current involvement in addressing violence against women? 5. What from your understanding were the objectives of this project? 6. What motivated your organization/institution to become a sector partner on this project? (In other words, what were your objectives in partnering?) 7. Do you think other communities in Cape TownDurban need this initiative? Why? 8. What was your organization/institution’s role in developing the social mobilization model? Implementation of Social Mobilization Strategy 9. What were different activities has your organization implemented as part of the strategy? 10. What capacity-building, general and supervision support was provided to your organization to ensure that people at different levels of the project were well trained to manage and implement the social mobilization strategy? 11. What do you consider as project constraints in implementation of this project? 12. What were some of the key enablers that supported project implementation? 13. What were the outcomes of the project (intended and unintended) from your perspective? Addressing Violence Against Women 14. What strategies were implemented to address violence against women? 15. Which settings were strategies implemented in? 16. What was the reason for the choice of these settings, over and above others – i.e. who was targeted? 17. Did the strategies address all the risk factors associated with VAW? 18. What other prevention activities could have been implemented? 19. What other types of activities do you consider as also important in providing a holistic response to VAW (care, support, legal aid, services provided by SAPS etc.)? 20. Do you think that the activities that were decided on for this project provide sufficient exposure for communities to effect enduring changing behavior in relation to VAW? 21. What activities were developed and implemented that address the intersection between VAW and HIV and AIDS? Longer-term Sustainability 22. What activities in relation to the social mobilization model have your organization/institution continued to implement beyond the end of the project (Sept 2012)? 23. What are the key enablers/barriers to continuing the project? 24. Do you think that the Networks are the most strategic drivers of such a project? 25. With the support you have received during this initiative, what have you been able to achieve better or differently as a sector partner in terms of your own strategic objectives to address violence against women? 26. How has the implementation of this project fit in with your wider goals and other activities? E.g.: 27. How easily did it fit in with your other management processes – e.g. around reporting or learning? 28. Has it helped or hindered you to pursue your own priorities or support different kinds of projects or social change? 29. Has it helped or hindered you to reach ‘hard to reach’/different kinds of beneficiaries and organisations? (What challenges do you face here?) 51 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 30. Has it caused confusion among staff or members? 31. Has it taken up more or less of your time than other project models? 32. How did the project contribute towards your organization/institution’s longer-term vision of addressing VAW? 33. Has working on this initiative with the other partners helped you to identify and share lessons on how to address violence against women from an organizational perspective? 34. What social mobilization activities is your organization continuing to implement (independent of this project)? Most Significant Change 35. Identify instances where there has been a significant change with regard to behaviour/actions in preventing VAW from your organization/institution’s perspective. 36. Do you think that the model that was developed (as it is currently) is a valuable model to share with others to address violence against women? 37. What is the most significant change experienced by your organization during the life of this project? 38. What is the most significant change that you as an individual working on this project experienced during the life of the project? 52 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 4. Key Informant Interview Guide for Other Sector Players Introduction/Context 1. What do we mean by violence against women? 2. Why is a focus on violence against women important? 3. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 4. How big is the problem here? 5. What are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 6. What is your organization/institution’s current involvement in addressing violence against women? 7. From your experience, what does a holistic approach to addressing VAW look like? 8. Could you share examples of institutional based responses to violence against women? What is your assessment of these? 9. Could you share examples of community-based responses? What is your assessment of these? 10. What are good indicators for assessing whether an initiative has effectively addressed violence against women? 11. What do you consider are some of the risks in developing community-based responses to addressing violence against women? 12. Do you know about the social mobilization model implemented by PCI and the KZN/WC Networks to address VAW? If yes, what do you know about it? If no, provide brief overview of the model. 13. What community services or resources do you think could enhance the effectiveness of the model? 14. What do you think would need to be considered when implementing such a model? (Consider enablers and inhibitors) 15. What are some of the challenges in getting shifting definitions, behavior and engagement of communities around VAW? 16. Do you have anything else to add in relation to responses to violence against women? 53 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 5. Key Informant Interview Guide for Program Beneficiaries General Questions 1. Do you know about the Prevention in Action (PIA) project? 2. Were you involved in the project? How? 3. What are your observations of the Prevention in Action (PIA) program? What do they do? Is it important? Why/why not? 4. Has this program helped you? How did it help you? Are you happy with the outcome? If you had another problem like this, would you go to the Community Influencers or Prevention in Action Groups again? If not, who would you go to? 5. What is the most significant change that happened to you as a result of your engagement or involvement with PIA? 6. Do you know of other processes the project is using that address violence against women? 7. What is your opinion of the effectiveness of the project? Probe: What contributed to any successes you have observed? What challenges have you observed? 8. Are there any organizations doing similar work? Is the work of the PIA project the same or different? 9. Have you seen the Community Influencers or Prevention in Action Groups involved in other situations such as yours? In what types of situations have they been involved? 10. Do you think PIA should be continued? Do you think you would do something differently in the future or the same? Why and how? What are the best practices from this process you observed? 54 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 6. Focus Group Discussion Guide with Community Engagers (CEs) Introduction/Contextual Information 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. How big is the problem here? 3. What are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. Do you think other communities in Cape Town/Durban need this initiative? Why? Questions about being a Community Engager (CE) 5. Why did you sign up to become trained as a CE? 6. If you think about the training you received, which part of the training was the most helpful to you in your work as a CE? 7. What in the training do you think needs to be changed? 8. What did you learn in the training that was easy to implement? Why? 9. What did you learn in the training that was difficult to implement? Why? 10. Was there anything else that you think the training should have covered? 11. Is there anything else that could have been provided by the project that would have helped you deliver your work better? 12. What are the most important things you have learnt from your experience as a CE? 13. Do people in this community talk about the project? What do they say? Questions about the Community Influencers (CIs) 14. What kinds of people did you approach to become CIs? Why? How did you recruit them? 15. What did you expect CIs to do with the skills/information you gave them? 16. What kinds of activities did they implement to prevent VAW? 17. Please give examples from your work with CIs and/or the community where you have seen the most significant change in terms of preventing VAW. 18. Have you any suggestions for improving the project and/or reducing VAW in this community? 55 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 7. Focus Group Discussion Guide with Community Influencers (non-VFZ) Introduction/Context 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. How big is the problem of VAW in your community? 3. What are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. Do you think other communities in Cape Town/Durban need this initiative? Why? Experience as a CI 5. Why did you sign up to become a community influencer (CI)? 6. What do you think the project is trying to achieve? 7. What did you think about the quality of the training by the community engagers? Explain. 8. What did you learn in the training that was easy to implement in your community? Why? 9. What did you learn in the training that was a challenge to implement in your community? Why? 10. Was there anything else that you think the training should have covered? Overall Thoughts on the Impact of the Project 11. What have you personally gained by being a CI? 12. What in this project has made the greatest difference to you/your family/your community? 13. Please give examples from your work with the community where you have seen the most significant change in terms of preventing VAW. 14. What do other people say about the project? 15. Have you any suggestions for improving the project and/or reducing VAW in this community? 56 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 8. Focus Group Discussion Guide with CIs (VFZ) Introduction/Context 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. How big is the problem of VAW in this community? 3. What do you think are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. Do you think other communities in Cape Town/Durban need this initiative? Why? Experience of Setting up the Violence Free Zones 5. Please explain how you created your Violence Free Zone. What was the process? Probe: How did you get consensus and buy in from the community? 6. What is your Violence Free Zone trying to achieve? 7. Do you implement activities in the Violence Free Zone to spread the message? 8. How significant is the use of the logo showing that an area is a Violence Free Zone? 9. Do you think the Violence Free Zones have been successful? What factors do you think helped/prevented the Violence Free Zones to be successful? Overall Thoughts on the Impact of the Project 10. What has been the most significant change you have seen in preventing VAW (might need to adapt this if the group have only reduced general violence and not VAW)? 11. What have you personally gained by being a part of this initiative? 12. What do other people in the community say about the Violence Free Zones? 13. Have you any suggestions for improving the project and/or helping to reduce VAW in this community? 57 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 9. Focus Group Discussion Guide for PAGs Introduction/Context 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. How big is the problem here? 3. What are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. Do you think other communities in Cape Town/Durban need this initiative? Why? Questions about PAGs 1. What is the role of the PAG in addressing violence against women? 2. How does the PAG function? 3. Which people are invited to join a PAG? 4. What training and/or support were provided to the PAG? 5. Which part of the training was the most helpful to you in your work as a PAG? 6. What in the training do you think needs to be changed? 7. What did you learn in the training that was easy to implement? Why? 8. What did you learn in the training that was difficult to implement? Why? 9. Was there anything else that you think the training should have covered? 10. What are some of the challenges that you encounter as a PAG? 11. What do you think is needed to ensure that the PAG continues it work? 12. What are some of the actions or activities that the PAG has organized to prevent VAW? Please give examples? 13. Has your PAG set up a Violence Free Zone? 14. If yes, explain how you created your Violence Free Zone. What was the process? Probe: How did you get consensus and buy in from the community? What is your Violence Free Zone trying to achieve? 15. If no, do you think the Violence Free Zones are a good way to prevent violence against women? What factors do you think would help the Violence Free Zones to be successful? Overall Thoughts on the Impact of the Project 1. What has been the most significant change you have seen in preventing VAW (might need to adapt this if the group have only reduced general violence and not VAW)? 2. What have you personally gained by being a part of this initiative? 3. What do other people in the community say about the project? 4. Have you any suggestions for improving the project and/or helping to reduce VAW in this community? 58 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE 10. Focus Group Discussion Guide for PACs Introduction/Context 1. Why do you think it is important to address violence against women in Khayelitsha/Inanda/Wentworth? 2. How big is the problem of VAW in this community? 3. What do you think are some of the factors that contribute towards the problem of VAW in Khayelitsha/Inanda/Wentworth? 4. Do you think other communities in Cape Town/Durban need this initiative? Why? Experience of Setting up the Violence Free Zones 5. Please explain how you created your Violence Free Zone. What was the process? Probe: How did you get consensus and buy in from the community? 6. What is your Violence Free Zone trying to achieve? 7. Do you implement activities in the Violence Free Zone to spread the message? 8. How significant is the use of the logo showing that an area is a Violence Free Zone? 9. Do you think the Violence Free Zones have been successful? What factors do you think helped/prevented the Violence Free Zones to be successful? Overall Thoughts on the Impact of the Project 10. What has been the most significant change you have seen in preventing VAW (might need to adapt this if the group have only reduced general violence and not VAW)? 11. What have you personally gained by being a part of this initiative? 12. What do other people in the community say about the Violence Free Zones? 13. Have you any suggestions for improving the project and/or helping to reduce VAW in this community? 59 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE APPENDIX 4: SCOPE OF WORK Detailed Task Description: Now in the fifth and final year of the project, FHI 360-UGM, at the request of USAID, is commissioning an external evaluation of our grantees. Partner organizations are non-governmental organizations (NGOs) working at national, provincial and local levels in South Africa, primarily implementing services related to services for orphans and vulnerable children (OVC), HIV care and support, HIV counseling and testing, and HIV prevention. These partners have received funding for a period of three to five years under PEFPAR, as well as both organizational and technical capacity building support. Creative Consulting is being contracted to execute the evaluation for Project Concern International (PCI), one of the UGM partners. The focus the project evaluation will be to:  Determine whether the program objectives under the PCI program under UGM were achieved  Evaluate the key program outcomes (both intended and unintended outcomes) Determine the key programmatic enablers and barriers to the success of the program (including the Violence Free Zones) in reducing the prevalence of physical and sexual violence against women most vulnerable to HIV infection in the KwaZulu-Natal and Western Cape provinces.  Determine the potential for continuity of the work done by networks under the program. The focus of the evaluation is to assess PCI’s effectiveness in addressing the needs of beneficiaries in targeted communities. The evaluators will be required to carefully consider the suitability and feasibility of design options that are likely to offer the best chance of establishing the value of the program in responding to the needs of targeted beneficiaries and communities. Both qualitative and quantitative data collection techniques should be employed. Data will be collected from various sources using appropriate data collection methods and tools for any given evaluation question. The final design to be employed will be determined after the contractor has had a chance to undertake a front-end analysis and is therefore able to select the best design option that specifies; which people or units will be studied; how they will be selected and the kinds of comparison that should be made. Data will be collected from three program sites for the partner. The evaluation will be undertaken in two stages and with expected outcomes for each stage as expressed below: Following award of the contract and the initial kick-off meeting with PCI and USAID on October 10 and 11, the consultants will refine an evaluation protocol which demonstrates:  Understanding the relationship between program stages and the proposed broad evaluation question  Understanding the context for program delivery and key factors that influence program implementation  Understanding the existing theoretical and empirical knowledge about the program and examining program theory  A comprehensive stakeholder analysis and determination of roles of key stakeholders in the evaluation  Balancing costs and benefits of the evaluation and advising on the most strategic questions to include in the evaluation  Developing the detailed evaluation protocol 60 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE Key Stage 1 Deliverables:  Key evaluation questions and linkages to program theory  Stakeholder analysis including their roles in the evaluation  Evaluation approach, design and sampling methods  Key measures and data collection tools to be used  Data analysis strategy including primary units for comparison  Evaluation work-plan including key activities and timeframes Stage 2: Implementation of the Evaluation The consultants will implement the partner evaluation following submission and approval of the detailed evaluation protocol:  Pre-test instruments  Train data collectors  Undertake the evaluation data gathering process  Develop field work notes  Prepare data for analysis  Clean data  Enter data into electronic data analysis systems  Undertake comprehensive data analysis  Formulate the findings Key Stage 2 Deliverables: Consultants will be required to prepare a range of reports on the findings of the evaluation and to participate in the provision of feedback and dissemination of key findings  Detailed written report including an executive summary with highlights of the evaluation and key findings (Draft due December 14, final on February 1)  Power Point Presentation providing summary of evaluation process and preliminary results (due January 9)  Brief paper (two-pager) targeting community audiences on the key findings from the evaluation (February 1) During the period of performance of October 5, 2012 to March 8, 2013, payment to Creative Consulting will be fixed price based on the receipt and FHI 360 technical representative’s written satisfaction/approval* with the deliverables listed below: Milestones Deliverables Due Date 1) Protocol Finalization Evaluation Work Plan, including key activities and timeframes submitted to FHI 360 Oct 15, 2012 Final Implementation strategy (evaluation approach) & methodology document Oct 26, 2012 Final evaluation protocol including key evaluation questions (and linkages to program theory), design and sampling methods, analysis of stakeholders and their role in the evaluation. Oct 26, 2012 61 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE Draft of all the data collection tools to be used based upon key evaluation questions. Oct 26, 2012 2) Data analysis strategy and IRB approval Data analysis strategy including primary units for comparison Nov 2, 2012 Obtain local/SA IRB approval Nov 2, 2012 3) Data collection (~Nov 5 to Dec 14) * Cannot start data collection w/o FHI 360 approval of tools Fieldwork training report highlighting training objectives, powerpoints and key challenges and successes from training Nov 9, 2012 Finalized data collection tools with the changes made from piloting- FHI to provide feedback by Nov 13 Nov 9, 2012 Final fieldwork notes from data collection phase. Nov 9, 2012 4) Detailed draft written report (including an executive summary with highlights of the evaluation and key findings) Draft written report due (FHI will provide feedback by January 25) Jan 11, 2013 5) Powerpoint Presentation providing summary of evaluation process and preliminary results Draft, copy-edited Powerpoint presentation submitted to FHI 360 Jan 18, 2013 Presentation by Creative to USAID and Partners (including FHI 360) **Subject to availability of USAID Jan 24, 2013 6) Second draft of report including an executive summary with highlights of the evaluation and key findings Second draft of report, incorporating feedback from first draft and presentation (FHI will review and provide feedback, aiming to have a final, approved report by February 26) February 15, 2013 7) Development of brief paper (two￾pager) for the partner, targeting community audiences on key findings from the evaluation, and submission of final report including an executive summary with highlights of the evaluation and key findings Two-page paper draft submitted to FHI 360 (FHI 360 will review and provide feedback within 2 business days) February 26, 2013 Final copy-edited two pager submitted to FHI 360 March 8, 2013 Final copy-edited evaluation report, incorporating feedback, submitted to FHI 360 March 8, 2013 62 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE APPENDIX 5: MER PIA DOCUMENTS PCI Prevention In Action Program to Reduce Violence Against Women MER PIA Documents October 2008 to January 2013 Date Document Title Comments 1 Jul-09 VAW Communications Strategy - July 2009 Incorporates Analysis Phase Findings 2 Oct-09 Pre-test findings for Billboard concepts One of several pre-testing reports 3 Feb-10 FINAL Findings of the Changing Faces Campaign Evaluation of 2009 16 Days Billboard Campaign 4 Mar-10 PCI Panel Survey Report - FINAL Findings from Baseline Survey 5 May-10 Action Media Workshop Report Participant based research to develop brand 6 Jul-10 PCI Pre-testing Report on Symbol/Slogan Led to Prevention In Action, etc. 7 Mar-11 PCI Action Meeting Report for Partners First analysis of actions by CE's, CI's and PAGs 8 Sep-11 PCI MER Summary Updated for Year 4 Contract with AED/FHI 360 9 Oct-11 All-Time PEPFAR Report PIA FY10-11 Final numbers for Year 2-3 CI's reached 10 Mar-12 PCI SA VAW Program PCI MER Plan FINAL Updated FINAL Document 11 Mar-12 KZN Network RDQA FINAL Report FINAL Data Quality Assessment by PCI 12 Mar-12 WC Network RDQA FINAL Report FINAL Data Quality Assessment by PCI 13 Jul-12 IAS 2012 Poster on Social Action Poster accepted for 2012 IAS Conference 14 Sep-12 PCI Salesforce Dashboard FY12 Final numbers for Year 4 CI's reached PLUS 15 Jan-13 Lessons Learned and a Model for Social Mobilization to Address Violence Against Women in South Africa Report from the internal evaluation 63 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE APPENDIX 6: PCI PIA PROGRAM TIMELINE PCI PIA Program Timeline Month Program Phase Communication Jan-Mar 2009 • Began Communication partner selection process Apr-Jun 2009 Analysis Phase - Conducted qualitative research and stakeholder meetings to develop strategy • Selected Ogilvy as communications partner in May Jul-Sep 2009 Engagement Phase - Launched of What's Keeping Violence Against Women Alive campaign during Women's Month (August); Program Communication Strategy agreed in September • Engagement brochure (Engagement support) • What's Keeping Violence Against Women Alive Collateral: WCN: network launch - 'Flipping the script' promo materials, girl child comic book and facilitation posters, KZN: bus campaign materials (i.e. branded bus and community dialogue) Oct-Dec 2009 Implementation Phase - Changing Faces Campaign conducted with Ogilvy • PR road show press kits and gifts • Radio spots (3) run from Oct to Dec • 'Changing Faces' giant billboards over 16 Days Jan-Mar 2010 2010 Strategic Directions: Due to findings from quantitative survey and budget cuts, program strategy revised • Draft Inspector General audit report on PCI (out of 5 GBV programs in SA) positive Apr-Jun 2010 Training of first set of CEs by both Networks and start of CEs facilitating Session 1 with CI's • PIA symbol (Action Media) Jul-Sep 2010 Debrief with CEs after Session 1 and then CEs reach 5,000 CIs with session 2 (over 4,000 KZN and less than 1,000 in WC) • CI toolkit distributed to all CI's attending Session 2 and Facebook page launched Oct-Dec 2010 16 Days focus on collection of action stories and ‘heroes’ gets some limited PR success… • 16 Days collateral, mini action brochure, KZN postcard campaign, poster kits, hand statues • CE appreciation days Jan-Mar 2011 2011 Strategic Directions: shifted focus to actions… • Inspector General report released officially with agreement by USAID to increase funding for PCI in FY 12 Apr-Jun 2011 • Old and new CEs trained in KZN and complete new set of CEs trained in Khayelitsha • Session 1 with 2nd round of CIs facilitated by CEs • Action meeting story collection • Story collection events (Car branding) Jul-Sep 2011 • Session 2 with 2nd round of CIs AND Session 3b with CIs reached in 2010 KZN • 7,000 CIs certified in 2010 (4,000 KZN and 3,000 WC) • CIs certified in FY 10 and FY 11 totals 12,000 to reach PEPFAR target • DVD Booklet #1 (distributed to CIs during Sessions 2 and 3b) • Story sharing events (women's month) • PIA t-shirts Oct-Dec 2011 • CEs trained for Session 3 • Session 3 with CIs facilitated • 16 Days Story Sharing events held (Three in KZN and one in Khayelitsha) • DVD booklet #2 (distributed to CIs during session 3) • 16 Days mobile story sharing events 64 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE Jan-Mar 2012 2012 Strategic Directions: Focus on CE/Os forming PACs. • Session 3 with CIs completed • CEs trained for Session 4 and Session 4 with CIs started • Develop Manifesto Poster • Version 1 of PIA program video Apr-Jun 2012 • Session 4 with CIs completed • CEs trained for Session 5 and Session 5 with CIs started • VFZs introduced during training of CE's for Session 5 in May • VFZ Materials • Jacket and ‘letter’ distribution to CIs Jul-Sep 2012 • Complete Session 5 with 6,000 CIs (4,000 in KZN and 2,000 in WC) to reach PEPFAR target for FY 12 • 50% of CI's attending Session 5 are active • 40 VFZs created by end of project (31 in KZN, 9 in Khayelitsha) • Launch of first VFZ in Wentworth on July 8 with U.S./KZN Government participation • Version 2 of PIA program video with VFZs Oct-Dec 2012 • KZN Network continues support to VFZs with USAID funding and to provide TA to expand PIA model in WC • WC Network to continue providing moderate TA to CE/Os in Khayelitsha via service providers as funding allows. 65 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE PERSONS CONTACTED Project Concern International SA Clayton Davis, Country Director, South Africa Warren Parker, Technical Consultant Clara Eder, Vice President for Strategic Information and Impact Ben van der Linde, Finance Manager Gil Lang, Communication Lead KwaZulu-Natal Network on Violence Against Women Cookie Edwards, Network Director Ntuthu Fuzile, Program Manager Zine Mdaki, MER Manager Western Cape Network on Violence Against Women Lungiswa Memela, Network Director Carmen Louw, Program Manager Dennis Vusani, Program Officer 66 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE REFERENCES Anderson, Bronwynne. Coloured boys in trouble: an ethnographic investigation into the construction of Coloured working-class masculinities in a high school in Wentworth. Thesis (PhD), University of KwaZulu Natal. Durban, 2009. Bachelet, Michelle. Good Practices and Remaining Gaps in the Prevention of Violence Against Women. Paper prepared for Human Rights Council Annual Day, panel discussion on Good Practices and Remaining Gaps in the Prevention of Violence against Women. Geneva, June 2011. Batliwala, Srilatha. Changing their worlds, Concepts and Practices of Women’s Movements. Association for Women’s Rights in Development. 2009. Brehm, V.M. NGOs and Partnership. NGO Policy Briefing Paper No.4. April 2011. Children’s Institute, National Strategic Plan – Priority Districts. Accessed from http://www.childrencount.ci.org.za/nsp-priority_districts.php, January 2013. Clohesy, Stephanie. Clohesy Consulting: Making the Case Tool. Women’s Funding Network. (Undated). Garone, D.B., Hilderbrand, K., Boulle, A., Coetzee, D., Goemaere, E., Van Cutsem, G., and Besada, D. Khayelitsha 2001-2011: 10 years of Primary Health Care HIV and TB Program. Southern African Journal of HIV Medicine, 12 (4). Injobo ne Bandla, CSIR, IPT and IDASA. Kwa Mashu and Inanda crime program prevention program. Report of first phase. (Undated). KwaZulu Natal Network on Violence Against Women. Accessed from http://www.womensnet.org.za/kwazulu-natal-network-violence-against-women, January 2013. Meer, S. Struggles for Gender Equality: Reflections on the place of men and men’s organisations. OSISA. 2012. Mercycorps. Community mobilization framework. In Guide to Community Mobilization, 2009. Michau, Lori. Approaching old problems in new ways: community mobilization as a primary prevention strategy to combat violence against women. Gender and Development, 15(1). March 2007. Nleya, Ndodana and Thompson, Lisa. Survey methodology in Violence prone Khayelitsha. IDS Bulletin 40 (3). National Department of Health. The National Antenatal Sentinel HIV and Syphilis Prevalence Survey, 2011. Pretoria, 2012. Nkani, Francis N. Factors that explain gender based violence amongst secondary school learners in the Inanda area. Thesis (Masters), University of KwaZulu Natal. Durban, 2006. Project Concern International. PCIs Mission – Disease Prevention, Community Health and Sustainable Development. Accessed from http://www.pciglobal.org/en/pci/pcis-mission-disease￾prevention-community-health-sustainable-development, January 2013. Project Concern International South Africa. APS 674-08-003 Technical Narrative. May 2008. Project Concern International. 2010 Strategic Directions. April 2010. Project Concern International. Monitoring, evaluation and reporting plan - Social mobilization to end gender-based violence: An essential HIV prevention strategy for South Africa, October 1, 2008 - September 30, 2012. March 2012. Project Concern International. Quarterly Report: July to September 2011. October 2011. 67 PROGRAM EVALUATION - SOCIAL MOBILIZATION TO END GENDER-BASED VIOLENCE Project Concern International. Quarterly Report: July to September 2012. October 2012. Project Concern International. PIA Program Timeline. October 2012. Project Concern International. Social mobilization programme to end violence against women: An essential HIV prevention program, South Africa – Communications Strategy. 2009. Project Concern International. Social mobilization to reduce violence against women (VAW) in South Africa, Strategic Directions for 2011. February 2011. Project Concern International. Final report of project implemented with USAID/PEPFAR funding by PCI and the KZN and WC Networks on Violence Against Women from 2009-2012. 2012 Thaler, Kai. Norms about intimate partner violence among urban South Africans: a quantitative and qualitative vignette analysis, CSSR Working Paper No. 302, January 2012. UNAIDS. Agenda for accelerated country action for women, girls, gender equality and HIV. Geneva, 2010 Vetten, Lisa. Addressing domestic violence in South Africa: Reflections on strategy and practice. Paper presented at the Division for the Advancement of Women, Vienna, May 2005. Accessed from http://www.un.org/womenwatch/daw/egm/vaw-gp-2005/docs/experts/vetten.vaw.pdf, January 2013. Wakefield, Melanie A., Loken, Barbara and Hornik, Robert. The use of mass media campaigns to change health behaviours. Lancet, 376 (9748): 1261–71. Western Cape Network on Violence Against Women. Accessed from http://www.wcnetwork.org.za/, January 2013. WHO. Why engage men in the fight to end violence against women and girls? 2009. Accessed from http://www.who.int/gender/topics/why_engage_men/en/index.html, January 2012.