December 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Victoria James (Team Leader), Ann von Briesen Lewis (Evaluation Specialist) and Dr. Sylvester Nandi (M&E Specialist). EVALUATION Midterm Performance Evaluation of the USAID/Tanzania Pamoja Tuwalee Project This document is available online and can be located through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: Global Health Program Cycle Improvement (GH Pro) Project 1299 Pennsylvania Avenue NW, Suite 1152 Washington, DC 20004 Phone: (202) 625-9444 Fax: (202) 517-9181 Cover Photo by Ann von Briesen Lewis MIDTERM PERFORMANCE EVALUATION OF THE USAID/TANZANIA PAMOJA TUWALEE PROJECT DECEMBER 2014 EVALUATION MECHANISM: AID-OAA-C-13-00113 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGMENTS The GH Tech Bridge IV evaluation team would like to acknowledge assistance provided by the Pamoja Tuwalee Program implementing partners: Africare, FHI360, Pact Inc. and World Education Inc. Partners provided key documents for review, facilitated access to beneficiaries and stakeholders and responded to emerging issues and questions. The evaluation team is grateful for support and cooperation from the Government of Tanzania during the evaluation. This support allowed the evaluation team to collect field data without sacrificing the necessary evaluative objectivity. The USAID/Tanzania team facilitated interviews with key stakeholders and provided technical support on the evaluation design and feedback on preliminary results. The evaluation team is also grateful to all respondents who provided insights into the evaluation process and the translators who assisted the team to capture them. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation i CONTENTS ACRONYMS..................................................................................................................................... iii EXECUTIVE SUMMARY.............................................................................................................. v 1. INTRODUCTION....................................................................................................................... 1 2. PROJECT BACKGROUND .................................................................................................. 3 3. EVALUATION METHODS and LIMITATIONS.............................................................. 7 4. FINDINGS.................................................................................................................................. 11 5. CONCLUSIONS...................................................................................................................... 48 6. RECOMMENDATIONS........................................................................................................ 52 ANNEX I. SCOPE OF WORK................................................................................................... 55 ANNEX II. EVALUATION METHODS AND LIMITATIONS........................................... 72 ANNEX III. PERSONS INTERVIEWED................................................................................ 80 ANNEX IV. WORK PLAN AND SCHEDULE ...................................................................... 91 ANNEX V. DATA COLLECTION INSTRUMENTS........................................................... 94 ANNEX VI. MAP OF TANZANIA AND PROGRAM AREAS...................................... 142 ANNEX VII. DOCUMENTS REVIEWED ........................................................................... 143 ii USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation FIGURES Figure 1: Map of Pamoja Tuwalee Implementation Zones and Regions ......................... 6 Figure 2: Evaluation Sampling Approach ........................................................................ 8 Figure 3: Evaluation Methodology ................................................................................. 10 Figure 4: Program Structure .......................................................................................... 12 Figure 5: Africare Progress against PEPFAR Indicators ............................................... 16 Figure 6: FHI360 Progress against PEPFAR Indicators ................................................ 21 Figure 7: WEI Progress against PEPFAR Indicators ..................................................... 27 Figure 8: Pact Performance against PEPFAR Indicators .............................................. 33 TABLES Table 1: Tanzania Social and Economic Indicators ......................................................... 3 Table 2: PT Implementing Partners, Zones and Regions ................................................ 6 Table 3: Education and Birth Registration Indicators for Sampled Regions .................... 8 Table 4: Alignment between PT and the NCAP II ......................................................... 11 Table 5: Africare MVC Beneficiaries by Type of Support Received .............................. 16 Table 6: Benefits Reported by Africare Beneficiaries .................................................... 18 Table 7: Savings for Africare SILC Groups ................................................................... 18 Table 8: Africare Partner Contributions ......................................................................... 19 Table 9: Amounts Invested by FHI360 in Supporting LGAs (Tsh) ................................. 22 Table 10: Overview of SILC Groups Supported by FHI360 ........................................... 23 Table 11: Statistics from the Child Helpline ................................................................... 23 Table 12: Challenges Outlined by FHI360 Beneficiaries ............................................... 25 Table 13: Emerging Issues from a Review of WORTH and SILC ................................. 29 Table 14: WEI LIMCA Group Performance ................................................................... 29 Table 15: Birth Registration Levels within the Northern Zone ....................................... 31 Table 16: Functional MVCCs in Pact-Supported Regions ............................................. 34 Table 17: New Businesses Formed in Pact-supported Regions ................................... 36 Table 18: Child Protection Cases by Partner ................................................................ 45 Table 19: Selected Sexual and Reproductive Health Indicators for Tanzania ............... 46 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation iii ACRONYMS CSO Civil society organization DSW Department of Social Welfare FGD Focus group discussion GBV Gender-based violence GoT Government of Tanzania IPG Implementing partner groups LGA Local Government Authority LIMCA Livelihoods Improvement for MVC Care M&E Monitoring and evaluation MKUKUTA II National Poverty Reduction Strategy MOHSW Ministry of Health and Social Welfare MoU Memorandum of Understanding MVC Most vulnerable children MVCC Most Vulnerable Children Committee NCPA National Costed Plan of Action for Most Vulnerable Children OVC Orphans and vulnerable children PEPFAR United States President's Emergency Plan for AIDS Relief PLHIV Person living with HIV PMO-RALG Prime Minister‘s Office – Regional and Local Governments PROMIS PEPFAR Reporting and Organizational Management Information System PT Pamoja Tuwalee REPSSI Regional Psychosocial Support Initiative RFA Request for applications RITA Registration, Insolvency and Trusteeship Agency SILC Saving and Internal Lending Communities TDHS Tanzania Demographic and Health Survey TSh Tanzanian Shilling UNDP United Nations Development Programme UNICEF United Nations Children’s Fund USAID United States Agency for International Development WEI World Education Inc. ZAMWASO Zanzibar Muslim Women's AIDS Support Organization iv USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation v EXECUTIVE SUMMARY This report provides findings, conclusions, lessons learned and recommendations from a mid-term performance evaluation of the Pamoja Tuwalee (PT) program for support to orphans and vulnerable children (OVC). EVALUATION PURPOSE The purposes of this evaluation are two-fold: 1. To assess the approaches used by each of the PT implementing partners and determine which approaches are most effective in achieving linkages across services and fostering sustainability; and 2. To document lessons learned that will inform the U.S. Agency for International Development (USAID) mission in Tanzania in designing approaches and focus areas for the follow-on program to support vulnerable children and youth. EVALUATION QUESTIONS 1. To what extent are PT program objectives likely to be achieved? 2. What are the challenges encountered so far? 3. Is the program addressing sustainability of the interventions? How? 4. In what ways has the PT program promoted linkages within and beyond HIV program areas? 5. Are there specific lessons from the program that can be applied in the second phase of the project and to other programs and countries within the United States President's Emergency Plan for AIDS Relief (PEPFAR)? 6. What is the private sector’s involvement and contribution to the program? 7. What is the efficacy of the zonal approach in PT program management and implementation? 8. To what extent and how have relevant gaps between males and females been closed? 9. What new opportunities for women and men were created? 10. What differential negative impacts on males and females (such as increasing the risk of gender-based violence (GBV)) were addressed or avoided? 11. What needs and gender inequalities emerged or remain? EVALUATION METHODS The evaluation used a mixed-method design to comprehensively respond to all evaluation questions outlined above. The evaluation team sampled Arusha (North East zone), Dodoma (Central zone), Kagera (Lake zone), Mtwara (Southern zone) and Zanzibar (Coast zone). The team collected primary data through key informant interviews, focus group discussions, household visits, consultative meetings and self￾administered key informant guides for sub-grantee staff. The evaluation team did not collect quantitative primary data and instead relied on the PEPFAR Reporting and vi USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organizational Management Information System (PROMIS). To facilitate systematic secondary data extraction, a template was developed with the specific partner indicators and the data requirements. FINDINGS To what extent are PT program objectives likely to be achieved? What are the challenges encountered so far? Geographic coverage Evidence from consultations and program data demonstrates that objectives are highly likely to be achieved. The program has ensured high levels of coverage for services to the most vulnerable children (MVC). Coverage has reportedly improved MVC identification, referrals and access to nutrition, health, education, psychosocial support and protection. Africare has the highest district coverage (100 percent), as it reaches all the 18 districts and 339 out of 501 wards (68 percent) in the Central zone. World Education Inc. (WEI) has the lowest district coverage (45 percent), while Pact has the lowest ward coverage (27 percent), as shown in the figure below. Improving the quality of life and well-being of OVC The PT program has contributed to improving the quality of life and well-being of OVC. MVC received food and nutrition support, have improved access to health, are now attending school and have received support to protect them from GBV. Africare provided food and nutrition support to 18,786 MVC, and FHI360 supported 29,376 MVC with food and nutrition support. Support from WEI ensured that 232 MVC diagnosed with severe malnutrition were immediately referred to clinics for treatment, resulting in access to Plumpy’Nut (ready-to-use therapeutic food). Beneficiaries included MVC living in high-risk environments (for example, 91 at-risk MVC in Bukoba). Children found on 100% 96% 95% 45% 68% 44% 27% 32% 0 10 20 30 40 50 60 70 80 90 100 Africare FHI Pact WEI % District Coverage % Ward Coverage USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation vii the streets were reunited with their families. Pact supported 38 cases of children in conflict with the law, and in six of these cases children were given alternative sentences for their offenses. Children are reportedly attending school due to improved income for caregivers who are part of economic strengthening groups, and access to health care has been enhanced by contributions to community health funds. Strengthening of the local government authorities All implementing partners have invested in strengthening local government authorities (LGAs) and Most Vulnerable Children Committees (MVCCs).The program has mobilized and strengthened LGAs and community-based responses to support MVC. All partners reported collaborating with LGAs to train and supervise MVCCs that are providing improved quality support to MVC. In the Northern zone, WEI signed memoranda of understanding (MoU) with 12 of the 16 councils covered by the project. The MoU have reportedly provided the basis for stronger collaboration with LGAs, resulting in improved support for MVC. In the Central zone, Africare supported all 19 district councils to plan, coordinate and monitor implementation of the PT program. Strong LGA capacities have reportedly resulted in prioritization on MVC needs in budgeting. Within the Southern zone, five districts released funds to support the provision of services to MVC and their caretakers in the form of capital for income-generating activities. Biharamulo district approved Tsh 6.3 million (US$3,778) to identify MVC caregivers living with chronic diseases, and the Karagwe district council provided grants to WORTH groups valued at Tsh 5 million (US$2,998). Capacity building of the MVCC Pact developed the MVCC Engagement Strategy, which reportedly has contributed to revitalizing MVCCs. The number of active MVCCs reportedly increased by 39 percent, from 966 to 1,346, in the third quarter of 2013. In the Coastal zone FHI360 supported LGAs in reviewing the stakeholders’ inventory to ensure comprehensive care and support to MVC through stronger coordination across partners. Strengthening the capacity of MVC support structures is facilitating access to services. Pact worked with District Child Protection Teams to ensure survivors of abuse accessed timely comprehensive services according to their needs. The organization further supported reunification of 76 children with their families. In addition, 53 perpetrators of child abuse were arrested and 14 convicted on child abuse charges. Pact also reunified 76 children with their families, while some neglected children were referred to temporary shelters at Upendo Daima, Starehe and Kuleana children’s homes. Improving capacities of LGAs and MVCCs is in line with the fourth expected outcome within the PT program design, which focuses on providing support to achieve “mobilized and strengthened LGAs as well as community-based responses to support Orphans and Vulnerable Children (OVC).” Although partners have documented progress toward capacitating LGAs and MVCCs, there are limitations in tracking and demonstrating any viii USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation improvements in conditions of MVC that may have resulted from enhanced LGA and MVCC capacities. Provision of services for survivors of violence Partners have contributed to reducing GBV and ensuring survivors receive support. As of 2013, WEI had supported 664 survivors of GBV to receive counseling and treatment. During the same period, Africare supported 947 survivors to receive counseling, while Pact supported 549 survivors who received counseling and treatment services. Statistics for supporting GBV survivors demonstrate progress; partners reported that prior to program support, there were weak reporting mechanisms and limited access to support services for survivors. Due to a unique design, FHI360 supported pilot projects for a child helpline to facilitate reporting abuse, as well as a one-stop center for GBV services. Helpline statistics showed high usage; it received 4,642 calls within the first quarter of its implementation. Although FHI360 has data on the numbers and types of abuses reported, there is limited outcome-level data demonstrating the percentage of calls that resulted in follow￾up and support for survivors. Additionally, there is limited evidence on potential linkages between the helpline and the one-stop center, though there is potential for synergy. Household economic strengthening support Beneficiaries of economic strengthening activities have higher individual savings compared to the food poverty threshold (an income less than Tsh 26,085 (US$15.76)1 per adult per month) and basic needs poverty threshold (an income less than Tsh 36,482 (US$22.04) per adult per month) (Tanzania Household Budget Survey, 2013). Program beneficiaries hold average savings of Tsh 60,891 (US$36.78). Beneficiaries reported using savings for supporting educational, health and nutritional needs of MVC. All economic strengthening models also include a mandatory percentage contribution by groups toward meeting MVC needs. All Saving and Internal Lending (SILC) and WORTH groups interviewed stated that they are now working on their own without support from the implementing partners, which demonstrates progress toward graduation. The figure below shows individual savings by economic strengthening groups. 1 Exchange Rate of 1 U.S. dollar: Tsh 1,582.03 (http://www.oanda.com/ 11 February 2014) USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation ix Is the program addressing sustainability of interventions and if so, how? The program is addressing sustainability; all partners’ program designs are based on the essential components of capacity building and skills transfer. Partners have prioritized working with the Government of Tanzania (GoT) as a sustainability strategy. Joint MVCC training between the Ministry of Health and Social Welfare (MOHSW) and FHI360, refresher trainings facilitated by Pact and the MoHSW and multi-sectoral school assessments executed by WEI in partnership with GoT structures all demonstrate implementation of the sustainability strategy. Partners have further adopted a training￾of-trainers approach to ensure capacity is institutionalized within the GoT and within sub-grantees. The table below shows the distribution of the 90 supported sub-grantees by implementing partner. Partner Africare FHI Pact WEI Number of sub-grantees 19 11 49 11 TOTAL 90 As a result of capacity building for sub-grantees, 30 out of 32 sub-grantees interviewed highlighted improved capacities in training of MVCCs and para-social workers. Additional capacities were reported in planning, budgeting and monitoring and evaluation (M&E). Through financial management training and joint follow-up visits to partners by Pact, 90 percent of partners improved on timely submission of monthly reports. In addition, questioned costs decreased. For example in Mara, where Pact identified over Tsh 23 million (US$13,794) in questioned costs prior to the support to partners, following the trainings and site visits partners managed to resolve nearly 90 percent of questioned costs by June and cleared all of them by the end of 2013 (Pact, 2013). Key informants acknowledged that partners are investing in strengthening MVCCs, which are based in the communities and closest to MVC. Reports show that 145,444 42,292 15,627 40,203 0 20000 40000 60000 80000 100000 120000 140000 160000 Africare WEI Pact FHI Individual Savings per Saving Cycle in Tsh Tsh x USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Africare supported formation or revival of 888 MVCCs, Pact supported 1346 and WEI supported 251. In what ways has the PT program promoted linkages within and beyond HIV program areas? The program has promoted and benefitted from linkages mainly through creating and strengthening referral chains. In the Northern zone WEI has started exploring ways that linkages can ensure MVC living with HIV are supported within the strategic unit that supports people living with HIV. Africare linked small-scale farmers with an organization2 that provides support, and 264 farmers received inputs, production-related technical support and market linkages. Pact used linkages and referrals to facilitate access to food and nutrition, education, shelter, health, vocational training and small livestock for 3,326 MVC in the Lake and Southern zones. WEI has initiated dialogue on linkages with Same District Council officials and Evangelical Lutheran Church in Tanzania-Pare Diocese’s Rights and Economic Justice program members. The Rights and Economic Justice program works to ensure children, especially MVC, are able to claim their rights, identify innovative solutions and receive appropriate support. Although discussions were at initial stages, the organizations intend to identify areas of collaboration as well as potential to access each other’s competencies. Are there specific lessons from the program that can be applied in the second phase of the project, and to other PEPFAR programs and countries? Capacity building is a gradual process, and aspirations should be realistic. The PT program has transformative aspirations, but there could be scope for achieving more if skills transfer and service delivery aims are critically analyzed (e.g., investigating how feasible it is to capacitate a civil society organization (CSO) with very low capacity levels and ensure it also delivers services). High staff turnover at different levels was cited, but there has been limited examination of effective capacity-building approaches in the context of high staff attrition. Capacity-tracking tools like the Comprehensive Institutional Strengthening Plan used by Pact are critical in objectively tracking organizational development progress. Both Pact and FHI have identified sub-grantees that potentially could be graduated to direct USAID funding. Communities have resources that need to be carefully mapped and harnessed to support MVC. The program has successfully revived and/or sustained MVCCs, which are local-level structures with potential to provide sustainable support to MVC. Lessons from Iringa showed that LGAs may not necessarily provide financial resources but can provide land, which will ensure beneficiaries have space to conduct their activities. 2Cheetah Development Cooperation USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation xi Capacity building and material support are not mutually exclusive components. The program has managed to provide capacity building while also providing services in order to transfer responsibility to households and the government. The program initially focused more on capacity building, but implementation lessons demonstrated that challenges facing LGAs, CSOs and MVC required material support to complement skills transfer and capacity building. Private sector partnerships can be successful in the context of well-developed implementing partner capacities. All partners have managed to implement activities classified as private sector partnerships, but they acknowledged having capacity limitations around effectively creating and nurturing such partnerships. All partners now have relationships with DAI, which is providing technical assistance in economic strengthening and whose experiences will be used in pursuing private sector partnerships. Referrals are effective in the context of available services. Partners have been successful in identifying and referring MVC, but in some instances services are not available at the referral centers. All partners have acknowledged that initial approaches focused more on providing referrals without ensuring availability of services. This experience has led to a shift toward providing referrals to locations where services are available, while strengthening referral tracking and follow-up systems. Coordination with and capacity building of government are central components of sustainable interventions. Partners’ work with government has initiated processes of active advocacy, in which the government is supported to assume responsibility of MVC, as opposed a process in which partners point out what government is failing to do for MVC. Trainings of MVCC and para-social workers have been done in collaboration with government, and engagement of LGAs has resulted in more investments in MVC activities. Involvement of government in training of MVCCs and their supervision, as well as in multi-sectoral school assessments, is reportedly strengthening capacity to assume full responsibility for supporting MVC. Expectations from government should be realistic and based on existing knowledge of available capacities. All partners are working to support the government, in recognition of its weak capacity to support MVC. However, partners also provide referrals to the government facilities, whose capacity gaps they have documented and are trying to address. Respondents from the evaluation explained that health fee exemptions are being facilitated, but health facilities sometimes do not have medication, which limits the effectiveness of referrals. Advocacy initiatives are likely to be effective if they transition from articulating deficiencies to working with government to address deficiencies. xii USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Partners have all prioritized working with government departments on training and actual implementation (for example conducting multi-sectoral school assessments). The approach has reportedly ensured stronger buy-in from the government. What was the private sector involvement and contribution to the program? All implementing partners have invested in training on resource mobilization and have supported community-based resource mapping. There is limited outcome-level evidence of private sector involvement. Pact-supported MVCCs successfully conducted resource-mapping exercises and articulated potential opportunities for private sector engagement. Africare linked farmers with Cheetah Development Cooperation, which provided inputs and production-related technical support. Improved production and enhanced quality of produce resulted in further linkages along the value chain as farmers secured stable markets, which reportedly provided high value for their produce. All implementing partners acknowledged technical gaps in private sector partnerships and are highly appreciative of DAI’s support. GENDER MAINSTREAMING Clear articulation of gender and age disparities All partner proposals articulate gender disparities and strategies for addressing them. Examples include a focus on nutritional support for children under 5, as well as economic empowerment (e.g., vocational training) for older MVC. Female caregivers benefitting from economic activities highlighted that such support has been critical, as men usually control household finances and often do not adequately prioritize financing needs of MVC. Prioritizing affirmative action Partners have adopted an affirmative action approach focusing on livelihoods strengthening for women, who are historically disadvantaged and constitute the majority of MVC caregivers. The approach has ensured that at least 60 percent of all SILC group members are female caregivers. Economic strengthening as an entry point The program uses economic strengthening groups as an entry point for other important issues like birth registration. Women in Arusha reported that such support has provided them with information on the importance of registering births as well as how to go about the registration process. Through economic strengthening groups, 1,671 children were supported to receive birth certificates, and 5,194 more will receive birth certificates in 2014 (WEI, 2014). CHALLENGES Weak livelihoods Weak livelihoods and high poverty levels constrain a community’s capacity for savings and MVC support. All partner baselines cited poverty as a key determinant of vulnerability, hence the prioritization of household economic strengthening. High levels USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation xiii of poverty reportedly sometimes caused MVC caregivers to prioritize other household needs at the expense of MVC needs. Overachievement of targets At mid-term all partners have surpassed almost all targets for PEPFAR indicators. However, evidence from the evaluation demonstrates improvements. Households in economic strengthening groups have more money than the basic needs threshold for Tanzania. The surpassing of targets raises concerns about the processes and evidence behind target setting. Low capacities of LGAs There are gaps in ensuring LGAs honor their funding obligations for MVC. Commitments through MoUs have been secured, but very few councils have funded their commitments. Local authorities acknowledged their responsibility to support MVC but cited limited financial resources as a critical challenge. Referrals to services that are not available The program prioritizes referrals and linkages, but these have been constrained by general service provision challenges within the country. For example, health fee exemption cards are only effective if services are available at the facilities. Limited capacities for private sector engagement Partners do not have adequate expertise for private sector engagement. As a result, support received has largely consisted of one-off donations that reportedly cannot contribute toward building community resilience. All partners have developed relationships with DAI, which will provide technical support in economic strengthening. Limited capacities of MVCCs The MVCCs provide strong structures for providing sustainable support for MVC, but they have weak capacity, which constrains the program’s efforts. In the Southern zone, for example, Pact documented that more than 70 percent of MVCCs were not functional prior to provision of support and had to be trained on their responsibilities before they could support MVC. xiv USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation RECOMMENDATIONS AUDIENCE CONCLUSION RECOMMENDATIONS USAID The zonal approach is consistent with capacity building and skills transfer, as it has promoted creation and maintenance of partnerships between implementing and LGAs, which are central to the program’s sustainability strategy. Continue prioritizing the zonal approach with clear indicators for efficiency, equity and accountability as outlined in the Request for Applications (RFA). USAID There were reported delays in disbursements, which affects program implementation. Prioritize timely disbursement of funds to partners to ensure they stay on track with implementation timelines. This will depend on availability of funding. USAID There is significant qualitative evidence of outcomes emerging from program activities. However, there is limited quantitative data for components like household hunger, dietary diversity, HIV, stigma knowledge, vulnerability and resilience. Plan to invest in an end-of￾program performance evaluation with scope for quantitative approaches like household economic assessment that provide objective data on progress made throughout program implementation. USAID Partners have documented key approaches, lessons learned and other methodological insights that can be useful to a broad range of partners. Pact has Comprehensive Institutional Strengthening Plans, and WEI’s WORTH+ approach is facilitating economic empowerment and promoting birth registration for children. FHI360 has piloted a one-stop center for GBV, while Africare has developed a successful model for private sector partnership. However, none of the partners reported having shared the approaches with other PT implementing partners. Prioritize sharing implementation experiences to strengthen learning and sharing of experiences. Specific learning meetings can be organized where partners can periodically share lessons learned and good practices. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation xv AUDIENCE CONCLUSION RECOMMENDATIONS USAID Partners acknowledge the importance of private sector involvement as part of their sustainability strategies but have weak capacity in private sector engagement. Africare successfully partners with Cheetah Development Cooperation, which successfully provided financial and technical support to farmers. All partners have relationships with DAI, which is now providing technical support on private sector engagement. Continue supporting partnerships with DAI that will provide technical support on strengthening private sector partnerships and economic strengthening. Use findings from a study being conducted by DAI on economic strengthening to determine the most effective approaches to adopt. Ensure private sector partnerships are designed to build on existing community capacities and livelihood options. For example, the partnerships for predominantly agricultural communities should focus on strengthening that existing livelihood source. Ascertain attribution of improved MVC health, education, nutrition and protection outcomes to savings and lending groups, beyond anecdotal evidence. Determine timing and approaches to scale-up community groups to larger microfinance lenders for real economic investment and growth. Partners Volunteers and para-social workers provide community￾level expertise and support. However, there are high levels of attrition as they are not paid. Consider using economic strengthening support as incentives for volunteers to ensure they are motivated and levels of attrition are reduced. Partners Agro-based livelihood strengthening has been successful in Iringa, but there have been challenges in Dodoma and Karatu where the agro-ecological conditions are different. Invest in different agricultural technologies like conservation agriculture in areas where agro￾ecological conditions are difficult for conventional farming. xvi USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation AUDIENCE CONCLUSION RECOMMENDATIONS Ascertain attribution of improved MVC health, education, nutrition and protection outcomes to saving and lending groups, beyond anecdotal evidence. Determine timing and approaches to scale-up community groups to larger micro-finance lenders for real economic investment and growth. Invest in monitoring the number and amount of loans from saving and lending groups that go to educational fees, and use this information to sensitize educators on the importance of MVC support. Partners Coordination with government has provided opportunities for sustainable provision of MVC services despite weak capacity. Continue nurturing and strengthening relations with the government to strengthen capacity for MVC support. Partners should continue investing in collaborative training and supervision of MVCCs and involvement of LGAs in multi￾sectoral school assessments. Partners Coordination of implementing partners through regional and district meetings has contributed to stronger sharing of information, and partners have platforms for exploring collaborative activities. Continue investing in regional and district-level meetings to further strengthen current coordination efforts. Partners Referrals are critical in broadening the range of services accessible to MVC beyond an individual’s capacity. However, the capacity of service providers is generally weak and MVC do not always receive the services. Ensure referrals are provided for services that are available, and develop strong mechanisms for referral tracking. Follow up to ascertain if MVC receive services for which they are referred. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 1 1. INTRODUCTION Consistent with the United States President’s Emergency Plan for AIDS Relief (PEPFAR) II OVC Guidelines and Government of Tanzania (GoT) policies, USAID Tanzania signed consultative agreements with four implementing partners (FHI360, Africare, Pact and World Education Inc.) to implement a program titled Pamoja Tuwalee (PT). The partners use a zonal approach to provide support and services to most vulnerable children (MVC). The program supports families, communities and local government agencies to strengthen their capacity to sustainably care for their own MVC. The goal of the program is “improving the well-being of most vulnerable children and their caretakers using a sustainable approach.” The overall target is to serve 318,107 MVC and 159,000 households in 21 regions of Tanzania by 2015. The PT program began on June 1, 2010 and ends on May 31, 2015. Its estimated budget is US$59 million. This report provides findings, lessons learned, conclusions and recommendations from a mid-term performance valuation of the PT program. EVALUATION PURPOSE The PT program design represents a transition from direct service provision to sustainable community support. The transition prioritizes capacity building and transfer of responsibility to the Government of Tanzania (GoT) and its structures responsible for MVC care. The performance evaluation focuses on learning and generating evidence on whether or not the new approach works. The purposes of this evaluation are two-fold: 1. To assess the approaches used by each of the PT implementing partners and determine which approaches are most effective in achieving linkages across services and fostering sustainability; and 2. To document lessons learned that will inform the mission in designing approaches and focus areas for the follow-on program to support vulnerable children and youth. EVALUATION QUESTIONS 1. To what extent are PT program objectives likely to be achieved? 2. What are the challenges encountered so far? 3. Is the program addressing sustainability of the interventions? How? 4. In what ways has the PT program promoted linkages within and beyond HIV program areas? 5. Are there specific lessons from the program that can be applied in the second phase of the project and to other PEPFAR programs and countries? 6. What is the private sector’s involvement and contribution to the program? 7. What is the efficacy of the zonal approach in PT program management and implementation? 2 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 8. To what extent and how have relevant gaps between males and females been closed? 9. What new opportunities for women and men were created? 10. What differential negative impacts on males and females (such as increasing the risk of gender-based violence) were addressed or avoided? 11. What needs and gender inequalities emerged or remain? EVALUATION AUDIENCE USAID, PEPFAR, implementing partners and the GoT will be the primary audiences of the evaluation. Results will provide evidence for making resource allocation decisions, exploring unintended consequences, shedding light on implementation processes and failures at any level, highlighting areas of accomplishment and potential, identifying emerging problems and building consensus on the causes of problems as well as potential responses. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 3 2. PROJECT BACKGROUND TANZANIAN CONTEXT Tanzania is the largest country in East Africa, covering 940,000 square kilometers, 60,000 of which are inland water. The country shares borders with eight countries: Kenya and Uganda to the north; Rwanda, Burundi, the Democratic Republic of the Congo and Zambia to the west; and Malawi and Mozambique to the south. Tanzania’s population is estimated at 44,928,923 with 43,625,354 living on the Tanzania Mainland and 1,303,569 in Zanzibar (Tanzania Population and Housing Census, 2012). An estimated 75 percent of the population resides in rural areas. Children below the age of 18 years make up 50 percent of the population (PHC, 2012). The table below shows social and economic data for Tanzania. Table 1: Tanzania Social and Economic Indicators Indicator Value Population 44.9 million (Tanzania Population and Housing Census, 2012) Percentage of population below 15 years 47 percent (Tanzania Population and Housing Census, 2012) Human Development Index value 0.476 (low human development) (UNDP, 2012) Human development rank 152 out of 187 (UNDP, 2012) Percentage of households classified as extremely poor 8 percent (Comprehensive Food Security and Vulnerability Analysis, 2012) Percentage of households vulnerable to food insecurity 8.3 percent (Comprehensive Food Security and Vulnerability Analysis, 2012) Percentage of households classified as chronically food insecure 1.7 percent (Tanzania Household Budget Survey, 2013) HIV prevalence 5 percent (6 percent women and 4 percent men) (Tanzania Malaria and HIV Survey, 2012) Women reporting ever having suffered physical violence 39 percent (Tanzania Demographic and Health Survey, 2010) Women reporting forced first sexual encounter 10 percent (Tanzania Demographic and Health Survey, 2010) Percentage of young women 15-19 who have already started childbearing 23 percent (Tanzania Demographic and Health Survey, 2010) Percentage of women with unmet need for family planning 25 percent (Tanzania Demographic and Health Survey, 2010) Percentage without toilet facility 14 percent (Tanzania Demographic and Health Survey, 2011) 4 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MVC IN TANZANIA Tanzania had an estimated 1.67 million MVC in 2010, with projection that the number would increase to 1.81 million in 2015 (MEASURE, 2011). Another study highlights the immense and diverse child protection needs in Tanzania, which are magnified by widespread poverty and the impact of HIV and AIDS (UNICEF 2011:8). One-third of households live in poverty. More than two million children are orphans (having lost one or both parents). One in five children is engaged in child labor. Only 2 percent of children with disabilities attend primary school. The Tanzania Violence against Children Survey (2009) concluded that nearly one in three girls and one out of six boys reported at least one experience of sexual violence prior to the age of 18. GOVERNMENT OF TANZANIA RESPONSE The National Poverty Reduction Strategy (MKUKUTA II, 2010) summarizes the basic factors causing poverty and vulnerability to include natural calamities; malaria, HIV/AIDS and TB; low incomes; lack of education and skills; lack of access to clean and safe water and lack of adequate shelter. However, the MKUKUTA II also points out that there are practices that have adverse effects on vulnerable groups, including customs, norms, taboos, values, unhappy marriages, domestic and gender-based violence, drunkenness and drug abuse. The GoT has prioritized supporting the needs of MVC and developed a National Costed Plan of Action (NCPA) for MVC (2007-2010). As of the end of 2010, more than 800,000 children were provided with services in health as well as broad social protection. In addition, Most Vulnerable Children Committees (MVCCs) were mobilized in up to 61 percent of the villages in Tanzania. Global evidence and guidance, together with the evaluation of the previous MVC NCPA I for MVC 2007-2010, emphasize the need for the MVC response to be refocused on sustainability approaches through support of households and government-led systems to coordinate service provision. To ensure realization of sustainability, the new national MVC plan for 2013–2017 was developed with an overall goal of establishing an MVC response that is government-led and community-driven and that facilitates MVC access to adequate care, support, protection and basic social services through increasingly mainstreamed and sustainable government-led systems. To achieve this, the second national MVC plan (NCPA II) is structured around four overall strategic objectives: 1. Strengthen the capacity of households and communities to protect, care for and support MVC. 2. Increase access to effective child protection services within a well-resourced child protection system. 3. Expand access to education, health and early childhood development services. 4. Strengthen coordination, leadership, policy and the service delivery environment. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 5 OVERVIEW OF PAMOJA TUWALEE PT is an OVC program in support of implementing the MVC NCPA I and the new MVC NCPA II. The program is implemented through four cooperative agreements with Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. PT is funded under PEPFAR II (2009-2014), which was authorized by the U.S. Congress in 2008, and which emphasizes local institutional capacity building, government ownership and sustainability. It also emphasizes a focus on outcomes rather than outputs. In terms of OVC, outcomes are achieved when the needs of a child are not just recognized and subsequently met, but when they are met in a sustainable fashion. For example, a one-off donation of commodities such as food or educational support, while useful, does not generally address the actual issues underlying OVC or household needs (RFA, 2010). Goal The program intends to improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions. The goal will be achieved through seven core strategies outlined below: 1. Provision of comprehensive services strategically aligned to OVC and their household needs 2. Sustainability 3. Coordination of all stakeholders 4. Local capacity building 5. Primary focus on the OVC household rather than individual OVC 6. Meaningful linkages and referrals to other AIDS, health and development services 7. Gender- and age-specific programming The Zonal Approach PT represents a new approach to project funding and implementation distinguished by two factors. Firstly, it comprises multiple cooperative agreements that were awarded based on a new “zonal” approach. Previously, implementing partners were spread across several regions and seldom covered all districts in a region. Evidence showed this to be an inefficient way to provide services, which favored urban centers and resulted in a deficiency in district-level services. A zone is defined as “three to four regions grouped together by USAID to form a bounded geographical area of OVC service delivery intervention” (RFA, 2009). The table below delineates the different zones, regions and implementing partners responsible under PT. Each zone includes several adjacent regions with varying degrees of resource and logistical challenges. 6 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Table 2: PT Implementing Partners, Zones and Regions Implementing Partner AFRICARE FHI360 WEI Inc./ Bantwana Lake-Pact Southern-Pact Regions Dodoma Dar es Salaam Arusha Mara Lindi Singida Coast Tanga Mwanza Mtwara Iringa Zanzibar (5) Manyara Kagera *Mbeya Njombe Morogoro Kilimanjaro Tabora *Rukwa *Ruvuma *Katavi *New expanded geographical coverage transitioned from DoD Implementing partners within specific zones are required to ensure district coverage within each region in the zone, as well as ensure that the zones’ data are entered in the national MVC data management system and that implemented activities are linked to the national coordination system. The map below shows different zones, regions and respective implementing partners. Figure 1: Map of Pamoja Tuwalee Implementation Zones and Regions USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 7 3. EVALUATION METHODS AND LIMITATIONS EVALUATION DESIGN The evaluation used a mixed-method design to respond comprehensively to all evaluation questions outlined above. A mixed-method evaluation design systematically integrates two or more evaluation methods at every stage of the evaluation process, drawing on both quantitative and qualitative data. This design has the highest potential to provide strong results and is endorsed by the USAID Evaluation Policy, which states, “Given the nature of development activities, both qualitative and quantitative methods yield valuable findings, and a combination of both often is optimal” (USAID, 2011). Design Strengths  The design allowed for triangulation; quantitative data from partners was complemented by qualitative accounts from consultations conducted during the mid-term performance evaluation.  There were multiple evaluation questions for the mid-term performance evaluation and these required a variety of methods. Some of the questions required more than one method to sufficiently answer all components.  The evaluation used multiple methods to answer the same elements of a single question, increasing confidence in the validity and reliability of the evaluation results. SAMPLING Four regions in the Tanzania Mainland were purposively selected according to these criteria: maturity of programs, HIV prevalence, rates of poverty and geographical coverage. In addition, Zanzibar was purposively sampled to ensure full representation of The United Republic of Tanzania. Selected regions were as follows:  Arusha (North East zone)  Dodoma (Central zone)  Kagera (Lake zone)  Mtwara (Southern zone)  Zanzibar The table below provides selected indicators in the sampled regions. 8 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Table 3: Education and Birth Registration Indicators for Sampled Regions Region Net primary enrolment Net secondary enrolment Percentage of children whose births are registered Arusha 84.3 27.7 25.3 Dodoma 67.7 15.8 5.9 Kagera 75.7 22.3 6.7 Mtwara 82.2 26.5 6.9 Zanzibar 78.7 Multi-stage random sampling was used to select two districts per region, one urban and one rural. Respondent-driven (snowball) sampling was also used to identify additional respondents who could provide critical information to the evaluation, but who would not have been identified initially. Expert case sampling was used to identify key policy and programming experts who could provide rich insights that shaped overall conclusions and recommendations. The figure below shows the sampling approach used. Figure 2: Evaluation Sampling Approach METHODS, APPROACHES AND TOOLS The evaluation team collected data using the following methods: Key informant interviews The evaluation team conducted key informant interviews based on interview guides. Strategic-level respondents formed the core of key informants at national, regional, USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 9 district and community levels. Respondents were drawn from USAID staff, USAID technical assistance partners, partner staff, stakeholders and community key informants (e.g., village or ward executive officers, community-level sub-grantee staff and community-based volunteers). A list of key informants is included in Annex III. Focus group discussions (FGDs) The evaluation team conducted FGDs to collect data from beneficiary groups in the implementation communities. Discussion participants were drawn from MVCC members, caregivers and children supported by the program. Household visits The evaluation team conducted household visits to observe and verify claims of livelihood improvements and asset accumulation. This dimension was added to ensure more credibility; beneficiaries for household visits were not notified prior to visits but were selected during group discussions. Self-administered interviews The evaluation team shared key informant interview guides with sub-grantee staff who completed them. The approach was necessitated by time constraints that prevented the evaluation team from conducting one-on-one interviews with all sub-grantee staff members. Secondary data The evaluation team did not collect quantitative primary data and instead relied on the PEPFAR Reporting and Organizational Management Information System (PROMIS). To facilitate systematic secondary data extraction, a template was developed with the specific partner indicators and the data requirements. LIMITATIONS  Limited quantitative outcome-level data: The evaluation collected qualitative primary data and used secondary sources for quantitative data. There was limited outcome-level data on components such as household hunger, dietary diversity, access to education, access to health, HIV knowledge, stigma, vulnerability and resilience.  Time: The evaluation design included site visits to the five regions (Mainland and Zanzibar) and two districts (urban and rural) within each region, which required more time.  Personnel: One member of the planned four-person team was unable to join the team, and this affected the initial plan for splitting into two teams of two team members. The figure below summarizes the evaluation methodology. 10 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Figure 3: Evaluation Methodology USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 11 4. FINDINGS PAMOJA TUWALEE DESIGN The PT design includes multiple partners with clearly defined responsibilities at different levels. All partners acknowledged that the design is clear and facilitates implementation of strategies aimed at improving the quality of life and well-being of OVC. The design is strong and provides scope for sustainability, as it is aligned to the NCPA II. The table below shows strategic objectives from the NCPA II and the corresponding objectives from the PT program. Table 4: Alignment between PT and the NCAP II Strategic Objective within NCPA II Corresponding Strategies within PT Strengthen the capacity of households and communities to protect, care and support MVC. Provide comprehensive services strategically aligned to OVC and their household needs. Enhance sustainability in care and support services. Support the OVC household rather than individual OVC; i.e., support interventions that strengthen the capacity of families, and secondarily that of OVC themselves, to better provide for long-term OVC needs. Increase access to effective, gender-responsive child protection services within a well-resourced child protection system that has the best interest of the child at its core. Provide comprehensive services strategically aligned to OVC and their household needs. Support all MVC/OVC by addressing issues of gender and age to respond to the differing needs of boys and girls. Enhance sustainability in care and support services. Expand access to education, health and early childhood development services. Ensure that there are meaningful linkages and referrals to other AIDS, health and development services. Strengthen coordination, leadership and the policy and service delivery environment. Support coordination of all stakeholders within the implementing areas to ensure linkages and comprehensive support. Each of the implementing partners (Pact Inc., World Education Inc., FHI360 and Africare) has its own strategic objectives, but all contribute to the overall objective of the program. Design strengths are evident in the centrality of capacity building for all partner programs. This is a direct response to findings from consultations on PEPFAR II OVC strategy development, which concluded that “although the increase in the number of MVC provided with critical services has been commendable, the manner in which these services have been provided is generally unsustainable in the long run” (RFA, 2009). Additionally, all implementing partners have prioritized household economic strengthening, which is central in sustainably capacitating households and communities to support MVC. The operational design also allows partner autonomy while facilitating contribution toward the program goal. Oversight is provided by USAID, and 12 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation implementing partners work with sub-grantees and LGAs. While MVCCs are the key local level structures responsible for supporting MVC, there are also volunteers. The figure below shows the operational structure for the program. Figure 4: Program Structure Within the program design, support is primarily provided for strengthening the household, which will result in support for MVC. This is in line with the shift away from supporting the individual child and toward supporting the household (RFA, 2009). Based on the design, key outcomes at mid-term demonstrate improvements at the household level, with additional outcomes in education and health, where improved income from economic strengthening groups is being invested to support MVC needs. The design further includes provision of technical assistance from specialized institutions. MEASURE provides technical assistance in monitoring and evaluation (M&E), and DAI provides technical assistance on economic strengthening. Technical assistance in M&E has contributed to strengthening data quality systems by reviewing partner M&E systems, identifying strengths and weaknesses, providing recommendations for improvements and mentoring. Areas for improvement related to M&E as documented by MEASURE are listed below (MEASURE, 2013):  A documented M&E training plan for staff at all levels of the reporting system was absent.  M&E plans do not clearly state the amount of funds to be used for M&E activities.  There was an absence of document retention policies outlining how long source documents should be kept before being destroyed. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 13  M&E functions are narrowly conceived in terms of routine data collection processing and analysis. Full application of M&E by all partners to guide decision-making and programming was still lacking.  Not all the relevant staff understood PEPFAR indicators very well.  Although partners have developed guidelines on ways to avoid double-counting, these were not disseminated well to lower reporting levels, especially for MVCCs, para–social workers and empowerment workers involved in filling in the primary data sources. While the strategic and operational components of the design facilitate achievement of results, there is scope for improving coordination and collaboration across implementing partners. Quarterly meetings convened by USAID provide platforms for sharing across partners, though the design of the program does not include built-in mechanisms that facilitate stronger coordination. Partner staff reported not having contractual obligations to invest in coordination and experience sharing. THE ZONAL APPROACH All four implementing partners, government and stakeholders such as UNICEF concurred that the zonal approach is working well. The approach was introduced to enhance three components: equity, accountability and efficiency. Equity All partners acknowledged that the approach has allowed them to be more focused within a specific geographic area and has further facilitated stronger reflection on the need to ensure access to services to those in need, as opposed to those residing close to where the partner has offices. All partners highlighted that working in a specified geographical zone has ensured they deliberately focus on both rural and urban districts, as they are solely responsible and accountable for delivering support within their zones of focus. For example, 75 percent of WEI’s districts are rural. All four implementing partners further outlined that the zonal approach has contributed to strengthening capacities of CSOs in specific geographical areas, as opposed to the previous approach where implementation would be done mostly by international non-governmental organizations. CSOs reported that the zonal approach has ensured they have better chances of receiving and delivering support within familiar communities, contexts and systems. Accountability Another salient outcome of the zonal approach relates to improved accountability across implementing partners. All four partners reported more incentive to achieve results, as they will be solely responsible for delivering PEPFAR/USAID support in a defined geographical zone. Two partners (FHI360 and WEI) mentioned that the zonal approach has increased innovation among partners, especially as their responsibility for the zone allows them to develop more sustainable networks with stakeholders and government. As partners are contractually answerable for results in specific geographical areas, they have invested in capacitating government structures for 14 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation sustainability. Partners highlighted where they previously focused on documenting and articulating structural weaknesses of MVCCs, they are now working toward capacitating those structures. Key informants from the GoT and UNICEF also mentioned that the approach strengthens coordination and ensures tailored, responsive approaches as partners have to consult with authorities within their zone to ensure approaches conform to the realities of specific contexts. In addition, LGAs in a specific region do not have to deal with many different implementing partners using different approaches to support the same beneficiaries. Accountability and standardization of approaches is critical, as all implementing partners highlighted that the capacity-building approach has been affected by other development actors providing direct material support. This reportedly creates conflicting perceptions among authorities and beneficiaries who are being supported by the program to transition from expecting material support to assuming responsibility for supporting MVC. Efficiency Another expectation from the zonal approach relates to efficiency as partners work in defined geographical zones, leverage existing strategic partnerships and invest in context-specific approaches. All partners acknowledged that the approach has potential for improving efficiency, but indicators of efficiency are not in place. Some implementing partners interpreted efficiency in relation to savings as they focus on a defined geographical area, while others interpreted efficiency in relation to potentially improved program quality as they work in a defined area with the same group of stakeholders. The zonal approach is supposed to improve issues of efficiency, equity and accountability, but all partners reported that stronger evidence of outcome-level changes would only be generated after a longer period of sustained investment within the specific zones. IMPLEMENTING PARTNER PERFORMANCE Africare Description Africare’s implementing approach has a strong capacity-building component. The organization uses household economic strengthening models as platforms for provision of sustainable direct support services to MVC. Between the first and second years, the project directly procured community health fund cards and birth certificates, paid school fees for MVC attending vocational education and facilitated all cases of child abuse through the justice system. When the savings groups matured and had saved sufficient resources through social and MVC funds, caregivers started procuring scholastic materials and uniforms for MVC, paying community health fund premiums, providing food rations and reconstructing shelter. Africare has developed an innovative partnership with a private sector company (Cheetah Development Cooperation) that is supporting farmers to access capital, increase production, reduce losses and access markets. The approach consists of three core components: USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 15 1. Capacity Development of LGAs: This component focuses on strengthening the institutional capacity of Tanzanian stakeholders. 2. Sub-granting to CSOs: This expands service provision to children and their households based on assessed needs. 3. Coordination: This strengthens the MVC response through establishing and supporting MVCCs at the community level. The approach further strengthens implementing partner groups (IPGs) at district and national levels. Objectives 1. Strengthen service delivery systems, ownership, planning, coordination, management and monitoring of community-level MVC interventions. 2. Ensure delivery of a comprehensive package of health and social services to MVC and their households that addresses their unique needs. 3. Support child protection systems and increase youth participation in addressing problems and issues affecting MVC. 4. Strengthen capacity of Tanzanian institutions to provide leadership in addressing MVC issues. Specific GBV objectives 1. Engage program beneficiary communities in prevention of GBV through community-level sensitization, outreach and resource mobilization. 2. Increase participation of children and youth, including girls, in preventing and addressing GBV within their communities. 3. Appropriately link communities with legal, psychosocial and clinical services to support survivors of GBV. 4. Strengthen the capacity of key community stakeholders, including MVCC members, para-social workers, community justice facilitators, teachers and faith and traditional leaders to take leadership in preventing and addressing GBV from the community to the district level. Progress toward achievement of objectives Africare has made significant progress toward achieving targets for key PEPFAR indicators. The number of people reached with GBV services was surpassed by 102 percent, while 96 percent of the target for the number of eligible clients who received food or nutrition has been achieved. Africare further surpassed targets for the number of eligible OVC provided with a minimum of one core care service (102 percent) and the number of community health and para-social workers who successfully completed a pre-service training program. Support has ensured MVC have access to community insurance and education services. Improved caregiver capacities reportedly contributed to improved reporting of child abuse, with 947 cases being reported and addressed, compared to none reported prior to the interventions. The table below shows the number of MVC who have benefitted and the type of support received. 16 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Table 5: Africare MVC Beneficiaries by Type of Support Received Type of support Number of MVC who benefitted Community health insurance coverage 20,290 Support to access primary education 20,965 Support to access secondary education 1,939 Vocational skills training 177 MVC caregiver houses renovated 190 Source: Africare, 2013 Africare is also on track (50 percent achieved) to achieve the target of the number of vulnerable households with at least one OVC/MVC, person living with HIV (PLHIV) or caregiver provided with a minimum of one economic strengthening opportunity or support. The figure below shows progress made against each of the PEPFAR indicators. Figure 5: Africare Progress against PEPFAR Indicators In line with the capacity-building approach, Africare successfully supported all its 19 district councils to plan, coordinate and monitor implementation of the PT program in the Central zone. All five Africare government stakeholders mentioned having benefitted from joint planning and implementation, especially in conducting MVC identification. Joint planning and implementation has reportedly contributed toward building capacities of government departments like the District Social Welfare Office and District Community Development Office to conduct MVC-supportive supervision to ward and village levels (Africare, 2013). USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 17 Support to district councils is consistent with one of the approaches recommended in the RFA, which prioritizes reinforcing the capacity of LGA to support OVC activities and to implement and manage the roll-out of the data management system and the NCPA. It is also consistent with the second objective of PT, which focuses on “strengthening the capacity of LGA and MVCC to support OVC and their households in a sustainable manner” (USAID, 2009). The organization has provided support for training ward executive officers who are now providing coordination support to village structures and cadres like MVCCs and community volunteers. Support ensures stronger coordination of MVC support at the local level as well as improved community willingness to assume responsibility for supporting MVC. A total of 888 MVCCs (694 in Iringa/Njombe and 194 in Dodoma/Singida) have been formed. Formation and strengthening of MVCC provide the basis for sustainability, as these are structures assigned to support vulnerable children and are based in communities; the GoT and UNICEF describe them as being “closest to MVC.” Africare has supported MVCCs in resource mapping to identify potential income sources, especially within their communities. Africare-supported MVCCs secured support from the municipal council for supporting projects aimed at economic strengthening to support MVC. In Iringa, six income-generating activity groups received additional financial support from the municipal council as start-up capital worth Tsh 3,000,000 (US$1,923.07) to expand their selected activities. Beyond support at the group level (MVCCs), individuals reported having received support from Africare-supported partners. Beneficiaries in all four FGDs mentioned now having the knowledge that caring for MVC is their responsibility. Additional benefits from FGDs are outlined in the table below. “Urbanization (losing a sense of caring) and donors made us too reliant. We have forgotten what it was like before to take care of ourselves and those who are vulnerable in our communities. With this project we are now going back to what we used to be.” Village Executive Officer, Dodoma 18 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Table 6: Benefits Reported by Africare Beneficiaries Issue Frequency of Reporting in the Four FGDs There is awareness that it is the community’s responsibility to look after MVC in their own communities. 8 times Child rights and protection is new and well understood by both communities and volunteers. 7 times Para-social workers are at every village and have assisted MVCs and child-headed households. Support ranges from MVC identification, providing referrals to services and providing community-based psychosocial support. 7 times Communities are recognizing the importance of education. “Monitoring is enabling those who are not going to school to be identified and assisted to go back.” 5 times Economic strengthening Africare supported 17,514 MVC caregivers who are now members of SILC groups. The approach has been broadened to ensure groups include members who are not MVC caregivers but who have overall ability to contribute to community resilience for supporting MVC. The table below shows the number of groups, total number of SILC beneficiaries and total cumulative savings. Table 7: Savings for Africare SILC Groups Number of Groups Members of Groups Cumulative Savings 1,196 28,583 Tsh 4,157,211,962 (US$2,511,380) The average number of members per group is 24, and this is within the recommended group size of between 20 and 25 (Africare, 2011). Individual savings are Tsh 145,444 (US$87.86). Africare has the highest individual savings across all PT implementing partners, but there has not been further investigation into factors behind the high individual savings. Average individual savings are higher than the food poverty threshold (an income less than Tsh 26,085 (US$15.76) per adult per month), and the basic needs poverty threshold (an income less than Tsh 36,482 (US$22.04) per adult per month) (Tanzania Household Budget Survey, 2013). “We can be proud that we have people in groups who would never have been able to build their own houses but now they have done it because of access to SILC loans. There is a woman who did not have anything, but now she has managed to build her own house, she is renting rooms out and this has increased her income at household level.” Female beneficiary, Dodoma USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 19 Unique Components Private sector partnerships Africare has a unique approach to private sector partnership, evident in the relationship with the for-profit Cheetah Development Cooperation. Cheetah specializes in agricultural investments to provide economic improvement to 200 households enrolled with two of the project’s sub-grantees, Iringa Mercy Organization and Ilula Orphan Program. The agreement involves organizing farmers into smaller teams, training them in business farming, providing them with loans for purchasing necessary equipment and supplies and linking them with markets for their agricultural produce. Sub-grantee staff and beneficiaries concurred that the partnerships facilitated an expansion of beneficiaries’ access to loans, markets and harvesting expertise. The primary lesson learned from the partnership is that priority should be given to building on existing livelihood options for communities. Cheetah Development Cooperation provided capital and technical expertise to enhance what farmers were already doing. Sub-granting to technical partners Africare has broadened sub-granting beyond small CSOs to institutions providing specialized technical expertise relevant to the project. Specialist partners include TAHEA (food and nutrition), UMATI (youth participation, child protection, HIV prevention and adolescent sexual and reproductive health), DAI IMARISHA (household economic strengthening), Cheetah (household economic strengthening) and the Regional Psychosocial Support Initiative (REPSSI). Qualitative evidence showed that kids’ clubs have improved MVC psychosocial well-being, while school health clubs are contributing to reductions in diarrheal diseases, especially among school children. Staff acknowledged that Africare does not have specialized expertise in all areas, and partnerships with specialized organizations will increase the quality of support. The table below shows changes recorded as a result of contributions made by each of the partners cited above. Table 8: Africare Partner Contributions Partner Activities and Outcomes TAHEA TAHEA conducted food and nutrition assessments and has identified and referred 35,545 MVC to receive food (referrals resulted in receipt of food). UMATI Youths have received ASRH information, 6,030 MVC accessed birth certificates and 947 cases were referred for and received legal support. Cheetah 200 farmers received inputs, training on farming, post-harvest handling and linkages to markets. REPSSI 187 kids clubs were formed, and schoolteachers and community volunteers were trained, resulting in 8,608 MVC understanding their rights and knowing where to report in case of abuse. 20 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Working with faith leaders The organization has prioritized working with faith leaders, as they play a key role in supporting MVC. Evidence from consultations during the evaluation and in Africare reports from Iringa demonstrates that buy-in from faith leaders will strengthen advocacy efforts and potentially contribute to overall program sustainability. FHI360 Description FHI360 implements the program in the coastal zone covering Dar es Salaam, Coast, Zanzibar and Morogoro. The FHI360 approach is based on two strategic components: Integration: Strengthen integration and enhance linkages at all levels to increase sustainability and produce better results. Empowerment: Empower stakeholders to improve performance and meet their own needs by building on their strengths. Objectives 1. Increase the capacity of communities and local governments to meet the needs of OVC and their households in an innovative, efficient and sustainable manner by enhancing their competencies to provide support and by improving communication, coordination and collaboration across sectors. 2. Increase the capacity of households to protect, care for and meet the basic needs of OVC in a sustained way by improving their caretaking, livelihood and health-seeking skills. 3. Increase OVC household access to comprehensive, high quality, age-appropriate and gender-sensitive services by creating integrated community-level referral networks that strengthen the continuum of care. 4. Empower OVC, particularly females, to contribute to their own well-being by improving their resilience, as well as their livelihood and self-care skills. Additional objectives 1. Accelerate start-up in the outgoing partner program areas; these are areas which were originally served by Deloitte Consulting Limited, i.e., Coast, Morogoro and Zanzibar. 2. Strengthen the capacity of MVC households headed by youth or elderly caregivers. 3. Pilot protection of MVC living or staying in the street within the PT program. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 21 Progress toward achievement of objectives FHI360 has made significant progress toward achieving targets for key PEPFAR indicators. Targets have been surpassed for some indicators, while the program is on course to achieve or surpass targets for the others. The program has surpassed by 62 percent (162 percent achievement) the target on the number of eligible clients who received food and nutrition services. Similarly, FHI360 has surpassed targets for the number of vulnerable households with at least one OVC/MVC, PLHIV or caregiver provided with a minimum of one economic opportunity or strengthening support (194 percent achievement). The organization successfully trained (pre-service) 2,329 community health and para￾social workers, representing 121 percent achievement against a target of 1,924. Nearly all (99 percent) of the OVC targeted to have been provided with a minimum of one core care service were reached. The figure below shows progress made against PEPFAR indicators. Figure 6: FHI360 Progress against PEPFAR Indicators “If the children’s club is to be stopped I will be sad because I like being part of it.” Boy aged 9, part of children's club The program successfully supported LGAs in reviewing and updating the stakeholders’ inventory, a list of all organizations supporting MVC in a district and the nature of support provided. Reviewing and continually updating the inventory reportedly contributed to ensuring comprehensive care and support is provided to MVC in a coordinated way. A total of five out of eight government stakeholders mentioned support for reviewing and updating the inventory, reportedly contributing to stronger coordination. Previously, different organizations would provide support to the same beneficiaries without platforms for leveraging potential implementation partnerships. 22 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Through the program, an MVC focal person for FHI360 sub-grantee WAMATA participated in a study tour to Ethiopia focused on improving nutritional status and economic strengthening at the household level through urban gardening and SILC groups. Knowledge from the tour was shared during volunteer meetings. Twenty-one (21) female-headed households and MVC caretakers had started putting the skills into practice, and 10 (one male and nine female) MVC caretakers had established their own gardens. The approach used by FHI360 has a strong advocacy focus, and qualitative accounts showed that some LGAs are budgeting for MVC. While budgeting demonstrates commitment, LGAs do not have adequate resources to increase support for MVC. Partners and stakeholders mentioned the need for continued advocacy for LGAs to provide resources but further recommended expanding the scope to include strong private sector involvement. In addition, all eight stakeholders cited the magnitude of need as a key challenge, especially as there are children fitting the definition of MVC who cannot be supported due to resource constraints. The table below shows the amounts invested by FHI360 in supporting LGAs. Table 9: Amounts Invested by FHI360 in Supporting LGAs (Tsh) 2010/11 2011/12 2012/13 2013/14 TOTAL 34,500,000 (US$20,292) 58,390,000 (US$35,020) 85,841,254 (US$51,485) 98,000,000 (US$58,778) 276,731,254 (US$165,977) 15,000,000 (US$8,996) 29,760,000 US$17,849) 40,880,000 (US$24,518) 59,894,536 (US$35,923) 145,534,536 (US$87,288) 45,496,500 (US$27,287) 65,260,000 (US$39,141) 116,855,000 (US$70,086) 227,611,500 (US$136,516) 455,223,000 (US$273,032) 94,996,500 (US$59,976) 153,410,000 (US$92,011) 243,576,254 (US$146,091) 385, 060,36 (US$230,950) 877,488,790 (US$526,296) Economic strengthening FHI360 successfully supported 1,819 MVC caregivers with economic strengthening capacity building and mentoring. Caregivers in turn supported 1,298 MVC within the Coast zone. Beneficiaries cited SILC training as having been critical in transforming their mindset from dependency toward innovation. Beneficiaries mentioned support for the formation and monitoring of groups eight times in three beneficiary FGDs. The approach for supporting SILC groups has expanded membership to include men, which has reportedly increased their buy-in. Although current data does not show significant trends attributable to involving males, key informants highlighted that men play significant roles in household budgetary allocations and women’s contributions to “The FHI approach ensures sustainability of results beyond the project, as the community understands their roles to support their MVCs” Focal Person, WAMATA PEMBA USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 23 SILC groups are likely to be consistent if they have supportive partners. FHI360 supported formation of 256 SILC groups with a total of 6,663 members, TSh 584,439,820 (US$353,061) in cumulative savings and TSh 40,203,750 (US$24,287) invested in supporting MVC. The table below shows the number of SILC groups formed, members in groups, cumulative savings and amounts invested in MVC. Table 10: Overview of SILC Groups Supported by FHI360 Number of Groups Members of Groups Cumulative Savings (TSh) Amount Invested for MVC (TSh) 265 6,663 584,439,820 40,203,750 Female Male (US$350,533) (US$24,113) 5,164 1,499 The average number of members per group is 25, and average individual savings amount to TSh 87,714 (US$52.99). Average individual savings are higher than the food poverty threshold (an income less than TSh 26,085 (US$15.76) per adult per month); and the basic needs poverty threshold (an income less than TSh 36,482 (US$22.04) per adult per month). Unique Components Investment in LGAs to review and update stakeholder inventory The program successfully supported 25 LGAs in reviewing and updating the stakeholders’ inventory to facilitate referrals for other services in order to ensure comprehensive care and support to MVC and their households. The inventory ensured availability of information on the partners working within specific areas as well as the nature of support they provide. Support will potentially improve coordination and is aligned to the zonal approach. Child helpline The program supported innovation through its child helpline. The helpline reportedly increased access to services and information among victims of child abuse as well as community members. Uniqueness is evident in the potential to expand the scope and the opportunity for children and communities to report incidents. The table below shows trends in children who have used the helpline and the forms of abuse reported. Table 11: Statistics from the Child Helpline Total Number of Calls Physical Violence Abuse: Rape and Other Sexual Violence Neglect Exploitation 4,642 62 43 80 70 Source: Quarter III Report–Sema Tanzania Child Protection Programme. October 2013. 24 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Piloting one-stop GBV centers FHI360 further supported piloting of one-stop GBV centers to ensure reliable, comprehensive and sustainable services to survivors of GBV and violence against children. The pilot is in its initial stages, and when fully operational it is expected to provide comprehensive services to survivors of GBV within one place. In addition, the organization has contributed to national processes, especially development of national operationalization guidelines for the centers. As part of the pilot, FHI360 trained government officials, including police officers and clinical staff from Amana hospital. At the time of the evaluation, trainings had just been completed and there was not yet outcome-level evidence of what had changed. Support to children living and working on the streets FHI360 is planning for a pilot program to support children living and working on the streets. Assessment of these children has been done in one region in collaboration with the GoT and other stakeholders. The program strategies focus on prevention and impact mitigation support. Planned support is aimed at preventing violence against children living on the streets as well as exploring ways of reintegrating them into their families and communities. Strong partnership approach The organization is pursuing a strong and unique partnership approach where sub￾grantee staff is part of annual planning processes. Six out of seven staff members from two sub-grantees (WAMATA Pemba and ZAMWASO) acknowledged the strong partnership approach as central to sustainable capacity building. Successfully working in Zanzibar Although all implementing partners work in specific zones, these are all located within Tanzania Mainland, which has similar government systems and structures across zones. FHI360 has successfully created synergies and worked in Zanzibar, which has a different religion (99 percent Muslim) as well as different systems and structures. Challenges Working with different systems and structures in Zanzibar Working in Zanzibar, with its different structures, systems and religion, posed challenges around the modalities for engaging authorities and implementing strategies that conform to the religious context. These challenges have required investments in engagement techniques like removing the component of interest in the operations of SILC groups, because interest is prohibited within Islam. Instituting a referral-tracking system The organization provides referrals to MVC, but there is no strong system that facilitates tracking to check if the MVC receive support from referred services. In one instance, a USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 25 one-stop center run by Save the Children is located in the casualty section, where other health emergencies are prioritized, resulting in longer waiting times. Resolving the conflict between the SILC methodology and dictates of religion FHI360 is implementing SILC in a predominantly Muslim context, where charging interest is prohibited. However, interest is a core component of the methodology and this raises challenges for beneficiaries. Beyond constraining the capacity of SILC groups to fully apply their methodology, there is potential to upset religious dynamics. That could potentially have broader negative impacts on the program, because its sustainability strategy is built around working with the government. Partners have developed strategies for addressing the challenges and in Pemba, SILC groups do not charge interests but use the collective system (collective business) to gain profit. Another strategy in Unguja involves two groups that do not charge interest but only save, provide loans and collectively contribute to MVC funds. Addressing corporate governance challenges within sub-grantees Some sub-grantees have corporate governance challenges, with founder members reportedly resisting institutionalization of transparent systems. FHI360 further outlined challenges around staff turnover. According to FHI360 staff, “CSOs are often the entry level for staff engaged in development work—as soon as they have acquired skills and experience, they often move on to international non-governmental organizations, donor agencies and other international organizations that offer better employment terms.” Partners highlighted that beyond training, investments need to be made to institutionalize learning as opposed to focusing only on individual staff members, who often leave once they are trained. Additional challenges are outlined below according to their frequency of mention in FGDs. Table 12: Challenges Outlined by FHI360 Beneficiaries Issue Frequency of Mention in Three FGDs High levels of vulnerability and project reaching small proportion 7 times Low volunteer motivation 6 times Challenges in transitioning from material support to capacity building 4 times World Education INC. (WEI) Description WEI operates in the Northern zone coverage area: Arusha, Kilimanjaro, Manyara and Tanga regions. The organization’s approach is based on four core components: 26 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 1. Advocacy (to ensure GoT and all its structures at all levels support MVC) 2. Multi-sectoral school assessments (jointly conducted with the Department of Social Welfare (DSW) as part of capacity building and skills transfer) 3. Child rights clubs (to ensure provision of psychosocial support within the school setting) 4. Economic strengthening (to strengthen household and community capacities to support MVC) WEI/Bantwana located its country headquarters and key personnel in Arusha, rather than in Dar es Salaam like the other partners. Proximity to GoT partners and better familiarity with local conditions reportedly allows the organization to adapt programming to local circumstances and increase penetration in a concentrated area. WEI has taken advantage of technical assistance available through PT. In addition to DAI Imarisha and MEASURE Evaluation, WEI also received technical assistance from FHI 360 through its FANTA3 food and nutrition project, and Mwanzo Bora nutritional program (under Africare as prime). WEI works closely with nine sub-grantees and is the process of expanding to 10. Objectives 1. Increase access to and use of comprehensive MVC services through community initiatives. 2. Strengthen human and organizational capacity of local community structures (MVCCs and community-based organizations) and LGAs to meet the needs of MVC. 3. Increase community awareness and engagement, children’s participation, and advocacy for social protection of MVC. Progress toward achievement of objectives WEI has surpassed targets for all the core PEPFAR indicators relevant to their PT program. The organization has reached 98,610 eligible OVC with a minimum of one core care service. This was against a target of 76,300, representing 129 percent achievement. WEI surpassed targets for the number of eligible clients who received food and nutrition services as well as the number of vulnerable households with at least one OVC/MVC, PLHIV or caregiver provided with a minimum of one economic opportunity by 18 percent (118 percent achievement for both indicators). The figure below shows achievements against key PEPFAR indicators. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 27 Figure 7: WEI Progress against PEPFAR Indicators Source: WEI M&E Data WEI has created strong relations with LGAs and signed MoUs with 12 of the 16 councils covered by the project. This has resulted in documented commitment from LGAs to support MVC. In addition, respondents highlighted that MoUs ensure continuity especially as they counteract high staff attrition within LGAs. Government stakeholders acknowledge the relationship as “formalized and institutionalized,” which is important for sustainability. Beyond signing MoUs, the partnerships with councils include MVC action plans committing finances for MVC support. All councils have committed to earmarking funds for MVC, but WEI reported that councils do not have resources to fund those commitments. In response to challenges facing councils, WEI is focusing more on economic strengthening for groups while consistently engaging councils to provide support such as land for groups engaging in economic strengthening activities. WEI has facilitated health fee exemptions for 8,991 MVC. The achievement is significant given that none of those MVC had previously benefitted. WEI also provided referrals to 1,052 severely malnourished MVC to receive nutritional support. However, there is no referral-tracking system to document whether or not children received those services. The organization further supported processes of MVC identification, resulting in the identification of 31,848 MVC. Psychosocial support The organization supports school-based child rights clubs, which have been successful as children attend school and then receive psychosocial services through clubs. Qualitative evidence pointed to low levels of school dropouts among club members, though there is limited program data on dropouts and the overall benefits of the 28 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation approach. Members of MVCCs outlined that the approach involves teachers, administrators and community members as leaders, and it further links MVC care to the broader community. Multi-sectoral school assessments WEI successfully worked with government in conducting multi-sectoral school assessments. The assessments include a special methodology that involves multi￾disciplinary teams working together to assess the vulnerability of children against indicators for nutrition, birth registration, access to health and education, protection and access to psychosocial support. Schools, LGAs and partners take the lead in conducting assessments with support and facilitation from WEI/Bantwana. The organization has prioritized strengthening referral tracking and this will be a core component of work during FY14. Economic strengthening The organization developed the Livelihoods Improvement for MVC Care (LIMCA) model for economic strengthening. The model is based on adaptation of the best features of SILC and WORTH formats. The table below shows the gaps in WORTH and SILC as well as how they are addressed within LIMCA. Innovations of the Multi-sectoral School Assessment  It reinvigorates the MOHSW’s school health program through joint implementation and helps MVC to remain in school and thrive through referrals and waivers to access health services.  It serves as a catalyst for identifying and registering MVC, removing a critical barrier for needed support. This is done through a scientific and objective assessment methodology that ensures MVC with the greatest need are prioritized.  It strengthens relations and coordination between schools, communities and LGAs, leveraging important district and community resources on behalf of MVC. The assessments include all community actors where previously support was not well coordinated.  It uses a low-cost approach for providing critical health services and referrals for other key services using locally available services and resources.  It fosters development of a simple system that more effectively ensures MVC receive services.  It generates, analyzes and disseminates important MVC data that contributes to district planning and resource allocation for MVC. Source: WEI Annual Report FY13 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 29 Table 13: Emerging Issues from a Review of WORTH and SILC Gaps (WORTH, SILC and VS&L) Solution (LIMCA) Groups were composed only of MVC caregivers. Broadened membership with a safeguard of at least 60 percent MVC caregivers to ensure broader community appreciation of the need to take responsibility and support MVC. Groups exclusively targeted women. Opened up to 40 percent of membership to men. This ensures support for MVC who are under the care of men only (fathers). It will further provide men with an appreciation of the importance of saving for purposes of supporting children. Community volunteers were assigned to cover several villages. Community volunteers reside in villages and support one or more groups within their villages of residence. No standard mechanism regulated that loans should be used for income-generating activities. A system was established that asks members to commit to using loans for income-generating activities. However, there is still no mechanism to ensure the commitments are honored. A total of 198 WEI LIMCA groups have been formed and have promoted internal savings by community members. In addition, 4,720 caregivers supporting 6,433 MVC are part of LIMCA. The table below shows an overview of LIMCA group performance. Table 14: WEI LIMCA Group Performance Number of Groups Members of Groups Cumulative Savings (TSh) Total Number of Loans Issued Value of Loans Issued (TSh) MVC Reported to be Supported 198 4,720 199,620,800 (US$119,728) 2,575 120,708,200 (US$72,397) 6,433 Female Male Girls Boys 4,349 371 3,225 3,208 The average number of members per group is 24, with an average individual savings of TSh 42,292.54 (US$25.55). The average amount borrowed as loans amounts to TSh 46,876.97 (US$28.32). Average savings and average loans are higher than the food poverty threshold (an income less than TSh 26,085 (US$15.76) per adult per month), and the basic needs poverty threshold (an income less than TSh 36,482 (US$22.04) per adult per month) (Tanzania Household Budget Survey, 2013). Unique Components School-based child rights clubs WEI’s approach of integrating psychosocial services into school-based clubs is innovative and ensures children have access to education and do not have to find time outside school days to attend the clubs, as opposed to other models not based in 30 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation schools. However, although this model has successfully reached children in school, it has not included approaches for reaching children out of school. Beyond ensuring support within school settings, the approach reportedly builds sustainability as it involves training both teachers and children to facilitate groups. WEI staff and stakeholders acknowledged that the school-based psychosocial services approach is a critical innovation as there is growing recognition of other vulnerability factors, such as poverty, that are broader than HIV and AIDS. Multi-sectoral school assessments The approach to conducting multi-sectoral school assessments has ensured practical collaboration with government. Strong collaboration is consistent with the program’s goal of developing government competencies at different levels. Multi-sectoral school assessment teams reportedly supported identifying eligible children and updating the MVC register quarterly, identifying and referring children in need of birth registration, screening children for health and nutrition issues and referring to services as necessary, and screening for child protection and psychosocial issues, referring to a trained school counsellor or to other services as needed (WEI, 2013). Advocating for health fee exemption WEI advocates for health fee exemptions while other implementing partners prioritize community health funds. Waivers through health fee exemption cards ensure access to services for MVC until they are 18 years old, while within the community health funds there is need for annual renewals. Annual renewals are meant to facilitate periodic reviews and potentially graduating some OVC whose vulnerability might have been reduced. Over the life of the project, 13,132 exemption cards have been processed and 8,991 have been distributed (WEI, 2013). To ensure availability of services, WEI has invested in advocacy with service providers and seven councils (Tanga City, Korogwe Town, Muheza, Pangani, Handeni, Same and Siha) have endorsed the provision of health fee exemption cards to MVC. Despite ensuring MVC have waivers, the project has no influence on availability of services, so children may have waivers for services that are not available. Supporting birth registration Eligibility for health fee exemption is dependent on whether or not an MVC has a birth certificate. The program has moved beyond only supporting access to exemption cards toward prioritizing access to birth registration so that MVC are eligible for exemption. Facilitating birth registration is significant in the context of low birth registration, as demonstrated by the table below from the 2011 Tanzania Demographic and Health Survey. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 31 Table 15: Birth Registration Levels within the Northern Zone Region Percentage of Children under 5 Years Whose Births Are Registered Arusha 25.3 Kilimanjaro 7.5 Manyara 22.7 Tanga 4.1 Source: TDHS, 2011 Challenges High levels of poverty within communities weaken livelihoods and limit the ability to save. Strengthening productive capacities of households through support in agriculture and vocational skills training was cited as a precondition for successful saving through SILC. Weak livelihoods increase the number of MVC and limit community capacity to support them. The overwhelming number of MVC versus the capacity of WEI and other partners to provide support was cited as a key constraint seven times in three focus group discussions with beneficiaries. High levels of poverty also mean there are multiple competing household needs and MVC needs may not always be prioritized. In Karatu there were challenges of substance abuse and female genital mutilation. These heighten the vulnerability of children and require investments in strengthening livelihoods as well as social and behaviour change communication around female genital mutilation to address the context that predisposes children to vulnerability. The challenges outlined above are compounded by lack of data on their magnitude, with studies concluding that the problems are much broader and pronounced than official statistics show (Mwema Children, 2011). There are gaps in ensuring LGAs honor their funding obligations for MVC. While WEI has MoUs with 12 districts, seven councils have endorsed the provision of health fee exemption cards to MVC. All six government stakeholders also cited limited fiscal capacities as the key factor behind limited LGA capacities to provide services. Addressing this gap is beyond the scope of the project, though continued advocacy was cited as critical by key informants to ensure LGAs are aware of their commitment and begin to provide funding even at a small scale. By design, cost for MVC birth certificates was expected to be covered through waivers by the Registration, Insolvency and Trusteeship Agency (RITA) or cost-sharing with district councils. However, RITA declined to grant any waiver or arrangements outside the provisions of the Births and Deaths Registration Act (CAP 108, R.E. 2002). “Even when children are treated or provided with 21 days Plumpy’Nut, they get better for a short period but the environment is still the same, sooner or later they become malnourished again. They live in a cycle of poverty.” Female Government Key Informant, Karatu 32 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Similarly, a presentation to the national IPG indicated that RITA has signed an MoU with the Prime Minister‘s Office – Regional and Local Governments (PMO-RALG) in which the LGAs will only play a facilitation role in birth registration in their areas of jurisdiction and not incur any other costs. Services are not always available even if an MVC has a health fee exemption card. In addition, there are gaps in the referral-tracking system where WEI or sub-grantees cannot track whether MVC receive services for which they would have been referred. The organization has prioritized strengthening referral tracking in 2014 but there are no clear plans to address supply-side challenges where services are not available. Although the LIMCA came out of a review of WORTH, SILC and VS&L, a rigorous review has not been conducted to ensure its fit for purpose. In addition, while the WORTH model was developed by Pact, a review of the model to come up with LIMCA did not involve Pact. Pact, Inc. Description The Pact program covers the Lake and Southern zones, including the eight regions of Mtwara, Lindi, Mwanza, Geita, Mara, Musoma, Kagera and Tabora. A modification was made in 2011 expanding program coverage to include the Southern Highlands zone, with its four regions of Mbeya, Ruvuma, Rukwa and Katavi. The program approach has a multi-level focus aimed at the child, the family, CSOs and government as outlined below. Child level: Facilitate access to essential health and social services, and identify and respond to the urgent needs of young children. Family level: Strengthen the capacity of caregivers to provide consistent and responsive care for young children by increasing their access to essential services through income generated from household economic strengthening activities. Civil society level: Strengthen the capacity of implementing partners’ empowerment workers and MVCCs to identify, address and meet the needs of young children. Government level: Strengthen local authorities to manage MVC programs and implement the NCPA II by building their knowledge and skills in identified capacity areas. Objectives 1. Increase community ownership and capacity to implement the NCPA, enabling MVC to access comprehensive care and support within their communities. 2. Strengthen the capacity of district authorities and local implementing partner organizations to manage the MVC program in their district or region. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 33 3. Replicate effective multi-sectoral coordination structures that include public￾private partnerships at district and village levels, with representatives engaged from all key stakeholders (children, families, community groups, service providers, local government and private sector). 4. Strengthen the capacity of families to meet the basic needs of their own MVC. Progress toward achievement of objectives Pact has surpassed targets for all PEPFAR indicators with the highest being 366 percent achievement for the number of people reached by an individual, small group or community-level intervention or service that explicitly addresses GBV and coercion. Figure 8: Pact Performance against PEPFAR Indicators Pact prioritized building the capacity of MVCCs, as they are the structures closest to MVC at local level. The organization developed a comprehensive plan, the MVC Committees Engagement Strategy, for improving the performance of MVCCs. Although Pact has developed mechanisms for measuring performance of MVCCs after supporting them, there are still gaps in measuring the impact of the conditions of MVC. Through the strategy, Pact works closely with focal persons from MVCCs, LGAs and sub-grantees. 34 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation “I am shining today because of WORTH, I can dress well, I can buy soap and lotion because of WORTH. I can meet my household needs because of worth. Before WORTH life was very difficult for us as a family. We did not have any money and could not build our own house. After the sensitization, and the information I received, I decided to join WORTH. I was not sure where I would get the money to save but I liked the idea. My first loan was Tsh 80,000 (US$48) in 2010. I was able to repay it in 3 months.” WORTH Beneficiary, Kagera In addition, Pact developed and delivered a comprehensive training curriculum to complement the DSW MVCC training curriculum. Launching of the MVC Committees Engagement Strategy has reportedly contributed to revitalizing MVCCs. The number of active MVCCs reportedly increased by 39 percent from 966 in the third quarter of 2013 to 1,346 in the fourth quarter of 2013 (Pact Inc., 2013). The table below shows progress made toward revitalizing MVCCs and demonstrates the weak capacities of MVCCs as outlined by the NCPA II. Table 16: Functional MVCCs in Pact-Supported Regions Region Number of MVCCs Number of Functioning MVCCs (evidence of MVCC meetings) Mwanza 461 118 (26 percent) Tabora 473 225 (48 percent) Mara 522 241 (46 percent) Kagera 568 193 (34 percent) Mbeya 284 To be determined3 Rukwa 323 To be determined Ruvuma 179 To be determined Katavi 204 To be determined Mtwara 521 328 (63 percent) Lindi 270 89 (33 percent) Total 3,805 1,194 (31 percent) Training for partners and LGAs As part of partner capacity building, Pact has invested in trainings for partners and LGAs across the three zones from Mwanza, Mbeya, Ruvuma and Mtwara. As part of skills transfer and capacity building of government, trainings are co-facilitated by the MOHSW. 3 These regions transitioned from DoD and assessments on the number of functioning MVCCs are yet to be conducted. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 35 Key informants from Pact and the government concurred that through trainings, district facilitators are developing capacities to deliver training to MVCCs while ensuring availability of localized support and mentorship. In line with the broad scope of challenges affecting MVCC, trainings focused on MVCC roles and responsibilities, resource mobilization, quality improvement, participatory M&E and service mapping. Despite trainings and capacity-building initiatives, there has been minimal tracking of the impact on livelihood conditions of MVC. Joint supportive supervision Beyond providing one-off training, the approach used by Pact includes joint supportive supervision with sub-grantees and LGAs. This involves scheduled visits to different MVCCs to assess how they will be working against what they are expected to do. Supportive supervision reportedly strengthens institutions and improves program implementation as MVCCs know their progress will be monitored and assessed. Supportive visits are structured and focus on organizational development roadmaps as well as Comprehensive Institutional Strengthening Plans. Support further focuses on developing practical partner skills in using reporting tools, as well as overall data quality management. Support for data quality management is critical, as the data quality assessment conducted by MEASURE (February 2013) concluded that “in most partners, not all the relevant staff understood PEPFAR indicators very well.” Economic strengthening Pact has invested in formation, mentoring and overall support for WORTH groups as part of economic strengthening support. While all PT implementing partners rely on unpaid volunteers, Pact works with empowerment workers who provide more structured support at the community level. Roles of empowerment workers include group visits, biweekly visits to MVCC, literacy volunteer training, literacy assessment, rolling out management committee training to leaders of WORTH groups and specialized trainings on good parenting, child protection, GBV, supplementary literacy and math and reporting. Key informants and FGD respondents highlighted that since empowerment workers are paid, they have incentives to work well and deliver results compared to volunteers who are used by other partners. However, key informants from Pact, sub-grantees and government all questioned the sustainability of the model, especially as the program is focusing on transferring capacity and responsibility to government structures. The WORTH approach has facilitated economic empowerment together with literacy, critical because 28 percent of women in Tanzania are illiterate (TDHS, 2011). Beneficiaries reported having improved ability to transact and to plan activities due to improved numeracy skills provided through WORTH. In addition, communities have demonstrated willingness to invest their own resources as evidenced by the number of new businesses formed. 36 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation “Before worth my favourite job in the community was weeding other people’s fields for money. I was the champion of this. When WORTH was being talked about, I was the first one to resist because I could not see how it would help me. But my friend even gave me money to join. She told me, you can’t lose it because it’s not yours. I started to realize this might be better than weeding and now I am at the forefront of work. I have managed to save TSh 61,400 and have borrowed loans that have assisted me in petty trade. I never lack money anymore.” 36-year-old female WORTH member The table below shows the number of new businesses formed within the different Pact￾supported regions. Table 17: New Businesses Formed in Pact-supported Regions Region New Businesses Formed Mtwara 338 Lindi 90 Tabora 293 Mara 602 Kagera 278 Mwanza 421 Total 2,022 Beneficiaries are also borrowing from WORTH and investing in livelihoods improvement as demonstrated by the quote below from a WORTH group member who borrowed TSh 100,000 (US$60.41) as a loan from her group for purposes of constructing a toilet. “Before we used a very dirty and unsafe toilet constructed with banana tree leaves and grass, but now we use a good and modern toilet at least and everyone in our family enjoys using it,” reported 69-year-old female beneficiary Julia Village, Mtonya Ward, Newala town, Mtwara. Unique Components Inclusion of literacy in WORTH The WORTH model is highly unique as it introduces literacy to economic strengthening activities. Beyond using economic strengthening as a rallying point, WORTH addresses the challenge of low literacy, which is reportedly prevalent among most MVC caregivers. Some caregivers reported facing challenges in transacting due to weak numeracy skills. Pact staff highlighted that numeracy skills are critical as petty trade is a key source of livelihood for the majority of vulnerable households. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 37 Capacity-building experience Pact has strong global, regional and national experience in capacity building. The organization has leveraged that experience to develop models and capacity-building approaches such as an adult literacy component to respond to literacy challenges in Tanzania. The WORTH model is a key example of models developed and refined through implementation in Nepal as well as in Tanzania through the Jali Watoto program. Capacity-building tools Experience and capacity-building tools have also been leveraged; these include the Comprehensive Institutional Strengthening Plan, which identifies capacity gaps and facilitates formulation of capacity-building roadmaps. Pact also has tools for developing organizational development roadmaps for partners. Pact has supported 43 partners to develop Comprehensive Institutional Strengthening Plans. In addition to supporting development of these plans, Pact has worked closely with local organizations to generate sufficient evidence of capacity to graduate partners for USAID funding. Challenges Sustainability of empowerment worker approach Empowerment workers provide constant support, but only as long as there is funding to pay them. The end-of-project support to empowerment workers could affect beneficiaries as well as the transfer of responsibilities. The empowerment workers have reportedly ensured provision of quality services to MVCCs, but there could be challenges if Pact fails to secure funding for them to continue. Sustainability challenges are further compounded by well-documented attrition challenges among unpaid community-level cadres like para-social workers and other volunteers. Sustainability of children’s clubs based outside schools Pact is supporting children’s clubs based within communities. Reports from the organization show that children’s clubs are providing MVC with access to sexual and reproductive health information as well as broader psychosocial support. However, sub￾grantee staff raised concerns that children’s clubs based within the community are less likely to be sustainable than school-based clubs as there is no other incentive (besides psychosocial services) to ensure children consistently attend. Respondents suggested using school-based clubs to reach children in school and integrating economic strengthening in clubs for children out of school. Emerging contextual challenges In Kagera, there has been an ongoing operation for removal and repatriation of illegal immigrants, targeting people from Rwanda, Burundi, DRC and Uganda. The region lies on the border and some of the repatriated people were WORTH group members. Transitioning from direct services to capacity building While Pact is transitioning from material support, some organizations in the same implementation areas are providing material support. For example, in Sikonge and 38 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Urambo Districts, an organization called PROSPER provides cash and direct service provision to community members who also work with Pact under PT. WORTH Groups have TSh 273,311,581 (US$165,108) in savings for 17,489 beneficiary households. This equates to individual savings of Tsh 15,627 (US$9.44), which is below the food poverty threshold and basic needs poverty threshold. The low individual savings are affected by high levels of poverty, which constrain capacities for supporting MVC. ENHANCING SUSTAINABILITY Sustainability analysis focuses on the design as well as operational approaches. An Enabling Design All partners’ program designs are based on capacity building and skills transfer, which is the key to sustainability. The RFA outlines working with GoT structures and entities as a central sustainability strategy. Examples of partnership with the GoT include joint MVCC training between the MoHSW and FHI360, refresher training facilitated by Pact and the MoHSW and multi-sectoral school assessments executed by WEI in partnership with GoT structures. Based on the design, multi-sectoral school assessments, along with training and supervision of MVCC, are likely to continue as these were jointly conducted with government staff. Transitioning from Direct Support to Capacity Building Capacity-building approaches are facilitating movement from supporting the individual (MVC) to strengthening structures (family, community, CSOs and LGAs). The approach is prioritized within the GoT and provides adequate scope for transferring responsibility to the GoT. Approaches have included the training-of￾trainer approach to ensure capacity is institutionalized within the GoT and among sub-grantees. Supporting institutionalization of competencies was cited by key informants as important, because when previous approaches focused on capacitating individuals, those who received such support would use their acquired skills to obtain employment in agencies that offered better remuneration. Caregivers who benefitted from economic strengthening support expressed confidence that they are likely to continue beyond the project, because they now understand how their groups should function. Strengthening the First Line of Support for MVC Stakeholders as well as the NCPA II characterize MVCCs as the first line of support for MVC. All partners have acknowledged the significance of the structure as it is based within communities and is closest to MVC. The MVCCs are likely to be essential to sustaining impacts from the program. All partners have prioritized reviving the MVCC “I was left with grandchildren that I could not support or able to pay school fees. Through being a member of SILC, I am now able to send them to school.” 62-year-old SILC group member, Dodoma USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 39 fund, which key informants cited as crucial in ensuring MVCCs have resources to carry out their activities. However, key informants acknowledged weaknesses that mostly arise from its volunteer-driven structure. Consultations on the development of the NCPA II concluded that support to MVC had not always prioritized working with existing support structures and recommended capacity strengthening of the structure as a core sustainability strategy. Working with Sub-grantees Working with sub-grantees has contributed to strengthening capacities of local partners. All partners are mentoring sub-grantees to develop capacities for them to receive direct USAID funding. Pact and FHI360 has also developed models for tracking institutional capacity in order to determine readiness for direct USAID funding as well as identifying specific capacity-building needs. The models have resulted in objective evidence of partner capacities based on clear indicators tracked over time for the following components:  Purpose and planning  Human resource management  M&E  Partnership and relations  Governance  Organizational sustainability  Financial management  Grants and compliance  Projects and services Sub-granting has further evolved from focusing on small CSOs toward involving organizations with specialist competencies like REPSSI, which provides technical assistance on psychosocial support to Africare and FHI360 partners. The approach has reportedly provided CSOs with opportunities to interact and get assistance from organizations with specialized competencies. Prioritizing Economic Strengthening Prioritizing economic strengthening is reportedly improving household capacities to support MVC. The work done by DAI (2013) concluded that expanding economic strengthening requires good partnerships with the private sector and development partners, strategic linkages to the right human and financial resources and appropriate investments to support innovations. The NCPA II outlines the importance of transferring responsibility to households and communities but also stresses the importance of strengthening the capacities of households, enabling them to effectively support MVC. Beneficiaries of three out of the four implementing partners hold savings that are more than the food poverty threshold and the basic needs poverty threshold (Tanzania Household Budget Survey, 2013). 40 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation SUPPORTING COORDINATION OF ALL STAKEHOLDERS Coordination is one of the strong components of the program. One of the objectives within the RFA focuses on increasing local ownership and coordination of all stakeholders including the for-profit sector to ensure efficiency and avoid duplication. Based on the program design, there should have been strong coordination of donors, USG programs in multiple sectors and other stakeholders to ensure efficiency. At another level, coordination was expected in the LGAs to facilitate strong implementation and management of the NCPA and data management system at the local level. The Zonal Approach All partners have been compelled by the program design (through the zonal approach) to invest in stronger coordination with the GoT. All four partners acknowledged that the zonal approach has strengthened coordination, especially as there is continuity and partners have more incentive to invest in stronger coordination within their zone of operation. Working alone in a specified zone is an incentive because it reportedly facilitates demonstration of impact, and inversely, partners will have to take responsibility for any negative outcomes. Government departments, on the other hand, are reportedly more willing to coordinate as they are not burdened with the responsibility of having to deal with multiple partners delivering USG support using different mechanisms within the same geographical zone. This has addressed concerns that emerged during consultations on PEPFAR II strategy development, which concluded that “PEPFAR has contributed to greater donor and implementing partner coordination but increased effort and mapping of services is needed to be more cost effective, avoid duplication and broaden district service coverage” (RFA, 2009). Joint Implementation with GoT Entities Partners have invested in strong coordination with the GoT. For example, Pact and FHI360 co-facilitate MVCC committee trainings with the MoHSW, and WEI conducts multi-sectoral school assessments with LGAs and other important contextual stakeholders. Joint implementation has reportedly contributed to providing practical skills to GoT entities; stakeholders mentioned having developed competencies to collect, analyze and use data from multi-sectoral school assessments, for example. Coordination with Sub-grantees Implementing partners have invested in coordination among sub-grantees. All partners have invested in capacity building to ensure partners graduate and potentially receive direct support from USAID. A total of 22 out of 32 staff members from 10 sub-grantees reported strong coordination among themselves as well as with GoT entities and other stakeholders. Stronger coordination is demonstrated through quarterly implementing partner meetings reported across all zones. Support for development of a stakeholders’ inventory by FHI360 has further contributed to coordination as partners have an understanding of other organizations working to support the same beneficiaries. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 41 LGA Investment in MVC Through coordination, local authorities are taking responsibility and prioritizing MVC in budgeting. Some councils supported by WEI have agreed to support MVC who have health fee exemption cards, while in Arusha local authorities have adopted MVC action plans that include funding commitments for MVC. However, all implementing partners reported challenges in ensuring LGAs honor their commitments to fund MVC activities. Investments in Information-sharing Platforms The program has invested in stronger coordination through quarterly national sharing meetings. Partners reported that these provide spaces for sharing implementation lessons while learning about innovation from other partners. Despite important lessons and expertise in capacity building, there is limited evidence of sharing between PT partners beyond the quarterly sharing meetings. In addition, partners highlighted that although they are implementing the same program, they sometimes do not see the incentive of sharing and learning beyond the confines of their cooperative agreement. They highlighted that development has become competitive and best practices and effective methodologies give a competitive edge. Sharing is sometimes viewed as potentially equipping competitors. CAPACITY BUILDING Capacity building is a central component of the PT program. The RFA (2009) outlines that while urgent needs often necessitate the direct, immediate provision of food, health and other basic social services to save the lives of vulnerable children, sustainability requires fortifying the abilities of OVC households and OVC themselves, communities, local government and indigenous institutions, including the for-profit sector, to continue supporting vulnerable children and their families after external assistance is no longer available. Sub-grantee capacities have been enhanced, resulting in improved management capacities. MVCCs have been revived and strengthened. Caregivers are investing in supporting access to food, protection, health and birth certificates for MVC. Implementing Partner Programs Capacity building is a central component in each of the partners’ program design. The capacity-building approach adopted reportedly introduced “a shift from dependency to development where more responsibility for supporting MVC is being shifted toward households, communities and the government” (government key informant interview). While there is still some resistance to the new approach, partners have largely succeeded in attaining buy-in from GoT entities and sub-grantees. Improved Management Capacities among Sub-grantees All implementing partners and sub-grantees reported improved organizational capacities as a result of capacity-building initiatives. In Mara, improved financial management capacity resulted in the elimination of all reporting challenges that had affected 90 percent of all Pact sub-grantees. In addition, all partners acknowledged that some of the “What is the incentive for us to share our best practice/methodologies”? Implementing partner staff 42 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation sub-grantees they support will be ready to receive USAID funding if capacity building continues. Improved Access to Services for MVC as a Result of Capacitated Caregivers Capacity building for caregivers reportedly strengthened their ability to support MVC. Economic strengthening groups are contributing to community health insurance and access to education for children. Groups have also been used as entry points to facilitate access to birth certificates for MVC. Tailored, Multi-level Capacity Building The capacity-building approach adopted by the program and partners has been tailored to the needs of recipients as opposed to being based on assumptions of homogenized capacity needs. Training-of-trainer approaches have been adopted to capacitate GoT staff at zonal and regional levels, and all partners have supported capacity strengthening of community-based groups like MVCCs along with WORTH, SILC and LIMCA groups. Development of Tracking Tools Some partners have developed tools for structured skills transfer; for example, Pact has a Comprehensive Institutional Strengthening Plan, and FHI360 has a Sub-grantee Organizational Maturity Model. The tools facilitate gathering of evidence to demonstrate progression by individual partners along pre-determined organizational development parameters. Tools have reportedly ensured partners have sufficient evidence for articulating levels of organizational maturity and making a case for graduating sub￾grantees for direct USAID funding. Case Study: Using WORTH Groups as Entry Points for Promoting Birth Registration The Juhudi WORTH+ group meets in Gereza East village, in Kagunda ward, Korogwe Rural. Until recently many of the children supported by the Juhudi group members did not have birth certificates. This meant they were unable to access free healthcare through the health fee exemption, secure their civic rights, open a bank account, take out loans or receive protection against underage abuses such as child labor and child marriage. The women said that they had never tried to register their children for birth certificates, because they believed the authorities would reject their applications. TEWOREC (PT program implementing partner in Korogwe Rural) gave the group members information on the importance of having birth certificates. All members of the Juhudi WORTH+ group have successfully applied for and received birth certificates for their children. “They [TEWOREC] told us birth certificates help in school registration and even in getting loans for higher education once our children complete secondary school,” reported Aziza, a WORTH+ Group member. Source: WEI, 2013 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 43 Scope for Improvement There is scope for improving mentorship and fostering overall sustainability and functionality beyond project support. This is especially relevant for Africare and WEI, which do not have structured capacity-building tracking tools. Capacity building will also take longer, because government and communities were used to receiving material support. Some partners in the same implementing areas are still providing material support, which negatively affects the project’s focus on capacity building. ENSURING MEANINGFUL LINKAGES AND REFERRALS The program has a specific focus on creating meaningful referrals and linkages. This emerges from the RFA, which cites the underutilized potential for linking OVC programs to prevention, care and treatment, and other health, education, and development interventions within a given region or district. Partners within the program are expected to facilitate meaningful linkages to create truly comprehensive services meeting a wider range of OVC needs. In addition, implementers are required to link with economic strengthening programs to develop or strengthen income-generating skills and activities within their areas. Linkages with Government Coordination with government has reportedly improved linkages and referrals. This is significant as government has overall responsibility for social service delivery. All partners outlined that their advocacy approaches with government have shifted away from articulating deficiencies toward supporting government to effectively provide services. Working with government has reportedly provided key entry points for meaningful linkages and referrals. Linkages with Other USAID-funded Programs The program has created and nurtured linkages and referrals with other USAID-funded programs; an important example is the partnership between Africare and Youth Net4 . The partnerships have facilitated access to scholarships to MVC identified by Africare. Youth Net provides scholarships and is supported by USAID. Scope for Improvement All partners reported providing referrals to government institutions. However, they also articulated weak capacities of government departments to provide adequate services. This limits the completeness and effectiveness of the referral linkages, especially if there are high chances that services for which MVC are referred are likely to be unavailable. Partners recommended continued advocacy with government to provide services while strengthening capacities of households to be able to support MVC. While all partners clearly articulated their approaches to referrals as well as specific referral paths, none provided a documented referral-tracking mechanism that provides information on whether or not MVC received services for which referrals were facilitated. There are plans to prioritize strengthening referral tracking for WEI, but there is no 4USAID-funded organization providing scholarships 44 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation information around how this will be done. Technical support from MEASURE can be leveraged to support partners in strengthening referral-tracking systems. ADDRESSING GENDER AND AGE The RFA outlines that “all aspects of program design……must consider issues of gender and age to ensure the differing needs of boys and girls at various developmental stages are addressed.” This provides a starting point for analyzing the extent to which program partners have addressed gender and age considerations. Clear Articulation of Gender and Age Disparities All partner proposals articulate gender disparities and strategies for addressing them. Examples include a focus on nutritional support for children under 5 as well as economic empowerment and vocational training for older MVC. The clear articulation demonstrates an understanding of the nature and scope of challenges and provides the basis for implementing partners to provide support that include gender and age considerations. Providing funds to women through the SILC and WORTH reportedly empowers women to participate in the decision-making process within the family. In addition, child rights clubs emphasize reproductive health strategies that protect girls from coercive sexual relationships, GBV and forced marriages (WEI, 2013). Prioritizing Affirmative Action Partners have adopted an affirmative action approach focusing on livelihoods strengthening for women, who are historically disadvantaged and constitute the majority of MVC caregivers. The approach has ensured that at least 60 percent of all SILC group members are female caregivers. The underlying hypothesis assumes that the individual savings outlined above are available to women, and studies have demonstrated that women are three times more likely to reinvest earnings in the family than men (Kristof and WuDunn, 2011). Emerging Opportunities for Women Access to micro-credit All beneficiaries in the three SILC FGDs reported improved economic status for women through savings and localized access to micro-credit. This is corroborated by the fact that individual savings for beneficiaries of three of the four implementing partners are higher than the food poverty threshold and the basic needs poverty threshold. Cash savings have reportedly contributed to diversifying savings options beyond crops, which are vulnerable to harsh climatic conditions. Literacy for transacting The program has contributed to improved literacy levels, which have reportedly opened more opportunities for women. Literacy reportedly improved women’s confidence to seek services, while numeracy facilitated their capacities to transact, important because petty trade forms the core of livelihood activities. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 45 Economic strengthening as entry points Economic strengthening groups have been entry points for other important issues like birth registration. Women in Arusha reported being provided with information on the importance of registering births as well as how to go about the registration process. Addressing GBV A strong focus on GBV has contributed to prevention as partners are engaged in creating communities that are less tolerant to GBV. Child protection data from partners showed an increase in reporting over FY12, when there were no reported cases. The table below shows the number of child protection cases reported and addressed for three of the four partners in FY13. Table 18: Child Protection Cases by Partner Partner Number of Child Protection Cases Reported and Addressed Africare 947 Pact 549 WEI 664 Referrals for GBV services have been strengthened, which will potentially strengthen reporting as survivors know they can access services. Partners have also started engaging men (though still minimally), as they have an important role in promoting gender equality and women’s empowerment. Engaging men is reportedly done through specific trainings as well as mainstreaming GBV messages in economic strengthening activities that involve men. FHI360 has also piloted the child helpline where 4,642 calls had been received by the time of evaluation. Building leadership capacities for women Improving literacy can also be used to support building leadership capacities of women. Women are already coming together and working toward common savings. The regular meetings can potentially be used to ensure women are empowered with leadership capacities to translate individual economic empowerment opportunities into wider upward mobility within community leadership spaces. Including sexual and reproductive health Partners and stakeholders outlined important opportunities for including sexual and reproductive health information within economic strengthening activities. There will be minimal additional cost, because women will be gathered for economic activities and information can be provided by the MOHSW. The table below demonstrates levels of unmet need, highlighting the critical importance of mainstreaming sexual and reproductive health. “I am appreciative of the project involving the police – before there was no collaboration, now because we are working closely together, we have been able to discover children who have been abused and tried to bring the perpetrators to book.” Police Gender Desk 46 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Table 19: Selected Sexual and Reproductive Health Indicators for Tanzania Indicator Percentages Women reporting ever having suffered physical violence 39 percent Women reporting forced first sexual encounter 10 percent Young women 15-19 who have already started childbearing 23 percent Women with unmet need for family planning 25 percent Source: TDHS, 2010 PRIVATE SECTOR PARTNERSHIPS Public-private partnerships are central to the PT program, and the RFA outlines that such a partnership “mobilizes and leverages the ideas, efforts and resources of governments, businesses and civil society.” Partners are expected to build alliances that will stimulate and expand the USAID OVC program’s reach and impact. Public-private partnership trainings All implementing partners have invested in training on resource mobilization and have supported community-based resource mapping. Pact-supported MVCCs have successfully conducted resource-mapping exercises and articulated potential opportunities for private sector engagement. Linkages with specialist organizations Linkages with private sector partnership development institutions like Cheetah Development Cooperation and DAI has opened opportunities for beneficiaries to expand their production and marketing potential. The partnership between Africare and the for-profit Cheetah has provided evidence of the strong potential inherent in public￾private partnerships. Cheetah specializes in agricultural investments, and has provided support to 200 households on a commercial, for-profit basis. Scope for improvement PT implementing partners have often taken a welfare approach to public-private partnerships, which has limited their effectiveness. The welfare approach involves requesting donations to support MVC without long-term sustainable commitments to support building their resilience. With the exception of Africare, all implementing partners provided examples where they successfully requested and received one-off donations from private sector entities. Africare, on the other hand, engaged an organization that specialises in small-scale agriculture financing. The organization provided loans to farmers and ensured these were paid back once farmers sold their produce. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 47 Case Study: Private Sector Partnership in South Africa The Hope worldwide South Africa OVC program consisted of three integrated, synergistic programs. In its 2007 semiannual report, the program reported to Pact that between October 2006 and March 2007 it had served a total of 7,231 OVC. Under the project, the Africa Network for Children Orphaned and at Risk (ANCHOR) was a unique public-private partnership operating in six sub-Saharan countries established to scale-up existing community OVC care and support programs. Private sector partners included Tiger Brands, South African Airways, Hollard and ABSA. Central to the partnerships was Hope worldwide South Africa’s capacity to articulate the reputational value to be derived by corporations if they support initiatives. Source: Khulisa Management Services. HOPE worldwide South Africa OVC Programmes. 48 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 5. CONCLUSIONS All key PT objectives are on course to be achieved both quantitatively and qualitatively. All partners have either surpassed or are on course to surpass targets for PEPFAR indicators. In addition, identification of MVC has been intensified while referrals have been provided. Advocacy efforts have been well established for initiatives like the health fee exemption cards and the community health fund, which contribute (in different ways) toward facilitating access to comprehensive services for MVC. Economic strengthening for caregivers has facilitated the transition from focusing on the individual MVC to strengthening the household and community. Individual savings from economic strengthening groups compare favorably with national benchmarks, and caregivers have demonstrated willingness to invest time and financial resources in collective community based savings. All partner programs prioritize capacity building and skills transfer, which are critical for enhancing sustainability. The zonal approach is enhancing coordination, and working with sub-grantees is contributing to local capacity building. Stronger coordination with government has opened pathways for referrals, and there has been joint implementation of initiatives between implementing partners, sub-grantees and the GoT. Partners have adopted diverse approaches for providing psychosocial support to MVC with school-based child rights clubs emerging as the most sustainable approach. In some instances partners have contracted REPSSI for assistance with mainstreaming psychosocial support. Economic strengthening initiatives have provided entry points for literacy, birth registration sensitization and potentially provision of sexual and reproductive health information. Models for private sector engagement have been tested and there is potential for strong public-private partnerships. Partners acknowledged their shortcomings in this area but highlighted their openness to capacity building that allows them to transition from a welfare-focused approach toward mechanisms that provide incentives for private sector partners. CHALLENGES AND EMERGING ISSUES Weak Livelihoods High levels of poverty and weak livelihoods constrain communities’ ability to save money and support MVC. All partner baselines cited poverty as a key determinant of vulnerability, resulting in prioritization of household economic strengthening. High levels of poverty reportedly meant MVC caregivers sometimes prioritized other household needs at the expense of MVC needs. Key informants outlined that the major challenge USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 49 they face relates to poverty and that families are unable to provide basic and supplementary needs. Overachievement of Targets At mid-term all partners have surpassed almost all targets for PEPFAR indicators. However, evidence from the evaluation demonstrates improvements as economic strengthening groups have more money than the basic needs threshold for Tanzania as outlined in the Tanzania Household Budget Survey, 2013. Surpassing targets raises concerns around the processes and evidence behind target setting. Low Capacities of LGAs There are gaps in ensuring LGAs honor their funding obligations for MVC. Commitments through MoUs have been secured, but very few councils have funded their commitments to support MVC. Local authorities acknowledged their responsibility to support MVC but cited limited financial resources as a key challenge. Referrals to Services That Are Not Available The program prioritizes referrals and linkages, but these are constrained by general service provision challenges within the country. For example, health fee exemption cards are only effective if services are available at the facilities. Limited Capacities for Private Sector Engagement Partners do not have adequate expertise for private sector engagement. As a result, support received has largely consisted of one-off donations that reportedly cannot contribute toward building community resilience. DAI will provide technical support for economic strengthening to all implementing partners. Limited Capacities of MVCCs The MVCCs provide strong structures for providing sustainable support for MVC but they have weak capacities, which constrain the efficiency of the program investing in capacitating them. In the Southern zone for example, Pact documented that more than 70 percent of MVCCs were not functional prior to provision of support, and investments had to be made in training them on their responsibilities before they were able to support MVC. Limited Examination of Why Specific Interventions Work in Specific Contexts There has not been sufficient enquiry to ascertain why more economic strengthening activities are in Iringa region, which has 61 percent of the SILC groups and 55 percent of the IGA groups. This is despite the partner having worked in Dodoma for 10 years but without similar levels of success. “There are high levels of poverty, which translates to high demand for support towards needs like food, education and health care.” PACT key informant, Biharamulo, Kagera 50 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation LESSONS LEARNED Capacity building is a gradual process, and aspirations should be realistic. The program has potentially transformative aspirations, but there could be scope for achieving more if skills transfer and service delivery aims are critically analyzed (e.g., how feasible is it to capacitate a CSO with very low capacity levels and ensure they also deliver services?). High staff turnover at different levels was cited, but there has been limited enquiry about effective capacity building approaches in the context of high staff attrition. Communities’ resources need to be carefully mapped and harnessed to support MVC. The program has successfully revived and sustained MVCCs, which are local structures with potential to provide sustainable support to MVC. Lessons from Iringa showed that LGAs may not necessarily provide financial resources but can provide land, which will ensure beneficiaries have spaces to conduct their activities. Capacity building and material support are not mutually exclusive components. The program has provided capacity building along with services, in order to transfer responsibility to households and government. The program initially focused on capacity building only, but implementation lessons demonstrated that challenges facing LGAs, CSOs and MVC required material support to complement skills transfer and capacity building. Private sector partnerships can be successful in the context of well-developed implementing partner capacities. All partners have implemented activities classified under private sector partnerships, but they acknowledged having capacity limitations around how to effectively create and nurture partnerships with private sector entities. All partners now have partnerships with DAI, which is providing technical assistance in economic strengthening and whose experiences will be used in pursuing private sector partnerships. Coordination with and capacity building of government are key components of sustainable interventions. Partners’ work with government has initiated processes of active advocacy where the government is supported to assume responsibility of MVC as opposed to where partners point out what government is failing to do for MVC. Involvement of government in training of MVCCs, their supervision and in multi-sectoral school assessments is reportedly contributing to strengthening capacities to assume full responsibility for supporting MVC. Referrals are effective in the context of available services. Partners have been successful in identifying and referring MVC, but in some instances services are not available at the referral centers. All partners have acknowledged that initial approaches focused more on providing referrals without ensuring availability of services. Through learning, there is a shift toward providing referrals where services are available while strengthening referral tracking and follow-up systems. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 51 Case Study: Facilitating Access to Education for OVC through Block Granting and Resource Exchanges The STRIVE Project, implemented by CRS starting in 2001, was supported by USAID with $2.49 million in funding to implement a two-and-a-half year pilot project ending in June 2004. It was aimed at testing innovative interventions that would allow resources to reach children at risk quickly and efficiently and would help determine what basket of support would best meet the needs of large numbers of children at risk in Zimbabwe. STRIVE initiatives included block grants to schools in exchange for the enrolment of a set number of OVC, resource exchanges (in which STRIVE purchased material items for schools in exchange for the enrolment of OVC), direct assistance scholarships, after-school programs and support and training for teachers. A block grant is a lump sum payment given to a school, which can then use the money to purchase materials, refurbish classrooms, strengthen infrastructure, etc. In return, the school agrees to enroll a pre-determined number of OVC who are exempted from paying fees for an agreed-upon time period. Local community members serve on committees to select the children who are to directly benefit from the intervention. Resource exchange works in the same way, except that STRIVE purchases the resource requested by the school instead of providing money to the school for it to make the purchase. The move to the block grants approach enabled STRIVE to keep overall beneficiary numbers (including both direct and indirect beneficiaries) relatively steady, and the overall beneficiary targets for education assistance were met in all years Support to Replicable, Innovative, Village/Community-level Efforts for Orphans and Other Vulnerable Children: End-of-Project Report. September 2008. Expectations from government should be realistic and based on existing knowledge of available capacities. All partners are working to support the government, whose weak capacity to support MVC is acknowledged. However, partners also provide referrals to the government facilities whose capacity gaps they have documented and are trying to address. Advocacy initiatives are likely to be effective if they transition from articulating deficiencies toward working with government to address deficiencies. Partners have all prioritized working with government departments in training and implementation (for example, conducting multi-sectoral school assessments). The approach has reportedly ensured stronger buy-in from the government. 52 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 6. RECOMMENDATIONS Audience Conclusion Recommendations USAID The zonal approach is consistent with capacity building and skills transfer; it has promoted creation and maintenance of partnerships between partners and LGAs, which are central to the program’s sustainability strategy. Continue prioritizing the zonal approach with clear indicators for efficiency, equity and accountability as outlined in the RFA. USAID There were reported delays in disbursements which affects program implementation. Prioritize timely disbursement of funds to partners to ensure they stay on track with implementation timelines. (This will depend on availability of funding.) USAID Partners have documented key approaches, lessons learned and other methodological insights that can be useful to a broad range of partners. Pact has Comprehensive Institutional Strengthening Plans, and WEI’s WORTH+ approach is facilitating economic empowerment and promoting birth registration for children. FHI360 has piloted a one￾stop center for GBV, while Africare has developed a successful model for private sector partnership. However, none of the partners reported having shared the approaches with other PT implementing partners. Prioritize sharing implementation experiences to promote learning. Specific learning meetings can be organized where partners can periodically share lessons learned and good practices. USAID Partners acknowledge the importance of private sector involvement as part of their sustainability strategies but have weak capacities in private sector engagements. Africare successfully partners with Cheetah Development Cooperation, which successfully provided financial and technical support to farmers. DAI is now providing technical support for private Continue supporting the partnerships with DAI, which will provide technical support on strengthening private sector partnerships and economic strengthening. Use findings from a study being conducted by DAI on economic strengthening to determine the most effective approaches to adopt. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 53 Audience Conclusion Recommendations sector engagement to all partners. Ensure private sector partnerships are designed to build on existing community capacities and livelihood options. An example is the partnerships for predominantly agricultural communities should focus on strengthening that existing livelihood source. Ascertain attribution of improved MVC health, education, nutrition and protection outcomes to savings and lending groups, beyond anecdotal evidence. Determine timing and approaches to scale-up community groups to larger microfinance lenders for real economic investment and growth. Partners The volunteers and para-social workers provide community level expertise and support. However, there are high levels of attrition as they are not paid. Consider using economic strengthening support as incentives for volunteers to ensure they are motivated and levels of attrition are reduced. Partners Agro-based livelihood strengthening has been successful in Iringa, but there have been challenges in Dodoma and Karatu where the agro￾ecological conditions are different. Invest in different agriculture technologies like conservation agriculture in areas where agro￾ecological conditions are difficult for conventional farming. Ascertain attribution of improved MVC health, education, nutrition and protection outcomes to savings and lending groups, beyond anecdotal evidence. Determine timing and approaches to scale-up community groups to larger microfinance lenders for real economic investment and growth. 54 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Audience Conclusion Recommendations Invest in monitoring the number and amount of loans from savings and lending groups that go to educational fees, and use this information to sensitize educators on the importance of MVC support. Partners Coordination with government has provided opportunities for sustainable provision of MVC, though government capacities remain weak. Continue nurturing and strengthening relations with the government to strengthen capacity for MVC support. Partners should continue investing in collaborative training and supervision of MVCCs and involvement of LGAs in multi￾sectoral school assessments. Partners Coordination of implementing partners through regional and district meetings has contributed to stronger sharing of information, and partners have platforms for exploring collaborative activities. Continue investing in regional and district implementing partner meetings to further strengthen current coordination efforts. Partners Referrals are critical in broadening the range of services accessible to MVC beyond an individual’s capacity. However, the capacity of service providers is generally weak and MVC end up not receiving the services. Ensure referrals are provided for services that are available, and develop strong mechanisms for referral tracking. Follow up to ascertain if MVC receive services for which they would have been referred. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 55 ANNEX I. SCOPE OF WORK PURPOSE The purpose of this evaluation is:  To assess the various approaches used by each of the Pamoja Tuwalee implementing partners to determine which approach is effective in achieving linkages across services and fostering sustainability; and  To document lessons learned that will inform the mission in designing approaches and focus for the follow-on program to support vulnerable children and youth. BACKGROUND The OVC Program Globally, unified goals and consensus on the urgency of the situation for orphans and vulnerable children (OVC) are provided through the United Nations General Assembly Special Session (UNGASS) goals on HIV/AIDS for OVC, specifically targeting articles 65, 66 and 67, and the Millennium Development Goals (MDG) to be achieved by 2015. In addition, global guidance, recommendations and evidence provide growing consensus on what works and how to focus responses: (a) HIV-sensitive rather than HIV-specific approaches; (b) interventions proven to improve the quality of life and outcomes for OVC, such as Early Child Development (ECD), keeping children and young people in school, strengthening families (in addition to the wider 6+1 package of OVC services described by OGAC5 ) and getting more money closer to families and children through economic strengthening of vulnerable households; (c) systematic strengthening of child protection rather than isolated interventions for specific vulnerable children (e.g., children living and working on the street); (d) child-sensitive social protection systems with sufficient institutional capacity and resources to implement social transfers (regular predictable transfers in cash or kind), social insurance such as health insurance, social services, policies, legislation and regulations that protect families’ access to resources and services (e.g., inheritance rights and anti￾discriminatory legislation); and (e) avoiding separation of children from families through early family-centered interventions, and reintegration of children where this is in the child’s best interests. Most Vulnerable Children in Tanzania and the Current Response In Tanzania the program support targets MVC, a subset of OVC. Using the existing national definition of MVC, MEASURE recently released Projected Numbers of Most Vulnerable Children (June 2011), which estimates that due to continued population growth, especially in rural areas, numbers of most vulnerable children are expected to increase nationally from approximately 1.67 million children in 2010 to approximately 1.81 million children in 20156 . Taking Evidence to Impact (UNICEF 2011 p. 8) highlights 5OGAC is the Office of the U. S. Global AIDS Coordinator. 6Caveats to these calculations are included in Annex 3. 56 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation the immense and diverse child protection needs in Tanzania, which are magnified by widespread poverty and the impact of HIV and AIDS, with one-third of households living in poverty. More than two million children are orphans (having lost one or both parents). One in five children is engaged in child labor. Only 2 percent of children with disabilities attend primary school. A recent national survey on violence against children found that nearly one in three girls and one out of six boys reported at least one experience of sexual violence prior to the age of 18. Domestic violence is commonplace, and many parents condone the behavior of teachers who beat their children to enforce discipline. Therefore, millions of children either suffer or are at serious risk of rights violations. The National Poverty Reduction Strategy (MKUKUTA II pp.17-18) summarizes the basic factors causing poverty and vulnerability: natural calamities; malaria, HIV and AIDS, and TB; low incomes; lack of education and skills; lack of access to clean and safe water; and lack of adequate shelter. However the MKUKUTA II also points out that there are practices that have adverse effects on vulnerable groups, such as customs, norms, taboos and values; unhappy marriages; domestic and gender-based violence; and drunkenness and drug abuse. As of November 2011, 849,051 MVC had been identified in Tanzania (November MVC Monthly Up-date). During the implementation of the NCPA for MVC (2007-2010) health, social and protection services were provided to more than 800,000 children by 2010, and Most Vulnerable Children Committees (MVCCs) were mobilized in up to 61 percent of the villages in Tanzania. The government agreed with key development partners (PEPFAR, UNICEF and the Global Fund to Fight AIDS, Tuberculosis and Malaria) to allocate regions equitably to implementing partners to avoid duplication and maximize scarce resources. Although these are considerable achievements, the national response to date has until recently been an emergency response, largely focused on the provision of material support from external sources rather than the establishment of systematic mechanisms that can be government-led and local-level initiatives that inspire communities to address the needs of their own vulnerable children. This picture is changing with a number of key policy and legislative changes that pave the way for a more systematic response, including the Law of the Child Act 2009, the launched Tanzania Social Action Fund III program, and the shift in development partner emphasis toward more sustainable solutions. Challenges ahead include insufficient evidence￾based learning and scientific evaluation of most promising practices (MPP) and service provision to establish quality standards based on cost-effectiveness. Responsive service provision needs to be balanced with adequate attention to prevention of vulnerabilities and early identification of children and households at risk. The government, which is widely considered to be very responsive, needs to take a stronger lead over the MVC response by allocating adequate resources to enable systems and services to be put in place and ensuring that policies and commitments made at the national level are translated into implementation at the local level. MVC outside of family care require urgent attention to ensure they receive adequate care and protection through government-led and regulated solutions that keep as many children as possible within family-based care. Most importantly, the government needs to lead a coordinated response across all ministries and sectors rather than allowing fragmented and often duplicative programming. Advocacy, awareness raising and support are needed at the USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 57 community level to re-establish a sense of ownership and care for MVC in their communities. Global evidence and guidance, together with the evaluation of the previous NCPA I for MVC 2007-2010, emphasize the need for the MVC response to be refocused on sustainability approaches through support of households and government-led systems to coordinate service provision. The role of civil society is still significant in the delivery of services and innovation of approaches. To ensure realization of sustainability, the new national MVC plan for 2013–2017 was developed with the overall goal of establishing an MVC response that is government-led and community-driven, and that facilitates MVC accessing adequate care, support and protection and access to basic social services through increasingly mainstreamed and sustainable government-led systems. To achieve this, the second national MVC plan (MVC NCPA II) is structured around four overall strategic objectives: 1. Strengthen the capacity of households and communities to protect, care and support MVC. 2. Increase access to effective child protection services within a well-resourced child protection system. 3. Expand access to education, health and Early Childhood Development (ECD) services. 4. Strengthen the coordination and leadership, policy and service delivery environment. THIMS 2011/2012 HIV Prevalence by Region 58 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Geographic coverage by Zones Partner: AFRICARE FHI360 WEI Inc./Bantwana Lake￾PACT Southern￾PACT Regions: Dodoma Dar es Salaam Arusha Mara Lindi Singida Coast Tanga Mwanza Mtwara Iringa Zanzibar (5) Manyara Kagera *Mbeya Njombe Morogoro Kilimanjaro Tabora *Rukwa *Ruvuma *Katavi *New expanded geographical coverage, transitioned from DoD Pamoja Tuwalee OVC Program The PEPFAR-funded OVC program supported the implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The program is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI), and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. In keeping with the MVC needs outlined in the NCPA, the purpose of the PT Program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). The goal is to achieve this objective through the employment of several cost-efficient, tightly integrated, sustainable, and interrelated strategies. The organizations are responsible for providing OVC services; strengthening coordination and collaboration with GoT entities and other donors at the zonal, regional and LGA levels; building capacity of local organizations, families and supporting groups; and collaborating with national-level entities serving the interests of MVC/OVC. In addition, the organizations are expected to work closely with local implementing partners to ensure that GoT policies and procedures are implemented at the LGA level and, conversely, to provide grass roots data on the impact of those policies and the needs of MVC/OVC and their households. The key approaches that are applied include the following:  Provide comprehensive services strategically aligned to OVC and their household needs. These comprehensive and quality services include health care (general health needs, specific health needs of HIV+ OVC), HIV prevention, education and vocational training, food and nutrition, protection, psychosocial support, shelter and care and economic strengthening based on OVC and household needs. The PT organizations use the national quality standard services delivery package for all interventions and provide comprehensive age￾and gender-appropriate care either through support of direct service provision or meaningful referral linkages (e.g., to other HIV/AIDS services, food assistance or economic strengthening support). USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 59  Enhance sustainability in care and support services. This is accomplished through empowering beneficiaries, communities and host GoT entities at all levels to ensure local ownership and capacity to continue the initiated program support. The PT organizations strive toward sustainability of the interventions through community and beneficiary involvement at all levels, building and strengthening capacity of local organizations and the local government authorities to ensure buy-in, including budgeting to support the interventions’ accountability. In addition, PT organizations are strengthening and building interventions on existing local structures such as MVCC to address MVC/OVC needs.  Support coordination of all stakeholders within the implementing areas to ensure linkages and comprehensive support. This ensures coordination with other donors’ programs as well as other USG programs in multiple sectors. In addition, the program enhances linkages with other stakeholders to ensure efficiency and avoid duplication of effort, which is critical to long term sustainability. PT organizations implement other efficient and/or cost-reducing donor-supported strategies such as mapping all GoT and donor activities in their zones of responsibility.  Build local capacity, taking into consideration the urgent needs which often necessitate the direct, immediate provision of food, medicine and other basic social services to save the lives of vulnerable children. This is achieved by sustainably fortifying the abilities of OVC and their households, communities, local government, indigenous institutions and the for-profit sector, to continue supporting vulnerable children and their families after external assistance is no longer available. Strengthen the ability of local organizations and graduate them to the technical and management level of international implementing partners.  Support the OVC household rather than individual OVC. Support interventions that strengthen the capacity of families and secondarily that of OVC themselves to better provide for long-term OVC needs. Also build the capacity of the caregivers within the household structure, i.e., parents, grandparents, older OVC and/or other family members. Identify locally relevant income-generating activities (e.g., group savings and loans, marketing of local agricultural output and strategic alliances with the for-profit sector), and if expertise exists within their organization or partnerships, implement these activities.  Ensure that there are meaningful linkages and referrals to other AIDS, health and development services, thereby using the potential for linking OVC programs to prevention, care and treatment and other health education and development interventions within a given region or district. Also ensure that MVC/OVCs who are exposed to HIV are referred to counseling and testing and, as necessary, to care and treatment services. Establish stronger linkages to prevention programs, especially age and gender appropriate and child-friendly prevention interventions, including those for older MVC/OVCs. Establish strong linkages and innovative partnerships with other programs that reach into OVC communities and household, such as home-based care, education and school feeding programs. 60 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation  Finally, PT organizations support all MVC/OVCs by addressing issues of gender and age to respond to the differing needs of boys and girls. It should be noted that in the course of implementation, there have been some key national and PEPFAR program policies, guidelines and strategic changes in care and support of OVC/MVCs. The changes necessitated some modifications of the program description and geographic program coverage expansion to some of the partners. Additionally, other strategic changes were made in order to respond to the baseline data findings and funding of specific initiatives that led to an increase of targets and expansion of the original scope. Aside from PEPFAR funding, the partners have received central initiative funds from AID/W to intensify gender and food and nutrition assessment interventions for their ongoing programs and expand geographic coverage to add a new zone that was originally supported by other U.S. Government agencies. Program Implementation and Geographical Coverage Africare The program covers the central zone which has four regions: Iringa, Njombe, Dodoma and Singida. Original Key Objectives Objective 1: Strengthen service delivery systems, ownership, planning, coordination, management and monitoring of community-level MVC interventions. Objective 2: Ensure delivery of a comprehensive package of health and social services to MVC and their households that address their unique needs. Objective 3: Support child protection systems and increase youth and participation in addressing problems and issues affecting MVC. Objective 4: Strengthen capacity of Tanzanian institutions to provide leadership in addressing MVC issues. Additional Objectives Addressing gender needs is part of the OVC programming; however, an additional four main objectives were added to intensify the GBV integration in the area with a high rate of violence and HIV prevalence. The overall goal was to ensure that vulnerable children and their caregivers, particularly program beneficiary children and women, are protected from all forms of gender-based violence. The integration of the GBV intervention into Africare PT supports and builds on the four OVC program objectives mentioned above, particularly, emphasis on child protection and participation. Specific GBV objectives include: Objective 1: Engage program beneficiary communities in prevention of GBV through community-level sensitization, outreach and resource mobilization. Objective 2: Increase child, youth and girls’ participation in preventing and addressing GBV within their communities. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 61 Objective 3: Appropriately link communities with legal, psychosocial and clinical services to support survivors of GBV. Objective 4: Strengthen the capacity of key community stakeholders, including MVCC members, para-social workers, community justice facilitators, teachers and faith and traditional leaders to take leadership in preventing and addressing GBV from the community to the district level. Family Health International (FHI360) The program covers the Coast zone in the mainland and Zanzibar Islands. The Coast zone covers Dar es Salaam, Morogoro, and Coast, and Zanzibar includes Unguja and Pemba. Original Key Objectives Objective 1: Increase the capacity of communities and local governments to meet the needs of OVC and their households in an innovative, efficient and sustainable manner by enhancing their competencies to provide support and by improving communication, coordination and collaboration across sectors. Objective 2: Increase the capacity of households to protect, care for and meet the basic needs of OVC in a sustained way by improving their caretaking, livelihood and health￾seeking skills. Objective 3: Increase OVC household access to comprehensive, high-quality, age￾appropriate and gender-sensitive services by creating integrated community-level referral networks that strengthen the continuum of care. Objective 4: Empower OVC, particularly females, to contribute to their own well-being by improving their resilience, as well as their livelihood and self-care skills. Additional objectives Support a smooth transition from the ended program to prevent service interruption as well as respond to the baseline report findings. Specific additional activities to the FHI360 program description include: Activity 1: Accelerate start-up in the outgoing partner program areas; these are areas that were originally served by Deloitte Consulting Limited, i.e. Coast, Morogoro regions and Zanzibar. This is to ensure smooth transitioning of OVC from the exiting partner to PT and uninterrupted services to OVC within these regions. Activity 2: Strengthen the capacity of MVC households headed by youth or elderly care givers. 62 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Activity 3: Pilot protection of MVC living and/or staying on the street within the PT program. Pact Inc. The program covers the Lake and Southern zones, including the regions of Mtwara, Lindi, Mwanza, Geita, Mara, Musoma, Kagera and Tabora. A modification was done in 2011 expanding the program’s coverage to include the Southern Highlands zone, which has four regions: Mbeya, Ruvuma, Rukwa and Katavi. Original Key Objectives: Objective 1: Increase community ownership and capacity to implement the NCPA, enabling MVC to access comprehensive care and support within their communities. Objective 2: Strengthen the capacity of district authorities and local implementing partner organizations to manage the MVC program in their district and/or region. Objective 3: Replicate effective multi-sectoral coordination structures that effectively include public-private partnerships at district and village levels, with representatives engaged from all key stakeholders (children, families, community groups, service providers, local government and private sectors). Objective 4: Strengthen the capacity of families to meet the basic needs of their own MVC. Pact’s award modification included not only geographical expansion into the Southern Highlands but also reviewing and adding targets and objectives to the original program description. The additional strategic objectives included: Objective 1: Increase community ownership and capacity to implement the NCPA. Objective 2: Strengthen the capacity of district authorities and local implementing partner organizations to manage the MVC program in their district and/or region. Objective 3: Replicate effective multi-sectoral coordination structures that include public-private partnerships at district and village levels, with representatives engaged from all key stakeholders (children, families, community groups, service providers, local government and private sectors). Objective 4: Strengthen the capacity of families to meet the basic needs of their own most vulnerable children. World Education Inc. (WEI) WEI/Bantwana supports the Northern Zone Coverage Area: Arusha, Kilimanjaro, Manyara and Tanga Regions. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 63 Original Key Objectives: Objective 1: Increase access to and utilization of comprehensive MVC services through community initiatives. Objective 2: Strengthen human and organizational capacity of local community structures (MVCCs and community-based organizations) and LGAs to meet the needs of MVC. Objective 3: Increase community awareness and engagement, child participation and advocacy for social protection of MVC. PT REPORTING OVC INDICATORS PT organizations report on both PEPFAR and their own developed program indicators to track impact of their interventions using both quantitative and qualitative tools. The organizations’ implementation data is channeled to the national level through the MVC database at the district level. Concurrently, the organizations are required to submit to the agreement officer representative their quarterly progress implementation reports and semiannual and annual reports of the program’s core PEPFAR targets using the PEPFAR web-based data system known as PROMIS. The overall expected results of the designed PT program were to attain the following:  Increased OVC geographical coverage in a coordinated manner in order to ensure efficient and non-duplicative efforts  Strengthened capacity of families to protect and care for orphans and other children made vulnerable by HIV and AIDS  Mobilized and strengthened LGAs and community-based responses to support OVC  Ensured access to essential and comprehensive services for orphaned and vulnerable children  Assurance of the government and continued local government plans to protect the most vulnerable children  Raised awareness and a created supportive environment for children affected by HIV and AIDS at all levels  Established viable sustainability plans for supporting of the human resources that oversee vulnerable children support at different levels PEPFAR indicators reported by the PT: Direct services:  Number of adults and children who received food and nutrition services during the reporting period  Number of vulnerable households with at least one OVC/MVC or PLHIV provided with a minimum of one economic support  Number of eligible OVC provided with a minimum of one core care service 64 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Training provided:  Number of community health and para-social workers who successfully completed a pre-service training program;  Number of health care workers who successfully completed an in-service training program. EVALUATION METHODOLOGY The evaluation team shall use facilitative methods and activities that will enhance collaboration and dialogue among counterparts, particularly the MOHSW, Ministry of Education, and Ministry of Labour and Youth Development, the local government’s authorities (PMO-RALG), UNICEF, CSOs and faith-based organizations. The evaluation team shall work under the supervision and guidance of the agreement officer representative for the PT program and the mission M&E Specialist of the Health Office. The agreement officer representative will organize all internal PT meetings, including linking the evaluation team with the Health Office Team Leader and other team members. This mid-term performance evaluation will address the following key questions:  To what extent are the PT program objectives likely to be achieved? What are the challenges encountered so far?  Is the program addressing sustainability of the interventions and how?  In what ways has the PT program promoted linkages within and beyond HIV program areas?  Are there specific lessons from the program that can be applied in the second phase of the project, and to other PEPFAR programs and countries?  What was the private sector involvement and contribution to the program? The evaluation team shall develop an evaluation design and data collection methods, using a mixed methods approach to gather both quantitative and qualitative information that is based on sound social science methods and tools used in a manner to minimize potential biases. The proposed evaluation design, data collection methods, tools to be used, and work plan will be submitted to USAID/Tanzania and discussed during the team planning meeting in Tanzania. The final evaluation design and work plan shall be presented to the agreement officer representative and relevant PEPFAR and Health Team members for comments. The PT agreement officer representative will arrange for an initial introductory meeting with appropriate MOHSW, DSW staff and PEPFAR Coordinator’s office at the outset of the process. Where necessary the agreement officer representative may participate in meetings with the GoT representatives and partners. A general list of relevant stakeholders and key partners will be provided to the evaluation team by the agreement officer representative at the time of arrival, but the evaluation team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 65 as to develop a comprehensive understanding of the program and services offered through the PT cooperative agreement. The final methodology, together with evaluation tools and work plan, will be developed as a product of the team planning meeting shared and approved by the mission and DSW, prior to application. Background Reading Prior to conducting field work, the evaluation team will review existing literature and data that will include:  The NCPA I & II, PEPFAR OVC and GHI guidelines, other related strategic documents and policies, medium-term strategic and business plans and M&E APR and SAPR reports  PEPFAR planning and strategic documents, including PEPFAR Country Operation Plan (COP)  2012 PEPFAR Guidance for OVC Programming  The Law of the Child Act (2009)  Zanzibar Children Act (2011)  Taking Evidence to Impact (UNICEF Report 2011)  Women and Children in Tanzania (2010)  National Survey report on violence against children  Tanzania Social Action Fund III  PT cooperative agreement documents including:  RFA  Implementing partners’ application proposals  Annual implementation plans  Implementing partners’ M&E plans  Implementing partners’ quarterly and annual progress reports  Implementing partners’ special reports  Environmental compliance reports  Baseline and other evaluation reports  Any other reports and documents reflecting MVC work in Tanzania All team members will review these documents in preparation for the initial team planning meeting. Team Planning Meeting in Tanzania  A two-day team planning meeting will be held in Tanzania before the evaluation begins. The first meeting will:  review and clarify any questions on the evaluation SOW  clarify team members’ roles and responsibilities 66 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation  establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion  review and develop final evaluation questions  review and finalize the assignment timeline and share with other units and the PEPFAR OVC TWG  develop data collection methods, instruments, tools and guidelines  review and clarify any logistical and administrative procedures for the assignment  develop a data collection plan  finalize the evaluation questions  draft the initial work plan to share with other units, the PEPFAR TWG, and OGAC  develop a preliminary draft outline of the team’s report  assign drafting responsibilities for the final report  The second planning meeting will be at USAID and will be the initial briefing with PEPFAR/Tanzania, the agreement officer representative, the mission M&E team, the mission gender focal person and other members of the Health Team to allow PEPFAR to present the team with the purpose, expectations and agenda of the assignment. The evaluation team will present an outline and explanation of the design and tools of the evaluation.  The third meeting will be with USAID and PEPFAR/Tanzania to present the preliminary findings and implementation challenges.  The final meeting is for the evaluation debrief on the summary of the data, draft recommendations and report.  The evaluation team will be accompanied by a member of staff from PEPFAR OVC TWG/Tanzania and/or DSW. The site visits will involve interviews with District Health Management Teams, sub-grantees, MVCC, MVC and targeted community beneficiary groups. The purpose of these site visits is to gain a better understanding of the technical competence of PT implementing partners’ staff and sub-partners, the constraints encountered in the various categories of activity implementation and key target audiences’ perception of their needs in order to provide quality services.  The evaluation team should outline key meetings to coordinate post-field visits in order to share findings and get final inputs before preparing the report. Data Analysis The report should include both qualitative and quantitative analysis of the achievements in relation to the objectives and targets for the output indicators for the cooperative agreement. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 67 Wrap up and debriefing At the conclusion of the field visits and key informant interviews, there will be a debrief meeting at USAID/Tanzania, and also with DSW and key MVC/OVC stakeholders. The purpose of the meeting will be to share preliminary findings and get final inputs before preparing the draft assessment report. TEAM COMPOSITION The mission proposes a team of four consultants: a team leader, a capacity development expert, an M&E expert and an OVC expert. The OVC expert can be HQ staff who works closely with the Tanzania community care team. The M&E expert should be local. All team members should have the following characteristics:  Master’s degree or higher level of education in a relevant technical area (experience working in OVC program is desirable)  Knowledge, skills and experience with USAID contracting and reporting requirements, policies and initiatives and tools, such as performance monitoring plans (PMPs) and results frameworks  Advanced written and oral communications skills in English  Expertise working in developing countries with decentralized health systems  Strong quantitative and qualitative analysis skills Additionally, the team members should together include the following individual levels of expertise: Team Leader/MVC/OVC (International): This person should have a minimum of 10 years of experience in public health, with technical knowledge and experience with MVC/OVC intervention, preferably community-based support. The team leader will be identified by the research firm/contractor and approved by USAID prior to the start of evaluation activities. The team leader will be responsible for (1) managing the team’s activities, (2) ensuring that all deliverables are met in a timely manner, (3) serving as a liaison between the mission and the evaluation team and (4) leading briefings and presentations. In addition the team leader must have these characteristics:  Excellent skills in planning, facilitation, and consensus building  Demonstrated experience leading an evaluation team  Excellent interpersonal skills  Excellent skills in project management  Excellent organizational skills and ability to keep to a timeline Monitoring and Evaluation: One person, preferably a local consultant, with relevant professional qualifications and skills and at least five years of experience in USAID M&E procedures and project and organizational management. This person should also have strong knowledge, skills and experience in qualitative and quantitative evaluation tools, 68 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation as well as skills in community program designing and monitoring. The mission M&E specialist can participate in the evaluation exercise. Capacity Building Expert: This person should have a postgraduate degree in organizational development or health systems. S/he should have at least five years of experience with institutional capacity building and organizational development in developing countries. S/he should be knowledgeable in program assessment and evaluation methodologies of community-based programs and work plan development and training in information monitoring systems. S/he should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting programmatic evaluations/assessments. A capacity building expert from OHA can participate. TIME AND LEVEL OF EFFORT PEPFAR/Tanzania anticipates that the period of performance of this assessment will be approximately 56 days. This would include preparation days, in-country work in Dar es Salaam and the regions and report writing and finalization. The following is a sample timeline. Task/Deliverable Duration Review background documents and offshore preparation work 3 days Travel to Tanzania 2 days international; 1 day for local team member Team planning meeting including meetings with community care ITT and health teams 2 days Information and data collection, key informant interviews and site visits 8 days Draft assessment report, in-country discussion, review of information and analysis 2 days Consult with the PT partners on the preliminary findings and clarifications 1 day Report preparation and writing (in country) (preliminary draft report due to mission before departure from country) 2 days Debrief meetings with community care teams and key stakeholders 0.5 day Depart Tanzania, travel 2 days PEPFAR community care TWG provides comments on draft report (team out of country) 5 days Team reviews comments and revises final report 2 days PEPFAR community care TWG completes final review 10 working days Additional time for site visits, report writing and finalizing 19 days (team leader) / 17 days (team members) Total days 56 (team leader) / 53 days (team members) LOGISTICS A six-day work week is authorized when team is working in country; local holidays are not authorized. The GH Tech will be responsible for all off-shore and in-country USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 69 logistical support including cost for transport. However, implementing partners and USAID will facilitate some logistics. This includes arranging and scheduling meetings (with exception to previously mentioned meetings with GOT and initial introductory meetings), international and in-country travel (including vehicle rentals), hotel bookings, working/office space, computers, printing and photocopying. In addition, the evaluation team leader is responsible for draft and final report development, as well as other eligible expenses associated with the completion of the assignment. EXPECTED DELIVERABLES The contractor deliverables shall include: 1. Inception report with tentative evaluation plan/schedule indicating which step/activity should occur and at which time to make sure that the evaluation team understands the assignment. 2. Evaluation design and methodology clearly articulating how evaluation questions will be answered during the pre-evaluation meeting. 3. Second briefing immediately after visits and data analysis to make sure that all questions have been addressed. 4. Completed draft of the evaluation report to USAID/Tanzania for presentation during the debrief meeting that will be held approximately three days before departure (see report format provided in “Reporting Requirements” below). After the debrief meeting, the evaluation team shall incorporate oral comments received from PMI and stakeholders. 5. Draft report should be completed prior to the team leader’s departure from Tanzania. The written report should clearly describe findings, conclusions and recommendations (using the report format provided in “Reporting Requirements” below). Community care team will provide comments on the draft report within five working days of submission. 6. A PowerPoint Slide Presentation of the key findings, issues, and recommendations before departure from Tanzania. 7. A final report that incorporates the team responses to mission comments and suggestions. The draft final report should be completed within five days after PEPFAR provides its feedback on the draft report, incorporating the comments received from the review of the draft and sent to the mission. The report format should be restricted to Microsoft products and 12-point type should be used throughout the body of the report, with one-inch page margins on all sides. The report shall not exceed 30 pages, excluding references and annexes. 8. List of all reviewed/cited sources in final report. The final product will be a working draft, as there will not be adequate time to professionally edit and format the final draft report prior to the GH Tech Bridge IV Project’s contract end date of March 21, 2014. 70 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation REPORTING REQUIREMENTS The findings from the evaluation will be presented in a draft report at a full briefing with PEPFAR/Tanzania and possibly at a follow-up meeting with key stakeholders. The format for the evaluation report is as follows:  Executive Summary: concisely state the most salient findings and recommendations (not more than 4 pages)  Table of Contents (1 page)  Introduction: purpose, audience, and synopsis of task (1 page)  Background: brief overview of OVC program in Tanzania, PEPFAR’s strategies and priorities, brief description of the Pamoja Tuwalee program and purpose of the evaluation (2-3 pages)  Methodology: describe evaluation methods, including constraints and gaps (1 page)  Findings/Conclusions/Recommendations: for each objective area (15-20 pages)  Issues: provide a list of key technical and/or administrative issues identified (1-2 pages)  Future Directions/Recommendations based on gaps or innovation model to be scaled up (2-3 pages)  References (including bibliographical documentation, meetings, interviews and focus group discussions)  Annexes: evaluation methods, schedules, interview lists and tables. These should be succinct, pertinent and readable, and they should include a list of documents consulted and the SOW. Report Contents The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an annex in the final report.  Evaluation findings will assess outcomes and impact on males, females and children.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 71  Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. MISSION CONTACTS Elizabeth Lema Community Care Unit Lead (OVC/HBC) Agreement Officer Representative, Pamoja Tuwalee Program Health Office USAID Tanzania Plot 686 Old Bagamoyo Road, Msasani P.O. Box 9130 Dar es Salaam, Tanzania Email: elema@usaid.gov Tel: +255 22 2294490 Ext. 4361 Cell: +0754292761 Alternate Moses Busiga Monitoring and Evaluation Specialist Health Office USAID Tanzania Plot 686 Old Bagamoyo Road, Msasani P.O. Box 9130 Dar es Salaam, Tanzania Email: mbusiga@usaid.gov Tel: +255 22 2294490 Ext. 4595 Cell: +255 659 269 188/ 0764 269 188 72 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation ANNEX II. EVALUATION METHODS AND LIMITATIONS EVALUATION METHODOLOGY The evaluation team proposes to use a mixed-method evaluation approach to comprehensively respond to all five evaluation questions above. A mixed-method evaluation approach systematically integrates two or more evaluation methods at every stage of the evaluation process, drawing on both quantitative and qualitative data. The proposed design has the highest potential to provide strong results and is endorsed by the USAID Evaluation Policy (2011), which states, “Given the nature of development activities, both qualitative and quantitative methods yield valuable findings, and a combination of both often is optimal.” (p. 4). Data collection methods will include literature review and content analysis, focus group discussions, structured and semi-structured key informant interviews and modified case studies. The team has developed discussion guides along with observation guides. The team proposes to conduct gender specific data collection to ensure confidentiality as well as creation of spaces that facilitate open dialogue on gender-specific issues. The team will collaborate on all design research and analysis and development of research tools with the USAID Tanzania mission. Using commonly shared tools, the team may split up during some site visits to efficiently expand coverage.The key evaluation questions in the design matrix (see Annex 4: Evidence and Data Sources) provide a common structure for all notes and data and are all keyed into a coding format to ease data management. Responses to the key evaluation questions and indicators within the five main areas (overall performance and impact, efficiency, coverage and design, sustainability and gender) form the core of the database. Information derived through other methods (literature review, direct observation, questionnaires, focus group discussions, group exercises, etc.) will also be coded to the key evaluation questions during data collection. To the extent possible, respondents and participants will be identified by gender, and all data collection and analysis will be undertaken with a gender perspective. DATA COLLECTION TOOLS Specific tools developed and adapted by the team include: Key Informant Interview Guides The evaluation team will develop key informant interview guides to collect information from respondents in strategic/leadership positions who can provide key insights around project effectiveness as well as learning. Key informants will be identified through purposive sampling, and the team will use respondent driven sampling to allow for identification of respondents who would not have been initially identified. Although the approach (key informant interview) will be the same, there will be varied guides focusing on USAID staff, USAID technical assistance partners, partner staff, stakeholders and community key informants (e.g. village/ward executive officer, community-level sub- USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 73 grantee staff and community-based volunteers). Stakeholder overview and analysis will allow the team to ensure that the breadth of participants will be covered. A draft list of key informants is included in Annex B. Justification: Using the key informant guide will facilitate collection of strategic information in a guided but unstructured way. FGD Guides The evaluation team will develop FGD guides for collecting data with beneficiary groups in the implementation communities. The FGD guides will be unstructured and will include a minimum of six and a maximum of 10 respondents. There will be four FGDs per region with respondents organised as follows: women only, men only, girls only, boys only and mixed youths (female and males). Justification: FGDs will ensure discussions are not personalized and will allow for deeper exploration of issues. The unstructured nature of the tools allows respondents to provide critical information that falls outside the strict M&E terms. Consultative Meetings The evaluation team proposes to conduct consultative meetings with regional, district level and village or shehia respondents. Meetings will be conducted where one-on-one interviews will not be feasible and will focus on collecting information from key stakeholders. The meetings will include respondents from different institutions including Government of Tanzania (Mainland) and Zanzibar, international and national non￾governmental organizations, faith-based organizations, civil society organizations and others who cannot be interviewed through in-depth key informant interviews due to time constraints. Secondary Data The evaluation team will not collect quantitative primary data and will rely on PROMIS. To facilitate systematic secondary data extraction, a template will be developed with the specific partner indicators and the data requirements. In addition to data from the extraction template, the evaluation team will verify data by using partner PMPs. INFORMED CONSENT The evaluation team will seek verbal informed consent from FGD participants for participation in the evaluation and taking of photos. The team will obtain written consent for any photos with identifiable individuals. In some cases, the team will rely on the sub￾grantees to assist in obtaining consent. Participants above 18 years will provide their own informed consent. Verbal informed consent from participants below 18 years (minors) will be provided by both their legal guardian (including school authorities) and the children. Facilitators will ask for consent prior to conducting discussions and provide potential respondents with opportunities to opt out if they are not comfortable. SAMPLING Four regions in the Tanzania Mainland have been purposefully selected by USAID according to these criteria: maturity of programs, HIV prevalence, rates of poverty and 74 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation geographical coverage. Zanzibar has been purposefully selected to ensure full representation of The United Republic of Tanzania. These regions include Kagera (Lake Zone), Dodoma (Central zone), Arusha (North East zone), Zanzibar and Mtwara (Southern zone). Overview of OVC Conditions in Sampled Regions Region Net Primary Enrolment Net Secondary Enrolment Percentage of Children whose births are registered Arusha 84.3 27.7 25.3 Dodoma 67.7 15.8 5.9 Kagera 75.7 22.3 6.7 Mtwara 82.2 26.5 6.9 Zanzibar 78.7 Source: Tanzania Demographic and Health Survey (2011) As directed by the scope of work, the evaluation team will use multi-stage random sampling to select two districts per region (one urban and other rural). Respondent￾driven/snowball sampling will be used to identify additional respondents who can provide critical information to the evaluation, but who would not have been identified by the mission and project partners. Expert case sampling will be used to identify key policy/programming level experts who can provide rich insights which can shape overall conclusions and recommendations. The figure below shows the proposed sampling approach. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 75 Sampling Approach 76 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation FIELD-LEVEL DATA COLLECTION APPROACH An evaluation team leader, evaluation expert and M&E expert will conduct data collection. Due to time constraints and the need to optimize utilization of expertise, there will be some interviews conducted by the whole team while the team will split for other interviews to ensure as many stakeholders as possible are consulted. Allocation of primary research responsibilities will be guided by team members’ areas of expertise, and these will include MVCC, economic strengthening, savings, caregiving, education and training and M&E. At regional level, the whole team will make a courtesy call to the Regional Secretary as well as the DED and thereafter split up to interview sub-grantees or other stakeholders. At the sub-grantee levels, the team will be provided with a comprehensive overview and thereafter use a combination of one-on-one interviews and self-administered interview guides for sub-grantee staff. This will ensure wide consultations while maintaining the depth and quality of data to be collected. Field Data Collection Approach DESIGN STRENGTHS  The design allows for triangulation where quantitative data from partners will be complimented by qualitative accounts from consultations as part of the mid-term performance evaluation.  There are different evaluation questions for the mid-term performance evaluation, and these require different methods sometimes more than one method to sufficiently answer all components. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 77  The mid-term performance evaluation intends to use different methods to answer the same elements of a single question, increasing confidence in the validity and reliability of the evaluation results. DESIGN LIMITATIONS AND ISSUES Some weaknesses are inherent to the overall evaluation design. These have the potential of threatening the internal validity of the evaluation. During the evaluation period, the team is unable to establish plausible counterfactuals against which one can measure effectiveness of the various approaches. However, the sample of 17 program objectives among four implementing partners provides sufficient variety and diversity of approaches to address the key evaluation questions. Sampling is a combination of purposive and random, as explained under 3.3 Sampling. The team will rely on secondary quantitative data provided by the implementing partners, and primary data qualitative data. Limitations of the evaluation include:  Time: The evaluation design includes only 22 days for site visits in five regions (Mainland and Zanzibar) and two districts (urban and rural) within each region. To accomplish this extensive travel, some data analysis and report writing will be undertaken after the team leaves Tanzania.  Geography: The national scope of the program includes all regions of Tanzania. The design includes visits to regions across the Mainland and Zanzibar, and two districts in each region. Travel time to cover all sampled areas raises challenges. Travel within Tanzania can be difficult due to fragile infrastructure, poor communications networks, weather and other circumstances.  Baseline Documents: Lack of complete pre-project baseline data for some project partners against which to measure progress toward achieving objectives may hinder validity.  Lack of household survey as part of midterm evaluation data collection methods: This limits the extent to which the evaluation will objectively measure program performance from project inception. All project partners conducted baseline household economy surveys, which could have been comparable with midterm household economy surveys to measure progress made. However, this will be mitigated through using partner M&E data.  Personnel: One member of the four-person team was unable to join the team due to an emergency. The three team members will divide up at all levels to ensure as many respondents as possible are covered. Team members may be assisted by translators, as needed.  Access may be limited by availability of key personnel. In all cases the team will attempt to follow up with key respondents through phone interviews or email if necessary.  Potential sampling bias: To minimize bias, four regions in the Tanzania Mainland have been purposefully selected based on maturity of programmes, HIV prevalence, poverty and geographical coverage, and also Zanzibar selected to represent the other side of the United Republic of Tanzania. 78 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation PROPOSED DATA MANAGEMENT/DATA ANALYSIS PROCESS The evaluation team will develop a data analysis plan after finalization of data collection tools (See attached analysis plan on how specific evaluation data will be analyzed). All evaluation data will be analyzed by data collection method then partner, region, district, gender and age group of program beneficiaries. Qualitative data analysis is proposed for primary data collected using key informant, FGD, observation, personal narratives, histories or case studies and collection guides. Content analysis, thematic analysis and quasi statistics will be used to analyze qualitative findings. FGD participants’ demographic information will be analyzed using quantitative descriptive statistics methods such as frequencies, percentages and averages. Routine monitoring and evaluation data from PROMIS, Partner Indicator Performance Tracking Tables and routine narrative reports will be quantitatively analyzed and presented in bar and pie charts, line graphs and tables. The evaluation team will triangulate all qualitative and quantitative analysis results in developing the debrief PowerPoint presentation and draft evaluation report. Feedback from USAID, PEPFAR, partners and stakeholders will be incorporated into the final evaluation report. The table below gives an overview of the proposed mixed data collection methods for each broad evaluation question. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 79 OVERVIEW OF PROPOSED DATA COLLECTION METHODS Evaluation Question Number Broad Evaluation Question Data Type Data Collection Methods Qualitative Quantitative KII FGDs Stories of Change Site Visits or Observations Document Review M&E Review 1 a. To what extent are the PT program objectives likely to be achieved? √ √ √ √ √ √ √ √ b. What challenges have been encountered so far in achieving PT program objectives? √ √ √ √ √ √ 2 Is the program addressing sustainability of the interventions and how? √ √ √ √ √ √ √ 3 In what ways has the PT program promoted linkages within and beyond HIV program areas? √ √ √ √ √ 4 Are there specific lessons from the program that can be applied in the second phase of the project, and to other PEPFAR programs and countries? √ √ √ √ 5 What was the private sector’s involvement and contribution to the program? √ √ √ √ 6 Conclusions and Recommendations √ √ √ √ 80 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation ANNEX III. PERSONS INTERVIEWED Organization Name Designation Area Mobile Number Email address USAID Tanzania Janean Davis PEPFAR Coordinator Dar es Salaam Elizabeth N. Lema Community Care SRU Lead Dar es Salaam 0754 292 761 Frank Eetaama USAID Palliative Care Specialist Dar es Salaam feetaama@usaid.gov Jacquiline Gaye USAID Community Care Advisor Dar es Salaam jgayle@usaid.gov Jennifer Erie USAID Gender Advisor Dar es Salaam jerie@usaid.gov Joan Mayer USAID-Iringa/Njombe jmayer@usaid.gov Mary Chale OVC Specialist Dar es Salaam mchale@usaid.gov Moses Busiga Monitoring and Evaluation Specialist Dar es Salaam mbusiga@yahoo.com Seth Greenberg Ag. Community Based Team Leader Dar es Salaam sgreenberg@usaid.gov UNICEF Birgithe Lund￾Henriksen Chief, Child Protections Dar es Salaam blundhenriksen@unicef.org Mbelwa Gabagambi Child Protection Specialist GOVERNMENT DEPARTMENTS: NATIONAL Ministry of Health and Social Welfare Dunford D. Makala Commissioner for Social Welfare Dar es Salaam 0655 596526 dunfordmakala@yahoo.co.uk Philbert Kawemama Head, Child Welfare and Protection Unit Dar es Salaam 0784 247 233 0654 501 502 0753 306 802 kawemama@yahoo.co.uk Jane Nyetabura Deputy Commissioner for Social Welfare Dar es Salaam 0713335454 GOVERNMENT DEPARTMENTS: REGIONAL Rebecca Gwambasa Regional Social Welfare Officer Kagera 0757 898 446 rebby24@yahoo.com USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 81 Organization Name Designation Area Mobile Number Email address Ministry of Empowerment, Youth, Women and Children Government of Zanzibar Mme. Muna Omar Senior Social Welfare Officer Zanzibar munaos@yahoo.com Prime Minister’s Office Evelyne P. Itanisa Regional Administrative Secretary (RAS) Arusha 0754 367 845 epitanisa@yahoo.com Prime Minister’s Office Smythies E. Pangisa Assistant Administrative Secretary Planning and Coordination Mtwara 0713 487 069 sepangisa@yahoo.co.uk Prime Minister’s Office Rehema Madenge Regional Administrative Secretary (RAS) Arusha Prime Minister’s Office Nkinda Regional Social Welfare Officer Arusha GOVERNMENT DEPARTMENTS: DISTRICT Arusha City Council Blandina Regional Development Officer Arusha Arusha City Council Daudi Felix Ntibenda District Commissioner Arusha Arusha City Council Dr. Bakari Salum City Medical Officer Arusha 0757 955 977 0783 211 768 bkrsalum@yahoo.com Arusha City Council Loyce Ndutu AG. City Education Officer Arusha 0755 497 884 Arusha City Council Rose William Maliya Nutrition Officer Arusha hkimbita1975@yahoo.com Chamwino District Council Adrian J. Jungu District Executive Director Dodoma 0755 516 319 jungu.adrian@yahoo.com Dodoma Municipal Council Robert M. Kikindu District Executive Director Dodoma 82 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organization Name Designation Area Mobile Number Email address Dodoma Municipal Council Euphrezia Anthony Municipal Social Welfare Officer Dodoma phrezia@yahoo.com Dodoma Municipal Council Inviolata Swai Health Secretary (CHMT) Dodoma Dodoma Municipal Council Martin Tumaini District Focal Person: MVC Dodoma hopekyara@yahoo.com Ministry of Health and Social Welfare Dr. Lucas Juma Kazingo District Medical Officer Karatu Ministry of Health and Social Welfare Efrazier Anton Social Welfare Officer Dodoma Ministry of Health and Social Welfare Elizabeth Herman District Social Welfare Officer 0787 208 486 Ministry of Health and Social Welfare Frazaiza Anton District Social Welfare Officer Dodoma Ministry of Health and Social Welfare Godlove Miho District Social Welfare Officer 0717 135 718 godlove78@yahoo.co.uk Ministry of Health and Social Welfare Michael Charles District Social Welfare Officer Biharamulo 0788 238 988 michaelbugeraha@yahoo. com PACT Mary Goreth Empowerment Worker Bukoba Police Amaza Nassoro 0755 991 909 Police Antonia J. Nazega Chairperson Police Gender Desk Dodoma 0754 069 630 Police Tekiemdbya Mvungi Police Gender Desk Dodoma 0754 889 570 Police Wevina Dume Police Gender Desk Dodoma 0754 837 322 Dr. Cyprian Chilowaka Director, Community Health Dodoma USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 83 Organization Name Designation Area Mobile Number Email address Frank Sule Community Development Officer and acting Council Director Karatu Linus Lukolela District Community Development Officer Biharamulo 0784 346 902 luco_linus@yahoo.com Rose William Maunga Nutrition Officer 0784 427 715 lkimbita1975@yahoo.com GOVERNMENT DEPARTMENTS: WARD AND VILLAGE AFRICARE Aisha Kidilleh MEO (Mtaa Executive Office) Mnadani, Dodoma Municipal 0713420131 AFRICARE Edith Mbelo Caregiver, Head of Primary MVCC Dodoma Ilolo Village AFRICARE Elias Kweli Volunteer AFRICARE Ivan Mateo Mangesa Village Executive Officer Dodoma Mlodaa Village AFRICARE Joseph Masadu Mwenyekitiwa Kijiji (Village Chairman Ilolo) Dodoma AFRICARE Martha Jerome Volunteer Dodoma AFRICARE Moses Mlema Caregiver Dodoma Mlodaa Village AFRICARE Moses Mlemwa Volunteer Dodoma, Chamwino 0685 124 605 AFRICARE Mwajuma Msagaa Volunteer 0766 303 920 AFRICARE Robert Caregiver, Volunteer for PT Dodoma Ilolo Village AFRICARE Robert Kuwayawaya Volunteer Dodoma AFRICARE Robert Maludes Volunteer Dodoma, Chamwino 0712 083 509 AFRICARE Sospeter Mwaluko Volunteer Dodoma, Chamwino 0657 016 804 AFRICARE Suzan Nanguka Mtendaji Kijiji/MEO Dodoma Ilolo Village 84 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organization Name Designation Area Mobile Number Email address FHI360 Ally Said Shehe MVCC Member: Religion Nyerere Shehia FHI360 Fauzia Ame Hamad Volunteer Nyerere Urban FHI360 Hamdu Shaka Hamdu MVCC Nyerere Shehia FHI360 Khamis Sabiri Nassor MVCC Volunteer Nyerere Shehia FHI360 Khatibu Kibweni Haji Volunteer Shehia yaShakani 0777 277 010 FHI360 Mariam Suwed Othman MVCC Volunteer Nyerere Shehia FHI360 Mwajuma Ismail Said Volunteer Nyerere Urban 0773 290 203 mwajumaismail@yahoo.com FHI360 Mwanahamis Abdallah Faki Volunteer Nyerere Urban 0773 613 389 FHI360 Mwanaid Abdallah Suleiman MVCC Secretary Nyerere Shehia FHI360 Mwatima Abdallah Haji MVCC Coordinator: MCH Nyerere Shehia FHI360 SubiraIddi Magambo MVCC Mlezi Nyerere Shehia FHI360 Uledi Khamis Khamis M Volunteer Nyerere Urban 0772 805 905 FHI360 Yumna Hassan Makame MVCC Teacher Nyerere Shehia Ministry of Health Mary Chitume Chamwino Health Centre Staff Chamwino Dodoma PACT Agnes Empowerment Worker Kagera Nyamahanga Village PACT Habibi Msham Samil Empowerment Worker Newala 0757 817 712 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 85 Organization Name Designation Area Mobile Number Email address PACT Happiness Rwassa Empowerment Worker Kagera Nyamahanga Village 0758 713569 PACT Lydia Leonard Ward Executive Officer Kagera Nyamahanga Village 0782 902590 PACT Rita Leonard Ward Executive Officer Kagera Nyamahanga Village PACT Salehe Bakazi Oritili Para-Social Worker Newala 0773 927 265 PACT Wolfrem Mathew Mahru Village Executive Officer Kagera Nyamahanga Village 0786 64604 WEI Stephen Irambo Headmaster of Bwarawani Primary School 0764 514 044 Aziza Katale Giti Volunteer Shehia yaDonge Mchangani 0773 849 021 Bimkubwa Abdi Mohamed Volunteer Shehia yaMwanakwe rekwe Mohamed Choum Bakari Volunteer Shehia yaKwawazee 0778 851 334 Ms. Mguma Msashaa Volunteer Raphael Titus Ward Councilor 0755 763 158 Zainab Ally Ward Executive Officer Pendo MVC assisted by MVCC Dodoma Municipal IMPLEMENTING PARTNERS AFRICARE Sekai P. Chikowero Senior Country Director Dar es Salaam 0783 654 844 schikowero@africare.org Herbert Mugumya Chief of Party PT Dar es Salaam hmugumya@africare.or.tz Conjesta Shao Deputy Chief of Party Dodoma 0754 972 920 86 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organization Name Designation Area Mobile Number Email address Fredy Turuka Project Coordinator UMATI Joyce Pandu Regional M&E Officer Dodoma 0764 929 214 jpandu@africare.org Datus Ng'wanangwa Chief of Party Dar es Salaam 0787 273 838 dngwanangwa@africare.org Donata Dodoma 0787 432 419 Aloyce Mkangaa Team Leader 0784 526 881 amkangaa@africare.org Felician Luchagula Senior M&E Advisor 0784 275 090 fluchagula@futuresgroup.co m Dickson Nyakitinga Mbita Economic Strengthening Technician Dodoma dmbita@africare.org Hadija Halidi Food & Nutrition Officer TAHEA Dodoma hadkit2007@yahoo.com FHI360 Priskila Gobba Program Director Dar es Salaam 0754 783 445 pgobba@FHI360.org Levina Kikoyo Deputy Program Director Dar es Salaam 0754 594 079 lkikoyo@FHI360.org Castor Kalemera Senior Technical Officer, M&E Dar es Salaam ckalemera@FHI360.org Shaibu Zanzibar Pamoja Tuwalee Regional Technical Officer Zanzibar Kenneth Chima Senior Technical Officer: OVC 0754 377 313 kchima@FHI360360.org Felista Mandari Technical Officer and M&E 0757 065 577 fmandari@FHI360.org Judith Masasi Senior Technical Officer: Child Protection 0754 784 414 jmasasi@FHI360.org Ipyana Lwinga Technical Officer and M&E 0757 065 577 ilwinga@FHI360.org PACT Todd Malone Country Director Dar es Salaam 0768 176 677 Tmalone@pactworld.org Linda Madeleka Deputy Chief of Party Dar es Salaam 0767 290 888 imadeleka@pactworld.org Daisy Kisyombe M&E Manager Dar es Salaam Norbert Nassay Zonal Manager Lake Zone 0754 531 611 nmassay@pactworld.org Kachocho Timanywa Senior Program Coordinator ktimanywa@yahoo.com kttimanywa@gmail.com USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 87 Organization Name Designation Area Mobile Number Email address Casimir Chipere Senior Technical Advisor Dar es Salaam 0757 615 641 cchipere@pactworld.org Anthony Binamungu Child Protection & GBV Specialist Dar es Salaam 0756 602 589 Jenipha P. Tarimo Economic Strengthening Coordinator Mtwara 0713 400 517 0755 918 919 jtarimo@pactworld.org Getrude Rweyemamu Economic Strengthening Coordinator Bukoba 0754 629 751 grweyemamu@pactworld.org Mtaki A. Mtaki Program Officer Mtwara 0786 947 780 mmtaki@pactworld.org Anthony Binamungu Child Protection & GBV Specialist Dar es Salaam 0756 602 589 Romuald Mwesiga Program Officer Bukoba 0755 813534 Deus Kapinga M&E Officer Dar es Salaam 0752 160682 Chela Ghanai MERL Officer Dar es Salaam 0767 850 021 Robert Kindishi MERL Officer Dar es Salaam 0755 909 192 Magawa Abdallah Program Officer Dar es Salaam 0763 490 230 Frida Chilimo Child Protection Officer Dar es Salaam 0762 604 366 Iseyongjee Health & Nutrition Intern Dar es Salaam 0769 210 199 Sherelle Wilson Program Advancement Intern Dar es Salaam Esther Ndyetabura Child Protection Officer - Intern Dar es Salaam 0715 170 901 WEI Lilian Badi Chief of Party Arusha 0754 371 747 lbadi@worlded.co.tz Dr. John Hillary Deputy Chief Of Party Arusha 0655 434 006 jhillary@worlded.co.tz Wences Msuya Health and Nutrition Coordinator Arusha 0715 717 990 wmsuya@worlded.co,tz Grace S.Muro Advocacy Coordinator Arusha 0754 265 474 gmuro@worlded.co.tz Deogratius Makoye Director of Finance & Administration Arusha 0715 283 212 dmakoye@@worlded.co.tz 88 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organization Name Designation Area Mobile Number Email address Patrick Ngowi Child protection and Psychosocial Support Coordinator Arusha 0782 444 355 pngowi@worlded.co.tz Collen Masibhera M&E Advisor Arusha 0654 700 784 cmasibhera@worlded.co.tz Annamaria Magige Field Technical Manager Tanga 0784 335 222 amagige@worlded.co.tz SUB GRANTEES CWCD Hindu Ally Mbaego Executive Director Arusha 0754 526 492 cwcdabehije@gmail.com Happy Josephat Malamsha Program Manager Karatu Saumu Hamza Accountant Karatu cwcdalbeliise@gmail.com Derrick B.Lema M&E Officer Karatu Basil Ezekiel CP Karatu Abbas Rashid Volunteer Karatu Kanankira Risael Volunteer Karatu Nelson K.Selestine Volunteer Karatu Spirian Wilbard Mwangua Program Manager 0764 308 871 Fatmah Alfiyo Senior Programs Manager 0658 648 881 Suzan Vendeline Accountant 0652 027 839 Marium Kassim Mkali M&E 0765 882 863 Rose Robert Field Officer 0698 264 583 Emmanuel Dominick Masanja CP/PSS Officer 0767 500 901 Ambakisye Mhiche Volunteer 0757 487 278 jaykmhiche@gmail.com Sauda Swai Secretary 0755 291 862 DCT Henry Mnyamvumi Project Coordinator Dodoma 0755 010 927 mnyamvumi@yahoo.com Dalidali Rashid Social Worker Dodoma 0712 005 676 dalikissimbi@gmail.com Essau Daudi Tandilla M&E Dodoma essau_daudi@yahoo.com Gasper Samuel M&E Officer Dodoma gaspersamuel689@yahoo.co m Sarah Ntandu Social Worker Dodoma 0713 705 712 ntandusarah@yahoo.com Loice Utanga Economic Strengthening Officer Dodoma 0716 954 496 loiceutonga@yahoo.com Osarah Economic Strengthening Officer Dodoma USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 89 Organization Name Designation Area Mobile Number Email address FAWOPA Hassan Saidi Program Technical Officer Mtwara makandehassani@yahoo. com Baltzan B. Komba Executive Secretary Mtwara fawopa_g@yahoo.com Rashid Zahir Treasure Board Member Mtwara fawopa@yahoo.com NEW￾NGONET Halima Naombunga Program Coordinator Newala 0784 533 458 0655 533 458 hnambunga@yahoo.com Bosco Jones Technical Officer Newala 0713 132285 0768 401 966 mwidade78@yahoo.com Shad Saidi M&E Newala 0715 959 933 nampeha@gmail.com Juma Chande Accountant Newala matekajuma@yahoo.com SHARING WORLDS Pemba Maiseli Program Coordinator Dodoma Suzana J. Mushi Social Development Officer Dodoma 0754 848 392 Aisha Kidilleh Mtaa Executive Officer (MEO) Dodoma 0713 420 131 0767 420 131 Ponsiano Mgomi M&E Officer Dodoma ponsiano@sharingworlds.org Upendo Meja Economic Strengthening Officer Dodoma Catherin Kigenda Project Officer Dodoma Mustapha Ramadan Accountant Dodoma Settembro Amani Volunteer TADEPA Dr. Jonathan Stephen Executive Director Bukoba 0755 908 450 jkisilaga@yahoo.com; tadepaprojects@gmail.com Mr. James Barongo Program Manager Bukoba Ms. Angela Ngaiza Accountant Bukoba Penina Petro M&E Officer Bukoba Kachocho R. Timanywa OVC Coordinator Bukoba WAMATA Queen Ali Said Coordinator Pemba 0773 94 1727 Omar Ali Coordinator Pemba Amina Haji Faum Focal Person Pemba 0777 417 947 Kassim Hamad Nassor Focal Person Pemba 0773 538 561 kassimnassor@yahoo.com Mwajuma Tahir Hamad Data Clerk Pemba 0777 904 009 damuurtalur@yahoo.com ZAMWASO Project Coordinator Zanzibar Focal Person Zanzibar 90 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Organization Name Designation Area Mobile Number Email address Focal Person Zanzibar HDT Irene C. Kanyankole Regional Program Manager Biharamuro 0787 374 577 irenek@hdt.or.tz OTHER STAKEHOLDERS Consultant Jonathan Watkins Social Work Consultant: Social Protection Trainer Zanzibar 44(0)773450 9792 0786 070 389 jonathan_watkins@mac.com DAI Colleen Green Chief of Party, IMARISHA Project 0782 987 062 Colleen_green@dai.com Deloitte Paschal Wilbroad Senior Capacity Building Officer 0767132 572 pwilbroad@deloitte.co.tz Futures Group Boney Mtabazi Senior Acting Technical Advisor Dar es Salaam bmutabazi@futuresgroup. com Futures Group Felician L. Luchagula Senior M&E Advisor fluchagula@futuresgroup. com Futures Group Prisca Mgombeni Community Care, M&E Officer 0767 224 461 0713 318 565 PMgomberi@futuresgroup. com Futures Group Zaddy Kibao Senior Technical Director 0754 812 080 zkibao@futuresgroup.com ICAP Grace Lugamba Psychosocial support Kagera IYF Eliflorida Mushi Education Partners 0763 676 763 e.mushi@iyfnet.org IYF Hannah Corey Program Manager (001) 410 951 1595 h.corey@iyfnet.org IYF Iris G. Kalavo Program Officer 0763 676 763 0754 821 742 i.kalavo@iyfnet.org Save the Children Alice Mushi Pemba Save the Children Zanzibar Translator Rita R. Karani Kagera Kararita2001@yahoo.com USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 91 ANNEX IV. WORK PLAN AND SCHEDULE WORKPLAN Travel day (if needed) to AFRICARE Dodoma or Nov. 11 TBD Site visits Dodoma Site visits Dodoma municipal Dodoma rural Chamwino FHI360 Travel to Dar Unguja Site visits Travel to Zanzibar Unguja site visits 17 18 19 20 21 22 23 Sunday off To Pemba Evening flight to Dar Dar: meetings with partners, Evening flight to Arusha (Africare and FHI360 WEI: Arusha city Arusha city Karatu rural Rest day 24 25 26 27 28 29 30 Travel day Kagera PACT Kagera urban Bukoba Kagera Rural Biharamulo Kagera Rural Return to Bukoba Fly to Dar es Salaam To PACT Mtwara urban Mtwara urban DECEMBER SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY 1 2 3 4 5 6 7 Rest day Mtwara Newala rural MtwaraNewala Rural Return to Dar KII with PACT in Dar, GoT meetings Team data analysis, follow up meetings Data analysis Data analysis 8 9 10 11 12 13 14 Rest day HOLIDAY Report preparation Draft PPT Report to implementing partners USAID mission debriefing Depart from Tanzania, USAID comments on debrief and reviews report Travel day 1 Return home 92 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Color Codes Zanzibar (four days including travel) MOH, MOE, District Social Welfare Officers, DHMTs, District Education Officers, partner’s regional office, interviews with sub-grantees, service beneficiaries, consultative meeting with key stakeholders, observations for each of the two districts Arusha (four days including travel) Regional level: RHMTs, Regional Education Officer, partner’s regional office District level: D/CHMTs, District Social Welfare Officers, District Education Officers, interviews with sub-grantees, service beneficiaries, consultative meeting with key stakeholders, observations Kagera (four days including travel) Regional level: RHMTs, Regional Education Officer, partner’s regional office District level: D/CHMTs, District Social Welfare Officers, District Education Officers, interview with sub-grantees, service beneficiaries, consultative meeting with key stakeholders, observations Dodoma (four days including travel) Regional level: RHMTs, Regional Education Officer, partner’s regional office District level: D/CHMTs, District Social Welfare Officers, District Education Officers, interview with sub-grantees, service beneficiaries, consultative meeting with key stakeholders, observations Mtwara (four days including travel) Regional level: RHMTs, Regional Education Officer, partner’s regional office District level: D/CHMTs, District Social Welfare Officers, District Education Officers, interview with sub-grantees, service beneficiaries, consultative meeting with key stakeholders, observations NOTES ON THE WORK PLAN The original plan in the SOW anticipated site visits to five regions, two sites each for 2-3 days within 12 days, including internal travel (not budgeted). However, initial consultations showed that this could not be accomplished within that timeframe. Implementing partners advised that a minimum of four days per region is required for minimum exposure and field review. The evaluation team agreed to extend their in￾country availability to December 13. The team schedule calls for the addition of a team member (PEPFAR OVC or DSW person per SOW) to join the team o/a November 21, and forming two sub teams, dividing for visits to Dodoma and Mtwara following joint WEI Lake Pact Southern Pact Africare FHI360 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 93 travel in North and Zanzibar. Two days are national holidays and meetings cannot be held. A debrief meeting will be conducted in-country, and thereafter report writing will be conducted offshore. 94 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation ANNEX V. DATA COLLECTION INSTRUMENTS MID-TERM PERFORMANCE EVALUATION OF FOUR OVC PROGRAMS USAID PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Institution_________________________________________________________ Designation/Position:………………………………………………………………………… Date:………………………………… Location:…………………………………………………................. Gender: Female Male Interviewer:…………………………………………………………………………………… o rl d E d u c a USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 95 1. What have been the key strengths of the PT program thus far? ………………………………………………………………………………………………… 2. What have been the key approaches or mechanisms that have produced those strengths? ………………………………………………………………………………………………… 3. Which objectives of the PT program are they unlikely to be achieved? (Ask for reasons behind likely non-achievement) ………………………………………………………………………………………………… 4. How does the program define sustainability? ………………………………………………………………………………………………… 5. What aspects of the PT program are likely to be sustained? (Ask for evidence) ………………………………………………………………………………………………… 6. How does the mission define “linkages across services”? (Ask for examples) ………………………………………………………………………………………………… 7. How would you characterize the levels of coordination at the government level (ask for evidence to support responses provided)? ………………………………………………………………………………………………… 8. What have been the positive or negative unintended consequences emerging from the PT program thus far? ………………………………………………………………………………………………… 9. What has been done to promote private sector involvement and contribution? (Ask for examples of any successes) ………………………………………………………………………………………………… 10. What have been the main challenges faced by the PT project thus far? ………………………………………………………………………………………………… 11. What have been the key lessons learned in implementation of the PT program thus far? ………………………………………………………………………………………………… 12. What recommendations can you provide for the following? Strengthening various partner approaches ……………………………………………………………………………………………......... Strengthening linkages across services ……………………………………………………………………………………………......... Fostering sustainability ………………………………………………………………………………………………..... 13. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? 96 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF FOUR OVC PROGRAMS OTHER DONORS PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Institution_________________________________________________________ Designation/Position:………………………………………………………………………… Date:…………………………………............................................................................... Location:………………………………………………….................................................... Gender: Female Male Interviewer:…………………………………………………………………………………… o rl d E d u c a ti o USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 97 1. How does the PT program coordinate/collaborate with your institution/organization? ………………………………………………………………………………………………... 2. How effective has been the coordination with the PT program? (ask for examples) ………………………………………………………………………………………………... 3. Which PT program implementation approaches have been effective and why (ask for evidence)? ………………………………………………………………………………………………... 4. Are there any weaknesses you have identified within the PT project? ………………………………………………………………………………………………... 5. Can you offer any examples of linkages between PT partners, the GoT, other stakeholders? ………………………………………………………………………………………………….. 6. What has been done to promote private sector involvement and contribution? (Ask for examples) What else should be tried? ………………………………………………………………………………………………….. 7. What are the main challenges that you have noted in relation to the PT program? ………………………………………………………………………………………………..... 8. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………..... 9. What recommendations can you provide for the following? Strengthening PT program approaches ……………………………………………………………………………………………......... Strengthening linkages across stakeholders ……………………………………………………………………………………………......... Fostering sustainability ………………………………………………………………………………………………..... 10. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? 98 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF FOUR OVC PROGRAMS IMPLEMENTING PARTNERS’ STAFF PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Institution :…………………………………………………………………………………… Designation/Position:………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Gender: Female Male Interviewer:…………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 99 1. What do you consider to be the greatest challenges facing MVC and their families in Tanzania? ………………………………………………………………………………………………..... 2. How is (implementing partner), through the PT project, addressing these challenges? ………………………………………………………………………………………………..... 3. What is (implementing partner)’s role within the PT project? What are the key project components? ………………………………………………………………………………………………..... 4. What progress has been made in relation to the following project components? i. Increasing the capacity of communities and local governments to meet the needs of OVC and their households; ……………………………………………………………………………………......... ii. Increasing the capacity of households to protect, care for and meet the basic needs of OVC in a sustained way; ……………………………………………………………………………………......... iii. Increasing OVC household access to comprehensive, high-quality, age￾appropriate and gender-sensitive services by creating integrated community-level referral networks that strengthen the continuum of care; ……………………………………………………………………………………......... iv. Empower OVC, particularly females, to contribute to their own well-being by improving their resilience, as well as their livelihood and self-care skills; ……………………………………………………………………………………......... v. Supporting MVC living and/or staying in the street within Pamoja Tuwalee program ……………………………………………………………………………………......... vi. Addressing gender-based violence and preventing and responding to child abuse, exploitation, violence and neglect? ……………………………………………………………………………………......... 5. If your organization were to discover that a child was being abused by a family member, community member or project staff, how would (implementing partner) address this issue? ……………………………………………………………………………………………......... 6. What have been the key strengths in (implementing partner)’s implementation approach for the PT program? (ask for example why they say they are key strengths) ……………………………………………………………………………………………......... 7. How effective has been the zonal approach in facilitating program management and implementation? ……………………………………………………………………………………………......... 8. How does (implementing partner) define sustainability? ……………………………………………………………………………………………......... 9. How is (implementing partner) ensuring the sustainability of activities under the Pamoja Tuwalee project? ……………………………………………………………………………………………......... 100 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 10. How would you characterize your working relationship with the GoT ministries at the various levels? Extent of interaction, cooperation, alignment? ……………………………………………………………………………………………......... 11. How can the relationship outlined above be improved so that it is more effective? ……………………………………………………………………………………………......... 12. Could you provide specific examples of how you think that your project is better because of its relationship with other PT partners? ……………………………………………………………………………………………......... 13. What has been challenging about working with other PT partners? ……………………………………………………………………………………………......... 14. How do you think that private sector collaboration and support for MVC activities could be enhanced? ……………………………………………………………………………………………......... 15. How does the (implementing partner) program ensure equal participation and benefit for the following groups: Women and girls (To what extent and how relevant gaps between males and females were closed) ……………………………………………………………………………………………… Youths ……………………………………………………………………………………………… 16. What new opportunities for women and men were created? ………………………………………………………………………………………………… 17. What needs and gender inequalities emerged or remain? ………………………………………………………………………………………………… 18. What are the main challenges that you have encountered implementing the PT program? ………………………………………………………………………………………………… 19. How have you addressed the challenges outlined above and what would help you to address those challenges more effectively? ………………………………………………………………………………………………… 20. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………… 21. What recommendations can you provide for the following? Strengthening PT program approaches ……………………………………………………………………………………………......... Strengthening linkages across stakeholders ……………………………………………………………………………………………......... Fostering sustainability ………………………………………………………………………………………………..... 22. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 101 MID-TERM PERFORMANCE EVALUATION OF THE 4 OVC PROGRAMS IMPLEMENTING PARTNERS’ SUB-GRANTEES PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Institution :……………………………………………………………………………………. Designation/Position:………………………………………………………………………… Date:………………………………………………………………………………………….. Location:…………………………………………………................................................... Gender: Female Male Interviewer:…………………………………………………………………………………… 102 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 1. What do you consider as the key challenges facing MVC and their families in Tanzania? ....................................................................................................................................... 2. How is your organization, through the PT project, addressing these challenges? ....................................................................................................................................... 3. What are the key components on your project, under PT? ....................................................................................................................................... 4. Describe your relationship with (implementing partner). How do you work together to support MVC? ....................................................................................................................................... 5. What do you like about that relationship? What has been challenging about that relationship? How could the relationship be improved? ....................................................................................................................................... 6. What support did you receive from (implementing partner) to strengthen your organization’s capacity to respond to MVC needs? ………………………………………………………………………………………………..... 7. How has your organization changed because of this support? ………………………………………………………………………………………………..... 8. Is there any additional support that you require that you have not yet received? ………………………………………………………………………………………………..... 9. What are the unique features and design strengths of the capacity building approach used by (implementing partner)? ……………………………………………………………………………………................ 10. Is the project on track to achieving results set out at inception? (ask for evidence and what components are on track) ……………………………………………………………………………………………......... 11. What have been the most successful components of your project? What have been the least successful components? ………………………………………………………………………………………………..... 12. Which components do you think will continue beyond the life of the project? ………………………………………………………………………………………………..... 13. Which do you think will not continue and why? ………………………………………………………………………………………………..... 14. To what extent do you collaborate and/or compete with the other (implementing partner) sub-grantees in this region? (ask for how collaboration has enhanced their work and challenges that have arisen) ……………………………………………………………………………………………......... 15. What is the private sector’s involvement and contribution to the program? ……………………………………………………………………………………………......... 16. How does your program ensure equal participation (in planning, implementation, M&E) and benefit for the following groups Women and girls (to what extent and how relevant gaps between males and females were closed) ……………………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 103 Youths ……………………………………………………………………………………………… 17. What new opportunities for women and men were created? ………………………………………………………………………………………………..... 18. What needs and gender inequalities emerged or remain? ………………………………………………………………………………………………..... 19. If your organization were to discover that a child was being abused by a family member, community member or project staff, how would you and you PT partner address this issue? ………………………………………………………………………………………………..... 20. What are the main challenges that you have noted in relation to the PT program? ………………………………………………………………………………………………… 21. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………… 22. What recommendations can you provide for the following? Strengthening approaches used by (implementing partner); ……………………………………………………………………………………………......... Strengthening linkages across stakeholders; ……………………………………………………………………………………………......... Fostering sustainability ………………………………………………………………………………………………..... 23. If you were redesigning the PT follow-on, what components would you include? ………………………………………………………………………………………………..... 24. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? 104 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS IMPLEMENTING PARTNERS’ BENEFICIARIES FGD GUIDE PEPFAR has been funding programs to support most vulnerable children and children affected by HIV and AIDS and under the National Costed Plan of Action for Most Vulnerable Children. The Pamoja Tuwalee project is implemented by several partners, and the purpose is to provide services that will improve the quality of life and well-being of most vulnerable children by empowering households and communities to provide care and support to children in the 21 regions (mainland and island). A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform PEPFAR in designing approaches and focus for follow￾on programs supporting vulnerable children and youth. You have been selected to participate in the evaluation, because you benefitted through support provided by (name of sub-grantee). Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Number of Participants:……………………………………………………………………… Gender: Female Male Interviewer:…………………………………………………………………………………… 1. Do you know how you were selected for this project? If yes, how or why? USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 105 ……………………………………………………………………………………………… 2. What support have you received from the partner (state name of responsible sub￾grantee)? (probe for methods used in delivering support) ……………………………………………………………………………………………… 3. What support has been most helpful out of all the support you mentioned above? ……………………………………………………………………………………………… 4. What support has been least helpful out of all support you mentioned above? ……………………………………………………………………………………………… 5. Have you received any support from private sector institutions? (if yes, ask for type of support) ……………………………………………………………………………………………… 6. What changes have you experienced because of the support received? ……………………………………………………………………………………………… 7. Out of all the changes you outlined above, what changes do you think will continue even without support? (ask why and for evidence) ……………………………………………………………………………………………… 8. Have there been any good or bad changes in your life which you did not expect but happened because of the project? ……………………………………………………………………………………………… 9. How does this project ensure equal participation (planning, implementation and M&E) and benefit for the following groups: Women and girls (to what extent and how relevant gaps between males and females were closed) ……………………………………………………………………………………………… Youths ……………………………………………………………………………………………… 10. What new opportunities for women and men were created? ……………………………………………………………………………………………… 11. What needs and gender inequalities emerged or remain? ……………………………………………………………………………………………… 12. How do you think the project could help you more? ……………………………………………………………………………………………… 13. What support would you like that you have not received? ……………………………………………………………………………………………… 14. Do you receive support from other programs? If yes, what type of support? ……………………………………………………………………………………………… 15. What have you learned since becoming involved in this project? ……………………………………………………………………………………………… 16. What recommendations can you provide for the following? Strengthening ways through which the project supports communities Linking different services for beneficiaries …………………………………………………………………………………………….... 17. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? 106 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation FOCUS GROUP DISCUSSION PARTICIPANTS DEMOGRAPHIC INFORMATION FORM Country:______ Province:___ District:__________ Community:________ Date:_________ Start Time__ Client Code Gender (M/F) Age (years) Living with Disability (Y/N) Level of Education (0-None, 1-Primary, 3- Secondary, 4- College/University and above) Household Head (Y/N) School Child (Y/N) OVC (Y/N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 107 MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS IMPLEMENTING PARTNERS’ GOVERNMENT STAKEHOLDERS PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Institution:…………………………………………………………………………………… Designation/Position:………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Gender: Female Male Interviewer:…………………………………………………………………………………… 108 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 1. What are the greatest challenges that you think MVC, their families and communities face? ………………………………………………………………………………………………..... 2. How do you work together with the Pamoja Tuwalee program to address these challenges? (ask for collaborative approaches) ………………………………………………………………………………………………..... 3. How has the Pamoja Tuwalee program supported MVC? ………………………………………………………………………………………………..... 4. Which types of support provided by the Pamoja Tuwalee has been most/least effective? ………………………………………………………………………………………………… 5. Which of the approaches used by the Pamoja Tuwalee program to support MVC, their families and communities have you liked the most/least? ………………………………………………………………………………………………… 6. What aspects or benefits of the program do you think will most/least likely continue beyond the life of the project ………………………………………………………………………………………………… 7. What do you think that PT could do to ensure that benefits continue beyond the life of the project? ………………………………………………………………………………………………… 8. How does the (implementing partner) program ensure equal participation and benefit for the following groups (participation in the planning of activities, implementation, monitoring and evaluation) Women and girls (to what extent and how relevant gaps between males and females were closed) Youths 9. What new opportunities for women and men were created? ………………………………………………………………………………………………… 10. What needs and gender inequalities emerged or remain? ………………………………………………………………………………………………… 11. If the project found out that a child or adult caregiver was being abused, how would that situation be handled?) ………………………………………………………………………………………………… 12. How would you recommend that a situation like the one outlined above be handled? ………………………………………………………………………………………………… 13. What has been done to promote private sector involvement and contribution toward assisting MVC and their caregivers? (ask for examples) ………………………………………………………………………………………………… 14. What are the main challenges that you have noted in relation to the PT program? ………………………………………………………………………………………………… 15. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………… 16. How is the project strengthening linkages across stakeholders? ………………………………………………………………………………………………… 17. How could the project strengthen linkages more effectively? USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 109 ………………………………………………………………………………………………… 18. Is there any additional information that you might want to share?  With whom else do you suggest we meet?  May we contact you to follow up if we have additional questions? Thank you for your time and ideas. 110 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS BENEFICIARY CASE STUDY DOCUMENTATION GUIDE PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. As a member of this community who benefited from the Pamoja Tuwalee Project, you have been selected to provide your story on experiences with regard to this project. The discussion comprises a series of questions and should take between 15 to 20 minutes to complete. Your participation in this discussion is entirely voluntary. Your responses will be treated absolutely confidential. Information you provide will only be used for purposes of the evaluation as stated. Country/Partner: _____________________________________ District Name: ______________________________________ Gender: ___________________________________________ Age (Optional): ___________________________________________ Intervention: __________________ Date: __________________ USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 111 Start Time: _____________End Time: ____________ Do you the storyteller want to have your name on the story (tick one) Yes [ ] No [ ] Name of storyteller*: * (If they wish to remain anonymous, don’t record their name or contact details.) Are you comfortable in us taking your pictures so that we can use them in the report and any publications for wider stakeholder dissemination (Yes/No), If “Yes” can you sign here as evidence of informed consent: _______________ Title of story_______________________________________________________ Name of person recording story: ________________________________________ THE COMMUNITY WILL ASSIST IN IDENTIFYING PROJECT CASE STUDIES DURING FGDs 1. Tell me how you (the storyteller) first became involved with Pamoja Tuwalee project, and what your current involvement is: ______________________________________________________________________ ______________________________________________________________________ 2. Describe your case story, who was involved, what happened, where and when? ______________________________________________________________________ ______________________________________________________________________ (a) What difference has it made/will it make? (Probe for changes related to the support that the beneficiary received from the project) ________________________________________________________________ ____________________________________________________________ 3. What did not work well during your participation in this project? ___________________________________________________________________ _____________________________________________________________ 4. Have you participated in other similar projects in the past, and how has this one differed from those? (Read the story to the storyteller to seek authenticity) ______________________________________________________________________ ______________________________________________________________________ 5. Any recommendations for the future? ______________________________________________________________________ ______________________________________________________________________ ______ 6. Do you have any other comments? ______________________________________________________________________ ______________________________________________________________________ (Read the story to the storyteller to seek authenticity) 112 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS CAREGIVERS FGD GUIDE PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Number of Participants:……………………………………………………………………… Gender: Female Male Interviewer:…………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 113 1. What support have you received from the program partner? (probe for methods used in strengthening Caregiver capacities) ……………………………………………………………………………………………… 2. What changes have you experienced as a result of the support received? (probe for capacities to support MVC) ……………………………………………………………………………………………… 3. Do you think the benefits from changes outlined above will continue even without PT program support? (ask for reasons for their responses) ……………………………………………………………………………………………… 4. What have been the positive or negative unintended consequences emerging from the program thus far? ……………………………………………………………………………………………… 5. How do you as caregivers ensure equal participation and benefit for the following groups (To what extent and how relevant gaps between boys and girls were closed): Boys ……………………………………………………………………………………………… Girls ……………………………………………………………………………………………… 6. What new opportunities for women and men were created? ……………………………………………………………………………………………… 7. What needs and gender inequalities emerged or remain? ……………………………………………………………………………………………… 8. What are the main challenges that you have faced in relation to support received from the program thus far? ……………………………………………………………………………………………… 9. What have been the key lessons learned in implementation of the program thus far? ……………………………………………………………………………………………… 10. What recommendations can you provide for the following? ……………………………………………………………………………………………… Strengthening the approaches used in supporting caregivers …………………………………………………………………………………………….... Strengthening linkages between different services …………………………………………………………………………………………….... Fostering sustainability of impacts ……………………………………………………………………………………………… 114 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation FOCUS GROUP DISCUSSION PARTICIPANTS DEMOGRAPHIC INFORMATION FORM Country:______ Province:___ District:__________ Community:________ Date:_________ Start Time__ Client Code Gender (M/F) Age (years) Living with Disability (Y/N) Level of Education (0-None, 1-Primary, 3- Secondary, 4- College/University and above) Household Head (Y/N) School Child (Y/N) OVC (Y/N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 115 MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS AFRICARE/FHI360 SILC BENEFICIARIES FGD GUIDE PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Number of Participants:……………………………………………………………………… Gender: Female Male Interviewer:…………………………………………………………………………………… 116 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 1. What support has your Savings and Internal Lending Community (SILC) received from Africare/FHI360? (probe for methods used in delivering support) ………………………………………………………………………………………………….. 2. What changes have you experienced as a result of the economic strengthening support you received? ………………………………………………………………………………………………….. 3. How much money did the SILC group have when it was formed? ………………………………………………………………………………………………….. 4. How much money does the SILC group have now? ………………………………………………………………………………………………….. 5. Do you think this SILC group will continue even without support from Africare/FHI360? (ask for reasons for their responses) ………………………………………………………………………………………………….. 6. Have you been supported with other services besides SILC? ………………………………………………………………………………………………….. 7. Did the project provide any support to address gender-based violence? (If yes, ask what form of support and how effective was it) ………………………………………………………………………………………………….. 8. Have you received any support from private sector institutions, e.g., banks? (If yes, ask for type of support) ………………………………………………………………………………………………….. 9. How does this group ensure equal participation and benefit for the following groups: Women and girls ……………………………………………………………………………………………… Youths ……………………………………………………………………………………………… 10. What have been the positive or negative unintended consequences emerging from the project thus far? ………………………………………………………………………………………………… 11. What are the main challenges that your SILC group has faced thus far? ………………………………………………………………………………………………… 12. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………… 13.What recommendations can you provide for the following? ………………………………………………………………………………………………… Strengthening the approaches used in supporting SILC groups ……………………………………………………………………………………………......... Strengthening linkages between SILC groups and other services ……………………………………………………………………………………………......... Fostering sustainability of SILC groups ………………………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 117 FOCUS GROUP DISCUSSION PARTICIPANTS DEMOGRAPHIC INFORMATION FORM Country:_____ _ Province:__ _ District:________ __ Community:________ Date:_________ Start Time__ Clien t Code Gender (M/F) Age (years) Living with Disability (Y/N) Level of Education (0-None, 1-Primary, 3- Secondary, 4- College/University and above) Househol d Head (Y/N) School Child (Y/N) OVC (Y/N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 118 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF THE 4 OVC PROGRAMS PACT WORTH BENEFICIARIES FGD GUIDE PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Number of Participants:……………………………………………………………………… Gender: Female Male Interviewer:…………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 119 1. How did you find out about the WORTH group? ………………………………………………………………………………………………….. 2. How did you join? ………………………………………………………………………………………………….. 3. Do you know anyone in your community that wanted to join a WORTH and was unable to join? Do you know why they were unable to join? ………………………………………………………………………………………………….. 4. What support has your WORTH group received from Pact? (probe for methods used in delivering support) ………………………………………………………………………………………………….. 5. How has your life changed for the better or worse since joining your WORTH group? How do you think that the WORTH has contributed to these changes? ………………………………………………………………………………………………….. 6. How much money did the WORTH group have when it was formed? ………………………………………………………………………………………………….. 7. How much money does the WORTH group have now? ………………………………………………………………………………………………….. 8. Has the WORTH group used any money to support MVC? (probe on how many children they have supported and on what services) ………………………………………………………………………………………………….. 9. How is the WORTH group managed? ………………………………………………………………………………………………….. 10. What do you like about how the WORTH is managed? What do you not like about how it is managed? What ideas do you have for managing it better? ………………………………………………………………………………………………….. 11. Do you think this WORTH group will continue even without support from Pact? (ask for reasons for their responses) ………………………………………………………………………………………………….. 12. Have you been supported with other services besides WORTH? If yes, what type of services or support and who has provided these? ………………………………………………………………………………………………….. 13. Did your WORTH group engage in any discussions about specific issues (e.g. health and HIV issues, parenting, child protection or gender-based violence issues, other issues?) ………………………………………………………………………………………………….. 14. If yes to question 13 above, what did you find most/least helpful about these discussions? ………………………………………………………………………………………………….. 15. Did the project provide any support to address gender-based violence? (If yes, ask what form of support and how effective it was). If you or someone you knew learned that someone in the WORTH were abusing their child or being abused by their partner, what would you do? ………………………………………………………………………………………………….. 16. Have you received any support from private sector institutions, e.g., banks? (If yes, ask for type of support) …………………………………………………………………………………………………. 120 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 17. How does this group ensure equal participation and benefit for the following groups: Women and girls (To what extent and how relevant gaps between males and females were closed) ……………………………………………………………………………………………… Youths ……………………………………………………………………………………………… 18. What new opportunities for women and men were created? ………………………………………………………………………………………………….. 19. What needs and gender inequalities emerged or remain? ………………………………………………………………………………………………….. 20. What are the main challenges that your WORTH group has faced thus far? ………………………………………………………………………………………………….. 21. What ideas do you have for addressing challenges outlined above? ………………………………………………………………………………………………….. 22. What have been the key lessons learned in implementation of the PT project thus far? ………………………………………………………………………………………………….. 23. What recommendations can you provide for the following? ………………………………………………………………………………………………….. Strengthening the approaches used in supporting WORTH groups ……………………………………………………………………………………………......... Strengthening linkages between WORTH groups and other services ……………………………………………………………………………………………......... Fostering sustainability of WORTH groups ………………………………………………………………………………………………….. USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 121 FOCUS GROUP DISCUSSION PARTICIPANTS DEMOGRAPHIC INFORMATION FORM Country:_____ _ Region:____ _ District:________ __ Community:________ Date:_________ Start Time__ Clien t Code Gender (M/F) Age (years) Living with Disability (Y/N) Level of Education (0-None, 1-Primary, 3- Secondary, 4- College/University and above) Househol d Head (Y/N) School Child (Y/N) OVC (Y/N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 122 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS PACT YOUTH FGD GUIDE PEPFAR has been funding OVC programs in support of implementation of the MVC National Costed Plan of Action I (NCPA I) and the new MVC NCPA II. The Pamoja Tuwalee project is implemented through four cooperative agreements with four organizations: Africare, Pact Inc., World Education Inc. (WEI) and Family Health International (FHI360).These are five-year awards (June 1, 2010 – May 31, 2015) with an estimated initial total budget of $59 million. The purpose of the Pamoja Tuwalee (PT) program is to provide services that will improve the quality of life and well-being of OVC by empowering households and communities to provide care and support in the 21 regions (mainland and island). This will be achieved through employment of several cost-efficient, tightly integrated, sustainable and interrelated strategies. A mid-term performance evaluation has been commissioned to determine which of the approaches being used by the PT partners are most effective in achieving linkages across services and fostering sustainability. The evaluation will further document lessons learned that will inform the mission in designing approaches and focus for the follow-on program supporting vulnerable children and youth. You have been selected to participate in the evaluation due to your involvement with the PT program. Your participation in the evaluation is voluntary. Information you provide will only be used for purposes of the evaluation as stated. Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:…………………………………………………………………………………………. Location:…………………………………………………................................................... Number of Participants:……………………………………………………………………… Gender: Female Male Interviewer:…………………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 123 What support have you received from the program partner? (probe for youth friendliness of methods used in delivering support) ……………………………………………………………………………………………… 1. What changes have you experienced as a result of the support received? ……………………………………………………………………………………………… 2. Do you think the benefits from changes outlined above will continue even without program support? (ask for reasons for their responses) ……………………………………………………………………………………………… 3. Have you received any support from private sector institutions? (If yes, ask for type of support) ……………………………………………………………………………………………… 4. What positive or negative unintended consequences have emerged from the support you received? ……………………………………………………………………………………………… 5. How did the program ensure equal participation and benefit for the following groups: i. Female youths (To what extent and how relevant gaps between male and female youths were closed) ……………………………………………………………………………………………… ii. Male youths ……………………………………………………………………………………………… 6. What new opportunities for young women and young men were created? ……………………………………………………………………………………………… 7. What needs and gender inequalities emerged or remain? ……………………………………………………………………………………………… 8. What are the main challenges that you have faced in relation to support received from the program thus far? ……………………………………………………………………………………………… 9. What have been the key lessons learned in implementation of the program thus far? ……………………………………………………………………………………………… 10. What recommendations can you provide for the following? ……………………………………………………………………………………………… Strengthening the approaches used in supporting youths …………………………………………………………………………………………….... Strengthening linkages between different services for youths …………………………………………………………………………………………….... Fostering sustainability of impacts for programs supporting youths ……………………………………………………………………………………………… 124 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation FOCUS GROUP DISCUSSION PARTICIPANTS DEMOGRAPHIC INFORMATION FORM Country:_____ _ Province:__ _ District:________ __ Community:________ Date:_________ Start Time__ Clien t Code Gender (M/F) Age (years) Living with Disability (Y/N) Level of Education (0-None, 1-Primary, 3- Secondary, 4- College/University and above) Househol d Head (Y/N) School Child (Y/N) OVC (Y/N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 125 MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS PACT WORTH GROUP OBSERVATION GUIDE Region:…...…………………………………………………………………………………… District:………………………………………………………………………………………… Date:………………………………………………………………………………………….... Time:.............................................................................................................................. Location:………………………………………………….................................................... Observer:..…………………………………………………………………………………….. 126 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Guide for Direct Observation: Savings and Internal Lending Groups Description of setting (indoor, outdoor, urban, rural, private home, office, school, etc.) ………………………………................... …………………………………………… …………………………………………… …………………………………………… Numbers of members present: Male__________ Female________ Leadership/speakers/officers: (male/female, age, etc.) …………………………………………… …………………………………………… …………………………………………… …………………………………………… Number/gender of speakers: please note number of vocal participants ……………………………………………… ……………………………………………… ……………………………………………… ……………………………………………… Describe process (How they go about their activities) …………………………………………… …………………………………………… …………………………………………… …………………………………………… Observe and document if there are any issues, altercations or debate ……………………………………………… ……………………………………………… ……………………………………………… ……………………………………………… Check their records and document funds dedicated to MVC. How much and what percentage is it of their total budget? …………………………………………… …………………………………………… …………………………………………… …………………………………………… Check for the number of loan applications submitted? ………………………………………........... ……………………………………………... ……………………………………………… ……………………………………………… Document any other observations: ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… …………………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 127 MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS PACT KIDS’ CLUB OBSERVATION GUIDE Region:.……………………………………………………………………………………. District:..……………………………………………………………………………………. Date:……………………………………………………………………………………...... Time:........................................................................................................................ Location:……………………………………………...................................................... Observer:…………………………………………………………………………………... Implementing Partner:.…………………………………………………………………... Sub grantee:……………………………………………………………………………… Name of Club:…………………………………………………………………………….. 128 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Guide for Observing Kids’ Clubs Setting: outdoors, at school, home, office ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………… Number and gender of leaders: (teachers, volunteers, para social workers, mentors?) Number____ Gender_____Female Male Number of children Boys_______ Girls________ Check if a register, attendance list is available and document information recorder? MVCs only? Yes/No……………… Percentage of MVC? Yes/No……….. Open to all? Yes/No………………….. Check and document the curriculum/topics/themes? ………………………………… ………………………………… ………………………………… Check and document if teaching, play materials are present? (Are they adequate? What is the quality?) …………………………………………. ………………………………………….. ………………………………………….. Observe and document if the leadership style and approach are fit for the purpose? ………………………………… ………………………………… ………………………………… Assess and document if there is evidence of engagement of children: Mastery of topics:……………………… …………………………………………… …………………………………………… Other comments: ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………… USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 129 MID-TERM PERFORMANCE EVALUATION OF THE FOUR OVC PROGRAMS INDICATOR DATA COLLECTION TEMPLATE 130 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Africare Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F P12.1.D: Number of people reached by an individual, small group or community-level intervention or service that explicitly addresses gender-based violence and coercion (GBV) P12.2.D: Number of GBV service encounters at a health facility P12.3.D: Percentage of health facilities with GBV services available (Mwanza Only) C5.1.D: Number of eligible clients who received food and/or nutrition services TZ OVC: Number of eligible OVC provided with a minimum of one core care service  Educational support  Economic strengthening support  Psychological, social & spiritual care support  Nutrition and food security support USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 131 Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F TZ ED: Number of vulnerable households with at least one OVC/MVC or PLHIV or caregiver provided with a minimum of one economic opportunity / strengthening support  OVC Households  HBC Households  OVC Caregiver Households  HBC Caregiver Households H2.2.D: Number of community health and para-social workers who successfully completed a pre-service training program H2.3.D: Number of health care workers and community and health and para￾social workers who successfully completed an in-service training program DIRECT SERVICES  Number of adults and children who received food and/or nutrition services during the reporting period.  Number of vulnerable households with at least one OVC/MVC or PLHIV provided with a minimum of one economic support 132 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F  Number of eligible OVC provided with a minimum of one core care service TRAININGS PROVIDED  Number of community health and para-social workers who successfully completed a pre￾service training program  Number of health care workers who successfully completed an in￾service training program Notes i. Provide data for indicators that are applicable to your program USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 133 FHI360 Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F P12.1.D: Number of people reached by an individual, small group, or community-level intervention or service that explicitly addresses gender-based violence and coercion (GBV) P12.2.D: Number of GBV service encounters at a health facility P12.3.D: Percentage of health facilities with GBV services available (Mwanza Only) C5.1.D: Number of eligible clients who received food and/or nutrition services TZ OVC: Number of eligible OVC provided with a minimum of one core care service  Educational support  Economic strengthening support  Psychological, social & spiritual care support  Nutrition and food security support 134 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F TZ ED: Number of vulnerable households with at least one OVC/MVC or PLHIV or caregiver provided with a minimum of one economic opportunity / strengthening support  OVC Households  HBC Households  OVC Caregiver Households  HBC Caregiver Households H2.2.D: Number of community health and para-social workers who successfully completed a pre-service training program H2.3.D: Number of health care workers and community and health and para￾social workers who successfully completed an in-service training program DIRECT SERVICES  Number of adults and children who received food and/or nutrition services during the reporting period.  Number of vulnerable households with at least one OVC/MVC or PLHIV provided with a minimum of one economic support USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 135 Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F  Number of eligible OVC provided with a minimum of one core care service TRAININGS PROVIDED  Number of community health and para-social workers who successfully completed a pre￾service training program  Number of health care workers who successfully completed an in￾service training program Notes ii. Provide data for indicators that are applicable to your program 136 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation PACT Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F P12.1.D: Number of people reached by an individual, small group, or community-level intervention or service that explicitly addresses gender-based violence and coercion (GBV) P12.2.D: Number of GBV service encounters at a health facility P12.3.D: Percentage of health facilities with GBV services available (Mwanza Only) C5.1.D: Number of eligible clients who received food and/or nutrition services TZ OVC: Number of eligible OVC provided with a minimum of one core care service  Educational support  Economic strengthening support  Psychological, social and spiritual care support  Nutrition and food security support USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 137 Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F TZ ED: Number of vulnerable households with at least one OVC/MVC or PLHIV or caregiver provided with a minimum of one economic opportunity / strengthening support  OVC Households  HBC Households  OVC Caregiver Households  HBC Caregiver Households H2.2.D: Number of community health and para-social workers who successfully completed a pre-service training program H2.3.D: Number of health care workers and community and health and para￾social workers who successfully completed an in-service training program DIRECT SERVICES  Number of adults and children who received food and/or nutrition services during the reporting period.  Number of vulnerable households with at least one OVC/MVC or PLHIV provided with a minimum of one economic support 138 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F  Number of eligible OVC provided with a minimum of one core care service. TRAININGS PROVIDED  Number of community health and para-social workers who successfully completed a pre￾service training program  Number of health care workers who successfully completed an in￾service training program Notes iii. Provide data for indicators that are applicable to your program USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 139 WEI Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F P12.1.D: Number of people reached by an individual, small group, or community-level intervention or service that explicitly addresses gender-based violence and coercion (GBV) P12.2.D: Number of GBV service encounters at a health facility P12.3.D: Percentage of health facilities with GBV services available (Mwanza Only) C5.1.D: Number of eligible clients who received food and/or nutrition services TZ OVC: Number of eligible OVC provided with a minimum of one core care service  Educational support  Economic strengthening support  Psychological, social & spiritual care support  Nutrition and food security support 140 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F TZ ED: Number of vulnerable households with at least one OVC/MVC or PLHIV or caregiver provided with a minimum of one economic opportunity / strengthening support  OVC Households  HBC Households  OVC Caregiver Households  HBC Caregiver Households H2.2.D: Number of community health and para-social workers who successfully completed a pre-service training program H2.3.D: Number of health care workers and community and health and para￾social workers who successfully completed an in-service training program DIRECT SERVICES  Number of adults and children who received food and/or nutrition services during the reporting period  Number of vulnerable households with at least one OVC/MVC or PLHIV provided with a minimum of one economic support USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 141 Indicator Year Overall Target Overall Achieved YR 1 YR 2 YR 3 M F M F Target Achieved Target Achieved Target Achieved M F M F M F M F M F M F  Number of eligible OVC provided with a minimum of one core care service. TRAININGS PROVIDED  Number of community health and para-social workers who successfully completed a pre￾service training program  Number of health care workers who successfully completed an in￾service training program Notes iv. Provide data for indicators that are applicable to your program 142 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation ANNEX VI. MAP OF TANZANIA AND PROGRAM AREAS USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 143 ANNEX VII. DOCUMENTS REVIEWED USAID and Other Global Documents PEPFAR Orphans and Other Vulnerable Children Programming Guidance. 2012. UNDP. Human Development Report 2012. UNAIDS. Global Report. 2010. UNAIDS. Global Report. 2013. UNICEF. Taking Evidence to Impact. 2011. UNICEF. 5 th Stocktaking Report. 2010. USAID. Evaluation Policy. January 2011. USAID. Evaluation Report Template. USAID. How To Note: Preparing Evaluation Reports. USAID/Tanzania. Final scope of work for the Global Health Technical Assistance Bridge IV Project: Midterm Performance Evaluation of the Four OVC Programs (Pamoja Tuwalee). October 18, 2013. USAID/Tanzania. Request for Applications (RFA) Number USAID-TANZANIA-10-004-RFA, Coordinated Orphans and Vulnerable Children Care Program. Initial Partner Assessment Report on the State of Economic Strengthening of PEPFAR Community Care Partners. 2011. PowerPoint Presentation. Orphans and Vulnerable Children (OVC) Programming In Global Fund HIV/AIDS Grants In Kenya. 2010. Country-specific Documents Children in Detention Assessment 2011 Final Multi Sector Task Force TNZ March 31 Final One Stop Centre Report Final Violence Against Children National Plan TSH version Global Fund Support To Tanzania: Achievements and Challenges. 2010. IMARISHA. PowerPoint Presentation on IMARISHA's work. September 2013. IMARISHA Partner Assessment Report, revised final MEASURE Evaluation. Evaluation of Four OVC Projects in Kenya and Tanzania. 2007. MEASURE Evaluation and Futures Group. MVC Zanzibar M&E Systems Strengthening Plan. Multi-sector National Plan of Action to Prevent and Respond to Violence against Children 2013- 2016. United Republic of Tanzania. April 2013. National Bureau of Statistics, Dar es Salaam, Tanzania and ICF Macro. Tanzania Demographic and Health Survey 2010. Calverton, MD, USA, April 2011. National Guidelines for Improving Quality of Care, Support, and Protection for Most Vulnerable Children in Tanzania National Plan of Action for the Prevention and Response to Violence against Children (2011- 2015) National Strategy for Growth and Reduction of Poverty II (NSGRP II or MKUKUTA II) National Survey on Violence Against Children, 2011 Nyangara F, Matiko C, Kalungwa Z. SAWAKA Jali Watoto Program Supporting Most￾Vulnerable Children, Tanzania: A Case Study. 2009. PEPFAR Country Operational Plan (COP) Guidance 2012 PEPFAR Tanzania Operational Plan Report 2012 Program Area Funding 05-29-2013 144 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Save The Children. Establishment of a One Stop Center at MnaziMmoja Hospital for Children, Women and Men survivors of Sexual, Physical and Emotional Violence. One Stop Center at MnaziMmoja Hospital, Stonetown, Zanzibar. Situational Analysis on Child Labour in Tanzania Mainland and Zanzibar, 2011 Summary of the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) HIV/AIDS Partnership Framework with the Government of the United Republic of Tanzania. March 2010. Tanzania Development Vision 2025 Tanzania Global Health Initiative Strategy 2010-2015. September 2011. Tanzania HIV and Malaria Indicator Survey 2011-2012 (THMIS III) Tanzania MDG Report 2010 Tanzania Social Action Fund (TASAF III) Terms of Reference for a Savings Study – final draft The Law of the Child Act, 2009 The Multi-Sector Task Force: A National Response to Violence Against Children, Tanzania Ministry of Community Development, Gender and Children The U.S. President‘s Emergency Plan for AIDS Relief; Guidance For Orphans And Vulnerable Children Programming. July 2012. The United Republic of Tanzania, Ministry of Health and Social Welfare, Department of Social Welfare. MVC IPG Monthly Update. August, 2013. The United Republic of Tanzania, Ministry of Health and Social Welfare, Department of Social Welfare. MVC IPG Monthly Update. November 2012. The United Republic of Tanzania, Ministry of Health and Social Welfare; Department of Social Welfare. MVC IPG Monthly Update. October, 2013. The United Republic of Tanzania, Ministry of Health and Social Welfare, Department of Social Welfare. MVC IPG Monthly Update. September, 2013. The United Republic of Tanzania, National Bureau of Statistics, Ministry of Finance (Dar es Salaam) and Office of Chief Government Statistician President’s Office, Finance, Economy and Development Planning (Zanzibar). 2012 Population and Housing Census: Population Distribution by Administrative Areas. March 2013. The United Republic of Tanzania Bill Supplement No. 0, 18th June, 2010: The Public Private Partnership Act, 2010 The United Republic of Tanzania, Prime Minister’s Office. National Public Private Partnership (PPP) Policy. November 2009. (‘‘Restricted Circulation’’) The United Republic of Tanzania, Ministry of Health and Social Welfare. National Costed Plan of Action for Most Vulnerable Children 2013-2017. The United Republic of Tanzania, National Bureau of Statistics, Ministry of Finance (Dar es Salaam) and Office of Chief Government Statistician President’s Office, Finance, Economy and Development Planning (Zanzibar). Population Distribution: Age and Sex. September 2013. The United Republic of Tanzania, Ministry of Finance, National Bureau of Statistics. Tanzania in Figures 2012. June 2013. The United Republic of Tanzania. Tanzania Statistical Master Plan 2009/10 to 2013/14 The United Republic of Tanzania, Ministry of Health. The Food and Nutrition Policy of Tanzania. 1992. The United Republic of Tanzania President’s Office, Planning Commission. National Post￾MDGs Development Agenda Consultations: The National Synthesis Report. May 2013. UNAIDS. Tanzania Case Study. 2012. UNDP. Human Development Report 2013: The Rise of the South: Human Progress in a Diverse World. Explanatory note on 2013 HDR composite indices, Tanzania (United Republic of) HDI values and rank changes in the 2013 Human Development Report. UNICEF Annual Report for Tanzania, 2010 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 145 UNICEF. Children and Women in Tanzania. 2010. UNICEF Cities and Children: The Challenge of Urbanization in Tanzania, 2012 UNICEF, CDC and Muhimbili University of Health and Allied Sciences. Violence Against Children in Tanzania: Findings from a National Survey, 2009. United Republic of Tanzania, 2011. World Food Programme in collaboration with the World Bank. Comprehensive Food Security and Vulnerability Analysis (CFSVA). 2012. Zanzibar Children’s Act. Bill Supplement to the Zanzibar Government Gazette, Vol. No. CXX No. 6405. March 2, 2011. AFRICARE ADP Mbozi Mid-term Organizational Capacity Assessment Report AFRICARE Final Household Economic Assessment Report AFRICARE Mid-term Performance Evaluation Indicator Data: PEPFAR AFRICARE Pamoja Tuwalee Program Mid-term Organization Capacity Assessment Analysis Tool. September 2013. AFRICARE Pamoja Tuwalee Project Quarterly Report (October 1, 2011 to December 31, 2011), Cooperative Agreement #621-A-00-10-00025-00 AFRICARE Pamoja Tuwalee Project, Cooperative Agreement #621-A-00-10-00025-00, Quarterly Performance Narrative Report, October-December 2012 AFRICARE Pamoja Tuwalee Project, Cooperative Agreement #621-A-00-10-00025-00, Quarterly Performance Narrative Report, January-March 2013 AFRICARE Pamoja Tuwalee Project, Cooperative Agreement #621-A-00-10-00025-00 Quarterly Performance Narrative Report, April-June 2012 AFRICARE Pamoja Tuwalee Project, Cooperative Agreement #621-A-00-10-00025-00 Quarterly Performance Narrative Report, July-September 2012 AFRICARE Pamoja Tuwalee Project, Quarterly report (January, 01 2012–March, 31, 2012), Cooperative Agreement #621-A-00-10-00025-00 AFRICARE Pamoja Tuwalee Savings Groups Contact Information AFRICARE PT FY2013: Final Year 3 Work Plan AFRICARE Tanzania Pamoja Tuwalee (UDUGU) OVC Project, Central Zone (Dodoma, Iringa and Singida regions) Cooperative Agreement # 621-A-00-10-00025-00. Quarterly Report: July– September 2010 AFRICARE Tanzania Pamoja Tuwalee MVC Program, Central Zone (Dodoma, Iringa and Singida regions), Cooperative Agreement # 621-A-00-10-00025-00. FY 2011 Quarterly 1 Report: October– December 2010 AFRICARE Tanzania Pamoja Tuwalee OVC Project Central Zone (Dodoma, Iringa and Singida regions), Cooperative Agreement # 621-A-00-10-00025-00. Quarterly Report: January 2011– March 2011 AFRICARE Tanzania Pamoja Tuwalee OVC Project Central Zone (Dodoma, Iringa and Singida regions), Cooperative Agreement # 621-A-00-10-00025-00. Quarterly Report: July 2011– September 2011 AFRICARE Tanzania Pamoja Tuwalee OVC Project, Central Zone (Dodoma, Iringa and Singida regions), Cooperative Agreement # 621-A-00-10-00025-00. Quarterly Report: April 2011–June 2011 AFRICARE RFA-10-004 Technical Proposal Afya Women’s Group Mid-term Organizational Capacity Assessment Report Allamano Centre Mid-term Organizational Capacity Assessment Report COCODA Mid-term Organizational Capacity Assessment Report Cooperative Agreement #621-A-00-10-00025-00: "THREE IN ONE," A Baseline Assessment of Pamoja Tuwalee Project In Dodoma, Iringa and Singida Region. December 2011. 146 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Cooperative Agreement #621-A-00-10-00025-00: A Baseline Assessment of the Impact of Commercialized Farming on Households Participating in the AFRICARE Pamoja Tuwalee MVC Project. Final draft. August 2013. Evangelical Lutheran Church in Tanzania Makete Mid-term Organizational Capacity Assessment Report FY 2013 AFRICARE Pamoja Tuwalee Project Year 3 Annual Plan (Narrative) Healthcare Providers Directory Household Economic Assessment (HEA): Key Findings For AFRICARE Household Income and Savings Associations (HISA) Facilitators’ Training Manual, 2nd Edition. April 2013. Iringa Mercy Organization Mid-term Organizational Capacity Assessment Report Ilula Orphan Program Mid-term Organizational Capacity Assessment Report Mapping of MVC/OVC services Providers in the Pamoja Tuwalee Program in Dodoma, Iringa, Njombe and Singida Regions. Volume 1: Introduction, Background and Report of Activities. MEASURE Evaluation, JL Consultancy, Ltd. AFRICARE Tanzania MVC Data Quality Assessment Report. MISO Mid-term Organizational Capacity Assessment Report Monitoring and Evaluation Handbook for AFRICARE Pamoja Tuwalee Project, Version 2.0. August 2013. Pamoja Tuwalee Branding Strategy and Marking Plan: Supporting the Government’s National Costed Plan of Action for Most Vulnerable Children Pamoja Tuwalee Cooperative Agreement Number: 621-A-00-10-00025-00, Quarterly Report, July–September 2013 Pamoja Tuwalee MVC Support Program, Cooperative Agreement Number: 621-A-00-10-00025- 00, Annual Report FY 2013 Pamoja Tuwalee MVC Support Program Quarterly Performance Narrative Report, April–June 2013 Pamoja Tuwalee Project: Cooperative Agreement # 621-A-00-10-00025-00, Annual Report FY 2012 TAHEA Mid-term Organizational Capacity Assessment Report Tanzania Food and Nutrition Centre. Household Food and Nutrition Security Baseline Survey for Dodoma, Iringa, Njombe and Singida. TAHEA Consultancy Report. July 2013. UDUGU Partnership For Children (TushikamaneKwaWatoto) in response to USAID-Tanzania￾10-004-RFA (Central Zone): June 2010–June 2015 FHI360 Appendix I: MVC Served with a Minimum of One Core Service Per Ward/Shehia during October to December 2012 Deloitte, Pamoja Tuwalee/FHI360 Sub-grantees Capacity Assessment Report; An Assessment to Determine Organizational, Institutional and Technical Capacity Strengths and Gaps. October 2012-February 2013. Family Health International Pamoja Tuwalee Program–Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, January to March 2013 Family Health International Pamoja Tuwalee Program–Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, July to September 2013 Family Health International Pamoja Tuwalee Program–Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, January To March 2013 Family Health International Pamoja Tuwalee Program–Coast Zone Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, April to June 2012 Family Health International Pamoja Tuwalee Program, Cooperative Agreement No. 621-A-00- 10-00027-00, Quarterly Performance Narrative Report, January to March 2012 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 147 Family Health International Pamoja Tuwalee Program Cooperative Agreement No. 621-A-00- 10-00027-00, Quarterly Performance Narrative Report, October to December 2011 Family Health International Pamoja Tuwalee Program Cooperative Agreement No. 621-A-00- 10-00027-00, Quarterly Performance Narrative Report, July to September 2011 Family Health International Pamoja Tuwalee Program Cooperative Agreement No. 621-A-00- 10-00027-00, Quarterly Performance Narrative Report, January To March 2011 Family Health International Pamoja Tuwalee Program Cooperative Agreement No. 621-A-00- 10-00027-00, Quarterly Performance Narrative Report, April to June 2011 FHI360 Revised FY 2012-2013 Work Plan–Final FHI360 Technical and Cost Proposal and Annexes Final Household Economic Assessment Report for FHI360 Pamoja Tuwalee FY13 Q4: July-September 2013, Quarterly Report, List of Appendixes Household Economic Assessment: Key Findings For FHI360 Pamoja Tuwalee Pamoja Tuwalee Program–Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, October-December 2012 Pamoja Tuwalee Program–Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00, Quarterly Performance Narrative Report, July-September 2012 Report on the Identification of Most Vulnerable Children Conducted in Ilala and Kinondoni Municipal Councils, Dar es Salaam Region. April 2011. Year Three Annual Work Plan: October 2012 to September 2013 Pamoja Tuwalee Program Coast Zone, Cooperative Agreement No. 621-A-00-10-00027-00 PACT Addendum, PACT Data Quality Assessment Final Report. March 2013. Advocacy and Lobbying Orientation for MVCC Building Monitoring, Evaluation and Reporting Systems for HIV/AIDS Programs. Module 1 of the Monitoring, Evaluation and Learning Series Publications. Second Edition. March 2009. Coordinated Orphans and Vulnerable Children Care Program: JaliWatoto Phase II for Lake Zone & Southern Zone. Technical Proposal Submission. CP Data Collation Sheet (October 2012-September 2013) Developing Advocacy & Lobbying Strategy for MVC Action Plan Inclusion in Village Plans Financial Management Training Materials Health Development Trust: The Experience in Capacity Building; Capacity Building in Tanzania: Myth or Reality Health Promotion Tanzania, Project Brief Household Economic Assessment (HEA): Key Findings for PACT Tanzania JaliWatoto MVC End of Program Evaluation, the Process Report, Final Report. February 2012. Kikoyo, L., Madeleka, L. and Matiko, C. Revised National Identification Process Steps, incorporating PT revision. February 2011. MEASURE Evaluation, JL Consultancy, Ltd. Pact-Tanzania MVC Data Quality Assessment Report. February 2012. MEASURE Evaluation, JL Consultancy, Ltd. Pact-Tanzania Pamoja Tuwalee Program Mini Data Quality Assessment Report. February 2013. Minutes of the meeting held at FHI360 (PamojaTuwalee Project) offices on 11 February 2011 NEW-NGONET. Brief Report on the Organization and Implementation of Pamoja Tuwalee Project. December 2013. Organizational Development Roadmap List and CISPs FY13 Pact Final Household Economic Assessment Report Pact JaliWatoto II, Technical Application Pact JaliWatoto & Anti-Stigma Initiative Final Evaluation Report, Award 621-A-00-06-00001-0 148 USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation Pact Mtwara Pamoja Tuwalee Program Overview Pact Pamoja Tuwalee Final Revised Annual Workplan FY13 Pact Field Guide for Data Quality Management. Module 2 of the Monitoring, Evaluation and Learning Series. Pact Standards for Programs Serving Vulnerable Children, A Practical Guide. Operationalizing Our Principles and Standards. September 2013. Pact WORTH Handbook, A Practical Guide to the PACT WORTH Program for Practitioners and Development Professionals. September 2013. Pamoja Tuwalee Annual Work Plan, October 2011–September 2012, Cooperative Agreement No. 621-A-00-10-00023-00 Pamoja Tuwalee Annual Performance Report, October 2010–September 2011, Lake and Southern Zones, Associate Award Number 621-A-00-10-00023-00 Pamoja Tuwalee Annual Workplan, FY13 October 2012–September 2013, Coordinated OVC Care Program, Cooperative Agreement 621-A-00-10-000-23-00 Pamoja Tuwalee Annual Workplan FY13, October 2012–September 2013, Coordinated OVC Care Program, Cooperative Agreement 621-A-00-10-000-23-00 Pamoja Tuwalee Baseline Evaluation Report, Award 621-A-00-10-00023-00 Pamoja Tuwalee Coordinated OVC Care Program FY13, Q2 Progress Report, January 2013 through March 2013, Cooperative Agreement 621 A-00-10-000-23-00 Pamoja Tuwalee Coordinated OVC Care Program Quarterly Progress Report, Q1 FY13 October 2012 through December 2012, Cooperative Agreement 621-A-00-10-000-23-00 Pamoja Tuwalee Districts and Number of Trainees to Participate in the ID Facilitators Training Project Performance Monitoring Plan (PMP) 1st October 2011 to 30th September 2012 Project Performance Monitoring Plan (PMP) 1st October 2012 to 30th September 2013 Project Performance Monitoring Plan (PMP) 1st October 2013 to 30th September 2014 Sub-grantee Liquidation Guide Women Ending Poverty, The WORTH Program in Nepal: Empowerment Through Literacy, Banking and Business 1999-2007 Village MVC Service Map Data Verification Process-KiSwahili WEI Annex 2 Roles and Responsibilities of District Child Advocacy Team Annex 4 Data Verifications Summary by District/Implementing Partner Annex 5 Summary of Quarterly Partners Technical Meetings FY12 Annex 6 Summary of Joint Institutional Assessment Results Annex 7 Capacity Building Training Conducted by WEI/BANTWANA Annex 8 Child Rights Club Guide/Curriculum Materials Annex 9 District Nutrition Action Plans Annex 10 Summary of MSSA Results by School/District Annex 11 Partner Performance against Performance Monitoring Plans in FY12 Annex A. A Look at the Data: PMP – Targets vs. Actual Achievement June 2010 – September 30, 2011 Approved WEI Work Plan. December 2012. Children Count, USAID/Tanzania RFA 10-004, World Education, Inc./Bantwana Initiative Household Economic Assessment (HEA+): Key Findings for WEI/Bantwana’s Pamoja Tuwalee Program in Northern Tanzania. December 2012. Intrahealth Tanzania. Working with Most Vulnerable Children; A “How-to” Guide for Creating a Comprehensive Para-social Work Program. 2013. Kagera Worth OVC Stories, Bwanjai Ward, Misenyi District, Kagera: “Falling in Love and Realizing a Dream Together” USAID/Tanzania Pamoja Tuwalee Project Mid-term Performance Evaluation 149 MEASURE Evaluation, JL Consultancy, Ltd. WEI Tanzania MVC Data Quality Assessment Report. February 2012. Pamoja Tuwalee WEI/Bantwana Performance Monitoring Plan Revised November 2011 For FY 2012 – Targets Pamoja Tuwalee WEI/Bantwana Monitoring and Evaluation (M&E) Plan June 1, 2010 – May 31, 2015. Revised February 2013. Pamoja Tuwalee WEI/Bantwana Annual Report October 1, 2012 – September 30, 2013 Fiscal Year 2013 Pamoja Tuwalee WEI/Bantwana Quarterly Report January 1, 2013 – March 31, 2013 Fiscal Year 2013 Pamoja Tuwalee WEI/Bantwana Quarterly Report October 1, 2012 – December 31, 2012 Fiscal Year 2013 Cooperative Pamoja Tuwalee WEI/Bantwana Annual Report June 1, 2010 – September 30, 2011 Pamoja Tuwalee WEI/Bantwana Annual Report October 1, 2011 – September 30, 2012 Fiscal Year 2012 Cooperative Pamoja Tuwalee WEI/Bantwana Summary of Approaches and Results. September 2013. Pamoja Tuwalee/WEI. Protecting Ourselves and Each Other, a Child Rights and Protection Booklet. Performance Monitoring Plan for Pamoja Tuwalee Output Indicators, WEI WEI/Bantwana. Pamoja Tuwalee Program List of Sub-partners (Sub-grantees). WEI Household Economic Assessment Report Final. July 2013. WEI Performance Monitoring Plan for Pamoja Tuwalee – FY 13 Output Indicators WEI Performance Monitoring Plan for Pamoja Tuwalee, Revised October 2013 Output Indicators WEI Success Story: A Hand-Up: WORTH Creates Economic Opportunities for Vulnerable Families WEI Success Story: Addressing Stigma & Discrimination through Child Rights Clubs: Paulina’s Story WEI Success Story: Birth Certificates WORTH+ Paying For! WEI Success Story: Child Rights Club Feeds Primary School WEI Success Story: Economic Strengthening: Blending Support, Persistence and Commitment for Success through SILC Groups WEI Success Story: Leveraging Existing Infrastructure and Policy to Advance the Rights and Well-Being of Tanzania’s Most Vulnerable Children WEI Success Story: LIMCA – Vehicle for Financial Growth WEI Success Story: Multi-Sectoral School Assessments Foster District Support & Collaboration around Critical Health Services in Schools WEI Success Story: Receiving Nutrition Treatment Through Advocacy WEI Success Story: TACAIDS, District Council and Pamoja Tuwalee: Joint Approach to MVC Identification Achieves Success in Ngorongoro District WEI Success Story: Teachers Advocate for MVC to Receive Health Care Services WEI Success Story: Teaching Children about Rights & Responsibilities: Getting a New Pair of Shoes from Gardening WEI Success Story: The Catholic Diocese Of Tanga Pasadit OVC Project: Economic Strengthening Support Bring Changes in MVC Family WEI Success Story: Tina Pius: The Lady with a Purpose WEI Technical Program Description Year 3 Work Plan Pamoja Tuwalee WEI/Bantwana October 1, 2012-September 30, 2013 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523