November 2012 This publications was produced request of the United States Agency for International at the Development independently by Ishrat Husain, Rachel Macharia, Mary O’Grady and Billy Pick 2 Final Evaluation Report on the ROADS to a Healthy Future (ROADS II) Project in East and Southern Africa Mary O’Grady, Independent Consultant and Team Leader Ishrat Husain, USAID/Washington Rachel Macharia, Independent Consultant Billy Pick, USAID/Washington DISCLAIMER: The contents of the following report are the responsibility of the authors and the opinions expressed herein are those of the authors only. Cover Photo: Boda Boda operators (cyclists) ferry passengers and goods in front of the ROADS SafeTStop Cente in Busia, Uganda. 3 ACKNOWLEDGEMENTS   This report is dedicated to the people of East, Central and Southern Africa. The ROADS II Evaluation Team wishes to thank all the organizations, community clusters, governments, private sector entities, and individuals, especially at USAID and FHI 360, who provided assistance and support to the Team during the evaluation, conducted from May through September 2012. 4 TABLE OF CONTENTS List of Abbreviations and Acronyms ……………………………………..………...6 Executive Summary………………………………………………………..…..…..…8 ROADS II Project Objectives and Introduction…………………………..………12 Main Evaluation Findings………………………………………………..…............13 Extension of Coverage………………………………………...………..….....13 Capacity Strengthening……………………………………..…………….......16 Innovation and Diffusion…………………………………..…………..…..…19 HIV Prevention……………………………………………………………… 22 Administration and Management……………………..………………....……23 Lessons Learned………………………………………………..……………………28 Conclusions and Recommendations………………..…………………………..…..31 Economic Strengthening……………………..…………………………………..…36 ROADS II Responses to Recommendations of ROADS I Evaluation in 2008…..45 Annexes Annex A: Scope of Work for the Evaluation………………………………...…....58 Annex B: Evaluation Methodology………………………….................................62 Annex C: Limitations of the Evaluation…………………………….………….....64 Annex D: List of Individuals Met or Interviewed………………….……………..66 Annex E: Evaluation Site Visit Schedule…………………………….…………...80 Annex F: List of Documents Reviewed……………………………….…………..85 Figures and Tables Table 1: ROADS II Results, Targets and Percent of Targets Achieved...….……...13 Figure 1: ROADS II Site Locations in East and Southern Africa, May 2012….….14 Figure 2: Total Number of ROADS Site from 2006-2012………………………....14 Figure 3: Growth of ROADS II Sites, Obligations and Expenditures FY09-12…...15 Table 2: ROADS II Budgets and Obligations Summary by Country, May 2012….15 Table 3: ROADS Training Data FY2006-FY2012………………………………...17 Figure 4: Group Savings and Loan Association (GSLA) Results, May 2012……...18 Figure 5: Community Clusters and Groups Growth from FY2006-FY2012……....20 Table 4: Community Clusters by Site in Rwanda, March 2012…………………....21 Figure 6: Djibouti USAID-Dubai Ports World-FHI 360 Partnership……………....22 Figure 7: Innovations and Best Practices of Corridors of Hope in Zambia…….…..22 Figure 8: Materials Developed by ROADS in Kenya on Alcohol and Anal Sex......23 Figure 9: ROADS LWA Bilateral Mission Buy-Ins FY2006-FY2012…………….24 Figure 10: ROADS Conception of Linkages among Issues, Behaviors and Impact.25 Table 5: ROADS Maternal, Neonatal, and Child Health Achievements, Rwanda...26 Figure 11 ROADS Results in Burundi Showing Cotrimoxazole Prophylaxis…...…26 Table 6: ROADS Economic Strengthening Activities by Country & Location……38 Table 7: GSLA Results in Mozambique, Rwanda, Tanzania and Zambia………….39 Figure 12: Volunteers Trained by ROADS in Rwanda Currently Active………….42 Figure 13 Comparison of GSLA Membership & Volunteers in Tanzania………….42 Figure 14: Comparison of GSLA Membership in Tanzania and Rwanda……….…43 Figure 15: Grassroots Capacity Building through Learning Approaches…………..43 Figure 16: Progression of ROADS M&E System……………………………....…..47 Figure 17: Diagram of ROADS Data Collection and Outputs………………….…..48 Figure 18: Data Use Flow Chart by STI Team for Sex Workers in Rwanda……….49 Figure 19: Organogram Showing ROADS Organizational Evolution…………...….50 Figure 20: ROADS Tanzania Team Organogram…………………………………...51 Table 8: Representation by ROADS on National Planning Bodies by Country….…53 5 ABBREVIATIONS AND ACRONYMS AOR Agreement Officer’s Representative (USAID) APHIA AIDS Population and Health Integrated Assistance (in Kenya) ARC American Refugee Committee ART Antiretroviral therapy ARVs Antiretrovirals BCC Behavior change communication BMS Behavioral monitoring survey BSS Behavioral surveillance survey CCP Center for Communication Programs of Johns Hopkins University CDC US Centers for Disease Control and Prevention CRS Catholic Relief Services DAI Development Alternatives, Inc. DFID UK Department for International Development DRC Democratic Republic of the Congo EAC East African Community ECSA East, Central and Southern Africa ES Economic strengthening FHI Family Health International FP Family planning FGD Focus group discussion FSW Female sex worker FY Fiscal year GBV Gender-based violence GHI Global Health Initiative GSLA Group Savings and Loans Associations HMIS Health management information system HTC HIV testing and counseling IGAs Income-generating activities INGO International non-governmental organization IOM International Organization for Migration IP Implementing partner (organization) IRC International Rescue Committee ISN Immediate social network Jhpiego Johns Hopkins Program for International Education in Gynecology and Obstetrics LAPM Long-term acting or permanent methods (of FP) LIW Low-income women LQAS Lot quality assurance sampling LWA Leader with associates M & E Monitoring & evaluation MARPs Most-at-risk populations MCP Multiple and concurrent sex partners MMC Medical male circumcision MNCH Maternal, neonatal and child health MOH Ministry of Health NACC National AIDS Control Council (Kenya) NASCOP National AIDS & STI Control Programme (Kenya) NGO Non-governmental organization NSP National Strategic Plan 6 OGAC Office of the US Global AIDS Coordinator OHA Office of HIV/AIDS (USAID Washington) OR Operations research OVC Orphans and Vulnerable Children PACE Pharmacists and Continuing Education of Howard University PATH Program for Appropriate Technology in Health PHDP Positive health, dignity and prevention PLHIV People Living With HIV/AIDS PHN Population, Health, and Nutrition PHSC Protection of Human Subjects Committee PLACE Priorities for Local AIDS Control Efforts PMP Program monitoring plan PSI Population Services International PwP Prevention with Positives ROADS Regional Outreach Addressing AIDS through Development Strategies ROADS II Roads to a Healthy Future Project SADC Southern African Development Community SBCC Social and behavior change communication SCM Supply chain management SDM Standard days method (of FP) Sida Swedish International Development Cooperation Agency SOTA State-of-the-art SRH Sexual and reproductive health STI Sexually transmitted infection TA Technical assistance USAID/EA United States Agency for International Development/East Africa USG United States Government VCT Voluntary counseling and testing WHO World Health Organization    7 EXECUTIVE SUMMARY The ROADS to a Healthy Future (ROADS II) Project is a five-year Leader with Associates (LWA) award with funding from USAID/East Africa (EA) starting on August 1, 2008 that also includes bilateral funding from USAID Missions and will end on July 31, 2013. This initiative was a direct follow-on program to the ROADS Project that began in 2005. In mid-2012 The ROADS II project is operational in the following countries in East, Central and Southern Africa: Burundi, Djibouti, Kenya, Mozambique, Rwanda, Tanzania, Uganda, and Zambia. Countries with previous ROADS programming included: Democratic Republic of the Congo (DRC), South Sudan, and Ethiopia. The objectives of the project are to: (1) Extend HIV and broader health services to underserved, most-at-risk mobile and community populations along the East, Central and Southern Africa (ECSA) transport corridors and waterways; (2) Build the capacity of indigenous partners to design, implement and manage programming of their own design over the long term, in line with USAID’s Global Health Initiative (GHI); and, (3) Identify, test and diffuse innovations throughout sub-Saharan Africa. ROADS II is managed by Family Health International (FHI) 360 with the following sub-partners: Development Alternatives, Inc. (DAI), Howard University/Pharmacists and Continuing Education, Jhpiego, Johns Hopkins University/Center for Communication Programs, North Star Alliance, Program for Appropriate Technology in Health (PATH) and Voice for Humanity. The ROADS Project is focused on developing and instituting African-led and African-owned response to health and development needs by targeting transport corridor communities. As a result, ROADS hopes to build effective programs in HIV/AIDS and health to increase the number of people served, innovate and use promising new practices and approaches to improve outcomes in health and HIV/AIDS and build African capacity to respond to future health and development challenges. To meet these expected results, ROADS carries out a portfolio of activities, including: economic empowerment, community based services and activities, clinical health services and SafeTStops. Key to this programming are program features such as integration of health services, community engagement with new and innovative models such as the community clusters, collaboration with government and private partners, use of a regional platform to test, diffuse and scale up innovations and then documenting and disseminating these interventions. The final, independent evaluation of ROADS II was conducted from late May through September 2012 by Ishrat Husain, Rachel Macharia, Mary O’Grady, and Billy Pick, all of whom have experience evaluating health and development programs in Africa. The purpose of this evaluation was to assess overall project performance and to obtain recommendations for the re-design of the project. In order to do this, the evaluation assessed the outcomes of both the ROADS I and II projects in strengthening the response to HIV/AIDS in vulnerable mobile and transient populations along transport corridors in East, Central and Southern Africa. It is important to note that the overall evaluation, while focused on ROADS II took the entirety of ROADS programming into account and are reflected in the evaluators overall findings. Conclusions, lessons learned and recommendations will provide information for decisions made about ROADS program components of a follow-on project and the potential scale-up of these components by USAID/EA and bilateral programs in the region. Evaluations of new innovations designed and implemented by ROADS, such as the community cluster model and the regional platform approach to cross-border health issues, will serve not only to inform the future re-design of the ROADS project, but to inform the design of broader services in the region. 8 The ROADS II Evaluation Team found that the project largely met or exceeded its targets across the program areas and interventions, both for the mobile populations it has served and the local community members, by mid-2012. The two monitoring and evaluation (M&E) indicators where targets were exceeded included sexual and reproductive health and family planning (SRH/FP) counseling services provided, which exceeded the target by 253%, and training to provide SRH/FP services, which exceeded its target by more than 100%. Both program areas were added to the HIV prevention program interventions originally designed to be covered by the ROADS project. Two indicators where targets were below 70% in mid-2012 included positive health, dignity and prevention (PHDP) or prevention with positives (PwP) interventions for people living with HIV (PLHIV), a newer HIV prevention and care intervention emerging internationally over the last few years, and delivering core care services to eligible adults in the target areas. Another indicator where ROADS II did not reach the target was delivering essential services to orphans and vulnerable children (OVC), where 78% of the target was met. However, comprehensive OVC programming requires more intensive and potentially daily service provision and support across a wide range of health and development sectors. ROADS also added interventions to its original programmatic purview, thus extending the range of health and development services provided, including gender-based violence (GBV) prevention, alcohol abuse counseling and support groups, nutritional support to families and individuals through the innovative design and training provision of community gardens and bio-intensive gardens. Importantly, ROADS provided economic strengthening through the creation, training, and support of local Group Savings and Loan Associations (GSLAs), a key development model that strengthens the economic resilience of communities, families, and individuals, with savings enabling greater access to health services. In mid-2012 5,800 individuals were members of sustainable GSLAs in four countries with savings to date totaling nearly $184,000. In terms of the capacity building of indigenous organizations to design and manage their own programs on a sustainable basis, ROADS largely exceeded its cumulative targets over the life of the project by mid-2012, a year before the project is set to end. Eight of the ten training targets have been exceeded by ROADS, with a few exceeded at nearly 200% or more of the target. As well as the training targets exceeded in the technical intervention areas, the number of individuals trained in strategic information exceeded the target by 170%, and the number of individuals trained in community mobilization exceeded the target by 190%, showing significant focus on building the capacity of communities to launch and monitor programs they have identified themselves as important to implement. ROADS has shown substantial leadership on and even foreshadowed the principles of the US Government’s GHI and USAID Forward. The project’s focus on country ownership and leadership, gender equity, and capacity building, including technical training and programmatic design and leadership, financial and grants management, and M&E is a forerunner of such approaches, which over the last few years have become a priority for USAID. Some 84% of the 1,307 indigenous volunteer groups that have worked through the ROADS’ ‘community cluster’ model over the life of the project had never participated in a donor-funded program prior to ROADS. Moreover, in FY2012, ROADS II allocated US$12,747,733 in sub-awards, with 82% ($10,467,587) going directly to communities through cluster sub-agreements; and, 18% ($2,280,147) going to international NGO (INGO) partners. 9 In addition to the economic strengthening model successfully developed by ROADS, the aforementioned community cluster model, also developed and promulgated by ROADS, is at the forefront of the innovations designed by the project, totaling 114 clusters across the ROADS countries as of mid-2012. The use of the cluster model by ROADS has contributed to an improved quality of life of the cluster members individually and at an organizational level. Through this approach ROADS has provided a unique model of development, which builds capacities at the grassroots level through horizontal and vertical learning processes, utilizing immediate social networks that previously were informal, and strengthening organizational and leadership skills to implement what formerly were ad hoc responses by communities to address their needs. The clusters show a significant range in their membership, including: low-income women’s clusters, men’s clusters, youth clusters and PLHIV clusters. Moreover, the cluster model has proven successful in mobilizing communities sustainably through the life of the project, as the Evaluation Team found that the clusters formed since the project initiated them were still in existence in 2012. The LWA award, the program funding mechanism used by USAID for ROADS II, was found by the Evaluation Team to be a robust and flexible model for a regional program. The LWA mechanism has enabled USAID Missions to buy into a high-quality program crossing borders and fostering community health and economic development. Although the LWA model of ROADS has been acknowledged as a substantial value-added mechanism in most countries, it has not been integrated into every bilateral program where it started. Likewise, the funding levels of ROADS by country have varied greatly, from $100,000 for a one-year program in Ethiopia to $25 million for five years in Zambia. A lesson learned evident to the Evaluation Team was that providing only one year of funding or funding on a very limited level, for a complex program will achieve little or no impact. Given this lesson learned, continuation of the ROADS II model for five years is strongly recommended by the Evaluation Team. Continuing the regional concept of the ROADS model will serve to advantage nationally in most countries and result in continuing regional impact, with a greater focus on relevant regional and national policy advocacy added to the model in the future. The rationales for continuing this important multisectoral and multidimensional health and development initiative in the medium term include the following: (1) Public Health Rationale: ROADS target groups are at the highest risk of HIV infection and for onward transmission of HIV/STIs nationally and internationally. (2) Human Rights Rationale: ROADS target groups remain underserved by all health services, and these groups are generally socially marginalized and stigmatized. (3) GHI Rationale: The ROADS approach is the GHI approach, encompassing HIV/STI, TB, malaria, maternal and child health (MNCH) and FP as an integrated program under one umbrella, which has adaptable intervention priorities based on the programmatic flexibility to address local needs, as well as local ownership by communities with an intensive focus on capacity building and sustainability. (4) Sustainable Development Rationale: ROADS uses a holistic development framework for programming, including economic strengthening, which is increasingly recognized as essential to overcome the inferior health conditions and socioeconomic vulnerabilities in sub-Saharan Africa. (5) Gender Rationale: ROADS takes a gendered approach to its interventions and prioritizes gender equity, increasing the access to essential services for women and men and increasing individual and community understanding of the related socio-cultural issues, including GBV, alcohol and substance use, and other behaviors increasing individual and community vulnerability to ill health and socioeconomic dissolution. (6) Structural Change Rationale: ROADS strengthens existing community structures, including CBOs and local 10 associations, to build and maintain social networks to support health, economic development, food security, and healthy behavioral and social norms as well as enforce laws and provide legal support, where needed. (7) Globalization Rationale: An adequate supply of food continues to be the single greatest need in many countries in sub-Saharan Africa, especially among PLHIV and people affected by HIV/AIDS. That ROADS II has increased its focus on food supply and nutritional support through a variety of initiatives, such as the bio-intensive individual and community gardens, highlights its flexibility as an initiative capable of encompassing a wider array of development options within the present global situation of rising food prices based on drought in many areas. (8) Humanitarian Rationale: In several ROADS countries, the number of refugees, another mobile population using transport corridors, is increasing. Future corridor initiatives also should recognize the current or potential need, where relevant, to create linkages with or incorporate humanitarian-focused programming into the overall response and intervention mix to be able to have a greater positive impact on the communities and individuals served. In the remaining year of ROADS, it would be helpful for the program partners, the bilateral programs, and USAID/ EA to work together to increase the program documentation and disseminate more widely the program’s best practices and lessons learned, including its most successful management approaches, enabling USAID as a whole, other donors, and mobile population and community-based programs to gain from the experience of ROADS. Lastly, ROADS should develop a viable exit strategy to transition community initiatives over the next year to be more sustainable, whether or not a related or the same mechanism will continue in the future; and, actions taken as part of this process should be documented. 11 INTRODUCTION The ROADS II Project is a five-year LWA award funded by USAID/East Africa and bilateral USAID Missions in East and Southern Africa on August 1, 2008 and ending on July 31, 2013. This activity was a direct follow-on to the ROADS Project, whose ceiling was exceeded before its completion date due to significant participation from various bilateral USAID missions. ROADS II picked up from the mid-point of the previous ROADS Project. The midterm evaluation conducted in January 2008 informed the ROADS II Project design. In FY 2004, USAID/East Africa (EA) issued a Request for Applications for “Strengthened HIV/AIDS Program in the Region,” also referred to internally as the Transport Corridor Initiative. The proposed program was designed to reduce HIV transmission, improve care for people living with HIV and AIDS, and to reduce the impact of this disease on the communities and mobile populations along the region’s major transport corridors by harnessing the resources, imagination and commitment of communities and the private and public sectors. On August 5, 2005, USAID/EA signed Cooperative Agreement No. 623-A-00-05-00320-00 with FHI entitled “Technical Assistance for Strengthening USAID/EA Response to the HIV/AIDS epidemic in East and Central Africa.” This project was referred to as the “Regional Outreach Addressing AIDS through Development Strategies (ROADS I) Project” and was branded “SafeTStop.” ROADS I implemented HIV prevention and AIDS care and support services in “hot spot” communities linked by major transport routes across nine countries in East and Central Africa: Burundi, the eastern Democratic Republic of the Congo, Djibouti, Ethiopia, Kenya, Rwanda, South Sudan, Tanzania and Uganda. Beyond building health services and community mobilization structures to promote increased use of HIV/AIDS and health services, a critical component of the project was economic strengthening to address vulnerabilities of the local communities through partnership with local and international businesses called “LifeWorks.” The ROADS II Project, a five-year LWA award was a direct follow-on program to the ROADS Project. In mid-2012 the ROADS II project is operational in the following countries: Burundi, Djibouti, Kenya, Mozambique, Rwanda, Tanzania, Uganda, and Zambia. Countries with previous ROADS programming included: Democratic Republic of Congo (DRC), South Sudan, and Ethiopia. Led by FHI 360, the ROADS strategic partners include DAI, Howard University/Pharmacists and Continuing Education, Jhpiego, Johns Hopkins University/ CCP, North Star Alliance, PATH and Voice for Humanity. The Roads to a Healthy Future (ROADS II) Project Objectives: • Extend HIV and broader health services to underserved, most-at-risk mobile and community populations along the ECSA transport corridors and waterways. • Build the capacity of indigenous partners to design, implement and manage programming of their own design over the long term, in line with the GHI. • Identify, test and diffuse innovations throughout sub-Saharan Africa. 12 MAIN EVALUATION FINDINGS EXTENSION OF COVERAGE Program Indicator Targets: The program targets largely have been met or have been exceeded annually and over the life of ROADS in some cases. The following table shows the ROADS II results against the targets from fiscal years 2009 to 2012, including six months of results for FY12. Table 1: ROADS II Results, Targets, and Percent of Targets Achieved 2009-2012 PEPFAR Indicators and ROADS II Results and Targets Reached FY09-FY12 Results FY09-FY12 Targets % Target Achieved Number of pregnant women with known HIV status (includes women who were tested for HIV and received their results) 185,131 214,602 86% Number of PLHIV reached with a minimum package of PwP interventions 26,784 57,358 47% Number of the targeted population reached with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet the minimum standards required 3,977,392 3,968,514 100% Number of MARPs reached with individual and/or small group level preventive interventions that are based on evidence and/or meet the minimum standards required 320,532 342,242 94% Number of individuals who received HTC services for HIV and received their test results 700,328 768,182 91% Number of eligible adults and children provided with a minimum of one care service 61,289 112,504 54% Number of OVC provided with a minimum of one CORE care service 29,732 38,037 78% Number of eligible clients who received at least 1 PLHIV care and support service 46,866 51,988 90% Number of people trained in FP/RH with USG funds 1,692 1,443 117% Number of counseling visits for RH/FP as a result of USG assistance 43,065 17,000 253% Number of people who have seen or heard a specific USG￾supported FP/RH message 108,900 133,500 82% The two M&E indicators where targets were exceeded were related to RH/FP which was an addition to the program interventions originally designed to be covered by the ROADS project. The targets were exceeded for the number of individuals who were trained to provide sexual and reproductive health/family planning services and the number of counseling sessions/visits. 13 The indicators where the percent of targets were the lowest, with both below 70%, were PHDP or PwP interventions for PLHIV, a newer HIV prevention and care intervention developed over the last few years, and delivering core care services to eligible adults in the target areas. Another indicator where ROADS II did not reach the target was in delivering essential services to OVC where 78% of the target was met. One explanation for this may be that, in interventions for OVCs, the delivery of services to these children is often needed on a daily basis, thus involving more intensively trained staff and wider community support, as well as extensive resources. Whether ROADS is the best possible mechanism to provide care and support to OVC should be made on a case-by-case basis depending on the country, the level of need, available expertise, local community capacity, and the overall adequacy of resources. Site Coverage: The ROADS II model in its current capacity has extended HIV-related and other health and development services to the underserved and most-at-risk and vulnerable populations (MARPS) and communities along transport corridors, waterways, and in other locations where the project works. Figure 1: ROADS II Site Locations in East and Southern Africa in May 2012 ROADS Sites May 2012 As a result of funding increases, ROADS II sharply increased its site coverage as shown in Figure 2. The number of ROADS sites increased from 30 to 50 from FY 2009 to FY 2010 and then stabilized in subsequent years. Figure 2. Total Number of ROADS Program Sites from 2006-2012 14 Resource Utilization: ROADS II has been efficient in the use of resources, as shown in Figure 3 below. More in-depth analysis should be conducted to identify whether improvement is needed in linking expenditures to results-based management and/or target coverage. Figure 3. Growth of ROADS II Sites, Obligations and Expenditures: FY2009-FY2012 Table 2, below, summarizes the ROADS II budgets and obligations by country from the start of the LWA award by USAID/EA in August 2008, including the bilateral Associate awards, through September 2014. The list shows a vast range in commitments, varying from $100,000 in Ethiopia for one year of programming to $25 million for five years of programming in Zambia, illustrating that some USAID Missions considered ROADS to be a more important programmatic opportunity than others. 15 Table 2. ROADS II Budgets and Obligations Summary by Country, May 28, 2012 USAID Mission Start Date End Date Total Commitment Total Obligated Obligation Mortgage LWA Total East Africa 1 Aug 08 31 July 13 $17,000,000 $8,301,789 $8,698,211 $183,000,000 Tanzania 6 Oct 09 30 Sep 13 $18,493,169 $10,989,741 $7,503,428 $164,506,831 Tanzania 29 Sep 08 31 Dec 09 $3,593,286 $3,593,286 $0 $160,913,545 Burundi 29 Sep 08 30 Sep 13 $15,351,804 $13,507,713 $1,844,091 $145,561,741 Rwanda 29 Sep 08 30 Sep 13 $14,900,000 $11,644,467 $3,255,533 $130,661,741 DRC 29 Sep 08 31 Dec 09 $800,000 $800,000 $0 $129,861,741 Ethiopia 30 Sep 08 31 De 09 $100,000 $100,000 $0 $129,761,741 Uganda 2 Oct 08 30 Nov 10 $4,685,000 $4,685,000 $0 $125,076,741 Sudan 21 Nov 08 31 Dec 09 $2,701,000 $2,701,000 $0 $122,375,741 Zambia 11 Sep 09 10 Sep 14 $25,000,000 $13,596,972 $11,403,028 $97,375,741 Mozambique 1 Sep 10 31 Aug 14 $14,059,026 $6,254,387 $7,804,639 $83,316,715 Kenya 1 Sep 10 31 Aug 14 $1,000,000 $1,000,000 $0 $82,316,715 TOTALS $117,683,285 $77,174,355 $40,508,930 $82,316,175 Framework: The use of a consistent framework by ROADS II to focus on mobile populations at high risk of HIV infection across countries has provided a basis for replication of the program activities and has increased the sharing of innovations and best practices across the project sites. ROADS’ SafeTStop Resource Centers are alcohol-free and provide HIV prevention, counseling and testing, STI diagnosis and treatment services, linked with public health systems, men’s discussion groups on these topics, and road safety, alcohol abuse, men as partners, financial literacy, and offer satellite TV, pool, and Internet access. Stakeholder Collaboration: ROADS II has prioritized collaboration with a variety of stakeholders across sectors, including national and local government bodies, unions, clinics, community-based organizations (CBOs), local associations, etc., enabling wider geographic coverage of interventions and population groups nationally and enhancing local buy-in, thereby ensuring high-quality standards and increasing the possibility for sustainability. Alignment with USAID Initiatives: ROADS has achieved greater Programmatic Coverage and Alignment with USAID Forward and GHI:  ROADS is aligned with and foreshadowed USAID Forward and the Global Health Initiative (GHI) with its focus on country ownership and leadership, gender equity, and capacity building focusing on technical training and programmatic leadership, as well as financial and grants management.  Some 84% of the 1,296 indigenous volunteer groups that have worked through ROADS’ Community cluster model over the life of the project had never participated in a donor-funded program prior to ROADS.  In FY 2012, ROADS II allocated US$12,747,733 in sub-awards, with 82% ($ 10,467,587) going directly to communities through cluster sub-agreements; and, 18% ($2,280,147) going to international NGO (INGO) partners. CAPACITY STRENGTHENING Community Involvement and Ownership: ROADS II has been largely community-driven. Community beneficiaries have described how the training they have received by ROADS and 16 the related activities they have implemented have changed their mindset, providing hope for the future. Table 3, below, shows the ROADS training results achieved to date and the percentages against the targets. Several training areas have exceeded the targets set, including: the number of individuals trained to provide prevention of mother-to-child transmission (PMTCT) of HIV at 190%; the number of individuals trained to promote HIV prevention (through abstinence and faithfulness messages and general prevention messaging) at 129%; the number of individuals trained to provide care for OVC at 104%; the number of individuals trained in collecting strategic information at 170%; the number of individuals trained in SRH and FP service provision at a whopping 225%; and, the number of individuals trained in community mobilization at 190%. One area where additional effort undertaken in the final project year could make a difference is training focusing on care of PLHIV, where the project has met less than 50% of its target. Table 3. ROADS Training Data FY2006-FY2012 Program Area/Indicators Cumulative Actual FY06-12 Cumulative Targets FY06-12 Results Achieved Against Targets (%) Number of Individuals trained in PMTCT 580 305 190% Number of Individuals trained to promote HIV prevention messages through Abstinence and Being Faithful (AB) 13,281 10,334 129% Number of Individuals trained to promote HIV prevention beyond AB ( MARPs/sexual prevention for general population) 13,521 9,291 146% Number of Individuals trained to provide care and support for PLHIV 4,784 11,240 43% Number of Individuals trained to provide care and support for OVC 2,092 2,020 104% Number of Individuals trained to provide HIV testing and counseling 1,264 1,840 69% Number of Individuals trained to provide ART services 256 n/a Number of Individuals trained in strategic information 4,604 2,710 170% Number of Individuals trained in stigma reduction 389 n/a Number of Individuals trained in FP/RH with USG funds 2030 903 225% Number of Individuals trained in community mobilization 8,824 4,640 190% Total Number of Individuals Trained 56,110 46,274 121% Furthermore, ROADS II is a robust model to instill community ownership of economic strengthening, forming a strong basis for financial sustainability although this area needs further exploration and development as time passes. The economic strengthening component of ROADS II was considered a major strength of the project by the Evaluation Team. Figure 4 shows the results of the economic strengthening activities undertaken by ROADS II by through establishing GSLAs as of May 2012. 17 In programming for economic strengthening, ROADS II has incorporated their lessons learned into new or different approaches, achieving greater programmatic effectiveness, i.e., focusing on GSLA rather than continuing to create some larger private sector-focused and less sustainable income-generating activities (IGAs) initiated by LifeWorks under ROADS I. Though the GSLA approach promises to continue the economic strengthening of communities, it remains to be seen whether health program interventions will continue to be implemented by communities after ROADS II ends. Figure 4. Group Savings and Loan Association (GSLA) Results, May 2012 Examples of other areas in which ROADS II has increased NGO/CBO capacity through capacity building and the involvement of communities in identifying priorities for intervention support include:  Prevention of GBV was initiated in many communities by ROADS II. Since GBV and gender-focused initiatives are a prominent new and growing focus of the US Department of State and USAID, this is an important achievement of ROADS II, which should be continued as additional gender-focused funding is made available.  Alcohol use can be linked to GBV and the practice of high-risk behaviors related to HIV transmission. Consequently, alcohol and substance abuse and the related counseling and support groups were an intervention priority identified by communities and ROADS II; these programs are supported by ROADS II in three countries.  Magnet theatre for community and peer education on various issues has become a mainstay intervention approach for social and behavior change communication (SBCC) on health issues in many communities supported by ROADS. It is important to underscore this successful approach as currently the value of SBCC programming is being questioned internationally. Yet many ROADS II community members 18 emphasized how important magnet theatre as a communication medium to foster social norm change has been to them.  Nutrition initiatives developed through ROADS training and support have been adopted by communities in several countries, strengthening individual, couples, and family health outcomes through bio-intensive gardening. GSLA member in her bio-intensive garden in Makambako, Tanzania  Family planning was enhanced at district hospitals and clinics in Burundi, Rwanda and Tanzania by training health care workers and making referrals; and, in some Rwandan communities, FP/RH services were initiated where Catholic clinics were not providing FP services.  Community participation has been utilized in developing relevant HIV prevention messages based on community identification of their specific needs and the development of ideas on how to make messages resonant to other community members, enhancing the quality of these interventions.  The capacity of health staff, community clusters, and peer educators to make referrals by developing referral systems, networks, and processes, including ‘back’ referrals.  Most training provided by ROADS has been fully collaborative with government, resulting in local capacity enhancement and increased national capacity, with greater likelihood of programmatic sustainability in the future.  The capacity of community members to gather and report data on a monthly basis has been built by ROADS and electronic M&E systems have been developed nationally, with ROADS II sites feeding data to district, provincial, and national health management information systems (HMIS), as well as to ROADS HQ and USAID. INNOVATION AND DIFFUSION 19 The cluster model using volunteers appears to be more sustainable than using paid staff to manage programs or providing incentives to community members to organize and undertake programmatic activities, a more common development approach. ROADS has been effective in delivering services through the cluster model, a holistic and flexible approach beneficial at the individual, community, and/or national levels. While utilizing the community cluster model for development was an innovation in ROADS I, the further evolution of this participatory approach for development has enabled:  The integration of various program areas, including tuberculosis (TB) and malaria, GBV, SRH and FP, and maternal, neonatal and child health (MNCH)  The ability to focus on special issues identified by communities for programmatic attention, including alcohol and substance use  The destigmatization of previously marginalized community members (PLHIV and sex workers). Figure 5 shows the number of ROADS community clusters from FY 2006 to FY 2012 (to date) and the number of groups involved in these clusters across the same period. Figure 5. Community Clusters and Groups Growth from FY2006-FY2012 The use of the cluster approach has contributed to an improved quality of life of the cluster members individually and at an organizational level. This unique model of development, which builds capacities at the grassroots level through horizontal and vertical learning processes, thereby utilizing immediate social networks that previously were informal and strengthening organizational and leadership skills to implement what formerly were ad hoc responses by communities to address their needs. Moreover, the cluster model has proven successful in mobilizing communities sustainably throughout the life of the project as the Evaluation Team found that all clusters formed in their respective country since project inception in 2006 were, for the most part, still in existence. Enhancing the understanding, acceptance, and inclusion of previously marginalized community members, such as PLHIV, into the community clusters has resulted in wider access to services by these community members. Moreover, in some areas they have been recognized as community assets, with the group and individual PLHIV no longer marginalized within the 20 community. This example highlights how ROADS management and support for the inclusion of a generally marginalized group based on a highly sensitive issue into the overall community has benefitted the community as a whole. An example of the clusters formed in Rwanda in Table 4, below, shows the types created, their locations, and the number of individuals involved, nearly 32,000 in 5 provinces. Table 4: Community Clusters by Site in Rwanda, March 2012 Sites # of organizations/groups involved Cluster LIW Youth PLHIV Union Fishers Church Total Kigali 3 22 15 30 24 0 0 91 Gatuna 3 127 51 13 0 0 1 192 Rubavu 4 4 1 30 0 7 0 42 Bugarama 3 16 19 11 0 0 0 46 Rusizi 4 22 14 20 0 9 0 65 17 191 100 104 24 16 1 436 # Members 9,054 8,048 5,743 7,881 1,178 4 31,908 A total of 60 community cluster steering committees (representing community memberships, including many women) have driven decision-making on programming US$10,467,587 in USAID funding, with ROADS providing the technical support to the clusters. Partnerships: Partnerships with a variety of stakeholders, including various levels of government, faith-based, gender-based, age-based and work-based networks and community￾based organizations, have played a key role in the success of the ROADS project. In Djibouti, ROADS has established an important partnership with the Government of Djibouti, USAID, and Dubai Ports (DP) World, which manages the Port of Djibouti. The port serves as sea access for Ethiopia, with some 1,000 trucks from Ethiopia passing through each day, transporting goods from Addis Ababa to and from the port, and supporting workers from Djibouti, Ethiopia, Somalia, and other countries. The alliance is the first of its kind in Djibouti, formed to protect the health of these workers and their families through the establishment of a SafeTStop Community Center. This partnership is now serving as a model for other ports such as Maputo, Mozambique and Dakar, Senegal. 21 Figure 6: Djibouti USAID-Dubai Ports World-FHI 360/ROADS Partnership HIV PREVENTION Medical male circumcision (MMC): MCC, an HIV prevention strategy that has been incorporated into the ROADS programming through the transport corridor project in Zambia, This is shown in Figure 7, below, which outlines the way in which MMC is incorporated into a large portfolio of interventions and best practices in Zambia. This MMC programming may serve as a model for replication in other countries. Figure 7: Innovations and Best Practices by Corridors of Hope III in Zambia Behavior Change Communication (BCC): For BCC messaging ROADS II in Kenya initiated a collaborative process with the National AIDS Control Council (NACC), the Ministry of Health (MOH)/National AIDS & STI Control Programme (NASCOP), and USAID-funded AIDS, Population and Integrated Health Assistance (APHIAplus) implementing partners to develop messages on the topics identified by target group members related to risk behaviors forHIV transmission, including: unprotected sex, fear of HIV testing and counseling (HTC), practicing anal sex without using condoms or appropriate lubricant, using alcohol and drugs, and, having multiple and concurrent sex partners (MCP). The messages were pretested at two ROADS sites, including Malaba and Mariakani, and at two non-ROADS sites, Machakos Junction and Salgaa, and were approved by the MOH, Department of Health Promotion, NASCOP and the NACC prior to dissemination. ROADS trained the APHIAplus partners to disseminate the messages and built their capacity to develop their own contextual messages in the future. Figure 8, below, shows two messages developed • DP World demonstrates long-term commitment to the health development of its host countries. • Establishing a SafeTStop Community Center near the port:  DP World: contributing US$100,000 to establish the Center  USAID: funding technical and management support through ROADS  FHI 360: preparing site for construction  Government of Djibouti: providing health care workers; will take on full management within 2 years • Offer an array of health and social services:  Malaria, TB, respiratory infections, first aid, HIV education, counseling and testing, linkages to other health and social services  Training for personal and professional development (e.g., language classes, computer literacy)  Recreation (e.g., satellite TV, billiards) • ROADS partners adapting model at ports of Maputo, Mozambique, and Dakar, Senegal.  Quality improvement for referral system for ART in Livingstone, Kazungula and Kapiri Mposhi sites  Training and involvement of volunteer lay counselors to provide psycho-social counseling  Design and implementation of Community Radio Programs in partnership with communities  Introduction of door-to-door HTC  Integration of FP/RH services in HTC service delivery  Incorporation of male circumcision at selected COH sites through partnership with Society for Family Health 22 collaboratively by ROADS in Kenya, highlighting innovative and relevant BCC messages for the African context, promoting awareness of HIV transmission through anal sex, a very high￾risk behavior for HIV and a culturally sensitive topic, and alcohol use. Figure 8: BCC Materials Developed by ROADS in Kenya on Alcohol Use and Anal Sex Gender Equality: A total of 441 (51%) of the cluster steering committee members (867) are women, indicating equitable decision-making in ROADS II program design and implementation and highlighting women’s leadership. This is a real achievement by ROADS focusing not only on gender inclusion, but on gender equality as many communities in sub￾Saharan Africa still engage with each other using a patriarchal and hierarchical structure, positioning men in most of the leadership roles. ADMINISTRATION AND MANAGEMENT The LWA award is a robust and flexible model for a regional program, enabling USAID Missions to buy into a high-quality program crossing borders and fostering community health and economic development. Although the LWA model of ROADS has been acknowledged as a substantial value-added mechanism in most countries, it has not been integrated into every bilateral program where it started. Figure 9, below, shows the extent of the bilateral buy-ins on an annual basis. Monitoring and Evaluation (M&E): ROADS II has developed a state-of-the-art M&E system, effective in tracking project results on the community, district, provincial, and national levels with future plans for additional improvements to their national M&E systems in countries and the project’s M&E system and databases. An unusual aspect of ROADS’ M&E system is the emphasis on data capturing by community-level data collectors, who, after training, have appreciated their new and important roles, becoming engaged with the process and involved with achieving the programmatic results. 23 Figure 9. ROADS LWA Bilateral Mission Buy-ins FY2006-FY2012 Decentralization: The LWA award model has worked well after the ROADS country teams were decentralized from ROADS HQ in Nairobi. The results have included the strengthening of partnerships with government and local organizations, resulting in stronger national and local responses to the health and development needs on the ground in countries, and increased communication across countries, and with USAID regionally and bilaterally. Leadership: The management structure of ROADS II is strong and has provided adequate leadership across countries, using mostly African and local expertise in management and technical areas. See the ROADS II management organogram on page 47 showing the changes made by ROADS based on the recommendations made by the Evaluation Team in 2008. In addition, some 97% of the ROADS II staff of 184 people in the country programs are national staff. Of the 3% of ROADS II staff who are not national staff, one-half are African, including the former Rwandan Country Manager in Tanzania; a Rwandan Country Manager in Burundi; and, the Kenyan STO/BCC officer in Mozambique. Consequently, while the multinational ROADS project is managed by a US-based non-governmental organization, FHI 360, the program is managed on the ground largely by Africans for Africans, embodying GHI and USAID Forward principles. Furthermore, some 94% of all the technical assistance provided through ROADS II is South-South, an important management decision and direction taken by ROADS management, which also is a growing focus for USAID. Program Integration: ROADS II serves as a model for program integration by integrating new or additional HIV prevention and other health and development program areas Figure 10, below, displays the linkages of various contextual issues and behaviors and emphasizes why program integration is so important to enhance and deliver much-needed services and to foment understanding of the need for individual, couples, and community behavior change and the adoption of healthier social norms. 4.50 2.80 2.30 1.90 2.50 2 1.80 3.80 4.80 9.40 15.30 16.40 19.5 21.20 0.00 5.00 10.00 15.00 20.00 25.00 2006 2007 2008 2009 2010 2011 2012 US$ millions by fiscal year Associate Award (Bilateral Mission) 24 Figure 10. ROADS Conception of the Linkages among Contextual Issues, High-Risk Behaviors, and Individual, Familial, and Societal Impacts *Evidence in ROADS sites indicates that girl children rarely "drop" out of school of their own accord, but rather are "pulled" out of school by parents/guardians, often for economic or gender-related reasons. Table 5 shows the ROADS II MNCH results in Rwanda, one of many programmatic foci by ROADS. In Rwanda, ROADS has had a direct impact on strengthening MNCH programs in the country through a combination of increasing the related services delivered and the training provided by as much as eight-fold or more. As illustrated in the table, ROADS has succeeded in overall program management, training provision, and technical service delivery of the MNCH program through program integrations. Ensuring these programs are not only integrated with one another, but with the greater service provider networker, has also proven successful. Establishing local clinics providing FP services where they previously have been unavailable or adding these services to existing clinics has filled a much-needed service gap, which has benefitted many women, adolescent girls, families, and PLHIV in these communities. ROADS was lauded by nursing staff and community members at one of the new community clinics visited by the Evaluation Team in Rwanda, both for initiating the service and the management finesse shown in gaining the acceptance of the new clinic in the area by the older, more established clinics unable to provide family planning services. 25 Table 5. ROADS Maternal, Neonatal and Child Health (MNCH) Achievements in Rwanda District # New ANC visits at facility # of 4 standard ANC visits # people trained in MNCH # deliveries by SBA # children < 12 months received DPT3 # children < 5 received Vitamin A Before Current Before Current Before Current Before Current Before Current Before Current GASABO NA NA NA NA NA NA NA NA NA 457 NA 69 GICUMBI 316 1,183 20 44 0 2 197 681 543 1,022 62 7,199 KICUKIRO NA NA NA NA NA NA NA NA NA NA NA NA RUBAVU 2,724 3,065 841 1,111 0 0 1,880 1,894 3,213 3,457 652 13,175 RUSIZI 3,686 8,979 465 3,686 0 8 2,087 5,029 1,801 10,450 913 38,429 Total 6,726 13,227 1,326 4,841 0 10 4,164 7,604 5,557 15,386 1,627 58,872 Another success of program integration lies in Burundi, where pregnant women testing HIV￾positive are initiated on ART for life, unlike many other countries, and as a ‘PMTCT plus’ country, a package of HIV prevention, treatment, care, and support interventions is provided to mothers, their children, and their families. When ROADS started in Burundi, only 14 PMTCT sites existed; yet by 2011 PMTCT services were offered at 82 health facilities, with 17 newly launched. Officials lauded ROADS for their management and capacity building of the government’s PMTCT teams, the Burundi Ministry of Health’s (MOH) main priority. Within its PMTCT minimum package in Burundi, ROADS provides preventive therapy using cotrimoxazole to infants exposed to HIV, with achievements shown below11. Figure 11. ROADS Results in Burundi Showing Cotrimoxazole Prophylaxis Management # of people who on cotrimoxazole prophylaxis: 2006‐2012 1,130 2,041 3,606 4,972 6,210 8,105 9,525 0 2,000 4,000 6,000 8,000 10,000 12,000 2006 2007 2008 2009 2010 2011 2012 Individuals on cotrimoxazole Program integration reaches beyond health to such areas as economic strengthening, which has, in an initiative focused primarily on health, and even more narrowly HIV/AIDS, has enhanced 26 the asset value of ROADS II programs within communities, making them more appreciated and locally sustainable as well as strengthening health programs themselves. “Estradas” (ROADS) Project Youth Cluster member in Munhava, Mozambique LESSONS LEARNED 27 Organization, Networking and Planning  Long-term Planning: Comprehensive programming requires strong country leadership and planned technical support, which should be flexible to enable evolution as needs change. Comprehensive programs are likely to show little impact with funding for only one or two years.  Government Involvement: Government engagement and ownership works best when ideas and initiatives begin at the community level rather than through external program initiation. In addition, strong community commitment and government support and coordination at the local level is key, e.g., holding quarterly review team meetings with all implementing partners (IPs).  Partner Networks: Establishing strong networks with local implementing partners, including government agencies, NGOs, CBOs, and community clusters is essential not only for comprehensive program implementation and coverage, but also for state-of-the-art training provision and necessary refresher training using locally adapted materials translated into the common local languages, and for overall initiative sustainability in the future.  Creative Partnerships: Bringing ‘non-traditional’ program partners together adds creativity and dynamism to programming, including peer education through cross-training and the design and development of resonant social and behavior change communication (SBCC) messaging.  Organization of Key Groups: Through the cluster approach, the anchor organization that is elected by the different cluster organizations as the cluster network center facilitates a Steering Committee with representatives from each organization. Selected members from each organization are trained to build the capacity of their other members in their communities. Taking this approach has worked very well for ROADS, one of its most important programmatic features that should be continued and replicated in the future.  Social Networks: Using the immediate social networks (ISNs) in addition to the classic peer education model and the ongoing peer education provided by CBOs and community clusters has resulted in reaching out to people who were not initially targeted, increasing program coverage. The ISN channel has proved to make learning more retentive as the learners, in turn, become coaches and mentors to those in their own ISN. 28 Truck drivers and their assistants discuss their programming needs and issues while awaiting government clearance at a SafeTStop Centre in Rusizi, Rwanda. Program Approaches  Branding: The ROADS branding of the SafeTStop Resource Centers has been critical to the establishment of these dedicated health provision, information dissemination, meeting place and leisure facilities catering to transport workers, other mobile population members, and local communities. Moreover, the SafeTStop branding across national borders has enabled international recognition of these sites as safe havens and rest stops for mobile populations as well as community HIV/STI/alcohol prevention centers.  Training: The shift by ROADS from classroom training to mentoring and coaching gave more favorable learning results as it is more participatory and encourages creativity and initiative within the community. Utilizing both cascade and horizontal learning approaches provides quicker and longer-lasting impact.  Strategies for Involving Youth: Young people are responsive to music and theatre as a way of awareness rising. More ideas need to be sought on innovative strategies that will be attractive to youth to build greater awareness of sexual and reproductive health issues, including HIV and STI prevention.  Information Dissemination: Information dissemination is crucial so that good practices and innovations can be shared and replicated, or refined, so that each project does not have to build from scratch using trial-and-error approaches. 29 Peer educators interact at a SafeTStop Centre in Tunduma, Tanzania  Motivation: Some households have moved up on the economic ladder through GSLAs. They advanced from households previously unable to respond positively to any external shock without reducing their levels of consumption, to households that had developed a financial safety net and were able to level out consumption peaks and troughs. This change provided encouragement for moving towards greater levels of risk management in their economic decision-making. Members reported feeling more confident in their economic future than their neighbors who had not formed a GSLA. Moreover, confidence is a key motivator for future investment to provide income growth opportunities. Another GSLA cluster of HIV post-test club members had improved their health status due to better nutritional and financial security, and they were thrilled with the contribution of the project. 30 CONCLUSIONS AND RECOMMENDATIONS Conclusions Continuing the regional concept of the ROADS model over the next five years will serve to advantage nationally in most countries and result in continuing regional impact. The rationales for continuing this important multisectoral and multidimensional health and development initiative in the medium term include the following:  Public Health Rationale: ROADS target groups are at the highest risk of HIV infection and for onward transmission of HIV/STIs nationally and internationally.  Human Rights Rationale: ROADS target groups remain underserved by all health services, and these groups are generally socially marginalized and stigmatized.  GHI Rationale: The ROADS approach is the GHI approach, encompassing HIV/STI, TB, malaria, MNCH, SRH and FP as an integrated program under one umbrella, which has adaptable intervention priorities based on the programmatic flexibility to address the local needs, as well as build local ownership by communities and a more intensive focus on capacity building and sustainability.  Sustainable Development Rationale: ROADS uses a holistic development framework for programming including economic strengthening, which is increasingly recognized as essential to overcome the inferior health conditions and socioeconomic vulnerabilities in sub-Saharan African countries.  Gender Rationale: ROADS takes a gendered approach to its interventions, including gender equity, increasing the access to essential services for women and men and increasing individual and community understanding of the related socio-cultural issues, including GBV, alcohol and substance use, and other behaviors increasing individual and community vulnerability to ill health and socioeconomic dissolution.  Structural Change Rationale: ROADS strengthens existing community structures, including CBOs and local associations, to build and maintain social networks to support health, economic development, food security, and healthy behavioral and social norms as well as enforce laws and provide legal support, where needed. The lack of addressing the relevant structural obstacles and issues remains a major impediment to increasing HIV prevention practices and gender equality and decreasing poverty in sub-Saharan Africa.  Globalization Rationale: In 2012 food prices are rising because of serious drought in many locations, having a direct impact on the availability of staple crops. Food supplies may become less secure in the foreseeable future based on this situation. Moreover, an adequate supply of food continues to be the single greatest need in many countries in sub￾Saharan Africa, especially among PLHIV and people affected by HIV/AIDS. That ROADS II has increased its focus on food supply and nutritional support through a variety of initiatives, such as the bio-intensive individual and community gardens, highlights its flexibility as an initiative capable of encompassing a wider array of development options in partnership with other entities―governments, non-governmental and community-based 31 organizations, and the private sector―depending on the needs locally, provincially, nationally, or regionally.  Humanitarian Rationale: In several of the ROADS countries, the number of refugees, another mobile population using transport corridors, is increasing. Future corridor initiatives also should recognize the current or potential need, where relevant, to create linkages with or incorporate humanitarian-focused programming into the overall response and intervention mix to be able to have a greater impact on the communities served―whether they are local community members or mobile individuals, families, and groups transiting program areas for political or economic reasons, or to access basic human needs and services for survival. Recommendations Continuation in Programming  ROADS has successfully integrated various components into the project. Whereas HIV prevention, care, and support are the main foci of the project, ROADS has successfully managed to add supportive strategies such as economic strengthening through formation of GSLAs, FP, MNCH care, and addressing social problems, such as alcohol and substance abuse and GBV that contribute to risk behaviors and HIV transmission. It is important to continue this wide ‘umbrella’ approach in the future as conditions on the ground evolve re: specific intervention needs and HIV prevention and other interventions also evolve.  The use of the cluster model as a vehicle to respond to shifting priorities in HIV prevention, e.g., MMC, or for provision of ART refills at SafeTStop Resource Centers, or to incorporate new program areas such as MNCH, SRH and FP, GBV, and alcohol and substance abuse counseling not only can mobilize communities to design and plan community responses, but also increase awareness and create demand for greater service utilization.  The use of GSLAs as a cost-effective development foundation is promising for long￾term efforts to establish economic resilience and move towards income growth. The GSLAs also can be expanded and improved. They have been shown to make a difference in the lives and livelihoods of individuals and communities by fostering economic growth and helping to reduce the stigma associated with poverty or ill health. Of particular interest to the Evaluation Team was that the GSLA mechanism for economic strengthening has led to considerable reduction in the stigma associated with one’s HIV status for PLHIV involved in GSLAs in rural Rwanda. Thus the approach appears to be an entry point worth trying in other locations in sub-Saharan Africa to lessen stigma toward and discrimination against PLHIV, which exists more or less worldwide, and its more central positive outcomes of community mobilization, partnership, economic strengthening, and self-reliance.  The approach taken by ROADS ensures that entry into a GSLA is accomplished through self-selection of like-minded persons, who are known and trusted to be safe savers and who are reliable. Thus a member’s identity is less connected with the person’s association to the HIV prevention or other program element and based more 32 on the mutual desire for economic self-reliance and sharing information on sustainable economic activities.  The horizontal and vertical learning processes used by ROADS have the potential to build capacity at the grassroots level and can be adopted as a capacity strengthening approach to use in the project activities where relevant. In terms of sustainability, the economic strengthening approach used by ROADS is developing lower-cost delivery approaches, positioning the provision of training and mentoring within the environment of the learner, rather than venue-based traditional classroom teaching. The horizontal approach of encouraging GSLA members to share their own best practices, identify additional good practices in their own communities, and learn from each other not only promotes ownership and reduces the reliance on external ‘experts,’ but builds sustainability of the knowledge transfer process and the initiative itself. At the same time, ROADS II has recognized that some specific and carefully focused technical support is still required to build on local knowledge as it develops and to introduce new thinking and techniques where appropriate. This mix of vertical and horizontal learning is a key strength of ROADS. Future Shifts in Programming  The use of communication technology (text messaging on cell phones) will strengthen the communication links between mobile men and their families and could be used to engage with youth creatively regarding youth-focused programs. Cell phones already are used in a few countries in Africa to share treatment information with healthcare providers, including updates on drug side-effects, the latest protocols to follow, and individual patient information on a confidential basis after implementing an ‘informed consent’ process with the patient. Using this type of technology, as well as networked laptops, with mobile population group members for healthcare access is not new. However, it generally has been implemented on a local or national basis, where it is available, and the technological configurations should be analyzed as potentially very important cross-border tools.  The pharmacy component of ROADS appears not to be an integral aspect of the program; it may be better suited to provision by government, or a health-system strengthening mechanism, depending on the location and the existing local capacity and resource availability.  There should be a focus in the future on a regional policy initiative in conjunction with a corridor program, using a regional body to initiate or change related policies with additional support and advocacy from the regional health community and NGOs. Monitoring and Evaluation  A number of studies are being undertaken during the current project to document the impact of the project. These studies include the study of cost-effectiveness of activities, the impact of economic strengthening activities on sexual behavior and health status, the study of the community cluster model, and population size estimations of ROADS target groups for coverage percentage determinations. Such studies should be continued as they provide information useful not only to this project, but also provide much￾needed information for other programs nationally and internationally. 33  The ROADS approach using volunteers rather than paid staff theoretically should result in a cost-effective program. Yet the upcoming study on the cost-effectiveness of ROADS across program areas, taking into account that costs differ by location and other relevant factors affecting costs, will be able to make a more comprehensive determination on cost effectiveness than achievable through this program evaluation. ‘Cost-effectiveness’ determinations involve a wide range of issues. For example, if using volunteers is unsustainable, such an approach will not be a cost-effective approach over the long term―if a programmatic objective is to build the capacity of communities to sustain programs on their own in the future.  Regional programs that cross national boundaries must be flexible and adapt to specific country contexts, targeting the highest priority needs in the country or the regions of the country where the program will be operational. Not only should there be a situational analysis conducted to determine the greatest needs and the current response, but baseline data should be collected to be able to gauge appropriate M&E annual and life￾of-program targets, specific and measurable indicators, and establish a results-based management approach using systematic M&E design and implementation methodology including the capacity to measure change.  In the remaining year of ROADS, it would be helpful for the program partners, the bilateral programs and USAID/EA to work together to increase the program documentation and disseminate more widely the program’s best practices and lessons learned, enabling other donors and mobile population and community-based programs to gain from the experience of ROADS. An evaluation should be conducted two years after ROADS ends on the impact of the program in building the capacity of communities to continue the various intervention areas without external support and a determination made of the reasons why or why not the continuation of specific interventions was possible. Sustainability  Where the decision needs to be made between intensifying current programmatic coverage versus adding more sites before the end of ROADS II or transitioning to a potential ROADS III, the determination should be made based on potential epidemic impact, prioritizing both locations and interventions based on the need for either increased geographic coverage or a more intensified response at existing sites.  ROADS should develop a viable exit strategy to transition community initiatives over the next year to be more sustainable, whether or not a related or the same mechanism will continue in the future; actions taken as part of this process should be documented.  The unique sustainability aspects of ROADS II can be replicated: the economic strengthening components all have sustainability elements, including financial, manpower, and organizational structure. These components mobilize domestic resources; rely on local capacity development and volunteerism, and the cluster model shows substantial resilience for enhancing community leadership and fostering new community initiatives. The three aspects working together increase the chances of sustainability, which needs further review and contextual exploration. 34 Community Cluster meeting in Kayanza, Burundi Low-income women’s group in Kamambe, Rwanda 35 Youth Cluster dancers, Munhava, Mozambique ECONOMIC STRENGTHENING Of all the ROADS II interventions reviewed and visited, the Evaluation Team was most impressed with the economic strengthening component and the approach used. Thus this special section of the evaluation report specifically highlights this intervention approach as an example to adapt and replicate in the future. The ROADS II economic strengthening approach provides a foundation for building the internal capacity of individuals, households, and communities to build, strengthen, and maintain their livelihoods through entrepreneurial activities, resulting in better individual and community health outcomes, including a stronger focus on the need for HIV/STI, malaria, and TB prevention, as well as less socioeconomic vulnerability. ROADS II rightly has made the improvements in economic status of the household as one of the key areas of its work. Secondly, the project has responded to some key recommendations of the evaluation in 2008 relating to the strategic direction of any economic strengthening component of the project. These activities have been innovative and effective in terms of impact, and they have been organized and implemented with cost-consciousness to ensure sustainability. The following information summarizes the approach of ROADS II to economic strengthening, the participants, geographical coverage and key activities, describes the status of the economic activities and provides comments on them. Approach: The project follows PEPFAR guidelines for economic strengthening. It lays down a pathway where households move from activities that decrease their vulnerability to those that improve quality of life and contribute to economic growth. Thus, the project has three components of economic strengthening:  Developing a financial safety net: mobilizing internal funding, developing social capital and governance and promoting savings discipline through the formation of savings and loan groups  Undertaking household production for consumption and the local market  Expanding production for the commercial market. 36 ROADS II undertakes the assessment of household vulnerabilities based on 65 question tools to determine the kind of activities required to improve the economic status. Development Alternatives Inc (DAI) and FHI 360 provide technical assistance to the local implementing partners in implementing economic strengthening activities. ROADS has a well-defined strategic framework for economic strengthening (ES), which offers a portfolio of interventions reflecting differing levels of household economic vulnerability among participants, as well as their differing capacities to manage risk, and move towards income growth. The implementation approaches vary in different countries, from direct implementation by FHI 360 technical officers (Tanzania); capacity building of national implementing partners (Zambia); and, capacity building of site-level local implementing partners (Rwanda and Mozambique). In all the countries where ES programming is evident, ROADS is shifting to more capacity building of implementing partners and away from direct implementation. The ROADS technical officers carry out regular monitoring and supervision of IP staff and conduct rapid evaluation and program reviews at community and household level. The main participants are:  Low-income women (LIW)  Youth (18-24)  Orphan and vulnerable children caregivers  Sex workers  Fishermen  People living with HIV/AIDS (PLHIV)  Men Men’s Cluster, Beira, Mozambique Community clusters based on the groupings mentioned above provide the platform for organizing economic activities. Economic activities are cross-cutting as they draw on members from different clusters on the basis of their interest in economic activities. For example, ROADS II Provides assistance to HIV/AIDS‐ affected individuals and households focusing on strengthening the conditions that underlie coping strategies, particularly:  Level of savings  Access to and availability of food  Diversity of income  Volume of income 37 members engaged in savings and loan or other economic activities may belong to multiple clusters such as PLHIV, low income women, or sex workers. Geographic Coverage: ROADS II has economic strengthening activities in four of the countries. In Kenya the economic strengthening activities started in Mariakani under ROADS I are continuing with technical assistance from ROADS. The Burundi program is focused mainly on clinical service provision augmented by HIV prevention and community support activities. Djibouti does not have an economic strengthening program yet, but it is under discussion. Uganda benefitted from limited ES interventions under ROADS 1, which were not included in the reduced ROADS II award for Uganda. The ES work in Mozambique is still developing and shows promise so far. The economic strengthening sites in four countries are listed below in Table 6. Table 6. ROADS Economic Strengthening Activities by Country and Location Rwanda Tanzania Zambia Mozambique Kigali Kahama Chipata Beira Rusizi Makambako Solwezi Bugarama Tunduma Kapiri-Mposhi Rubavu Port of Dar Livingston Gatuna Kazungula Nakonde Chirundu ROADS conducts a Household Economic Assessment at sites prior to implementation, using a fixed choice questionnaire on the key aspects of the household income sources, asset levels, experience of hunger, access to savings and credit, access to farm land and utilization of that land, and access to health and education services. Focus group discussions on these issues build a more complete picture of the economic profile of ROADS beneficiaries. There are no data to determine population coverage at the site level. Activities: Based on the PEPFAR guidelines stressing an ‘economic pathway’ approach to developing household economic resilience, the three main activities mentioned below range from simple saving mobilization to expanding businesses on a large scale:  Formation of Group Savings and Loan Associations (GSLAs)  Small businesses start-up  Expansion of businesses on a commercial scale. The details of each of these activities are summarized below:  Mobilizing Financial Safety Nets through Savings ROADS’ support specifically focuses on empowering beneficiaries and their families to build group savings-based safety nets in partnership with trusted friends and relatives. The GSLAs are the focal point for all FHI 360 economic livelihoods programs and the hub for the majority of all community-level economic discussions and capacity building. Access to training and advanced levels of technical support depend on the commitment of the GSLA members to form and effectively manage their GSLA group. Using an adaptation of an international standard manual developed by VSL Associates, ROADS trains focal persons from implementing partners in the methodology. After training, the trained focal persons provide a modular training course to groups of between 15 and 30 people, meeting within the household environment rather than through venue-based training. The training program includes governance, record-keeping, and managing savings and loans procedures. The ROADS technical officers thereafter support the focal persons to collect 38 financial data and to analyze the financial performance on each group, as well as to monitor individual groups’ adherence to the agreed procedures. Considerable progress has been made in the formation and sustaining of GSLAs especially during the last two years in four countries as shown in Table 7: Table 7. GSLA Results in Mozambique, Rwanda, Tanzania and Zambia Category Mozambique Rwanda Tanzania Zambia Total Number of GSLAs 10 134 96 49 289 Number of Members 216 2,936 1,743 905 5,800 Savings $ 10,495 124,498 38,711 17,359 183,746 Value of Outstanding Loans $ n/a 39,428 34,694 10,042 84,164 Loans 135 1,246 855 264 2,365 GSLAs started in Tanzania in April 2010; in Zambia in February 2011; and, in Rwanda in May 2011. They have started more recently in Mozambique. GSLA meeting, Bugarama, Rwanda  Maximizing Household Production: By instilling a performance-based culture into programming, ROADS provides both direct technical support in the areas of common economic activity, including small plot horticulture, small livestock, and small business /income generating activities (IGAs) to GSLA members who are adhering to their group processes, saving regularly, and utilizing loans with appropriate repayment. ROADS uses standard adult-learning approaches that ensure short repeated learning opportunities based within the environment of the learner, rather than through classroom-based ‘top-down’ teaching. Consequently, ROADS encourages the trainees to make the best use of the assets 39 available to each of them. This approach is a vital precursor to more income growth-oriented activities. Through the use of modified best practices in behavior change and peer education, the cluster members learn skills on how to engage in livelihoods-focused community dialogues, emphasizing the values of peer accountability to adopt productive behaviors for stabilizing and expanding food security and household resilience. Most importantly, the households learn how to identify and better utilize their household surplus, their savings and their assets to improve land, care for livestock and engage in petty trading outputs. Once the GSLA is consolidated and functioning smoothly, ROADS provides additional technical assistance on more advanced foundation economic livelihoods areas.  Increasing Market Readiness: As production improves and households consider more advanced commercial activities, ROADS provides technical support to help interested and motivated project beneficiaries gain skills in business initiation (through the concepts of market dynamics and the analysis of commercial risk) and prepare, plan, and improve their access to local markets. Although only some of ROADS’ beneficiaries will be able or wish to participate in more market-oriented commercial activities, it is important that clusters learn the basic skills and knowledge to allow them to make solid commercial choices.  The GSLAs have contributed to accessing informal financial services: In all the discussions with the stakeholders mentioned above, the Evaluation Team members received a clear and unanimous message that the GSLAs are fulfilling a useful function by providing internally sourced access to credit for economic activities by and stronger financial security for vulnerable populations. Thus the GSLAs contribute to the long-term goal of lifting these community members out of poverty. Most of the GSLA members had access to financial services for the first time through the formation of a GSLA. Further, the GSLA approach introduced the orientation of developing a savings discipline, a key behavior change necessary for subsequent movement towards income growth. The GSLAs also equip individuals to understand and access formal microfinance and/or banking systems where they exist locally. The GSLAs have been the main source for funding businesses by the members of GSLAs. The anchor or coordinator provides guidance to the members on organizing income-generating activities (see examples in the section on businesses). A critical component of the GSLA methodology is its accountability to all the GSLA members and the transparency of record-keeping. On a visit made by the Evaluation Team to a Youth Cluster steering committee in Kigali, the anchor organization president said, “When we started the GSLA some of the members were skeptical. Now that we have closed the first phase, there is such high demand [to become members] by the whole general population.” Mobilizing Savings: Impoverished community members, including unemployed youth, low income women, sex workers, and others are saving and borrowing money as part of GSLAs. The ROADS II experience shows that even impoverished people can save some amount of money if proper mechanisms are available. Thus, GSLAs can be an important source of mobilizing domestic savings and reducing dependence on outside assistance. In addition to the focus on savings, the GSLAs have inculcated a culture of asset-building. In the FSW group in Rusizi, Rwanda, the 25 members started by contributing as little as $0.3 per person. At the time of the evaluation, the minimum individual savings total was reported to be USD$36.70, while the maximum individual savings total was $78.50. The PLHIV GSLA in Bugamara, Rwanda, reported their savings for one year, which totaled $349, and they had made loans to 12 members, each borrowing $25 at an interest rate of 15%, from which these 40 individuals had started their own small and growing businesses. The GSLA members apply for a loan, discuss the purpose of the loan with the other GSLA members, and are subject to a review of any previous loans and repayment history by their peers before any new loan is issued. The Evaluation Team learned that non-PLHIV community members wanted to join the PLHIV GSLA because it was thriving. Thus the success of the PLHIV GSLA had lessened the usual stigmatization and discrimination by the community toward PLHIV. Introducing Democracy, Transparency and Accountability at the Grassroots Level: The GSLAs, which have between 15 and 30 members, are organized and run democratically, with a governance structure and the use of transparent procedures. The GSLA members can be sex workers, PLHIV, or unemployed youth, etc. A key aspect is the self-selection, whereby members join the GSLA after a personal self-assessment for suitability and the selection of other members with which to save. Thus one’s HIV-status or other identity becomes secondary to the identity of a saver/loanee, reducing both stigma and organizational hierarchy and fostering equality. Every month the accounts are discussed with the GSLA members and their views are taken into account for improving the system. The cash is put in a box with three persons holding separate keys, ensuring that the money cannot be accessed easily by any individual. The procedures for dealing with default or inadequate meeting attendance of members also are in place. Part of the savings plan is a social welfare contribution to cater for emergencies and death. In Makambako, Tanzania, the Evaluation Team visited a GSLA that was sharing their funds after a completing their savings cycle, which lasts from 9 to 12 months as decided by the members. The members agree individually whether to start another savings cycle or to withdraw from the group. The current GSLA evidence from ROADS shows a retention rate of more than 95%, indicating a sustainable demand for the financial services offered within the groups. In Makambako, the GSLA members sat in a circle and referred to a common sheet of paper with the summary of each individual’s savings and loans through the year. A member’s name would be called, and s/he would go to receive their money at the center of the circle where the cash box was placed and the three key holders sat. When receiving his or her money, each member would tick the summary sheet; and, the savings and loans booklet of each respective member would be updated accordingly. Promoting and Sustaining Volunteerism: The functioning of GSLAs is based on volunteering by members of the group. The members have a sense of shared goals and purpose, understanding the benefits of their efforts. In both Rwanda and Tanzania, the Evaluation Team found very high retention rates as shown in Figures 12 - 14, below. 41 Figure 12. Volunteers Trained by ROADS in Rwanda Currently Active Further, the GSLA monthly meeting attendance rates, an ongoing indicator of members’ interest, has been high. The Evaluation Team observed that for the groups that had GSLAs and economic empowerment activities, the issue of sustaining the volunteerism was not a key concern emerging in discussions the Team had with GSLA members. Figure 13. Comparison of GSLA Membership with Community Volunteers in Tanzania Introducing Low-cost Learning and Capacity Building Processes: ROADSII has adopted a low-cost and decentralized training approach, using ROADS technical officers at each site to 42 train the focal persons for GSLAs on how to organize financing and provide information to members on business activities. The focal person arranges for a visit of members to a successful GSLA site to learn about and adapt the approach to their community. Figure 14. Comparison of GSLA Memberships in Tanzania and Rwanda The household production vertical and horizontal learning process is shown in Figure 15, below. Figure 15. Illustration of Grassroots Capacity Building through Learning Approaches Through the innovation of ROADS’ Household Production Guide, the GSLA members are provided with a series of simple worksheets, based on best practices in land use, animal husbandry, water use and small business performance. The selected facilitators encourage GSLA members to identify, discuss, and adopt key ‘productive behaviors’ that are aimed to 43 maximize the utilization of household assets. Each GSLA group discussion encourages the organic adoption of productive behaviors at the household level. These productive behaviors guide the households in determining their own solutions and empowering both individuals and GSLA groups to adapt the available guidance to the local context and their own needs by identifying local resources and the means and available assets to systematize the economically productive behavior over the long-term. Such a peer-driven process instills a mindset in the group and within communities that technical assistance and solutions to common household￾related problems can be found within, decreasing the reliance on external inputs, expertise, and funding. Starting Small Businesses Improves Nutrition and Social Status of Vulnerable Groups: The evaluation team visited two households that have started kitchen gardening on the basis of learning the best practices through the ROADS II GSLAs. ROADS Economic Strengthening Adding to the Household Food Basket Peninah has a kitchen garden, growing Chinese (green leafy vegetables) from which she earns an income of $3.20 per week. She bought three chickens from some of her GSLA income, and she bought a pig from another source of income. Peninah sees the benefits of this GSLA activity and is preparing another piece of land to start a kitchen garden. According to Peninah, the food is nutritious and organic. She uses manure, no fertilizers, and grows a weed with the vegetables that keeps away the pests and insects. In another household, the GSLA member had several kitchen gardens growing different vegetables. She had delivered a baby the previous day and was home with her husband. Her husband said he was happy helping her with the chores, as the earnings helped her to become somewhat economically independent, and she was contributing to the family income. She said most of her produce was bought by her neighbors, and her earnings totaled $1.70 per week. Some of her neighbors started kitchen gardening after seeing how she productively utilized the little gardening space she had. The PLHIV GSLA group in Bugarama, Rwanda, has been in existence for a year. The group had saved $350.00 and issued loans of about $300.00. One of the group members, a widow with six children, said, “After being trained in the kitchen gardening, I started growing vegetables that we eat. We sell the surplus, and every week I earn RwF2000 from the vegetables. I managed to save $30.00, and I bought a goat for $27.00 and used the remaining $3.00 to buy food.” Commercial Businesses Are Emerging, Contributing to Improvements in the Quality of Life: The team visited one of the households in Makambako, Tanzania, that is ready to take its poultry business to scale starting from a very small scale. Albertina’s Story Albertina has tried various businesses in the past. Initially, she was an illicit alcohol brewer. Next she tried sunflower seed pressing to extract oil. According to Albertina, the pressing involved too much work for too little return. On joining the ROADS II LIW Cluster, Albertina was trained in kitchen gardening and poultry rearing. Shortly afterward she joined a GSLA and opted to start rearing chickens, which was her preference. Albertina borrowed $38.00 from the GSLA and pays back $1.20 on a weekly basis. Currently, she has 66 chickens and sells two trays of eggs daily, earning $3.20. Regarding record keeping for her business, Albertina said her records are not up-to-date in her books yet. But they were clear in her mind, and she would write them down as soon as she had settled her crying child and finished the evening chores. 44 The Evaluation Team also visited an impressive LIW Cluster meeting in and using a small warehouse opposite a truck stop outside Kigali, Rwanda, which has started international exporting of their handsome, well-made, wide range of handicraft products. It is important to note that this space was donated to the community by the Government of Rwanda and for the use of the Community Clusters. Basket Weaving by Women outside Kigali, Rwanda A women’s sector leader linked a local women’s group to ASOFERWA, which is an NGO and an anchor organization for ROADS in Kigali. The women’s group joined the ROADS Women’s Cluster, and ASOFERWA helped them to achieve quality assurance in weaving beautiful baskets and creating other handmade products to sell in overseas markets. So far, with ASOFERWA’s assistance, the cluster has established markets in Japan and the USA. Moreover, the Government of Rwanda is helping them set up a factory to produce high-quality paints to use on their baskets and other crafts products. ROADS II has contributed by training the women in HIV/STI prevention, which is important because of their workshop base across from a heavily trafficked truck stop, and GSLA methods for economic strengthening of them individually and of their business enterprises. The cluster members said the training they had received from ROADS has changed their perceptions, enhancing their self-efficacy and opening their minds to other healthier options. The cluster spokeswoman informed the Evaluation Team that the financial returns from the weaving depended on personal diligence: “The more you weave, the more you make.” The earnings of the cluster averaged between $133.00 and $166.00 per month. To date, the cluster reported to have saved $4,160.00 through their GSLA and earnings of $4,660.00 from their weaving. While their weaving business is designed to make money, they said they have learned new skills from ROADS, which has helped them in various ways and has made them feel dignified. Fifteen of the women reported they had stopped practicing sex work, and they were concentrating on weaving and their GSLA for economic empowerment. “Money is important, but we have received a lot of other good education on our self￾worth and dignity,” said the women. 45 ROADS II RESPONSES TO KEY AREA RECOMMENDATIONS FROM the ROADS I EVALUATION IN 2008 During the ROADS I Project evaluation conducted in January 2008, the Evaluation Team identified key areas where they felt the project could focus additional effort and recommended actions to be taken during the final two years of the project. These recommendations were aimed at establishing that the project activities outlined in the annual workplans would be able to demonstrate a significant impact on the target populations, as well as being sustainable, replicable, and capable of being expanded during the next phase of the project. The ROADS II Evaluation Team reviewed the extent to which these recommendations were addressed by the ROADS II Project in the period since the last evaluation. The key areas that were deemed to require immediate action by the 2008 team are highlighted below. Recommendations were made and these are also addressed in the relevant key area sections. Several of the recommendations made by the previous Evaluation Team were addressed structurally when the ROADS II project procurement mechanism changed from a Cooperative Agreement managed by a single Agreement Officer Representative (AOR) in Nairobi, Kenya, to become a Leader with Associates (LWA) award with bilateral USAID Mission AORs having greater management and oversight responsibilities. Other recommended actions remained relevant to the successful delivery of activities under the current iteration of the ROADS project and are addressed below in the Responses. Key Area Recommendation The first key area identified for action by the 2008 Evaluation Team was the need to scientifically verify the basic premises of the project. The team determined that the premises for the project design needed to be verified with solid qualitative and quantitative evidence. The project’s premises at that time posited that: (1) a comprehensive set of services provided at the branded SafeTStops would help change the behavior of truckers and other target populations and that these services would be synergistic and cost-effective; (2) the cluster model of community mobilization would be organizationally sustainable being based on the views and voices of the target communities; (3) the model would also be financially sustainable as peer educators would earn income through Lifeworks, thereby mitigating the need for regular funding commitments; and, (4) that LifeWorks itself would become self-sustaining over a defined period of time due to its grounding in core business principles, and that further income and employment generation would help reduce high-risk behaviors among target groups and lead to improved treatment and care outcomes. In order to collect and analyze data to establish the evidence base for the project’s activities, the Team recommended that the ROADS project strengthen and expand its system for monitoring and evaluation (M&E) to be able to measure project outcomes and impact on the target communities and populations. It was felt that increased capacity to analyze data related the various aspects of the program also would provide evidence to inform scaling up and expansion of the program to other sites in additional countries. At the time of the first ROADS evaluation, there had been no systematic collection of quantitative baseline-indicator data prior to the implementation of project activities in individual countries. Therefore, the team identified the need to strengthen the collection of baseline quantitative data for both new activities and in new countries. Similarly, outcome monitoring for determining which objectives had been met was not available, limiting the ability of program managers to improve ongoing program performance. 46 The Evaluation Team recommended that the ROADS Project enter into a contract with a reputable operational research institution to design and carry out studies that could provide qualitative and quantitative evidence for program effectiveness. An additional technical recommendation was made for the ROADS team to consider the use of outcome monitoring surveys that use Lot Quality Assurance Sampling (LQAS). The advantage of LQAS is that it requires smaller samples and is less expensive. As a corollary to the M&E recommendation, issues of data quality were identified for some PEPFAR indicators. For instance, in 2008 in countries such as Burundi and Rwanda, there were systems at the facility level to track and monitor clients on ART, but there were no systems to track the number of people who were lost to follow-up or died. The M&E data that flowed upward from the project sites to the FHI office in Nairobi was neither site-specific nor disaggregated by sex. Furthermore, data was not readily available at the headquarters for clients who were referred by the project. The Evaluation Team also recommended that training should continue to be provided to members of the M&E steering committees of the clusters to update their knowledge on the definition of indicators and the processes of collecting, analyzing, and reporting data from the activities implemented. Response One of the most notable successes of the ROADS II Project has been the strengthening of the Strategic Information team and the emphasis on improving M&E structures and systems. The project has conducted an analysis of past, present, and future M&E needs as articulated in Figure 16, below, for ROADS/Rwanda. Each ROADS country office now has dedicated M&E Officers. Selected countries also have Data Managers based at individual sites to provide regular and immediate support to local Implementing Partners (IP), or based at the country office to maintain databases and collect information for national and regional reporting. In addition, ROADS II has strengthened its IP M&E systems through the training of staff and deployment of Data Management staff. Figure 16. Progression of ROADS M&E System 47 Despite not partnering with an outside institution to conduct operations research (OR), ROADS’ research agenda has been strengthened significantly. It is quite possible that the project’s decision to utilize FHI360’s existing resources and develop ROADS in-house capacity for research may result, in the long term, in greater local capacity to conduct operations research and program evaluation. This, however, is a longer term, secondary gain in capacity building that remains to be seen. As part of the project’s scientific research agenda, ROADS II has strengthened its site and technical assessment tools and conducted qualitative and quantitative studies to address the scientific validity of the project’s premises and inform ongoing programming. The project has conducted behavioral monitoring surveys (BMSs) in several countries, which provide a baseline to estimate project contributions to observed outcomes, as well as site-level outcome data for use by all stakeholders. Each country has developed assessments/operations research/evaluations relevant to the activities specified in the Associate awards. For instance in Burundi, these activities consisted of OR on the Rationale of ART drug use (2009); an MCH/FP baseline assessment (2009); a Standard Days Method (SDM) evaluation (2010); a PMTCT quality evaluation (2011); and, a BSS baseline among FSWs in Karusi (2012). In addition, there is multi-country OR, such as the ongoing assessment of the cluster community￾organizing model being conducted in four countries. The project has developed a regional integrated database replacing the multiple formats previously used. The database has been designed to meet the data needs of this complex, multi￾faceted project, yet also has been proven easy to use at the IP level. The Strategic Information team is populating the database with service statistics to better correlate inputs and outcomes. The project has revised M&E plans and developed country-specific program monitoring plans (PMPs) in response to the ROADS I recommendations and emerging data needs. This has included revision of tools and development of new tools and guidelines. Figure 17, below, provides an example of the database source flow. Figure 17. Diagram of ROADS Data Collection and Outputs 48 The project’s M&E team has strengthened its data quality assurance mechanism through development of a participatory data quality assessment checklist and guide. These are used periodically to assess the quality of data at the IP level. The IP skills in data quality assessment have been strengthened for routine data-quality assessments during implementation. Figure 18 illustrates the flow for an IP working with FSWs and STI management at a health facility in Rwanda. Figure 18. Flow Chart of Data Use by STI Intervention Team for SWs in Rwanda Discussions are ongoing with the internal FHI 360 ethics committee (PHSC) on ways to routinely use LQAS through less time -demanding processes (without compromising ethical requirements) to effectively measure project outcomes. Among the evaluation activities pending IRB approval, there is one designed that will use LQAS. In addition to the efforts related to utilizing the LQAS methodology, the ROADS team has been using the PLACE method to identify locations and circumstances determining high-risk behaviors (meeting new sex partners or having risky sex) that occur among fisher folk in Mwanza, Tanzania. The Behavioral Monitoring Surveys (BMS) conducted to date have been cross-sectional, descriptive studies among selected project target populations, including truckers, FSWs, youth and women in low-income settings/households. Repeat studies have been conducted in Zambia, and the team plans to repeat the Tanzania BMS in the five sites where the baseline study was conducted. Key Area Recommendation During the previous evaluation, the team found that Health Officers working in bilateral USAID programs felt that there was inadequate interaction and communication between the Nairobi-based ROADS I project staff and the bilateral Missions. The team felt that two factors contributed to the problem. The first factor was the absence of dedicated ROADS Project field staff at FHI country headquarters offices in the capital cities. At the time, the ROADS Project had only recently hired staff in South Sudan. In the DRC, while ROADS had staff in the FHI field office in Bukavu, the ROADS Project did not have a staff member in the FHI country office in Kinshasa who was capable of interaction with USAID Mission staff. At the time of the 2008 evaluation, the process of posting staff in Kampala and Dar es Salaam was still ongoing. Second, bilateral Missions felt that simply having staff present was adequate. In 49 the case of Kenya; the ROADS Project was headquartered in Nairobi and yet communication with USAID/Kenya remained challenging. This was often due to a heavy workload of Health Officers, especially in the larger PEPFAR focus countries. In these countries, the ROADS Project constituted a tiny portion of a much large portfolio, and it was perceived to be difficult for a busy Health Officer to give time and attention to a small program over which they had no managerial oversight. The Evaluation Team identified the need to increase interaction with the Health Officers and national IPs. ROADS was enjoined to find ways to enhance interactions with Missions and IPs. The recommendations emerging from the discussions with the Health Officers and other concerned parties included:  Organize a meeting of Health Officers and HIV point persons to report on and discuss ROADS and regional health issues at a time when both would be available to attend it, e.g., at a SOTA or PEPFAR Implementers Meetings.  Prepare a short progress report focused on each country and send it via e-mail to the Health Officers and the point persons.  Appoint a focal person in all FHI country offices that have substantial ROADS project activities in order to maintain regular contact with the Missions.  Have ROADS staff be represented at all Population, Health, and Nutrition (PHN) Implementers meetings and also at thematic group meetings (e.g., for OVC, family planning) at the various USAID Missions. Response The advent of the LWA awards under ROADS II allowed bilateral USAID Missions to address the communication issues through increases in ROADS national staffing and having bilateral Agreement Officer Representatives (AORs). As part of the LWA, ROADS has expanded its staff from 58 in 2008 to 184 in 2012, decentralizing key functions to fully staffed country teams (technical, program, finance) linked through the regional platform. ROADS Country Managers liaise regularly with USAID AORs and other USG staff and partners via monthly and quarterly meetings, site visits, etc. While the LWA award is still managed at USAID/EA in Nairobi, Kenya, the change provided more opportunities for communication, giving the bilateral AORs more oversight responsibility via the Associate awards. Figures 19 and 20 illustrate the evolution of the decentralized structure of ROADS. Figure 19. Illustrative Organogram Showing ROADS Organizational Evolution 50 Figure 20: ROADS Tanzania Team Organogram The FHI360 ROADS management team also organized annual AOR meetings to bring together USAID counterparts from across the region to discuss lessons learned, best practices, and new horizons (Dar es Salaam, 2010; Kigali, 2011). Key Activity Area The Evaluation Team felt the Project should increase advocacy for the adoption of regional policies and initiatives for addressing the needs of mobile, transnational populations and the communities through which they pass (e.g., the regional adoption of ROADS models). The ROADS I Evaluation Team felt that Project staff needed to conduct more thorough data analysis to help stakeholders to better understand the implications of regional transport networks on their national HIV programs and to demonstrate how the transnational nature of the ROADS Project impacts the local HIV transmission factors. Close contact and/or collaboration with in-country partners to share experiences and coordinate program activities were determined to be extremely important. The team felt that the ROADS I project was trying to develop these contacts, but more had to be done on a systematic and regular basis. At that time, although there had been several activities aimed at enacting regional policies (e.g. HIV prevention and alcohol abuse, alcohol abuse, and ART adherence), the ROADS Project primarily and appropriately focused on country-level activities. The team believed that the project had generated enough experience to disseminate lessons learned across borders and should begin sharing experiences among various clusters and associations to improve quality and effectiveness. In addition, they recommended that the project should strengthen cross￾country activities, such as joint trainings, exchange of best practices such as SafeTStop branding, and cross-country visits by the field staff and volunteers. In a related recommendation, the team also felt that the project should be getting the stories out more vigorously and rapidly. The team felt that the ROADS Project staff could devote more effort to timely documentation and dissemination of its experiences. The ROADS Project would benefit from visibility as its activities and experiences were perceived as valuable for 51 strengthening the HIV/AIDS prevention activities in communities with mobile populations. They recommended linking with the local and international media to share the experiences with the general public in addition to professionals and program planners. Response ROADS has continued to work with the East, Central and Southern Africa Health Community (ECSA-HC). The project participated in a Health Ministers Meeting in Mbabane, Swaziland, in February 2009, presenting on alcohol and ART loss-to-follow up resulting in passage of resolutions on each issue. They also participated in a GBV Technical Experts Meeting in Dar es Salaam, Tanzania, in September 2009 and an M&E Technical Experts Meetings, also in Dar es Salaam in August 2011. The project was enlisted by USAID/Southern Africa to participate in a Southern African Development Community (SADC) donor meeting to share project approaches in transport corridor programming in September 2011, attended by Sida, DFID, the World Bank and IOM. ROADS also participates in East African Community (EAC) Regional HIV and AIDS Partnership Forums (2009-2011). The project continues to collaborate with the private sector, including Dubai Ports World (Djibouti), General Motors (Kenya with potential expansion in Southern Africa), Equity Bank (Kenya, Tanzania) and Airtel (Kenya). ROADS has also received other donor funding, e.g., UNICEF funding in Djibouti to conduct youth￾focused programming in PK 12 and Balbala. ROADS also organized cross-country exchange visits to accelerate shared learning: • In September 2011, a Government of Mozambique delegation visited ROADS II Rwanda sites to examine community-led and -owned programs (Provincial Health Director, Sofala Province; Deputy Health Director, Maputo City; Chair; Ministry of Transport HIV Committee) • In August 2009, local partners from four Tanzania sites visited Busia, Kenya, to examine the community-based alcohol counseling programming • In October 2008, representatives of South Sudan partners PSI, IRC and ARC visited Busia and Malaba, Kenya-Uganda to examine community organizing thru clusters. Also, as mentioned earlier in this report, 94% of all technical assistance through ROADS II is South-to-South. Community partners attend ROADS regional meetings and are supported by the project to attend donor forums (e.g., a Busia, Kenya, IP presented at the PEPFAR Southern and Eastern Africa Technical Consultation on Alcohol and HIV Prevention, April 2011). ROADS II has developed various ways to disseminate program accomplishments. For instance, a SafeTStop exhibit was organized at the US Embassy/Nairobi in May 2012, which was attended by the Ambassador and other USG officials. The project has generated and presented abstracts and presentations at national and international forums, including:  “Strengthening Broad-based Community Engagement in HIV Service Delivery,” AIDS 2012, Washington, July 2012)  “New Directions in Transportation and Development in Africa,” Africa House Lecture Series, New York University, December 2011  “Capacity Building to Promote Locally Driven, Sustained Action in Historically Underserved Transport Corridor Communities,” (OHA Partners Meeting, Washington, DC, September 2011)  “Behavior Change for Key Agricultural Practices to Promote Improved Quality of Life for Vulnerable Persons,” 6th International Conference for Exchange and 52 Research on HIV and AIDS, Kigali, Rwanda, June 2011 (awarded Best Abstract, Category C, Socioeconomic Impact and Socio-Economic Support). ROADS II has integrated itself into a range of national planning bodies, including MOH, Transport, Gender and PEPFAR TWGs as shown in Table 8, below. Table 8. Representation by ROADS on National Planning Bodies by Country As part of ROADS II, the project team has formalized the branded ROADS package for easy adoption and franchising along the corridors. The Kenya Ministry of Transport has adopted SafeTStop branding and replicated it along key roads and highways in the country. The SafeTStop logo has been translated into French, Portuguese, and Amharic (an Ethiopian language) in order to be recognizable to populations across countries. The SafeTStop Resource Center model has been replicated across transport routes. For instance, Dubai Ports World is in the process of working with FHI360 and the Government of Djibouti to establish a SafeTStop Community Center in Djibouti, with plans to expand an adapted model in ports in Mozambique and Senegal. Key Area Recommendation One of the most pressing issues that the 2008 Evaluation Team identified was the prospect of whether to expand the project to new sites/different corridors or to consolidate program activities at the existing sites. The Team felt that the ROADS Project should consolidate the model and demonstrate impact before undertaking expansion to new sites. This would allow for the development of a functional model that could be replicated/franchised elsewhere based on evidence that the model had an impact on HIV transmission, access to care and treatment, and mitigation of the disease on affected populations. The team felt that there were still many 53 challenges with the existing sites and, therefore, that it might be prudent to solidify the program at the existing sites over the following two years, document the successes of ROADS methods, and consider geographic expansion and replication during the next phase of the project. The Evaluation Team found that the ROADS Project had functioned best in countries where: (i) it was able to hire an adequate number of expert field staff commensurate with project activities, as in Rwanda, Burundi, and Kenya; (ii) where missions have provided appropriate resources to fund the project activities, as in Tanzania, Burundi, Rwanda, and Uganda; and, (iii) where the ROADS Project implemented its core set of activities, i.e., prevention with MARPs, such as truckers and LIW/FSW, linkages to voluntary counseling and testing (VCT)/STI/ART services through the SafeTStops, community empowerment through the cluster model, and LifeWorks. The Evaluation Team also felt that the ROADS Project staff should rethink implementation in countries where factors were not in place to adequately roll out the model. For example, in countries with small amounts of funding, such as Ethiopia, supporting discrete elements of the ROADS model would not contribute to demonstrating impact of the model overall. In South Sudan where the ROADS Project received extensive funding, the Team felt efforts should be made to provide resources for staff commensurate with activities, as well as to have discussions with the USAID Mission to refocus on the core ROADS model activities (HIV prevention, SafeTStops, cluster models) as opposed to overseeing HIV activities implemented through sub￾grantee programs, such as Population Services International (PSI), International Rescue Committee (IRC), American Refugee Committee (ARC). The Evaluation Team recommended that the Project work with USAID/EA to discuss with bilateral USAID Missions in Kenya, Rwanda, Burundi and Uganda the feasibility, funding, and result expectations for expansion of activities during the last two years of the ROADS I project. They also recommended that ROADS should consider the possibility of limiting its activities in Ethiopia and Djibouti after discussions with the Missions and management. Response As mentioned above, the shift from a Cooperative Agreement to a Leader with Associates award significantly changed the approach that bilateral USAID Missions took when considering the ROADS model. The ability of the ROADS project to scale-up the entire package of activities was limited to a certain extent by country-specific PEPFAR strategies. Some Missions continued to utilize the ROADS II LWA mechanism, while others decided to provide similar services through bilateral mechanisms. For instance, the ROADS-II Project Associates Award between USAID/South Sudan and FHI/360 lasted just over one year from November 21, 2008 to December 31, 2009. While the ROADS II Project in South Sudan promoted peer education and community mobilization activities, and generated significant uptake of HIV services, including HIV counseling and testing in Juba and Greater Yei, the Mission decided to issue a bilateral procurement that included many of the ROADS I activities. The bilateral was awarded to FHI360, so there was some continuity. Yet the same cannot be said for USAID/Uganda where ROADS II did not receive significant ongoing support, nor has USAID/Ethiopia seriously engaged the project. On the other hand, bilateral USAID Missions in Rwanda, Burundi, Tanzania, Zambia, Mozambique, and the DRC (in process) have made substantial buy-ins through the LWA award, as shown earlier in this report in Table 2. Djibouti has continued to receive assistance from USAID/EA through ROADS II and now has additional PEPFAR funding to expand the scope of ROADS activities. 54 In some cases, the scope of work for ROADS through the Associate awards has departed from the SafeTStop and cluster models. For example, a central component of the current LWA award for ROADS/Kenya is the national level development of BCC messages for vulnerable groups in collaboration with NACC, MOH/NASCOP and APHIAplus partners. ROADS/Kenya has developed messages with focusing on specific risky behaviors identified by the target groups during the rapid assessment including unprotected sex; fear of HIV testing and counseling (HTC); anal sex without condoms and appropriate lubricants; alcohol and drug abuse; and multiple and concurrent sexual partnerships (MCP). Many other components of ROADS model have been devolved by the USAID/Kenya Mission to the APHIAplus partners in each province. The ROADS Project team’s response to the Evaluation Team’s recommendation to consolidate and improve the evidence base has been discussed above, but ROADS II also made technical and programmatic changes based on assessments and epidemiology. These include improved targeting of risk and vulnerable groups, particularly young women. Under ROADS I, youth clusters reached significant numbers of young people, but not necessarily those most at risk. Under ROADS II, the project has made a more concerted effort to identify and reach most-at￾risk young people, particularly younger, highly vulnerable female sex workers (Burundi, Rwanda, and Tanzania). Adapting strategies from the Avahan program in India, ROADS II Rwanda has increased the number of “hidden” FSW reached with prevention, care, and support services from 596 in 2009 to 2,139 in 2011, showing more than threefold growth. In another technical area, ROADS also has tried to negotiate with government, regional, and donor programs to address continuity of ART provision for truckers transiting through multiple countries in order to avoid missed doses, ensure adherence and prevent drug resistance. In the current ROADS II Kenya Associate Award, FHI 360 has negotiated with MOH/NASCOP and USAID to pilot ART refills and other selected clinical HIV services at two SafeTStop Resource Centers in Rift Valley Province. This pilot is the first of its kind in East and Central Africa and will be evaluated for cost, adaptation, and scale-up in Kenya and elsewhere, working closely with the EAC. ROADS II is also negotiating with Reach Out Mbuya, a USAID/Uganda clinical partner, to provide ART re-supply at the SafeTStop Resource Center in Mbuya-Kinawattaka, Kampala. Key Area Recommendation In 2008, the Evaluation Team recognized the need to undertake a financial analysis on the cost of the comprehensive SafeTStop model and of the various individual components (e.g., branding, cluster formation, income generation, counseling and testing outreach). While the Team recognized that it may have been somewhat early to undertake a financial analysis of the program model based on having only recently been established in many sites, they felt there was a critical need for making a case for program expansion during the next phase. The USAID Missions that proposed funding the ROADS II Project needed to be able to make budget decisions based on appropriate information. They recommended that this financial analysis should occur within the final two years of the ROADS I project. Response For a variety of reasons, including the early transition in mechanisms from ROADS I to ROADS II, and the ensuing revamping of the decentralized project staffing and expansion to additional countries, e.g., Mozambique, a formal financial analysis did not occur. Nonetheless, the ROADS II Project has undertaken efforts to document cost aspects of the model to enhance reliability. The team tracks programming costs, including costs of establishing and supporting clusters. Currently, the ROADS SI team and an FHI 360 health economist are developing a 55 cost-tracking system for specific services supported by the project. New sites in Tanzania (Chalinze, Ilula) will serve as baseline sites to launch the cost-tracking system in FY 2012. Key Area Recommendation The 2008 Evaluation Team also recommended strengthening the employment generation component of the project: They felt that LifeWorks had been successful and in high demand among the target populations due to extreme poverty in their communities. The Team recommended that LifeWorks consider expanding local entrepreneurship activities, particularly among the vulnerable members of the youth cluster, especially in countries with a limited private sector. The team also recommended that the project could possibly apply for some of the increased funds being made available by the US Congress for micro-enterprise activities; intensify the search for local and international markets; and, conduct a cost-effectiveness analysis of its operations and determine the impact of economic activities on the high-risk behaviors among the vulnerable groups. Most importantly, the project needed to determine the applicability of the LifeWorks approach in multiple contexts beyond Kenya. Response The ROADS Project management team undertook a strategic review of LifeWorks in FY 2009. The review found that the LifeWorks Shukrani Production Facility in Mariakani was an opportunity unique to that site. The company has since transitioned from being an NGO sub￾grantee to an independent registered private company, with a strategic partnership that is being developed with an allied company for sustainability. More importantly, the review led to adaptation of USAID’s Economic Strengthening pathway approach to working with HIV￾affected households, based on: (i) strengthening internal savings and loans; (ii) maximizing household production; and, (iii) preparing for commercial readiness. This transition was described in greater detail in the previous Economic Strengthening section of this report. Key Area Recommendation Finally, the Evaluation Team felt that ROADS should make use of new information technology for program improvements. New information technologies, such as cell phones and solar computers, were seen as presenting new opportunities to strengthen employment and income￾generation, as well as health programs. The ROADS Project was asked to explore increased linkages across USAID sectors, such as trade and economic growth as well as collaborations with other donors. Also, the ROADS Project was enjoined to develop the interest and understanding of youth clusters in information technology. Response The ROADS project has made use of IT in many areas of the project. The majority of SafeTStop Resource Centers visited by the team had Internet access on site both as a way to attract clients and to disseminate HIV prevention, care, and treatment information and health messages. For example, through the partnership with Dubai Ports World, the SafeTStop Resource Center in Djibouti has been equipped with a solar internet kiosk to help mobile workers stay in touch with their families while away from home. ROADS II has developed virtual technical teams (HIV prevention, economic strengthening, M&E) to rapidly share critical information, standards and tools with transport corridor communities across countries. In Rwanda, the IP M&E teams use iPads for service provision, and iPads and PDAs for research and evaluation. 56 As part of the LWA award, ROADS also has supported the redesign of the AfriComNet web site, now highly interactive receiving 77,000 hits per month on average. Under guidance from the Board, the network continues to send weekly e-updates and other electronic features. 57 ANNEX A. SCOPE OF WORK FOR THE EVALUATION I. BACKGROUND In FY 2004, in response to the needs of these vulnerable and underserved populations, USAID/East Africa (EA) issued a Request for Applications for “Strengthened HIV/AIDS Program in the Region”, also referred to internally as the Transport Corridor Initiative. The proposed program was designed to reduce HIV transmission, improve care for people living with HIV and AIDS, and to reduce the impact of this disease along the region’s major transport corridors by harnessing the resources, imagination and commitment of communities, the private and public sectors. On August 5, 2005, USAID/EA signed Cooperative Agreement No. 623-A-00-05-00320-00 with Family Health International (FHI) entitled “Technical Assistance for Strengthening USAID/EA Response to the HIV/AIDS epidemic in East and Central Africa.” This project was referred to as the “Regional Outreach for Addressing AIDS through Development Strategies (ROADS I) Project” and was branded “SafeTStop”. ROADS I implemented HIV prevention and AIDS care and support services in “hot spot” communities linked by major transport routes across nine countries in East and Central Africa: Burundi, the eastern Democratic Republic of the Congo, Djibouti, Ethiopia, Kenya, Rwanda, South Sudan, Tanzania and Uganda. Beyond building health services and community mobilization structures to promote increased use of HIV/AIDS and health services, a critical component of the project was economic strengthening to address vulnerabilities of the local communities through partnership with local and international businesses called “LifeWorks.” The Roads to a Healthy Future (also known as ROADS II) Project, a five-year Leader with Associates (LWA) award funded on August 1, 2008 and ends on July 31, 2013. This activity was a direct follow-on to the ROADS Project whose ceiling was exceeded well before its completion date due to significant participation from bilateral USAID missions. Roads to a Healthy Future picked up from the mid-point of ROADS and the mid-term evaluation conducted in January 2008 informed the ROADS II design. The ROADS II project is currently in the following countries: Burundi, Djibouti, Kenya, Mozambique, Rwanda, Tanzania, Uganda and Zambia. Countries with previous ROADS programming included: Democratic Republic of Congo (DRC), South Sudan, and Ethiopia. However, since a mid-term evaluation was conducted in January of 2008 for ROADS I, the bulk of this evaluation will focus on ROADS programming from that date forward, but should strive to see the project as a holistic project, since 2005. II. PURPOSE OF EVALUATION The main objectives of this evaluation were to assess overall project performance, and to obtain recommendations for the re-design, or follow-on of this project. The evaluation assessed the outcomes of the ROADS I and II projects in strengthening the response to HIV/AIDS in vulnerable mobile and transient populations along transport corridors in East, Central and Southern Africa and the additional value in integrating Family Planning/Reproductive Health (FP/RH), Maternal and Child Health (MCH) and nutrition interventions into the ROADS projects. Conclusions, lessons learned and recommendations will provide information that will allow USAID/East Africa to: (i) make decisions about the design and specific components of a follow-on project, and, (ii) determine what aspects of the project can be scaled-up by USAID/EA and bi-lateral programs in the region. Specifically, the evaluation will identify ideas on potential models, approaches, activities, and even options for mechanisms. 58 It is important to note that the ROADS project evaluation looked specifically at the achievements of ROADS, the challenges the components of the model and made recommendations for continued programming and re-design. Other studies are being undertaken by ROADS and USAID/EA that are critical pieces of both the ROADS Project and HIV/AIDS and broader health and development needs along the transport corridors. One such study will be ROADS’ internal evaluation of their community organizing model (the community cluster approach), and this evaluation will evaluate the role of the community cluster organizing model in creating and sustaining community ownership of interventions that increase demand and access to HIV prevention, care and treatment in ROADS II project sites in 4 selected countries. Another critical assessment is one that USAID/EA in collaboration with EAC and through AIHD, a regional research institution, is implementing will more thoroughly document services and needs along the wider transport corridor in and around the 5 EAC countries. Specifically, the aim is to document and provide a regional perspective on HIV prevalence and provision of integrated health, HIV/AIDS and reproductive health services at cross-border communities within Eastern Africa. The results of both of the evaluation and the AIHD studies will be integral to design and implementation of broader services in the region, will help inform and shape specific aspects of ROADS and any future re-design and programming, as well as address some critical questions vis à vis sustainability and a locally/regionally owned program/activities/response. Dates of the evaluation site visits and meetings were from Monday May 28th – July 15th, 2012 (with the evaluation report development extended by the ROADS II AOTR through September 2012 due to previously scheduled team members’ schedules). The specific questions to be addressed by this evaluation included (updated with peer review): 1. As a regionally planned and targeted set of activities, in which areas has the ROADS project contributed the most and what key lessons have been learned, both positive and negative? 2. Integration: What was the value-added of the integration components of various project components (FP/RH, MCH, nutrition, economic strengthening etc.) and the lessons learned? 3. To what extent has the ROADS II project increased capacity of local and CBOs to ensure sustainability without project support? 4. To what extent did the ROADS project collaborate with local and other relevant government authorities and with what results? 5. What lessons have been learned in the expansion of the ROADS II project that can inform what aspects of the project can be scaled up or replicated at a bilateral and/or regional level? 59 6. What can available secondary data and epidemiological analyses on the dynamics of the HIV/AIDS epidemic in the region tell us about the strategic role of the ROADS Project? 7. What are some specific ways in which the project has harnessed and utilized innovative approaches and methods in their work and with what results? 8. How effective is the Regional Leader with Associate award model, including but not limited to the following areas: a. Management: Has the implementing partner provided adequate leadership to the Associates to meet country specific needs? b. Communication: What has been the experience regarding sharing and dissemination of best practices and lessons learned between the leader and associates? c. Program: What is the value-added for the bi-laterals of this mechanism? d. Monitoring and Evaluation: What systems and practices are being utilized and how effective have they been in tracking project results and informing future programming? 9. What are the key recommendations for future programming? The Evaluation Team made changes to the specific evaluation questions in a team discussion meeting held to review, clarify, and simplify some of the wording. Also, because the relevant epidemiological data is so sparse or non-existent in the countries where most of the ROADS sites are located, there currently is no way to determine the local or national epidemiological impact of ROADS on a location by location or a country by country basis. Thus the Evaluation Team decided to eliminate the original question 6. as a key evaluation question, as it is largely unanswerable without substantial additional epidemiological data on the national, provincial, and local levels and relatively accurate population group size estimates, which currently do not exist for any of the relevant countries. The evaluation report nonetheless addresses the strategic role of ROADS regionally and within countries where relevant, including in the Recommendations section. The following are the questions refined and used by the Evaluation Team that were disseminated during the evaluation: 1. What have been the major contributions of the ROADS project? 2. Which approaches or methods used by ROADS have been innovative? What have been the results? 3. What key lessons have been learned in ROADS implementation, both positive and negative? 4. To what extent has ROADS:  Increased the capacity of local NGOs and CBOs to ensure programmatic sustainability?  Collaborated with national, provincial or local government authorities? 5. How useful is the ROADS cluster model for:  Integrating various program areas with HIV prevention to add value, such as:  FP/SRH?  MCH? 60  Nutrition?  Economic strengthening?  Integrating other program areas in the future (e.g., MNCH, etc.) or other human rights issues (e.g., MSM)?  Delivering cost-effective interventions?  Ensuring program sustainability? 6. How effective is the Leader with Associate (LWA) award model in the following (or any other) areas:  Management: Has FHI 360 provided adequate leadership to the associates to meet your country-specific needs?  Program: What has been the value-added of this mechanism in your country?  Communication: What has been the experience regarding sharing and dissemination of best practices and lessons learned across the program area?  Monitoring and Evaluation: What M&E systems and practices are being utilized, and how effective have they been in tracking project results and informing program implementation? 7. What are your key recommendations for future programming? 61 ANNEX B. EVALUATION METHODOLOGY This was a descriptive evaluation that included the following activities: Team Briefing: The Evaluation Team held a preliminary briefing with the USAID/EA/RHH Office staff in Nairobi to review the statement of work, revise the SOW and/or develop additional key assessment questions as necessary, and finalize the schedule. The Evaluation Team also held an interview with the RHH Director to discuss the main priorities for the evaluation and expectations for future planning. The ROADS II AOTR on the RHH team joined the Evaluation Team for the site visits, meetings, and interviews held in Burundi, Kenya, and Mozambique. Document Review: The RRH staff provided the Evaluation Team with a package of briefing materials related to the ROADS Projects, both the Leader and Associate awards. The Team members also collected additional documents and materials during the sites visits to the various countries, and the RHH ROADS II AOTR was copied electronically on many of the documents submitted to the Evaluation Team for their review. (A comprehensive list of background documents is included as Attachment 3 and will be updated as a comprehensive list in the final Evaluation Report.) Examples of such documents include: Semi-annual performance reports, special presentations, country and technical briefs, technical frameworks, site assessment briefs, training materials and toolkits, as well as documented best practices. FHI360 Briefing: The Evaluation Team held in-depth preliminary meetings with the ROADS Team, which includes FHI360 as the Prime and key consortium members. The first meeting was held with the Nairobi-based FHI360/ ROADS II Team and consortium members. Subsequent preliminary meetings were held with country-specific ROADS II implementing teams. FHI360 compiled a packet of relevant materials, including the M&E framework and tools, site assessment reports, studies, BCC/messaging materials, ROADS country program descriptions, and photos of relevant project aspects. The ROADS II Team was requested to meet or communicate with the Evaluation Team as needed. Other Key Informant Interviews: The Evaluation Team met key stakeholders (a preliminary list of key stakeholders is included below and will be included as Attachment 5 as it is developed by the evaluation team) to conduct qualitative, in-depth interviews. The interviews were loosely structured but followed a list of key discussion issues and questions as a guide. Whenever possible, the interviewers were conducted face-to-face informants and interview notes were taken. In addition to interviewing the key stakeholders, various USAID and ROADS staff were included in the country visits to conduct interviews with key informants, community members, and government and community stakeholders. As it was impossible to interview all key stakeholders in person in ten countries, the Evaluation Team conducted telephone interviews with informants in the countries/sites not visited. Country Site Visits and countries of focus:  The ROADS evaluation shall include three different types of programming of the ROADS project:  Project activities directly funded by USAID/East Africa: Djibouti, Uganda, Burundi (FP/RH), and Tanzania (FP/RH).  Programs managed through USAID bilateral (country) missions: Tanzania, Kenya, Rwanda, Burundi, Mozambique and Zambia.  Project sites that began as ROADS I but have since been subsumed under bilateral country programs (Uganda, Kenya, DRC, and South Sudan). 62  USAID/EA in collaboration with the bilaterals selected 1 or 2 countries from each type of program described above, and the evaluation team shall look at these countries more in￾depth than the others and this shall include site visits as well.  The selected countries to be visited by the Evaluation Team were: Burundi, Kenya, Mozambique, Rwanda, and Tanzania (one-half of the Evaluation Team went to Mozambique, and one-half went to Tanzania due to time constraints).  In addition to interviews with USAID staff and other key national stakeholders in these countries, the Evaluation Team was expected to visit at least two to four overall ROADS sites as stated above to assess the implementation of activities and to interview local participants (including the beneficiaries of the program) and stakeholders. The sites were selected in consultation with the USAID/EA and the ROADS team. Stakeholders are broadly defined to include (but not limited to): relevant USAID bi-lateral and regional Mission staff, relevant regional bodies, government counterparts (national and district levels), representatives from community based organizations, ROADS Project staff, community members, health care workers, and beneficiaries of the program and representatives from key target populations. 63 ANNEX C. LIMITATIONS OF THE EVALUATION There were several limitations associated with this evaluation:  The qualitative nature of the evaluation meant that the impact of the project can only be determined through inference based on the available data, which is limited, the feedback from USAID and relevant government staff in the countries visited or through teleconferences or email with USAID staff in the countries the Team was unable to visit, and impressions gleaned during the site visits, which were very limited given the wide geographic coverage of the program. Moreover, the lack of an overall project baseline study by location precluded the use of more rigorous study methodologies. However, the availability of some site-specific assessments previously undertaken by the program, some behavioral monitoring surveys, and some feedback from population group members themselves permitted limited investigation and review of the impact of the project on the target populations. Consequently, the Evaluation Team recommends that a separate impact evaluation be undertaken in the future if the AIHD study, mentioned above, does not elicit adequate impact information.  Additional limitations include the complexity and size of the program; the number of countries involved (ten) and the inability to visit most of the project sites, even in the limited number of countries the team was able to visit; the language barriers across the countries; the very limited time allotted for the assessment with Evaluation Team members working together (three weeks); and, the various logistical challenges involved in any evaluation, and especially this one given the very short time frame, the range of program interventions, the broad spectrum of the target groups, and the large geographic spread. The logistical imitations included: travel, such as limited flight availability and duration of travel to disparate and far-flung project sites via air to cities and/or by road to rural areas; the costs due to the types of travel and accommodation needed for ten or more people at times; the availability of USAID program, government, and ROADS II program and implementing partner staff and target group members for meetings, interviews, international conference calls, etc).  Comparisons between country programs, including the impact and strategic role of the project within or compared to bilateral country programs was not possible to determine in a systematic way or based on the data and the amount of information available about each. Each country program is different across the ten countries where the project is or has been operational, and each ROADS program in each country differs somewhat from the other countries due to the range of interventions implemented, the specific needs by location, range of geographic coverage, provincial prioritization based on government buy-in and, indeed, differing national priorities. Moreover, the project started at different times, including in different years in the countries. Thus there was an inability to determine the strategic role of the project in each of the ten countries in an in-depth, accurate, and fair manner, while basic agreement was reached on the strategic role of the project in some of the countries through meetings, discussion, information and data review, and basic analysis.  The evaluation took place a year before the projected end of the program; thus, final data was not available for any aspect of the program in the countries where implementation continues.  The Evaluation Team requested a significant amount of additional data and information than was initially provided to the Team. The additional data and information required 64 extensive time and effort to compile by various ROADS II staff members in the countries and at ROADS HQ. Thus the additional data and information gathering and compilation time extended the amount of time needed for data and information analysis off-site and individually by the Evaluation Team, beyond the initial expectation for the end of the evaluation and the submission date of the final report. 65 ANNEX D. LIST OF INDIVIDUALS MET OR INTERVIEWED BY THE EVALUATION TEAM Organization Name Title/position US Agency for International Development/East Africa and USAID/Kenya USAID Kristen Ruckstuhl Prevention Advisor, USAID/EA Julia Henn Health Team Director, USAID/EA John Power Deputy Mission Director, USAID/EA Wairimu Gakuo Strategic Information Advisor, USAID/EA René Berger HIV/AIDS Team Leader, USAID/Kenya Emma Mwamburi Prevention Advisor, USAID/Kenya Organization Name Title/position Burundi FHI 360/ROADS Dr Antoine Barutwanayo Family Planning/Gender-Based Violence (FP/GBV) Program Officer Dr Bède Matituye Clinical Services Program Officer Didace Ngabonziza Program Assistant Dr Martin Ngabonziza Country Director Eslon Nduwayo Executive Assistant Emmanuel Nikoyagize Community Program Officer Thierry Nininahazwe FP/GBV Program Assistant Majoric Nshimirimana M&E Technical Officer Kayanza Province Concilie Gahungere Technical Advisor of the Provincial Committee for the Fight against AIDS Jean Claude Mpawenimana Governor of Kayanza Province National Council for the Fight against AIDS (SEP/CNLS) Appolinaire Kavungerwa Monitoring, Evaluation and Planning Expert Consolate Nduwarugira Civil Society Reinforcement Expert Dr Damien Nimpagaritse Technical Director Thérèse Ntahompagaze Prevention/PRIDE Project /Global Fund Expert Albert Ntiringaniza Pharmacist Alexis Nzeyimana Civil Society Expert Dr Jean Rirangira Executive Secretary of the National Council for the Fight against AIDS Kayanza Health Center Bénigne Bashushana Laboratory Technician Spès Mbonabuca Head of Kayanza Health Center Consolate Nduwayezu Head of FP Services Tétine Solange Niyibizi Antenatal Care Provider Kayanza Health Center Gender￾Based Espérance Bantegeyahaga Provincial Coordinator of Family Development Center Marie Goreth Gakobwa GBV Community Volunteer Sabine Habiyambere GBV Community Volunteer Marie Goreth Kayobera GBV Community Volunteer 66 Organization Name Title/position Burundi Violence Unit Sylvie Kayobera GBV Community Volunteer Marie Goreth Minani GBV Community Volunteer Lucie Niyibaruta GBV Community Volunteer Gertrude Niyibigira GBV Community Volunteer Pélagie Ntawe GBV Community Volunteer Domitile Ntuyahaga GBV Community Volunteer Immaculée Singirankabo GBV Community Volunteer Kayanza Health Center Family Planning Unit Anne Marie Citegetse FP Community Health Worker Siméon Hakizimana FP Community Health Worker Bernadette Nahimana FP Community Health Worker Eugénie Niyonsaba FP Community Health Worker Marie Nsaguye FP Community Health Worker Richard Sindihebura FP Community Health Worker RBP+ Kayanza Evelyne Bigirimana HBC Community Volunteer Christine Bizimana Care Services Officer Joséphine Bucumi HBC Community Volunteer Marcienne Bucumi HBC Community Volunteer Oscar Ciza HBC Community Volunteer Abel Hicumusi HBC Community Volunteer Daphrose Hitimana HBC Community Volunteer Mariane Hatungimana HBC Community Volunteer Elie Kabonetse Chief of RBP + Kayanza provincial office Concilie Kayobera HBC Community Volunteer Aline Mukeshimana HBC Community Volunteer Ancile Mukeshimana HBC Community Volunteer Béatrice Mukeshimana HBC Community Volunteer Savela Nahabandi HBC Community Volunteer Emmanuel Ndayizeye Accountant Azela Ndereyimana HBC Community Volunteer Sostène Ndereyimana Project Supervisor Julienne Nganyirimana HBC Community Volunteer Cyprien Nibaruta HBC Community Volunteer Spéciose Ninteretse HBC Community Volunteer Caritas Ntahonkiriye HBC Community Volunteer Celestin Ntakarutimana HBC Community Volunteer Josélyne Ntakirutimana HBC Community Volunteer Anastasie Ntanyungu HBC Community Volunteer Agnès Nyabenda HBC Community Volunteer Floride Nyabenda HBC Community Volunteer Elie Nyandwi HBC Community Volunteer 67 Organization Name Title/position Burundi Goreth Nyandwi HBC Community Volunteer Vénérande Nyanzira HBC Community Volunteer Aloys Nzigamasabo HBC Community Volunteer Samuel Sebaganji HBC Community Volunteer Marie Rose Tangishaka HBC Community Volunteer Organization Name Title/Position Kenya Nairobi FHI 360/ROADS Chris Degnan Associate Director - Private Sector Initiatives Leanne Kamau Associate Program Officer Boniface Kitungulu Associate Director - Strategic Information Dorothy Muroki Project Director Patrick Muthee Associate Director - Contracts and Grants Eunice Mutisya Behaviour Change Communications Officer Missie Oindo Senior Program Officer Tom Owaga Associate Director - Finance Robert Ritzenthaler Deputy Project Director PATH John Waimiri Project Director, Operation ARIFU, Uniformed Services Daniel Were Technical Advisor Howard University Andrew Maranga Regional Field Coordinator DAI Eliud Wakwabubi Livelihoods Advisor Mariakani FHI 360/ROADS George Dzombo Program Officer Ministry of Health Hussein Dima District AIDS and STIs Coordinator (DASCO) Dr. Lizzy Gathua District Pharmacist Dr. David Mang'ongo District Medical Officer of Health (DMOH) Wellington Mulewa Pharmacist Edward Mwamuye District Health Education Officer Ray Vespus Pharmacist APHIA Plus partners Mariakani Youth Centre Sadik Chimera Coordinator Solidarity with Women in Distress (SOLWODI) George Odhiambo Coordinator PSI Sila Mulwa Coordinator Mariakani Magnet Theatre Salim Swaleh Coordinator 68 Organization Name Title/position Burundi Pharmacies Immam Chemist Mariam Ibrahim Pharmacist Mariakani Boaz Obote Pharmacist Venture land Matano Maraga Pharmacist Immam Chemist Rose Samba Pharmacist Kenya Name Title/position Palm Land Chemist Joyce Wambui Pharmacist ROADS implementing partner - Kenya Long Distance Truck Drivers &Allied Workers Union (KLDTDAWU) Romano Gichinga Assistant Coordinator Abubakar Kabocha Truck driver Joseph Kahindi Truck driver Ruth Karanja Coordinator Paul Mwangi Truck driver Jorum Mwaura Truck driver Salim Mwero Truck driver Athman Nganga Truck driver Tonny Nganga Truck driver Fed Salim Truck driver Felix Senja Truck driver John Wamalwe Truck driver Volunteers trained in general peer education Maria Akinyi Community Volunteer Sammy Charo Community Volunteer Jemmimah Kilonzi Community Volunteer Mutuku Kitili Community Volunteer Omar Komu Community Volunteer Felister Mbula Community Volunteer Matano Mthami Community Volunteer Lucy Mwendee Community Volunteer Rehema Ndzame Community Volunteer Esther Njoki Community Volunteer Mutisya Wambua Community Volunteer Edith Wangari Community Volunteer Grace Wendy Community Volunteer Volunteers trained in Alcohol and Substance Abuse Hermissi Abubakari Community Volunteer Maria Akinyi Community Volunteer Salama Chuda Community Volunteer Patrick Chitty Community Volunteer David M Dalu Community Volunteer Hamza Juma Community Volunteer 69 Organization Name Title/position Burundi Jemmimah Kilonzi Community Volunteer Joyce Mkalu Community Volunteer Agnes Mtua Community Volunteer Charles Munene Community Volunteer Ali Nyota Community Volunteer James N. Nyota Community Volunteer Abubakari M.Tayari Community Volunteer Organization Name Title/Position Mozambique Maputo FHI 360 Paulo Ambasse M&E Assistant FHI 360/ROADS Paulo Araújo Program Officer Dr. Jorge Blanco Country Director Caximo Caximo M&E Officer Shirley Ko Roads Country Manager Maltez Mabuie Community & Mobilization Officer Daniel Mamberro Site Coordinator – Munhava Stella Manjate Site Coordinator – Luis Cabral and Trevo Julião Matsinhe Economic Strengthening Officer Nilton Mazitemba Site Coordinator – Ressano Garcia Stephen Mucheke Senior Technical Officer - BCC Ana Paula de Sousa Gender Officer Maria Isabel Torres Clinical Officer Ministry of Health Dr Marina Kangenge Provincial Director of Health - Sofala Ministry of Transport Luis John Department of Transport Cordinator, Chief of AIDS Department - Sofala Munhava-Beira Youth Cluster Margarida André Peer Educator Zito Costa Peer Educator Armais Draiva Coordinator, Videc Youth Group Ancha Emilia Peer Educator Luis Francisco Peer Educator Claudina Frijão Accountant José João Peer Educator Isabel Joaquim Peer Educator 70 Organization Name Title/Position Mozambique Parafino José M&E Assistant Izequiel Lavo Peer Educator Helena Luís Peer Educator Mário Mavundo Peer Educator Zelia Silva Peer Educator Arminda Victor Peer Educator Women’s Cluster Feliciana Alberto Peer Educator Rosa Antonio Peer Educator Cristina Chane Peer Educator Alcinda Chiposse Peer Educator Quinita Fernando Peer Educator Joana Filipe Accountant Maria João Peer Educator Felizarda José Peer Educator Cristina José Peer Educator Claudina Luisa Peer Educator Paulo Simango M&E Assistant Pedro Viajeiro Coordinator, Luis na Comunidade Group Accumulating Savings and Credit Associations (ASCAS) Ana Maria Antóno Member Maria Berta Member Inácia Ferrão Member Dominga João Member Paulina Lourenço Coordinator, Luis na Comunidade Group Rita Merça Member Elisa Luis Pereira Member Naquene Paulina Member Isabel Sebastião Member Mariana Francisco Soares Member Maria Verónica Member People Living with HIV (PLHIV) support group Ana Maria António Peer Educator Vitor João Amade Peer Educator Olga Bizique Peer Educator Ermelinda Fernando Peer Educator Inácio Ferrão Foia Coordinator, Grupo de Apoio Luisa Fernando Peer Educator Maria José Peer Educator Flora José Peer Educator Cristina José Peer Educator 71 Organization Name Title/Position Mozambique Amélia Nhamato Peer Educator José Roque Peer Educator Men’s Cluster Isabel Alberto Coordinator, Fambizanai Mens Group Santos Alberto Peer Educator Chico Arnaldo Peer Educator Alberto Charles Peer Educator José Charles Peer Educator António Dango Peer Educator Imen Fred Accountant José Manuel Peer Educator Soia Manuel Peer Educator Victor Manuel Peer Educator Lourenço Sola Peer Educator Maria Vicente Peer Educator Trevo Women’s Cluster Nomelia Mazuze Coordinator, Capaz Women Group Leila Abdula Peer Educator Albertina Abilio Peer Educator Rosa DeolindaDeolinda Peer Educator Francisco Domingos Accountant Rosita Guambe Peer Educator Palmira Jeremias Peer Educator Elsa Macuacua Peer Educator Isaura I.Monica Peer Educator Abel Mungoi Peer Educator Rosa Namburete Peer Educator Izidro Nhangala M&E Assistant Beatriz Novele Peer Educator Alzira Sebastiao Peer Educator Hortencia Upiliamo Peer Educator Julia Vasco Peer Educator Lucrencia Vasco Peer Educator Verginia Zita Peer Educator Youth Cluster Albino Mondlane Coordinator, Ajota Pejota Youth Group João Vilanculos M&E Assistant Narciso Zavala Accountant 72 Organization Name Title/Position Rwanda Kigali FHI 360/ROADS Anne Marie Ayinkamiye Technical Officer - Community-Based Prevention and Mitigation Jean Paul Balinda Technical Officer - Community-Based Prevention Didier Rukabu Kamali Country Manager Jean Baptiste Mugabo Technical Officer - Economic Strengthening Protais Ndabamenye Chief of Party and Francophone Countries Coordinator Phocas Ntahorugiye Technical Officer - Nutrition Anastase Nzeyimana Technical Officer - FP/MCH Rwanda Bio-medical Center (RBC) Dr. Anita Asimwe Deputy Director General Florida Mutamuriza In charge of Private sector and Civil Society Organizations (CSOs) Dr. Sabin Nsanzimana Head of HIV/AIDS,STIs and other Blood Borne Infections Division Program Coordinator Mr. Emmanuel Rusine Coordinator for Regional Project and Great Lakes Initiative on AIDS (GLIA) Focal Person Kicukiro District Emmérence Gatera Head of the Health Unit Jean Damascène Kayiranga Executive Secretary of Kanserege Cell Theophile Niwemutoni Social Affaires in Charge in Kigarama Sector Clauthilide Mukamanzi District HIV Activity Coordinator Florence Uwayisaba Vice Mayor - Social Affairs and Health ROADS implementing partners ASOFERWA (Low Income Women’s Cluster) Flavie Kanyemera Technical Assistant in charge of Care and Support Chantal Kwizera Technical Assistant in charge of Prevention Joy Kwizera Economic Strengthening Focal Person Claudette Muhimpundu Director of Administration and finance Cassilde Mukamakombe Technical Assistant in charge of Prevention Jacques Niyonshuti Accountant Appolinaire Nshimiyimana Managing Director Livin Rurangwa Technical Assistant in charge of M&E 73 Organization Name Title/Position Rwanda Ngabo J.M Rwurira- Economic Strengthening Focal Person Louise M Tuyisenge Technical Assistant in charge of MVC Program Association des Démobilisés pour le Développement, la lutte contre le SIDA et la Promotion de la Culture Rwandaise (ADSPCR) Youth Cluster Said Mazimpaka Mechanics Association representative Gloriose Mukeshimana Economic Strengthening Focal Person Sandra Mutabazi Peer Educator Mariam Nikuze Membre du club des jeunes filles Jonathan Niyonzima ADSPCR Coordinator Adele Nyirabititaweho Cashier Simon Turikunkiko ADSPCR Representative Gikondo FP Post Naomie Musabirane In charge of the Secondary post Violette Kantamati Representative of Community Volunteers Pascal Muyango Kayitaba FP Supervisor Marie Aimée Umutesi FP Provider Gatsata Community Center Philbert Romanus Mlyuka Truck drivers representative Sandra Mutabazi Peer Educator Chantal Nyakubyara Peer Educator Goretti Nyirabarigira M. Low Income Women Cluster Steering Committee Rusizi District Authorities Marie Alice Batamuriza Kamembe Sector Social Affairs Patrick Muturutsa Data Manager of District Health Unit Emmanuel Ndamuzeye Head of District Health Unit Françoise Nirere Vice Mayor - Social Affairs Peter Pokezi Tara Cell Executive Secretary Herman Rwagasine Muganza Sector Social Affairs Coopérative des Femmes de Mururu (COFEM) - ROADS implementing partner Low Income Cluster Jeanne D'arc Mukakayijuka Technical Assistant in charge of MVC Program Innocent Niyonsenga Technical Assistant in charge of Prevention Fabien Ntagwabira Program Coordinator Djohari Nyiraneza Economic Strengthening Focal Person Antoinette Nyirazaninka COFEM Representative Abishizehamwe (PLHIV Cluster) Jean Minani Technical Assistant in charge of MVC Program Phenias Mutabazi Program Coordinator Jean Damascene Economic Strengthening Focal 74 Organization Name Title/Position Rwanda Ndayizeye Person Beatrice Nyiranzeyimana Twizigamire GSLA President Ibrahim Uwimana Tinyuka Cooperative Representative Gihundwe HC Desiré Baritonda Data Manager Imelde Imaniliho Head of the Health Center Bugarama Islamic Health Center Shamilla Nyirandagijimana Deputy Head of Health Center Rusizi Health Center Ferdinand Macumu Deputy Head of Health Center Vestine Nyirandikubwimana Health Provider at Tara Secondary Post Organization Name Position/Title Tanzania FHI 360/ROADS Abdallah Mkumbwa Site Coordinator - Port of Dar es Salaam Melchiade Ruberintwari Country Manager Fredrick Ogenga Technical Officer - Economic Strengthening Ministry of Health (MOH)/Tanzania AIDS Commission (TACAIDS) Dr Raphael Kalinga Director of Evaluation and Research Port of Dar es Salaam COTWU (T) Samson Kabana Truck Driver/Peer Educator Mariki Kondo Truck Driver/Peer Educator Friday Likalala Truck Driver/Peer Educator Samson Lussumo Resource Centre Coordinator Mrace Mahena Truck Driver/Peer Educator Hilda Makene VCT Counsellor Magreth Millinga Truck Driver/Peer Educator Yusuf Mituka Truck Driver/Peer Educator Chisano Mwampashi Truck Driver/Peer Educator Nicholaus Mwashala Truck Driver/Peer Educator Monica Mwinuka Truck Driver/Peer Educator Mwanaharusi Nassoro Truck Driver/Peer Educator Bokorani Upendo Bakari Abdul Peer Educator Teddy Antoni Community Volunteer Hoyce Chabuluma Community Volunteer Miski Chaurembo Peer Educator Kazumari Chitwanga Community Volunteer 75 Organization Name Position/Title Tanzania Stephen Gregory Peer Educator Focal Person Hussein Jaha Community Volunteer Andrew Jimmy Community Volunteer Nuru Juma Peer Educator Fidea Kalindaga Peer Educator Hamida Kapilima Community Volunteer Sybus Kikungwe Coordinator Rachel Luhega Community Volunteer Warange Marungu Community Volunteer Ramadhani Mataluma Community Volunteer May Mchomvu Peer Educator Ramadhani Mohamed Peer Educator Wastara Omary Peer Educator Doto Potopoto Peer Educator Madina Abdul Qadir Peer Educator Zalia Yasini Community Volunteer Tanzania Youth AIDS Control Program (TYACP) Youth Cluster Ally Abdallah Community-Based Distributor Fatuma Fidas Community-Based Distributor Shabani Harife Community-Based Distributor Tatu Kitenge Community-Based Distributor Lilian Magembe Community-Based Distributor Rehema Makanjila Peer Educator Focal Person Safiel Mgonja Coordinator Hafidh Mohammed Community-Based Distributor Latifa Moshi Community-Based Distributor Hadija Mtegame Community-Based Distributor Jamila Nguche Community-Based Distributor Pili Rashid Community-Based Distributor Pendo Warange Community-Based Distributor Khadija Wawa Community-Based Distributor Temeke Hospital Cristabella Mwanda Maternal Child Health (MCH) Coordinator Makambako Local government and government departments Godwin L.Benne Township Executive Officer￾Makambako Township Authority Averino M.Chaula District Social Welfare Officer Emilia Chiwanga Secretary To The District Medical Officer Michael K.Haule Council HIV/AIDS Coordinator Dr. Maria Lupenza Njombe District Medical Officer 76 Organization Name Position/Title Tanzania Nicko Mandele District Community Development Officer Secilia Maseko District Reproductive and Child Health (RCH) Officer Tasilo B. Mdamu District AIDS Council Coordinator A.S.Mwampanga District Co-operative Officer Jonathan Siha Njombe District TB & Leprosy Coordinator Eng. Steven Siha Ag. District Executive Director - Njombe District Council Makambako Health Center Rose Chungu PLHIV HBC Tracking Coordinator Ritha Gadau PLHIV HBC Tracking Coordinator Hosea Kaguo PLHIV HBC Tracking Coordinator Christina Kisakali PLHIV HBC Tracking Coordinator Florence Mgaya CTC-Nurse Dr. Magreth Msasi Doctor in-charge Dr. John Salila Comprehensive Treatment Center (CTC) Head Mlimani GSLA Raheli Kalinga Chairperson, Evangelical Lutheran Church in Tanzania (ELCT) Southern Diocese Regina Kalinga GSLA member Luth Kibiki GSLA member Rose Kibiki GSLA member Upendo Kibiki GSLA member Jester Kigahe GSLA member Elizabert Kimbawala GSLA member Diana Kinyunyu GSLA member Elekia Kinyunyu GSLA member Sophia Kinyunyu Secretary, ELCT Southern Diocese Janet Kisoso Accountant, ELCT Southern Diocese Aneda Makweta GSLA member Mariam Malega GSLA member Bosco Mayanga GSLA member Leokadia Mdemu GSLA member Blantina Mdugo GSLA member Pielina Mfilinge GSLA member Rejina Michael GSLA member Ester Mkola GSLA member Sijali Mkula GSLA member Queen Mlamka GSLA member 77 Organization Name Position/Title Tanzania Rebeca Mlonganile GSLA member Onesmo Myinga GSLA member Neema Ndendya GSLA member Meck Ngili GSLA member Furaha Nyagawa GSLA member Jonisia Nyaulingo GSLA member Scola Peter Sanga GSLA member Frida Simime GSLA member PLHIV Cluster Rehema Arone PLHIV cluster member Paulina Chaula PLHIV cluster member Sophia Chaula PLHIV cluster member Rose Chungu PLHIV cluster member Asheli Gadau PLHIV cluster member Ritha Gadau PLHIV cluster member Ayubu Homange PLHIV cluster member Hosea Kaguo Chairperson, JUHUDI Post Test Club (PTC) Frola Kawogo PLHIV cluster member Jester Kigahe PLHIV cluster member Pelesi Kihungu PLHIV cluster member Fednand Kilamlya PLHIV cluster member Enelika Kilasi PLHIV cluster member Christina Kisakali PLHIV cluster member Maria Kiyaulilo PLHIV cluster member Tumaini Kunyumba PLHIV cluster member Leokadia Kyando PLHIV cluster member Eliza Licheka PLHIV cluster member Scola Lupande PLHIV cluster member Regina Matama PLHIV cluster member Venelanda Matandala PLHIV cluster member Dominika Mavika PLHIV cluster member Chezalina Mawona PLHIV cluster member Lozadei Mbata PLHIV cluster member Amina Mbogela PLHIV cluster member Nazalena Mbugi PLHIV cluster member Ainesi Mfilinge PLHIV cluster member Agrista Mgaya Accountant, JUHUDI PTC Fabiola Mgaya PLHIV cluster member Jastine Mhavile PLHIV cluster member Scola Mhelela PLHIV cluster member Enea Mhepela PLHIV cluster member 78 Organization Name Position/Title Tanzania Asha Mlango PLHIV cluster member Emelia Mng’ong’o PLHIV cluster member Andrew Mpagike PLHIV cluster member Ambokile Msigala PLHIV cluster member Yusuph Mtelewa PLHIV cluster member Grace Mtweve PLHIV cluster member Frida Mwakasala PLHIV cluster member Venace Ngimbuchi PLHIV cluster member Joshua Nyamba PLHIV cluster member Duwani Nyome PLHIV cluster member Helena Pila Secretary, JUHUDI PTC Semeni Salehe PLHIV cluster member Amina Sanga PLHIV cluster member Hezron Sanga PLHIV cluster member Ritha Sanga PLHIV cluster member Victoria Sanga PLHIV cluster member Miraji Sengele PLHIV cluster member Fatuma Tagalile PLHIV cluster member Lemija Waya PLHIV cluster member Staff and volunteers, SafeTStop, Mariakani, Kenya 79 ANNEX E. EVALUATION SITE VISIT SCHEDULE ROADS Project Evaluation: Schedule for Mozambique Site Visit May 30th ‐ June 3rd, 2012 Time Theme and Activities Day 1: Wednesday May 30th 07.50 ‐ 10.45hrs Evaluation Team travel to Maputo from Nairobi; hotel check‐in at Serena Polana Hotel 12.00 ‐ 13:00hrs Briefing with ROADS Mozambique (Estradas); USAID/Mozambique to attend 13.00 ‐ 14.00hrs Lunch Break 14.00 ‐ 16.00hrs Sexual prevention Visit to Trevo for discussions with female sex workers and low‐income women involved in ESTRADAS community cluster organizations Day 2: Thursday May 31st   6.45 ‐ 10.30hrs Flight to Beira 11.30 ‐ 12.30hrs ROADS community organizing approaches  and participation of local leadership   Meeting with local community leaders at a community center.   12.30 ‐ 13.30hrs Lunch Break 13.30 ‐ 14.30hrs Sexual prevention cont’d  Visit Beira SafeTStop Resource Centre     Meet and hold discussion with youth cluster committee 14.30 ‐ 15.30hrs Sexual prevention cont’d Discussion with sex workers   15.30 ‐ 17.00hrs Immediate  Social Networks Discussion with women cluster at the cluster office     Day 3: Friday 1st June   8.30 ‐ 9.30hrs Government collaboration   Discussion with  the Direcao Provincial de Saude (Provincial Director of Health) at Beira Provincial Hospital 9.30 ‐ 11.30hrs Male Involvement Discussion with the Beira  Men‘s Cluster at the community center 11.30 ‐ 12:45hrs HIV Prevention and Care for PLHIV Meet and conduct discussion with a PLHIV support group at the community center 12.45 ‐ 14.00hrs Lunch Break 14.00 ‐ 16.00hrs   Multi‐sectoral involvement of Government of Mozambique   Discussion with the Ministry of Transport / transport companies Steering Committee for HIV Prevention   21.05 ‐ 22.40hrs Flight to Maputo Day 4: Saturday  June 2nd     80 ROADS Project Evaluation: Schedule for Tanzania Site Visit May 29th to June 3rd, 2012 Time Theme and Activities Day 1: Tuesday, May 29th   22.10 ‐ 23.25hrs Flight Nairobi to Dar es Salaam (Ishrat Hussein) Day 2: Wednesday, May 30th 9.00 ‐ 10.00hrs Briefing with FHI 360 Tanzania ROADS key staff; country programme overview 10.30 ‐ 11.30hrs Briefing with USAID Tanzania 11.30 ‐ 13.00hrs Sexual Prevention Strategies ‐  FSW Intervention (regional lessons)  Discussion with female sex workers in Port of Dar 13.00 ‐ 14.00hrs LUNCH 14.00 ‐ 17.30hrs Sexual Prevention Strategies cont’d… ‐ SafeTstop Resource Center Visit to Port of Dar Resource Centre: discussion with truckers Addressing Root Causes: Gender Based Violence,  Alcohol  Abuse    Discussions with clusters on GBV and FP  Discussions with clusters on alcohol counseling   18.05 – 19.20hrs Flight Nairobi to Dar es Salaam Day 3: Thursday, May 31st 7.00‐16.00hrs Drive to Makambako Day 4: Friday, June 1st 9.00‐13.00hrs Care and Support Continuum  Visit to PLHIV cluster  Economic Strengthening    Visit to Makambako Health Centre 13.00‐14.00hrs LUNCH 14.00‐17.30hrs Collaboration with local leadership, Government of Tanzania    Njombe District Headquarters meeting with the ministries of Health, Community Development / Cooperative Development, Agriculture and Livestock officials Day 5: Saturday, June 2nd   7.00‐16.00hrs Drive to Dar es Salaam Day 6: Sunday, June 3rd   08.25 – 09.40hrs Flight to Nairobi (Ishrat Hussein) 81 ROADS Project Evaluation: Schedule for Burundi Site Visit June 3rd ‐ 6th, 2012 Time Theme and Activities Day 1: Sunday,  June 3rd 11.35‐01.45 Arrival times of USAID team from Nairobi (KR: 15.30‐17.25) (WP: 23.50‐01.45) (M O’G: 15.30‐17.25) (IH: 10.55‐ 11.35) Day 2: Monday , June 4th 8.00 ‐ 9.00hrs Presentation on ROADS Burundi Program 9.15 ‐ 10.00hrs Briefing with USAID at Burundi Mission 10.15 ‐ 11.00hrs Meeting with Ministry  of Public Health and the Fight Against HIV/AIDS /   SEP CNLS ( National AIDS Commission) 11:00 ‐ 14:00hrs Travel to Kayanza, Hotel Check‐in and Lunch 14.00 ‐ 16.30hrs HCT, PMTCT, PT, FP/GBV  Visit to Kayanza Health Center    Interact with the community on  FP/GBV at Kayanza Health Center Day 3: Tuesday, June 5th 8.30 ‐ 9.15hrs Local Collaboration with Government of Burundi Meeting with Kayanza Province Governor and Head of Provincial Health Office at Governor’s office. 9.15 ‐ 11.00hrs HIV and Health Service Integration Visit Kayanza Hospital   11.00 ‐ 13.00hrs Care and Support and / or FP/RH in the context of GBV  Presentation on home based care at RBP Plus office  Interact with PLHIV volunteers in Muruta commune 13.00 ‐ 14.00hrs Lunch at Paradise Hotel , Kayanza 14.00 ‐ 16.00hrs Departure for Bujumbura    Day 4: Wednesday, June 6th 08.10 – 08.45hrs 12.25 – 16.25hrs Flight to Kigali (WP + MO’G) Flight to Nairobi: (KR) 82 ROADS Project Evaluation: Schedule for Rwanda Site Visit June 6th to 9th, 2012 Time Theme and Activities Day 1: Wednesday, June 6th 08.10 ‐ 08.45hrs 09.35 – 10.45hrs Flight Bujumbura to Kigali (Billy Pick + Ishrat Hussein) Flight Nairobi to Kigali (Wairimu Gakuo) 10.00 ‐ 11.00hrs Briefing with FHI 360 Rwanda ROADS key staff; Country Program overview 11.00 ‐ 12.30hrs Briefing with USAID Rwanda 12.30 ‐13.00hrs Government‐led programming  Visit to RBC: Meeting with RBC Office government   13.00 ‐ 14.00hrs LUNCH 14.00 ‐ 15.00hrs Government‐Community planning and implementation processes  Visit to Kanserege Cell Office     Discussion with local government officials  Discussions with clusters in Gikondo   15.00 ‐ 17.30hrs Integrated services – Family Planning  Visit to Gikondo Family Planning post  Discussion with FP clients  Cluster discussion on lessons from introducing new technical area into programming Day 2: Thursday, June 7th 9.00 ‐ 13.00hrs An integrated M&E platform  ROADS M&E system overview  Visit to peer educators / Health Centre: data tool development, data collection process  Visit to cluster IP office: data entry and analysis processes, examples of key learning from data analysis and how to use data 13.00 ‐ 14.00hrs LUNCH 14.00 ‐ 17.30hrs Working with MARPs and hidden populations  Truck driver parking visit  Visit to HTC service targeting truck drivers and local populations; group discussion with men  Visit to Gatsata Youth Cluster: Girls intervention Day 3: Friday, June 8th 06.55 ‐ 07.30hrs Flight to Rusizi (check‐in by 04.55) 9.00 ‐ 10.30hrs Local collaboration with GoR Visit to District HQ: Meeting with ministries of Health, Community Development / Cooperative Development, Agriculture and Livestock officials 10.30 ‐ 13.00hrs Applying innovation: reaching FSWs  Discussions with FSW group   83  Visit to Gihundwe Health Centre 13.00 ‐ 14.00hrs LUNCH 14.00 ‐ 17.30hrs Care and support continuum Drive to Bugarama:  Visit to Bugarama Islamic Health Centre  Economic Strengthening  Visit to care and support groups (PLHA cluster, OVC) Day 4: Saturday,  June 9th 9.00 ‐ 12.00hrs Cross border services    Drive to Rusizi II border    Visit HTC services  Discussion with truck drivers 12.00 ‐ 13.00hrs LUNCH 15.25 – 15.55hrs Return flight to Kigali 17:00 – 18.00 Evaluation Team Meeting with USAID/Rwanda Mission Director Dennis Weller 21.55 – 23.30hrs TBD Return flight to Nairobi (Billy Pick + Mary O’Grady) Onward to US (Ishrat Hussein) ROADS Project Evaluation: Schedule for Kenya Site Visit June 10th to 11th, 2012 Time Theme and Activities Day 1: Sunday 10th June 18.30 ‐ 19.30hrs Flight Nairobi to Mombasa Day 2: Monday 11th June 08.00 ‐ 09.30hrs Travel to Mariakani 09.30 – 11.00hrs Country‐led development process for BCC messages  Discussion with GoK and targeted community participants in national message development process 11.00 ‐  11.45hrs Linking with pharmacies and drug‐shops along the transport corridor  Discussion with pharmacy network members 11.45 – 13.00hrs Adoption of key BCC messages by APHIAplus partners  Discussion with other USG partners in consistent local utilization of national messages 13.00 – 14.00hrs Lunch Break 14.00hrs Travel to Mombasa 16.10 – 17.10hrs Flight to Nairobi 84 ANNEX F. LIST OF DOCUMENTS REVIEWED 1. Regional overview brief 2. Mid-term evaluation recommendations and ROADS response 3. Country briefing documents a. Program briefs (Burundi, Djibouti, Kenya, Mozambique, Rwanda, Tanzania, Uganda and Zambia) b. Site assessment summaries (Burundi, Djibouti, DRC, Ethiopia, Kenya, Mozambique, Rwanda, South Sudan, Tanzania, and Uganda) c. UNAIDS Epidemiological factsheets for the relevant countries, as available 4. Technical proposals a. ROADS I b. ROADS II 5. Technical briefing papers a. Sexual prevention b. HTC c. HBC/OVC d. PMTCT/ART e. FP/RH/GBV and MNH f. Economic strengthening g. Gender h. Alcohol i. Capacity building 6. M&E Building the evidence: ROADS evaluation agenda 7. ROADS project awards 1.1 ROADS I technical proposal 1.2 Mid-term evaluation report and ROADS’ response 1.3 ROADS II technical proposal 1.4 LWA 1.5 Associate Awards (Burundi, Kenya, Mozambique, Rwanda, Tanzania, Uganda and Zambia) 8. Briefing documents a. ROADS regional and country briefs b. Site assessment summaries (Burundi, Djibouti, DRC, Ethiopia, Kenya, Mozambique, Rwanda, South Sudan, Tanzania and Uganda) c. Behavior Monitoring Surveillance (BMS) summaries 9. Selected strategic frameworks 10. USAID/EA progress reports (FY 2006, 2007, 2008, 2009, 2010, 2011 and semi-annual for FY 2012) 11. Project monitoring and evaluation presentations and documents 12. ROADS II training and operations manuals 13. Various ROADS intervention, situational, and technical assessments in specific districts in several countries 85 14. ROADS II presentations made to the Evaluation Team by FHI 360 ROADS II staff and project implementing partners, including: Development Alternatives, Inc. (DAI), Howard University/Pharmacists and Continuing Education (PACE), and the Program for Appropriate Technology in Health (PATH) 15. Status and achievements of country program PP presentations made to the Evaluation Team in the countries where site visits were made 16. Selected abstracts and presentations made to national and international fora 17. Selected learning stories 18. Tables and graphs specifically developed at the request of the Evaluation Team 19. Selected photos and films from ROADS sites, including clinics, interventions, clusters, community members, and individual beneficiaries 20. Various communication materials used by ROADS in the local languages in countries to disseminate information and for BCC messaging.