Performance Evaluation of Phase 1 of the Rural Road Development Project in Tanzania October 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A for the Data for Development Activity. PERFORMANCE EVALUATION OF PHASE 1 OF THE RURAL ROAD DEVELOPMENT PROJECT IN TANZANIA Submitted: August 30, 2018, Revised September 27, 2018 USAID/Tanzania, Data for Development Project Contract Number: AID-OAA-1-15-00024/AID-621-TO-17-00005 Prepared by: Ritu Nayyar-Stone, Team Lead and Evaluation Specialist, NORC at the University of Chicago Jacob Laden, Evaluation Advisor, Data for Development Zoe Grotophorst, Qualitative Specialist, NORC at the University of Chicago Daud Siwalaze, Monitoring and Evaluation Expert, Data for Development Bahati Tenga, Subject Matter Expert, ME&A Submitted by: Brenda L. Pearson, Chief of Party ME&A (Mendez England & Associates) USAID/Tanzania Data for Development (D4D) Project Plot 122 Mwai Kibaki Road, Mikocheni A, Dar es Salaam, Tanzania Email: bpearson@engl.com Cover Photo: Wamidakawa-Dihombo Road, Mvomero, Tanzania Credit: Evaluation Team DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government i ABSTRACT The performance evaluation of Phase I of the Roads Development Project utilized a document review, key informant interviews, focus group discussions, and a non-representative quantitative survey to investigate the socio-economic effects of 10 roads rehabilitated in the Kilombero, Kiteto, Kongwa, and Mvomero districts of Tanzania. Evaluation questions included: 1) the demographic reach of the rehabilitated roads; 2) changes in prices of agricultural inputs, consumer goods, and passenger transport; 3) changes in transport costs to market; 4) changes in vehicle operational costs; and 5) capacity development of district engineers and the sustainability of project activities. An estimated 402,209 individuals were affected by project activities. The rehabilitation of the roads increased settlements along the road and improved the quality of life of citizens by improving access to hospitals, schools and drinking water. The availability of agricultural inputs increased, and transportation costs decreased. The project enhanced the capacity of district engineers in planning and designing roads as well as maintaining them by newly established road user associations (RUAs). The rehabilitated roads may not be sustained because local government authorities failed to provide routine maintenance on the rehabilitated roads for more than three years. RUAs were late to be registered and only 1 in 4 RUAs has been contracted. The main constraints to project success include the lack of an enforcement mechanism to require that funds be spent on routine maintenance, and the lack of reference materials and equipment provided to RUAs in order to execute routine maintenance tasks. ii TABLE OF CONTENTS Abstract .................................................................................................................................... ii Acronyms.................................................................................................................................. v Executive Summary .................................................................................................................. i Evaluation Purpose And Evaluation Questions................................................................................ i Project Background..................................................................................................................... i Evaluation Methods And Limitations ............................................................................................ii Findings And Conclusions...........................................................................................................ii 1. 1.0 Evaluation Purpose & Evaluation Questions ............................................................ 1 1.1 Evaluation Purpose ......................................................................................................... 1 1.2 Evaluation Questions...................................................................................................... 1 2. 2.0 Project Background..................................................................................................... 2 2.1 Tanzania’s Transportation And Infrastructure Context....................................................... 2 2.2 Program Description ...................................................................................................... 2 2.3 Program Theory Of Change And Logical Framework......................................................... 5 3. 3.0 Evaluation Methods & Limitations.............................................................................. 5 3.1 Structured Desk Review ................................................................................................. 6 3.2 Key Informant Interviews................................................................................................ 6 3.3 Focus Group Discussions................................................................................................ 7 3.4 Capacity Building............................................................................................................ 8 3.5 Evaluation Limitations..................................................................................................... 9 4. 4.0 Findings & Conclusions ........................................................................................... 9 4.1 EQ 1: What Has Been The Estimated Demographic Reach Of The 10 Rehabilitated Roads Totaling 179.7 Km In The Four Targeted Districts? ...................................................................... 9 4.2 EQ 2: How Has Targeted Roads’ Rehabilitation Affected Prices Of A Number Of Goods And Services Such As Agricultural Inputs, Consumer Products, And Passenger Transport Costs? ...........13 4.3 EQ 3: How Has Targeted Roads’ Rehabilitation Affected Transport Costs To Markets Of Specific Agricultural Products Such As Maize, Rice, And Horticultural Products? ............................21 4.4 EQ 4: What Effects, If Any, Have Road Rehabilitation Had On Operation Costs Of Vehicles?26 4.5 EQ 5: How Did The Project Enhance The Capacity Of District Engineers To Plan, Develop/Rehabilitate And Maintain Roads? How Have Efforts In Phase 1 Been Sustained? If Not, What Constraints Were Not Overcome To Achieve These Results?.....................................................27 4.6 Unexpected Results Of Project Activities.........................................................................36 6. 5.0 Recommendations......................................................................................................38 Annexes ...................................................................................................................................43 Annex I: Evaluation Statement Of Work.....................................................................................44 Annex II: Data Collection Instruments .......................................................................................53 iii Annex III: Sources Of Information .............................................................................................95 Annex IV: Disclosure Of Conflicts Of Interest ..........................................................................100 Annex V: Evaluation Team Biography .......................................................................................104 TABLES Table 1: Summary Information on Roads Activities Undertaken in Phase I of the RRDP............................... 4 Table 2: KIIs Conducted by Project Stakeholder Type and District..................................................................... 6 Table 3: Breakdown of FGDs by FGD Type, District, and Number of Participants......................................... 7 Table 4: Demographic Information of Respondents to Questionnaire................................................................ 8 Table 5: Estimated Demographic Reach of the 10 Rehabilitated Roads Totaling 179.7km........................... 10 Table 6: Road Use by Gender and District............................................................................................................... 11 Table 7: Modes of Transportation for Commercial Purposes Before and After Road Rehabilitation........ 12 Table 8: Modes of Transportation to Access Social Services Before and After Road Rehabilitation......... 12 Table 9: Number of Processing Facilities................................................................................................................... 16 Table 10: Average Time to Primary and Secondary School by District (Minutes).......................................... 18 Table 11: Average Travel Time to Markets to Sell Produce (Minutes).............................................................. 23 Table 12: Travel Time to Markets to Buy Specific Goods (By District) ............................................................ 25 Table 13: Gender of RUA Leaders by District......................................................................................................... 30 Table 14: Condition of FARIL Roads (June 2018).................................................................................................... 31 FIGURES Figure 1: Map of RRDP Evaluation Locations.............................................................................................................. 3 Figure 2: Logical Framework Underpinning RRDP Phases 1 and 2 ....................................................................... 5 Figure 3: Average Price of Agricultural Inputs Adjusted to 2016 ........................................................................ 14 Figure 4: Overall Agricultural Yield (Ton) by Site Visited...................................................................................... 15 Figure 5: Amount of Cultivated Land by Site Visited.............................................................................................. 15 Figure 6: Travel Time to School................................................................................................................................... 18 Figure 7: Average Travel Time in Minutes to Hospital (Emergency).......Error! Bookmark not defined. Figure 8: Travel Time to Community Meetings....................................................................................................... 21 Figure 9: Transportation Methods Used to Access Markets to Sell Produce .................................................. 22 Figure 10: Travel Time to Markets to Buy Specific Goods................................................................................... 24 iv ACRONYMS Acronym Description CRB Contractors Registration Board DE District Engineer EQ Evaluation Question ET Evaluation Team FARIL Fixed Amount Reimbursement Implementation Letter FGD Focus Group Discussion FTF Feed the Future G2G Government-to-Government GoT Government of Tanzania IRRIP Irrigation and Rural Roads Infrastructure Project KII Key Informant Interview LGA Local Government Authority ME&A Mendez England & Associates NBS National Bureau of Statistics PE Performance Evaluation PPRA Public Procurement Regulatory Authority RRDP Rural Road Development Project RUA Road Users Association SOW Statement of Work TARURA Tanzania Rural and Urban Roads Agency TZS Tanzanian Shillings USAID U.S. Agency for International Development USG U.S. Government VEO Village Executive Officer WEO Ward Executive Officer v i EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this qualitative performance evaluation (PE) was to examine socio-economic effects of the rehabilitation of a set of roads in the Kilombero, Kiteto, Kongwa, and Mvomero Districts, implemented by CDM Smith as part of Phase 1 of the Rural Roads Development Project (RRDP) from 2012-2016 (Agreement numbers: 621-IL-14-00004, 621-IL-14-00007, 621-IL-14-00008, and 621-IL-14- 00009). Five key evaluation questions (EQ) were examined: (1) What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four targeted districts? (2) How has targeted roads’ rehabilitation affected prices of a number of goods and services such as agricultural inputs, consumer products, and passenger transport costs? (3) How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? (4) What effects, if any, has road rehabilitation had on operation costs of vehicles? (5) How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? If not, what constraints were not overcome to achieve these results? USAID is the primary audience for this evaluation. The evaluation results will serve as a first approximation of effects from this type of intervention, inform on how communities perceived changes following the completion of rehabilitated roads, and help establish a list of sectors where changes have been identified by beneficiary communities. The findings will also be useful to the Government of Tanzania (GoT), including Tanzania Rural and Urban Roads Agency (TARURA) and local government authorities (LGAs). PROJECT BACKGROUND Tanzania has been designated a priority country for the U.S. Government’s (USG) Feed the Future (FTF) Initiative, which aims to address the root causes of global hunger by sustainably increasing agricultural productivity to meet the demand for food, supporting and facilitating access to markets, and increasing incomes for the rural poor so they can meet their food and other needs. The Partnership for Growth Constraints Analysis (2011), which served as a basis for the USG and GoT Joint Country Action Plan (2012-2016), indicated that inadequate rural roads are one of the two key binding constraints to private investment and economic growth – the other being the supply of electric power. On this basis, USAID agreed to support the GoT’s efforts to address that major binding constraint through investments in rural roads to increase access to markets. Phase 1 of RRDP was an FTF activity implemented by CDM Smith from 2012 to 2016, that provided technical assistance and capacity building to the GoT to carry out improvements on up to 179.7 km of rural roads1 in Kilombero and Mvomero districts in the Morogoro region, and Kiteto and Kongwa districts in the Manyara and Dodoma regions, respectively. The rural road program has been divided into what is informally termed as Phase 1 and Phase 2. Phase 1 focused on roads that were identified and selected under the “Accelerated Process,” and funded by USAID via individual government-to-government (G2G) agreements taking the form of fixed amount reimbursement implementation letters (FARIL) with each district’s LGA. Phase 2, under the Rural Road Development Project (RRDP), supports the same four LGAs to rehabilitate 304 km of roads with construction that began in April 2017. This evaluation focuses solely on Phase 1 activities. 1 Rural roads in this context are defined as district collector and feeder roads being 4.5 meters or 6 meters in width and falling under the responsibility of LGAs. ii EVALUATION METHODS AND LIMITATIONS This PE utilized mutually reinforcing quantitative and qualitative methods to answer the evaluation questions (EQs) listed in the Statement of Work (SOW). To answer EQ1, the evaluation team (ET) collected quantitative population data for communities along the rehabilitated roads from district officials. To answer EQ2-5, the ET used qualitative data collection and analytical methods, including: (1) desk review of project documents, (2) key informant interviews (KIIs) with project beneficiaries and stakeholders, and (3) focus group discussions (FGDs) with beneficiaries. In total, the ET conducted 52 KIIs (47 with beneficiaries in the four project districts and five with CDM Smith, USAID, and TARURA regional coordinators); and 13 FGDs (all with beneficiaries in project districts). Although not specified in the SOW, the ET supplemented its qualitative data with two types of quantitative data: (1) village-level data on prices and travel time to key services and markets collected from a paper survey conducted with FGD participants (N=117), and (2) district-level agricultural production and cultivation area data collected from district agricultural officers. All data collection was completed May 28 - June 22, 2018. There are several important limitations inherent to the design selected for this evaluation, including: (1) time and location constraints, which only allowed the ET to visit seven of the 10 roads; (2) lack of baseline data to compare to current findings; (3) selection bias of qualitative participants; (4) recall bias due to the ET having to ask about events and perceptions that occurred several years earlier; (5) response bias/social desirability bias; and (6) sampling bias related to the purposive sampling of qualitative participants. The ET attempted to minimize these biases by providing anonymity guarantees and holding interviews in a private setting, as well as providing quotas to village leaders recruiting qualitative participants. This evaluation deepened the capacity of several individuals and organizations – ME&A’s local subject matter expert, the IP and USAID. FINDINGS AND CONCLUSIONS EQ 1: What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four targeted districts? Key Findings Conclusions The population along the 10 rehabilitated roads at the start of the project activities was 355,628 people – 49 percent men and 51 percent women – covering 95 villages within 34 wards. At the baseline in 2012, prior to the start of any rehabilitation, the population that would be affected by the roads’ rehabilitation was 355,628. By 2016, when all 10 FARIL roads were rehabilitated, the estimated population for these areas increased to 402,209, an 11.58 percent increase. The number of settlements along the rehabilitated roads has increased. Roads are being used more frequently for taking produce to the market, to buy consumer goods, and to access social services. Some villages at the intersection of a FARIL road with a regional road (Engusero, in Kiteto District, or Namawala, in Kilombero District) have now become town centers. An estimated 46,581 individuals had moved to these wards by the end of the project, based on population projections by the National Bureau of Statistics (NBS) for 2016. The number of settlements increased along the FARIL roads and the quality of homes along the rehabilitated roads visited by the ET improved due to ease of accessing markets and movement along the roads. There are more vehicles on the road and users of the road can now have access to additional modes of transportation (cars, buses, and motorcycles). Slightly more males reported using rehabilitated roads to sell agricultural products, while a larger percent of females use the roads to access health services. iii EQ 2: How has targeted roads’ rehabilitation affected prices of a number of goods and services such as agricultural inputs, consumer products, and passenger transport costs? Key Findings Conclusions 2A: What are the perceived effects of the road rehabilitation on prices of goods and services, agricultural inputs, consumer products, passenger transport costs? Agricultural inputs have become more available but also more expensive. Stakeholders reported a diversification and increase in production, which was confirmed by examining the change in total hectares of cultivated land before, during and after road rehabilitation, and overall agricultural yield, in each site visited for this evaluation. Overall, the perceived effect of road rehabilitation was that it increased the availability of a variety of agricultural inputs, while also increasing their prices. There was mixed perception regarding the effect of road rehabilitation on prices of consumer goods with the prices of some goods increasing, while others decreased. Finally choices regarding passenger transportation increased while prices of transportation decreased. Communities perceive that increases in demand and quality are responsible for increasing agricultural input prices. Communities perceive some consumer products have increased in price (e.g., sugar, banana, chicken, kerosene, and agricultural produce), while others have decreased (e.g., beans, rice). There are more options for passenger transportation and the fare has decreased. 2B: What are the perceived effects of beneficiary communities on access to social services such as schools, health services, water, etc.? Overall the perceived effect of the rehabilitated roads on social services is that it increased access to schools (primary and secondary), as well as to hospitals and water. It has also decreased travel time to primary and secondary schools, the hospital and community meetings. Beneficiaries cite the availability of school materials and increased supervision by school extension officers. School attendance, travel time to school, school choice, and supervision by extension officers have all improved after roads rehabilitation. The same can be said about the availability of medical supplies. In three out of four districts visited, roads rehabilitation has enabled communities to have access to clean drinking water. The most common use of rehabilitated roads is to access health services. On average, the travel time to hospitals has decreased from nearly three hours to just one hour. This is perceived as having reduced risks to pregnant women. Of all four districts, Mvomero has experienced the greatest decrease in travel time to hospitals. Rehabilitation has given communities greater access to drinking water. iv EQ 4: What effects, if any, have road rehabilitation had on operation costs of vehicles?? Key Findings Conclusions In all project districts, vehicles experienced improved fuel Roads rehabilitation has reduced efficiency of up to 50 percent after roads rehabilitation. overall vehicle operating costs, especially in the areas of fuel efficiency and maintenance, as vehicles spend less time on the roads and less time being repaired. The reduction in transporters’ operational costs has consequently lowered the price of passenger transport and the price charged to producers in order to transport their goods to market. In all districts visited, transporters reported that roads rehabilitation reduced their overall maintenance costs as the frequency of repairs decreased. EQ 5: How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in phase 1 been sustained? If not, what constraints were not overcome to achieve these results? Key Findings Conclusions 5A: How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? CDM Smith developed a prioritization tool that uses technical criteria to identify roads for rehabilitation. Council managers have been trained on the tool and have it in their possession for future projects. The project effectively enhanced the capacity of district engineers to plan roads by producing and implementing a prioritization tool and process to objectively select roads for rehabilitation. The project also improved the capacity of district engineers in technical drawing and budgeting through training and hands￾on mentorship. The project’s capacity building activities were least effective for district engineers’ contract management skills, potentially because of district engineers’ limited knowledge and experience with contract management and their failure to attend trainings held by CDM Smith. The project enhanced district engineers’ capacity to maintain roads by creating RUAs that they can engage to conduct routine maintenance on FARIL roads. CDM Smith built district engineers’ capacity to develop technical inputs for solicitations, ensure adherence to procurement regulations, and estimate expected budgets and forecasting for implementation of rehabilitated roads. This was viewed by all stakeholders questioned as a positive step to reduce political influence. Though CDM Smith provided classroom training on contract administration to district officials, a lack of basic contract management knowledge and tendency to delegate attendance to subordinates challenged its effectiveness. CDM Smith developed the framework for setting up RUAs, worked with district officials to identify communities to host RUAs, conducted a training of trainers for district officials on labor-based technology, and then oversaw the administration of that training to paraprofessionals. The labor-based technology training of paraprofessionals by district officials was challenged by the instructors’ lack of experience and knowledge of labor-based technology, and the long distances traveled by participants. v EQ 5: How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in phase 1 been sustained? not, what constraints were not overcome to achieve these results? If 5B: How have efforts in Phase 1 been sustained? None of the rehabilitated FARIL roads the ET visited had undertaken any routine maintenance since their construction, a period of more than three years. Phase 1 efforts have not been sustained because LGAs failed to provide routine maintenance on the rehabilitated roads for a period of more than three years. The registration process for RUAs was unexpectedly long, yet despite all being registered, only one in four RUAs has been contracted. Contracting with RUAs is essential to the long-term sustainability of the rehabilitated roads. A few RUAs are contributing to their own sustainability by partaking in other income-generating activities and adding their earnings to their group bank accounts. However, since RUA members include individuals from vulnerable groups, there is a risk that RUAs not contracted soon could have difficulty holding on to their members. District engineers used routine maintenance funds for other purposes and only did periodic maintenance on the rehabilitated roads. RUA contracting was delayed because RUAs had to register at the national level with the Contractors Review Board (CRB) in order to be eligible for contracts. Neither CDM Smith nor USAID were initially aware of this requirement. Registration was lengthy and cost each RUA 105,000 TZS. Although all RUAs are now registered with CRB, only nine out of 38 RUAs have maintenance contracts with These RUAs are in Kilombero and Mvomero. TARURA. Only Kongwa and Kilombero have budgeted in a way that will allow them to work with all RUAs on their roads in the upcoming year. In areas where RUAs have not been contracted, members are losing hope and groups are experiencing dropouts. RUAs are taking sustainability into their own hands, holding regular meetings, requiring dues from members, and adding to bank accounts by participating in other income￾generating work. vi EQ 5: How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in phase 1 been sustained? If not, what constraints were not overcome to achieve these results? 5C: If not, what constraints were not overcome to achieve these results? Districts depend on the Roads Fund Board for 100 percent of their road maintenance revenue. They are expected to set aside their own revenues to supplement funds from the Road Fund Board, but none have done this. The main constraints to project success include the lack of an enforcement mechanism to require that Roads Fund Board money be spent on routine maintenance, the lack of reference materials and equipment provided to RUAs in order to execute routine maintenance tasks, issues of payment to RUAs, and potential of weak project management at TARURA due to limited personnel. The lack of reference materials and equipment in RUAs is likely the simplest constraint to overcome. Maintenance funds from the Roads Fund Board are often used by districts to expand their road networks instead. Paraprofessionals lacked detailed reference materials or guides showing how to conduct the routine maintenance tasks. Contracted RUAs lack the equipment and safety gear required for a community-based organization to register with the CRB to undertake routine maintenance. Contracted RUAs are paid 8,000 TZS per day worked, due upon the completion of each task. The daily rate is set by the TARURA council manager. Various stakeholders noted RUAs’ low daily rate as a challenge. TARURA is understaffed. 1 EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of the performance evaluation of Phase I of the Rural Road Development Project (RRDP) was to examine the socio-economic effects of the rehabilitation of a set of roads in the Kilombero, Kiteto, Kongwa, and Mvomero districts in the Morogoro, Manyara, and Dodoma regions of Tanzania. The evaluation was conducted by a team assembled by ME&A under the Data for Development Activity. The results of the evaluation will serve to determine a first approximation of effects from this type of intervention, inform USAID personnel on how communities perceived changes following the completion of rehabilitated roads, and help establish a list of sectors where changes have been identified by beneficiary communities. Furthermore, the lessons learned and data collected during the implementation of this qualitative evaluation will contribute to the baseline for the second phase of the rural road development project. EVALUATION QUESTIONS The performance evaluation provides a substantive analysis of the following five evaluation questions (EQs), also listed in the evaluation Statement of Work (SOW) in Annex I. 1. What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four targeted districts? 2. How has targeted roads’ rehabilitation affected prices of a number of goods and services such as agricultural inputs, consumer products, and passenger transport costs? a. What are the perceived effects of the road rehabilitation on prices of goods and services, agricultural inputs, consumer products, passenger transport costs? b. What are the perceived effects of beneficiary communities on access to social services such as schools, health services, water etc.? 3. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? a. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? b. How do beneficiary communities perceive their access to markets to have changed (if at all) since the construction of the road? 4. What effects, if any, have road rehabilitation had on operation costs of vehicles? 5. How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in Phase 1 been sustained? If not, what constraints were not overcome to achieve these results? 2 PROJECT BACKGROUND TANZANIA’S TRANSPORTATION AND INFRASTRUCTURE CONTEXT In recent years, Tanzania has devoted resources to improve its trunk roads (main roads), particularly those linking upcountry regions with the ports and neighboring countries to form a regional transportation network. Notable transport routes are the central and northern corridors. Despite the substantial investment in trunk roads, rural roads (consisting primarily of district collector and feeder roads) have not received sufficient attention. It is estimated that 70 percent of the market costs of commodities emanates from costly and delayed transport. It is also estimated that between 20 percent and 40 percent of agricultural produce is lost during post-harvest operations, which can be attributed to the delay in transport of produce from farms to processing and storage facilities. Rural roads are critical to market access and some studies have demonstrated how higher poverty rates correlate with increased transportation distance to key arterial roads. USAID’s efforts to improve road infrastructure have occurred alongside the Government of Tanzania’s (GoT) own sustained efforts including Tanzania’s Vision 2025 and the National Strategy of the Transport Policy of 2011-2025, which focus on the development of infrastructure, transportation, and the country’s roads system as a means to economic development. PROGRAM DESCRIPTION Tanzania has been designated a priority country for the U.S. Government’s (USG) Feed the Future (FTF) Initiative, which aims to address the root causes of global hunger by sustainably increasing agricultural productivity to meet the demand for food, supporting and facilitating access to markets, and increasing incomes for the rural poor so they can meet their food and other needs. The Partnership for Growth Constraints Analysis (2011), which served as a basis for the USG and GoT Joint Country Action Plan (2012-2016), indicated that inadequate rural roads are one of the two key binding constraints to private investment and economic growth – the other being the supply of electric power. On this basis, USAID agreed to support the GoT’s efforts to address that major binding constraint through investments in rural roads to increase access to markets. Phase 1 of RRDP was an FTF activity implemented by CDM Smith from 2012 to 2016, that provided technical assistance and capacity building to the GoT to carry out improvements on up to 179.7 km of rural roads2 in Kilombero and Mvomero districts in the Morogoro region, and Kiteto and Kongwa districts in the Manyara and Dodoma regions, respectively (Agreement numbers: 621-IL-14-00004, 621-IL-14-00007, 621-IL-14-00008, and 621-IL-14-00009). RRDP has been divided into what is informally termed as Phase 1 and Phase 2, also known as IRRIP 1 and IRRIP 2. Phase 1, the focus of this evaluation, included the rehabilitation of roads that were identified and selected under the “Accelerated Process” and funded by USAID through IRRIP via individual government-to-government (G2G) agreements taking the form of fixed amount reimbursement implementation letters (FARIL) with each district’s local government authority (LGA). These roads were prioritized in 2012 by district engineers (DEs)3 on the basis of existing LGA road development priorities and an awareness of the general aims of the FTF program but no formal prioritization guidance. These four agreements, listed in Table 1 below, totaled $5,515,946 million to cover 179.7 km of road rehabilitation and upgrades, which were completed in January 2016. The locations of the rehabilitated roads in Tanzania is shown in the map in Figure 1. 2 Rural roads in this context are defined as district collector and feeder roads being 4.5 meters or 6 meters in width and falling under the responsibility of LGAs. 3 In July 2017, the Tanzania Rural-Urban Road Agency (TARURA) was established to address rural road challenges. At that time, district engineers became council managers under TARURA. 3 Figure 1: Map of RRDP Evaluation Locations 4 Table 1: Summary Information on Roads Activities Undertaken in Phase I of the RRDP Name of Roads Region District Implementing Partner No. of Roads & Lengths Amount (USD) Olboroti-Matui Engusero-Njiapanda Engusero-Orkine Manyara Kiteto Kiteto LGA 3 roads – 76.8km 2,060,667 Pandambili-Njoge Suguta-Kibaigwa Dodoma Kongwa Kongwa LGA 2 roads – 41.0km 1,069,878 Wamidakawa-Dihombo Langali-Nyandira Morogoro Mvomero Mvomero LGA 2 roads – 24.6km 973,276 Namwawala-Mofu1 Mpanga-Ngalimila Mgudeni-Mhelule Morogoro Kilombero Kilombero LGA 3 roads – 37.3km 1,412,123 TOTAL 179.7 5,515,946 Note: 1 Roads written in bold are ones visited by the evaluation team. Phase 2, under RRDP, supports the same four LGAs to rehabilitate 304 km of roads with construction and started in April 2017. Phase 2 focuses on roads that were short-listed by DEs in 2012 and later selected through the use of a prioritization tool developed under Phase 1 after the necessary socio-economic and road condition data input had been collected by each LGA and their DEs. Unlike Phase 1, USAID is funding Phase 2 activities under one G2G agreement with the Roads Fund Board. Both Phase 1 and 2 are comprised of three components of road rehabilitation: 1. Design: Works include the preparation of road designs including all drawings, specifications and cost estimates to upgrade basic earth roads to all-weather passable gravel roads. 2. Supervision: Works include the supervision and oversight of construction contractors’ mobilization, implementation, quality control, material testing, and completion. 3. Construction: Works include the supply and installation of all equipment and materials necessary for completion of the road improvement The technical assistance and capacity-building provided under Phase 1 (IRRIP 1) consisted of: ● Guidance and oversight on road prioritization and the piloting of a scoring tool to add rigor to the process. ● The production of a rural roads design guide and a template for unit rate analysis disseminated to DEs and LGA staff. ● The provision of a five-days course on the procurement of consultancy services to enhance the capacity of DEs and LGA staff at each of the LGA on procurement procedures, including the relevant legal framework and regulations, the evaluation of expressions of interest and proposals, and the preparation of terms of references. ● Sponsorship for DEs and LGA staff at each of the LGA to attend a five-days course on construction contracts administration, which covered topics such as the relevant legal framework and regulations; the management of various stages of the contract; risk management; dispute resolution; and time, quality, and cost controls. ● The production and dissemination to LGAs of a guide on how to constitute, register, and manage Road Users Associations (RUAs) and a guide on carrying out routine, emergency, spot, and periodic maintenance of various types of rural roads. ● The provision of training of trainer courses for district officials to build the capacity of paraprofessionals.4 4 Paraprofessionals refer to the three people in each RUA – two from the community (one man and one woman) and one member from the village council – who received training from district officials in labor-based routine maintenance. They lead the rest of the RUA members in routine maintenance tasks. 5 ● On-the-job training and mentoring of DEs and relevant LGA staff. In addition, Phase 1 also provided USAID with quality assurance for the road rehabilitation/construction which was undertaken through G2G agreements. PROGRAM THEORY OF CHANGE AND LOGICAL FRAMEWORK The RRDP is premised on the following theory of change and logical framework (Figure 2Figure 2 below): If the condition of rural roads is improved and capacity of local government authorities involved in managing rural roads infrastructure is increased, then farmers will have greater access to agricultural inputs and markets, rural commerce will be increased, and access to and delivery of social services will be improved in a more sustainable infrastructure sector. Figure 2: Logical Framework Underpinning RRDP Phases 1 and 2 Given the focus of the EQs, the team examined if: (i) the sub-project purposes were achieved; and (ii) institutional capabilities for rural roads infrastructure and maintenance services improved. EVALUATION METHODS & LIMITATIONS The performance evaluation of Phase I of the Rural Road Development Project utilized a mix of mutually reinforcing qualitative and quantitative methods, allowing the team to triangulate findings. The evaluation drew on data collected via a review of existing project documents, as well as focus groups and interviews with various groups of beneficiaries and project stakeholders. The fieldwork for the PE was conducted between May 28, to June 22, 2018. The evaluation relied on data gathered in affected communities through largely qualitative approaches due to a lack of baseline and monitoring data in the social sectors – the focus of EQs 1 through 4. Some data, relevant for EQ 5 was collected and shared by CDM Smith, and this was used where applicable The Data for Development project conducted the evaluation using a participatory and collaborative approach, engaging the USAID Mission throughout the evaluation, meeting with CDM Smith to better understand the Phase 1 project, and keeping them informed about the field work. CDM Smith also provided contact information for various Ward Executive Officers (WEOs) and Village Executive Officers (VEOs) in districts where the project was implemented, but the implementers 6 were not involved in data collection activities for the purpose of maintaining objectivity and for ensuring respondent/beneficiary confidentiality. The evaluation also took into consideration the local context by considering the opinions and recommendations elicited during the key informant interviews (KIIs) and focus group discussions (FGDs), and undertaking quantitative descriptive analysis of the results from a two-page questionnaire asked to community members, producers, and RUA members who participated in FGDs across the four districts visited for field work. The team also visited seven of the 10 FARIL roads5, documenting the current condition of the road through photographs and notes – see Table 14 in EQ5. Evidence of routine maintenance (e.g., RUA members working on the road, newly dug ditches, cut grass, etc.) was solicited from key informants for all 10 FARIL roads. STRUCTURED DESK REVIEW The ET obtained project documents from CDM Smith and USAID/Tanzania and reviewed and coded them to provide a shared understanding of the project, guide the development of the instruments, and enrich this report. The process followed was to open each file received in sequence and read through it, using an Excel spreadsheet to code information considered important based on the EQs to be answered and the evaluation’s focus on RRDP, IRRIP1 [as opposed to IRRIP2, although some documents covering IRRIP2 were also coded out of an abundance of caution (e.g., DROMAS Training Report)]. When relevant information was identified, the text was summarized and added to the Excel spreadsheet, along with important metadata associated with the entry. These metadata included: 1) the document name, reporting year, reporting period and page number; 2) the relevant project stage(s); 3) the relevant district(s); and 4) the relevant EQ(s). The resulting spreadsheet allows for filtering by EQ, district, and project phase to facilitate analysis, while also allowing easy reference to the original text if needed. See Annex 2: Data Collection Tools. Documents reviewed include: the rural roads design guide, the working paper on rural road prioritization and the prioritization report produced by each of the four LGAs, training guides on routine maintenance operations and the creation and management of RUAs, reports on the various training events provided to DEs and relevant LGA staff, and reports and checklists on the bidding process and works done at each LGA. See Annex 3: Sources of Information. KEY INFORMANT INTERVIEWS The ET conducted a total of 52 KIIs, including 47 across the four project districts (Kilombero, Mvomero, Kiteto, and Kongwa) with a range of project stakeholders and beneficiaries, plus an additional five KIIs with CDM Smith staff in Morogoro, USAID staff in Dar es Salaam, and regional coordinators of the Tanzania Rural and Urban Roads Agency (TARURA) in Morogoro and Dodoma (see Table 2). KIIs were predominantly used to answer EQs 1 and 5 dealing with the demographic reach of the FARIL roads and activities implemented by CDM Smith. EQs 2, 3 and 4 dealing with the effect of the road rehabilitation on beneficiaries along the road were predominantly answered using FGDs. A table mapping the EQs and instrument protocols to qualitative data sources can be found in the Evaluation Design Matrix in Annex II. Table 2: KIIs Conducted by Project Stakeholder Type and District KII Type District and Region Total Kilombero, Morogoro Mvomero, Morogoro Kiteto, Manyara Kongwa, Dodoma Council managers 1 1 1 1 4 Paraprofessionals 2 2 3 2 9 RUA leaders 1 1 1 1 4 Village Leaders (e.g., WEOs, VEOs) 2 2 1 1 6 5 As shown in Table 1 the 7 roads visited by the ET include: (1) Namwala-Mofu, (2) Mgudeni-Mhelule, (3) Wamidakawa￾Dihombo, (4)Langali-Nyandira, (5) Engusero-Njaipanda, (6) Engusero-Orkine, and (7) Suguta-Kibaigwa. 7 KII Type District and Region Total Kilombero, Morogoro Mvomero, Morogoro Kiteto, Manyara Kongwa, Dodoma Agricultural processors 3 3 1 5 12 Transporters 3 3 3 3 12 TARURA regional coordinators - - - - 2 CDM Smith staff - - - - 2 USAID staff - - - - 1 Total 12 12 10 13 52 The KIIs in Kilombero district were conducted in the villages of Mofu and Ihenga, those in Mvomero district were conducted in the villages of Nyandira and Dakawa, those in Kiteto district were conducted in the villages of Orkine and Engusero, and those in Kongwa district were conducted in the villages of Iduo and Kibaigwa. These villages were purposefully selected by the ET in consultation with USAID to capture information from a variety of beneficiaries. They were considered to have sufficient numbers of processors or specific producers of horticultural goods to enable meeting with key informants or ensure sufficient number of discussants for FGDs. Village leaders assisted the ET in recruiting agricultural processors, transporters, paraprofessionals, and RUA leaders for KIIs. To ensure that both gender perspectives were included, the ET requested one male and one female paraprofessional, at least two female agricultural processors, and at least two female transporters. All KIIs used semi-structured, open-ended interview guides tailored to the interviewee’s stakeholder group. After field work was completed, additional clarifying questions were shared and discussed with CDM Smith in July 2018. FOCUS GROUP DISCUSSIONS The ET conducted a total of 13 FGDs across the four project districts (Kilombero, Mvomero, Kiteto, and Kongwa). FGDs were held with three stakeholder groups: RUAs members, agricultural producers, and community members. FGDs were used to get feedback from beneficiaries on questions regarding access to market and social services, price of transportation costs, and prices of agricultural inputs and consumer goods. The use of FGDs enabled getting feedback on common issues across a diverse group of people. As shown in Table 3 below, one FGD of each type was held in each district (although not always in the same village or town), with the notable exception of Kiteto district, where the ET added one additional FGD to explore a developing hypothesis for EQ 5 (i.e., that RUA members’ perceived ownership of the road declined when the RUA was located nearby a market center). Table 3: Breakdown of FGDs by FGD Type, District, and Number of Participants FGD Type District and Region Total Kilombero, Morogoro Mvomero, Morogoro Kiteto, Manyara Kongwa, Dodoma Community membersn…..// 1 [N = 10] 1 [N = 15] 1 [N = 11] 1 [N = 10] 4 Agricultural producers 1 [N = 10] 1 [N = 17] 1 [N = 9] 1 [N = 10] 4 RUA members 1 [N = 10] 1 [N = 10] 2 [N = 18 total] 1 [N = 9] 5 Total 3 3 4 3 13 In each district, the ET began participant selection by first contacting the district’s TARURA council manager. After discussing the evaluation’s design, selected villages, and stakeholder groups to be interviewed, the Council Manager connected the ET with WEOs and VEOs for each of the selected villages. The ET supplied the village leaders with the following criteria for each stakeholder group: 8 ● Community members: 10-12 participants, 50 percent male/50 percent female, 30 percent youth, geographical representation of the village (i.e., at least one member from each of the hamlets), and those living along the road before rehabilitation ● Agricultural producers: 10-12 participants, 50 percent male/50 percent female, 30 percent youth, and those living along the road before rehabilitation ● RUA members: 10-12 participants, 50 percent male/50 percent female, 30 percent youth, only RUA members from the selected village, and those living along the road before rehabilitation Using these criteria, the village leaders identified and invited FGD participants, secured a suitable location for the research, scheduled the FGDs, and ensured that participants showed up. On the first day in each site, the village leaders and ET held a welcome meeting to introduce the evaluation to participants and make them comfortable with participating. The FGDs were conducted by Swahili-speaking team members who were knowledgeable of the program content and Tanzanian context in transport and agriculture; and had previous experience in conducting FGDs to solicit responses from the participants by asking neutral probing questions and without introducing their own biases. Participants in FGDs were also requested to fill a one-page questionnaire (double-sided) with close￾ended questions on: (i) their use of roads; (ii) quality of roads before, during, and after rehabilitation; (iii) cost and modes of transportation; and (iv) prices of goods. The ET received 117 completed questionnaires from all FGD participants, with 56 percent of respondents being male and 44 percent female. Additional demographic information on the respondents is shown is Table 4 below. Table 4: Demographic Information of Respondents to Questionnaire Category Characteristics Gender Male (56%); Female (44%) Age 18 – 35 (40%); 36 – 49 (42%); 50 – 65 (16%); 66+ (2%) Location Kilombero (26%); Mvomero (25%); Kiteto (26%); Kongwa (24%) Occupation Teacher (2%); Health worker (1%); Farmer (94%); Vendor (26%); Transporter (5%); No job (1%) Marital Status Married (72%); Divorced (10%); Widowed (8%); Not married (10%) Source: Evaluation questionnaire to focus group discussants; N = 117. CAPACITY BUILDING6 This evaluation deepened the capacity of several individuals and organizations – our local subject matter expert, the implementing partner (IP), and USAID. The local subject matter expert/consultant deepened skills in evaluation methods and analysis in the following ways: (i) improved skills in the development of an evaluation matrix that includes EQs, sub-questions, protocol questions and sources of data, (ii) enhanced skills in qualitative analysis using an Excel sheet to capture key findings and feedback by themes, and source of the finding, and (iii) increased analytical capacity by using the coded Excel sheet to summarize key findings by research question. The ET also worked closely with the IP in the design of the evaluation, sharing the instruments and getting contact information for key informants in the field. An initial meeting to understand the activity being evaluated, included explanation on how the evaluation was designed and would be conducted. Field work was followed by additional clarifying questions and a Skype call to get feedback and an understanding of specific project activities. Finally, there were several consultations with USAID staff, during the design stage providing feedback on the evaluation SOW. Data for Development exchanged several emails highlighting the advantages of a quantitative survey in the absence of a baseline for the activity and developed a design within the timeline and budget specified 6Although the original Scope of Work did not require documenting capacity building activities with the Government of Tanzania, these activities are being included in this report as per discussions and written exchanges with the USAID COR and the Senior M&E Advisor to USAID/PPL/LER. 9 by the Mission. A detailed debriefing at the end of the field work highlighted project gains and challenges. EVALUATION LIMITATIONS There are several important limitations inherent to the design selected for this evaluation: ● Time and location constraints: The timeline of the evaluation and accessibility of each site limited the ET to visiting seven of the 10 FARIL roads. Interviews and FGDs were conducted along six roads and are therefore indicative but not generalizable across all 10 roads. However, the team gathered some data about all 10 FARIL roads through conversations with TARURA regional coordinators and council managers in each district. ● Lack of baseline data: No baseline data was collected at the start of the Phase 1 road rehabilitation. Thus, the ET cannot compare current findings with the situation prior to the rehabilitation. The questionnaire instrument sought to collect information about perceptions prior, during, and after road rehabilitation, but this approach may introduce some level of recall bias (see below). The ET also collected some information directly from LGAs due to the lack of baseline data. ● Selection bias: There is a possibility of selection bias, (i.e., those respondents who accepted to be interviewed or participate in discussions may differ in some important dimensions from those who did not, for example in terms of their attitudes and perceptions, affiliation with government/non-government structures, socio-demographic characteristics, and experience). Additionally, the WEOs and VEOs in some districts, specifically Mvomero and Kongwa, did not give sufficient notice to participants. This resulted in some distant sub￾villages not being represented in community and producer FGDs. To mitigate this, the ET asked the same evaluation protocol questions to different beneficiaries, to get multiple and diverse perceptions and feedback. ● Recall bias: KIIs and FGDs relied on self-reports about events and perceptions that dated back to several years. Thus, some data may be inaccurate due to lapses in memory. ● Response bias: KIIs and FGDs relied on self-reports about events and perceptions that may be biased due to social desirability or to respondents wanting to provide the answers they thought interviewers ‘wanted to hear.’ To mitigate this limitation, the ET outlined confidentiality and anonymity guarantees to all participants and ensured that KIIs and FGDs took place in a private setting. ● Sampling bias: The non-random selection of persons interviewed, and sites visited undermines the external validity of the results (i.e., the ability of results to be generalized to the rest of the population). The two-page questionnaire was also filled by focus group participants (community members, producers, and RUA members) who were not selected randomly and not statistically representative of individuals affected along the FARIL roads. Thus, their feedback, while illuminating and highly relevant, is not generalizable. ● Presence of other development programs: This evaluation is not able to fully separate the effects of RRDP from those of other development projects operating in project areas. FINDINGS & CONCLUSIONS EQ 1: WHAT HAS BEEN THE ESTIMATED DEMOGRAPHIC REACH OF THE 10 REHABILITATED ROADS TOTALING 179.7 KM IN THE FOUR TARGETED DISTRICTS? FINDINGS Demographic Reach Based on the Population and Housing Census report, which was conducted in 2012 in Kilombero, Mvomero, Kiteto and Kongwa districts, the estimated demographic reach of the 10 rehabilitated roads was 355,628 people – 49 percent men and 51 percent women – covering 95 villages within 34 wards, as depicted in Table 5 below. Based on population growth estimates by the Tanzanian 10 National Bureau of Statistics (NBS), the demographic reach increased to 402,209, an 11.58 percent increase. Table 5: Estimated Demographic Reach of the 10 Rehabilitated Roads Totaling 179.7km District Roads Wards 2012 Villages 2012 Total 2012 Total 2016* Mgudeni – Mhelule (10.0km) 1 3 7,158 8,096 Kilombero Mpanga – Ngalimila (10.0km) 3 10 41,936 47,429 Namwawala – Mofu (17.3km) 2 4 23,637 26,733 Mvomero Langali – Nyandira (5.2km) Wami Dakawa – Dihombo (19.4km) 6 3 20 13 35,652 39,565 40,322 44,747 Engusero – Orkine (19.4km) 4 12 58,959 66,682 Kiteto Olboroti – Matui (32.0km) 3 7 36,380 41,145 Engusero – Matui – Njiapanda (25.5km) 5 8 31,547 35,679 Kongwa Suguta – Kibaigwa (25.0Km) Pandambili – Njoge (15.2km) 4 3 14 4 53,128 27,666 60,087 31,290 Total 34 95 355,628 402,209 Source: Population data by ward for 2012 was collected from TARURA Council Managers and other district officials. Endline data for 2016 was computed using population growth projections by the Tanzania National Bureau of Statistics. Via FGDs with community members in all sites visited, and KIIs with LGA officials in Kilombero and Mvomero, the ET learned that people have started to migrate to the villages along the FARIL roads, buying lands and building permanent residences (4 of 4 FGDs with community members and 4 of 6 KIIs with village leaders). In an FGD, one community member in Kiteto noted that residents who abandoned their houses in the village to spend most of their times in farming areas can now return to the village centers to live and commute daily to their farms for work. According to a village leader in Mvomero, governments and private institutions are also acquiring land and establishing permanent locations after road rehabilitation (1 of 2 FGDs with village leaders). In a KII, the ET was told that in at least one area along the road, Mofu ward in Kilombero district, the number of villages within the ward prior to road rehabilitation was four, and now there are seven (1 of 4 KIIs with council members). Thus, some communities have seen hamlets turn into villages and villages change into small town centers (3 of 4 FGDs with community members).7 The quality of houses was also reported to have improved in Mvomero and Kilombero district. Before rehabilitation, people used to have grass-thrashed house, or houses made of soil or clay bricks; after rehabilitation, houses have been improved and have corrugated roofs or cement bricks in their structure (KII with community members and producers in Mvomero; KII with council manager in Kilombero). Specific areas noted to have changed and become small town centers include Ihenga and Namwawala villages in Kilombero district (KII with council manager); Nyandira village in Mvomero district (KII with council manager); Engusero and Matui villages in Kiteto (FGD with community member); and Masinyeti village in Kongwa district (KII with council manager). In a KII with the council manager in Kiteto, the ET was told that Engusero village increased in size after roads rehabilitation since it connects all three of Kiteto’s rehabilitated roads. Many traders and middlemen are said to have migrated to the village which has now a large, well-known open market 7 Population growth confounds this increase, so it is not possible to attribute the increase in communities along the road purely due to the rehabilitation. 11 (or “Gulio”) with traders from different parts of the country, including Dar es Salaam and Dodoma, selling consumable goods, and farmers and livestock keepers selling their agricultural products. Engusero has also experienced growth in new businesses, especially those selling beverages and construction materials. Also in Kiteto, the village of Matui now hosts a transporter center, gas station, and its own “Gulio” (1 of 4 KIIs with council managers). “Before road rehabilitation, there were only three settlements (houses) along the road at the beginning of Orkine – Engusero road. Those settlement were just near to the FARIL Road Sign. Now there are lots of settlements along the road extended towards the neighbor village of Ngipa.” [Community member, Kiteto] Use of the Rehabilitated Roads Community members and producers reported using roads more frequently after rehabilitation because they are now passable throughout the year instead of being used with difficulties only during the dry season (4 of 4 FGDs with community members, 4 of 4 FGDs with producers). Another reason for increased frequency of road use is that the FARIL roads usually provide a shortcut to the district rural road network in accessing neighboring towns and regions. As noted by one transporter in a KII in Kiteto, “We use the Engusero-Orkine road more frequently, because it provides a shortcut to Kibaigwa and beyond, including Dar es Salaam.” One community member in Mvomero noted, “We use [the] road more frequently for transporting consumable goods such as sugar, salt, cooking oil and other agricultural goods which are not available in our village.” The roads are also being used to transport agricultural products from open markets commonly known as “Gulio” located in the village center or from collection point established by buyers and farmers along the roads to new markets where products can be sold for better prices (4 of 4 FGDs with producers). One village leader noted that producers are changing from subsistence to commercial livelihood activities. Additionally, it was mentioned that producers are becoming transporters (4 of 12 KIIs with processor, 3 of 4 FGDs with producers). (See EQ 3 for more detail). The reported benefits of accessing social services due to the rehabilitated roads, include students being able to reach secondary school, access to referral health services, ability to visit relatives, and attending community meetings and religious and other ceremonies in the neighboring villages (3 of 4 FGDs with community members). For primary health care, community members citied the availability of health service providers, drugs, and equipment and supplies in dispensaries and health centers (4 of 4 FGDs with community members). Under primary education, benefits included an increase in primary teachers, regular supervision of educational extension officers from the district, and availability or easy supply of educational materials (4 of 4 FGDs with community members). The effect on access to education is explained more in Question 2b. Table 6: Road Use by Gender and District Road Use Kilombero Mvomero Kiteto Kongwa Male Female Male Female Male Female Male Female Work 8 4 9 9 12 12 4 3 Getting agricultural inputs 8 5 10 4 11 9 9 2 Selling agricultural products 13 9 13 8 15 13 16 10 Going to the market 11 12 9 11 11 10 13 7 Health services 12 8 14 13 15 13 10 7 Schools 8 7 11 8 13 9 7 10 Other 4 5 9 6 2 3 0 0 Source: Evaluation questionnaire to focus group discussants. N = 117 Table 6 above shows the use of roads by male and female focus group discussants, confirming the use of roads for commercial purposes and to access social services. More males use the road to sell agricultural products compared to females. In Kilombero and Mvomero more females use the road to go to the market, and in Kongwa more females use it to go to school. The number and types of vehicles using the road has changed after road rehabilitation (4 of 4 FGDs with community members, 4 of 4 KIIs with village leaders, 4 of 7 KIIs with processors). The quality 12 of passengers’ cars that can now use the rehabilitated roads has improved. Before road rehabilitation, the types of passenger cars that could access specific villages along the FARIL roads during dry season were just Canter trucks commonly known as “Chai Maharage.” Now, there are minivans, “Noahs,” luxury buses, and sedans (KII with village leader). “There are more vehicles compared to before. Now, there are more buses and we have lorries. Before, we used tractors to carry goods and have some very low-quality buses.” [Processor, Kiteto] “We also have different vehicles on the road. Now we have Noahs, lorries, Fuso, Canter, and semi￾trailers. We didn’t have any of those before. [Village leader, Kiteto] Table 7 and Table 8 show modes of transportation used for commercial purposes and to access social services before and after road rehabilitation. Prior to road rehabilitation, the primary mode of transportation was by foot, bicycle or ox-cart; after rehabilitation, motorcycles, buses, and cars have become more dominant. (The two most used modes of transportation have been noted in red.) Motorcycles are widely used after rehabilitation and are used for traveling to near-by communities, going to the farm, and supporting youth movement to schools and elders to neighboring areas. The use of cars, and to a limited extent trucks, also increased after rehabilitation. Vehicles now in use – cars, mini-trucks (carrying a maximum of 1 ton), like the Suzuki Carry, commonly known as “kirukuu,” other trucks specifically for transportation of agricultural products, minivans, and bajaji (three-wheelers) – are all higher load bearing compared to bicycles and ox-carts. Table 7: Modes of Transportation for Commercial Purposes Before and After Road Rehabilitation Mode of Transportation Markets to buy produce Markets to sell produce Buying agricultural inputs Before After Before After Before After By foot 29% 10% 26% 4% 47% 42% Motorcycle 0% 43% - - 13% 36% Bicycle 59% 16% 20% 6% 35% 9% Truck 0% 4% 14% 23% 1% 1% Car 10% 18% 10% 33% 4% 12% Ox-Cart 1% 0% 23% 1% - - Source: Evaluation questionnaire for focus group discussants. N = 117 Table 8: Modes of Transportation to Access Social Services Before and After Road Rehabilitation Mode of Transportation Hospital (emergency) Community meeting Primary school Secondary school Before After Before After Before After Before After By foot 10% 3% 81% 53% 91% 81% 87% 70% Motorcycle 12% 47% 1% 27% 0% 5% 0% 5% Bicycle 23% 6% 17% 15% 8% 14% 12% 23% Truck 5% 3% - - - - 1% 0% Car 16% 33% 1% 4% - - 0% 1% Ox-Cart 32% 0% - - - - - - Ambulance 0% 6% - - - - - - Source: Evaluation questionnaire for focus group discussants. N = 117 CONCLUSIONS At the baseline in 2012 prior to the start of any rehabilitation, the population that would be affected by the roads’ rehabilitation was 355,628. By 2016, when all 10 FARIL roads were rehabilitated, the estimated population growth for these areas increased to 402,209, an 11.58 percent increase. The number of settlements increased along the FARIL roads and the quality of homes along the rehabilitated roads visited by the ET improved due to ease of accessing markets and movement along the roads. Roads are being used more frequently for taking produce to the market, to buy consumer goods, and to access social services. There are more vehicles on the road and users of the road can now have access to additional modes of transportation (cars, buses, and motorcycles). 13 Some villages at the intersection of a FARIL road with a regional road (Engusero, in Kiteto District or Namawala in Kilombero District) have now become town centers. EQ 2: HOW HAS TARGETED ROADS’ REHABILITATION AFFECTED PRICES OF A NUMBER OF GOODS AND SERVICES SUCH AS AGRICULTURAL INPUTS, CONSUMER PRODUCTS, AND PASSENGER TRANSPORT COSTS? 4.2.1 Sub-question 2a. What are the perceived effects of the road rehabilitation on prices of goods and services, agricultural inputs, consumer products, passenger transport costs? FINDINGS Availability of Agricultural Inputs Producers in all four FGDs (across all four districts) noted that agricultural inputs have become more available after roads rehabilitation because more traders can bring agricultural inputs to the villages with more ease. The lower cost of passenger transportation (discussed in EQ 1) has also enabled producers to access market centers to buy inputs not sold in their villages. In two of the four FGDs with producers in Kilombero and Kiteto districts, it was noted that prior to roads’ rehabilitation agricultural inputs like seeds and manure fertilizer were shared between farmers but rarely sold, and thus the agricultural inputs market was limited (2 of 4 FGDs). “These things were not previously sold. Many agricultural inputs were free – like manure and seeds – and given to each other. Pesticides were not previously used.” [Producer, Kilombero] “Before road rehabilitation maize seeds were not for sale, communities were just sharing one another. After road rehabilitation, maize seeds started to be marketable.” [Producer, Kiteto] In Mvomero district producers reported in an FGD that there are more shops in the villages selling a variety of improved seeds, fertilizers, and pesticides. In Kilombero district producers noted that other farming implements, such as hoes, machetes, and rakes, are also more available and tractors are easily accessible. Another producer in Kilombero said “Yes the prices have changed, because most people now are able to get things outside the villages as well as the number of shops have increased.” Processors in Kilombero district also told the ET that the road rehabilitation increased the availability of inputs such as packaging material, fuel or electricity, and spare machine parts needed by agricultural processors in the most remote communities of Mofu and Ihenga (2 of 3 KIIs with processors). However, in communities such as Dakawa (Mvomero district) that were already connected to larger market centers by roads other than the FARIL roads, the rehabilitation did not affect their ease of accessing inputs. Prices of Agricultural Inputs As seen in Figure 3, below, according to FGD participants8 the average price of agricultural inputs such as seeds (N=77), fertilizers (N=39,) and pesticides (N=60) has increased after road rehabilitation in 2016 as compared to before road rehabilitation in 2013. 8 Four FGDs with community members, 5 with RUA members, and 4 with agricultural producers. 14 Figure 3: Average Price of Agricultural Inputs Adjusted to 20169 Source: Evaluation questionnaire for focus group discussants community members, producers and road user association members10. For seeds N=77, for fertilize N=39, for pesticides N=60. Notes: a. All FARIL roads were rehabilitated between June 2014 to July 2015 b. Before rehabilitation is 2013 and after rehabilitation is 2016 for all FARIL roads. Prices for 2013 adjusted by inflation to 2016 prices. In an FGD with producers in Mvomero district, two reasons were suggested for this increase in prices. One producer noted that “we are now using more quality seeds, organic and non-organic manure, and pesticides to grow our horticultural products, so the prices have gone up.” According to another producer in the same FGD “there are so many customers now so the demand is high which leads to high prices of agricultural inputs.”11 An increase in cultivated land across all sites visited by the ET (see Figure 5 below) may also be driving a price increase. Changed Production in the Market Roads rehabilitation has helped producers diversify production. Based on FGDs with producers, the ET learned that in Mofu, Kilombero district, producers are now growing watermelon and horticultural products because they are sure about transporting them to Ifakara town where they can easily be sold. Producers in Nyandira, Mvomero district, were growing only green beans, potatoes, and tomatoes before the roads rehabilitation, but now they also produce eggplant, zucchini, cabbage, potatoes, beets, and carrots. In Orkine, Kiteto, and Iduo, Kongwa districts, sunflower has been introduced as a cash crop since roads rehabilitation, whereas before producers grew only peanuts, nuts, and black-eyed peas, and only in quantities needed to feed their families and the community. In all sites visited by the ET, producers said agricultural production for specific crops had increased (4 of 4 FGDs with producers). Quantitative data collected from district agricultural officers reinforces this. Figure 4 shows the overall increase in agricultural production (ton) in sites visited by the ET. The sites are not comparable with each other due to the variety of crops produced in each site, but within each site there is an increase in production after 2015/2016 when the road rehabilitation was 9 The increase in price of agricultural inputs was mentioned in 13 of 13 FGDs (4 FGDs with community members, 5 with RUA members, and 4 with agricultural producers). Two reasons for this price increase were suggested by producers in 1 of 4 districts visited by the ET (i) increase in demand for agricultural inputs, and (ii) increase in quality of agricultural inputs. Independent data collected by the ET on the increase in cultivated land and increase in agricultural yield in sites visited also supports the idea of increase in demand of agricultural inputs. 10 At the time of the evaluation, the exchange rate was 1 USD to 2,284.36 TZS. 11 Note that agricultural inputs like seeds and manure fertilizer were shared between farmers but rarely “sold” before road rehabilitation. Therefore, any cost, will seem as an increase. 15 completed. The greatest increases can be seen in Mofu, Kilombero district and Dakawa and Nyandira, Mvomero district. Note that increase in yield could also be caused by other factors such as higher quality seeds, use of pesticides to prevent crop loss, better weather patterns, rains, etc. Thus, increase in agricultural yield cannot be attributed purely to road rehabilitation. Figure 4: Overall Agricultural Yield (Ton) by Site Visited Source: Compiled by evaluation team using data provided by District Agricultural Irrigation and Cooperative Officers in Kiteto, Kongwa, Kilombero, and Mvomero. The ET learned that production of cabbage is especially high in Nyandira, Mvomero district, with customers coming directly to farms to buy it, as well as beans, radishes, and beets. In Mofu, Kilombero district, the production of watermelon and tomatoes for sale has increased as these products are now able to be transported to markets. In Orkine, Kiteto district, production of sunflowers, maize, and beans have all increased after road rehabilitation. In Iduo, Kongwa, sunflower production has been increasing after the crop’s introduction to the area in 2014-2105. Areas of rice cultivation such as Mofu, Kilombero district, and Dakawa, Mvomero district have also seen production increase since roads rehabilitation. Figure 5 shows the increase in cultivated land before (FY 2013-2014) and after (FY 2015-2016) roads rehabilitation in sites visited by the ET. Each site’s District Agricultural Irrigation and Cooperative Officer provided information for the total land used for agricultural production. Since different crops are grown in each site, only the trend in each location can be considered. The figure shows a sharp increase in total area cultivated (hectare) in each site after completion of road rehabilitation. Figure 5: Amount of Cultivated Land by Site Visited Source: Compiled by evaluation team using data provided by District Agricultural Irrigation and Cooperative Officers (Kiteto, Kongwa, Kilombero and Mvomero) on total hectares cultivated from 2013 – 2018. 1,003 3,87714,346 10,30021,808 2,807 1,32714,350 12,50028,480 4,795 8,54920,575 16,700 47,971 5,571 3,582 32,575 25,000 58,880 7,216 5,500 35,575 22,190 72,675 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 Orkine, Kiteto Iduo, Kongwa Mofu, Kilombero Dakawa, Mvomero Nyandira, Mvomero Tons 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018 16 Along with an increase in production, higher sales were also noted by producers in Mvomero and Kilombero (2 of 4 FGDs) and by processors (4 of 12 KIIs). One producer in Mvomero noted “Yes, sales have increased. Previously, traders and buyers could not buy produce in large quantities because of the transportation. Now, buyers can buy as much as they want without worrying about how to transport the produce.” Two processors in Kilombero reported a doubling of sales compared to before rehabilitation (2 of 3 KIIs). One processor noted: “Before the amount of cereal processing for community consumption, as well as for sales purposes, was low because people from neighboring village such as Mofu were not able to access our factories. The rate of processing was 10-20 people per day. Now days, I process the goods of 50 people per day.” According to the second processor, “Over the last 2 years since roads rehabilitation, sales were very good. In a day, I was able to sell 500,000 TZS. Before, it was only maybe 100,000 or 200,000 TZS.” In rural areas like Ihenga, Kilombero, two processors interviewed said that they turned their shops into collection points, gathering produce for external buyers for a fee. Larger processors in peri￾urban settings like Kibaigwa, Kongwa (2 of 5 processors interviewed), and Dakawa, Mvomero (2 of 3 processors interviewed), have collection points within their facilities so that producers can bring their goods directly to the processor for sale. These processors are also focusing on how to add value to agricultural products, package them, and sell the processed goods to further markets. Across all districts, both rural and peri-urban, nine of 12 processors interviewed have also become transporters, carrying their processed goods to market and picking up goods to be processed from farms. Finally, Table 9Table 9, below shows the number of processing facilities in each site visited by the ET. With an increase in sales, the number of processing facilities has increased by at least 33 percent across all locations visited by the ET, with an increase of 77 percent in Iduo, Kongwa. Table 9: Number of Processing Facilities Village, District 2013/2014 2017/2018 Orkine, Kiteto 8 12 Iduo, Kongwa 2 9 Mofu, Kilombero 12 39 Dakawa, Mvomero 5 8 Nyandira, Mvomero NA NA Source: District Agricultural Irrigation and Cooperative Officers in Kiteto, Kongwa, Kilombero, and Mvomero. Note: Processors are generally engaged in sunflower and cereal production; Nyandira, Mvomero, does not have any processors because its primary crops are horticultural products. Consumer Goods The ET received mixed feedback on the prices of consumer goods after roads rehabilitation. Consumers indicated a decrease in the price of beans and rice (1 of 4 FGDs); producers in Mvomero and Kiteto said that due to improved roads and transportation options the prices of maize, rice and beans had decreased (2 of 4 FGDs); and one para-professional and one transporter indicated a decrease in the price of salt, student’s stationary, and rice (2 of 21 KIIs). “Before, we had little access to many items because of poor transportation. Those items that we did have access to were very expensive. For example, 1 kg of salt was 1,000 TZS and now it is only 500 TZS. Even students’ stationery was expensive before, but not now.” [Paraprofessional, Kilombero] “Prices for crops that we don’t grow here have decreased due to the accessibility of roads. There are more cars transporting rice and beans to our village, introducing competition in the market which results in a decrease in the price of rice and beans.” [Producer, Kiteto] On the other hand, other community members referred to an increase in the price of sugar, banana, chicken, kerosene, and agricultural products (3 of 4 FGDs); and processers indicated that the prices of maize, rice and horticultural products such as Chinese vegetables and greens had increased (2 of 12 KIIs). One community member in Nyandira, Mvomero, said the following to explain the increased price of sugar, “After rehabilitation, we buy sugar at the same price as in Dar es Sala am,that is 2,600 TZS 17 per kg. But before rehabilitation, they used to charge us a transportation fee. We would buy sugar at 1,200 TZS whereas in town sugar was sold at 1,100 TZS. Passenger Transportation Before roads rehabilitation, passenger transportation was very limited along FARIL roads. People had to walk long distances in order to access the nearest possible means of transport, typically a heavy truck or tractor carrying cargo on the rough road. After the road rehabilitation, community members have more options for transportation, including minibuses, minivans (commonly known by their brand name “Noah”), motorcycles, sedans, and even luxury buses (3 of 4 FGDs). Before rehabilitation, the only means of transport was truck which could be accessed from Langali. The price for truck was not specific, relied on condition of the road (seasons) and number of passengers it could carry on top of the agricultural products, now we no longer use trucks for passengers transport, we use Motorcycle and minbuses (Noah) as public transport [Community member, Kilombero] According to village leaders, the large number of vehicles on the road has broken the monopoly drivers once had and simultaneously decreased operational costs as vehicles last longer and require fewer repairs. All of these factors have led to a reduction in passenger fare of about 50 percent (2 of 6 KIIs). “Passenger fare has been reduced because of more option for passenger transport and the reliability of road, unlike before when we had no reliable forms of transport and no Noahs, only vehicles that carried goods. You could go either by truck but they could get stuck and stay there for a whole week. You’d be sleeping there, and you’d have to push the truck. It was very difficult till you got out.” [Village leader, Kilombero] “The fare is low now compared to before. For those who use motorcycles, it was 15,000 TZS from Suguta to Kibaigwa before. Now, it is only 7,000 TZS or 8,000 TZS.” [Village leader, Kongwa] CONCLUSIONS Overall, the perceived effect of road rehabilitation was that it increased the availability of a variety of agricultural inputs, while also increasing their prices. Stakeholders reported a diversification and increase in production, which was confirmed by examining the change in total hectares of cultivated land before, during and after road rehabilitation, and overall agricultural yield, in each site visited for this evaluation. There was mixed perception regarding the effect of road rehabilitation on prices of consumer goods with the prices of some goods increasing, while others decreased. Finally, choices regarding passenger transportation increased while prices of transportation decreased. 4.2.2 Sub-question 2b. What are the perceived effects of beneficiary communities on access to social services such as schools, health services, water, etc.? FINDINGS Access to Schools Rehabilitation of the FARIL roads had a positive effect on children’s access to schools (3 of 4 FGDs with community members, and 2 of 6 KIIs with village leaders). For some communities, such as Mofu, Kilombero, accessing schools during the rainy season was extremely difficult prior to the roads rehabilitation because of flooding. Now, schools are easier to access and all are accessible by foot throughout the year. However, one village leader in Kiteto noted “For now, it’s not a problem to get to school on foot. For those who have vehicles, the rain season is still a problem.” “Before, people were thinking about how to reach school so some never got an education. Now, they are sure of reaching school.” [Community member, Kilombero] “Before, during the rainy season, small kids had to stay home because of the rain. Now, students can go to school all the time.” [Community member, Kilombero] Mofu Primary School, Kilombero 18 In all districts visited by the ET, road rehabilitation decreased students’ travel time to primary school. In Kilombero, the travel time to primary school decreased by two thirds. In Orkine, Kiteto, travel time to the primary school in Kiteto town took an entire day, so parents had to find accommodation for their children closer to the school. Lacking the means to do so, many parents chose not to send their children to school. After rehabilitation, those children can live at home and commute only two hours to the same school (FGD with community member). “Yes, before road rehabilitation, children in Kidimu and Mwaya sub-villages would use 3 hours by foot to reach Misheni Secondary School. But after road rehabilitation, most use bicycles and spend one hour to reach a school. Some are given rides by their relatives who have motorcycles and spend less than 20 minutes to reach school.” [Community member, Kilombero] As shown in Figure 6Figure 6 and Table 10: Average Time to Primary and Secondary School by District (Minutes) , the impact on travel time to school is greater at the secondary school level because these schools are located farther from village centers. Before roads rehabilitation, students used to spend an average of 85 minutes traveling to secondary school. Now, it takes students only 36 minutes. Travel time to primary school has also decreased but to a far lesser degree. Figure 6: Travel Time to School Source: Evaluation questionnaire for focus group discussants. N = 103 for primary schools and N=95 for secondary schools Table 10: Average Time to Primary and Secondary School by District (Minutes) Source: Evaluation questionnaire for focus group discussants In Kibaigwa, Kongwa district, children can now reach special schools, and private schools are also now accessible (KII with village leader and FGD with community members). In all districts, roads rehabilitation has had a significant positive effect on student attendance (4 of 4 FGDs with community members). The impact has been especially great for communities along Namwawala￾Mofu Road in Kilombero and Langali-Nyandira Road in Mvomero. In these two areas, schools closed for significant periods of time each year – weeks or months at a time - during the rainy season. Now, schools are accessible throughout the year by various modes of transportation and student attendance has increased (2 of 2 FGDs with community members). Average time to primary school Average time to secondary school District [N] Before rehabilitation After rehabilitation N Before rehabilitation After rehabilitation N Kilombero [N=24] 66 45 20 171 50 15 Mvomero [N=29] 19 18 27 118 27 24 Kiteto [N=30] 32 26 29 51 38 30 Kongwa [N=27] 50 41 27 41 33 26 19 “Student attendance is continuing to get better compared to the time before road rehabilitation of roads since they are walking for a short distance.” [Community member, Kongwa] Prior to roads rehabilitation, teachers were often delayed getting to work and sometimes missed school entirely during the rainy season. Both teacher timeliness and attendance has increased since roads rehabilitation, with some discussants noting an increase in teachers coming to live in their communities because of the improved social services (2 of 6 KIIs with village leader and 1 of 4 KIIs with council managers). “On Namwawala-Mofu Road, many teachers live on the other side of Luwipa River away from the school. Before road rehabilitation, Mofu ward was acknowledged a remote area, whereby civil servants were paid a hardship allowance to motivate them to work in the difficult environment. Living far from school made it challenging for them to come to teach in our schools during the rainy season. But right now, after building that bridge, they can come freely to school at any time and get back to their homes, and some have moved closer to school, as rehabilitated road, provides accessibility which have improved many other social services in the villages.” [Village leader, Kilombero] In Tanzania, DEOs are required to conduct quarterly supportive supervision to selected primary and secondary schools, and WEOs are mandated to conduct regular inspections and supervisions to all primary and secondary schools within their jurisdictions. Prior to roads rehabilitation, these visits were infrequently done. After roads rehabilitation, village officials in all districts visited noted that supervision has increased (6 of 6 KIIs). Community members in Kiteto also observed school improvement in terms of the number of teachers in schools and the quality of materials (1 of 4 FGDs). “Road rehabilitation has also increased the quality of education provided in our primary school, because we easily get school stationeries and education extension officers are often coming to school for supervision.” [Community member, Kiteto] “The quality of primary education in schools along rehabilitated roads will improve because now education officers are able to go for supervision throughout the year. Because it’s a must for supervision to be done from time to time in order to know if the kids are taught, if there is sufficient equipment like books, chalk, blackboards, and so that any problem or challenge can be reported and solved.” [Village leader, Kilombero] Access to Health Services Access to health services was one of the major uses of FARIL roads mentioned by respondents during FGDs and KIIs (4 FGDs with community members, 5 of 6 KIIs with village leaders, 4 of 4 KIIs with council managers). A village leader noted “Orkine-Engusero Road has one health center that everyone goes to. The road has helped with travel time. Before roads rehabilitation, it took 3 hours by car or 5 hours by foot. Now, it takes about 30 minutes by motorcycle or 45 minutes by Noah” (KII with village leader in Kiteto). In an FGD with community members in Kongwa, one participant noted a 50 percent decrease in time needed to access the closest health facility: “The nearest health center for the people of Suguta and Iduo is Mlali Health Center. Before road rehabilitation, Suguta people used 4 hours to reach it. But after road rehabilitation, Suguta and Iduo people use only 2 hours by car.” As Figure 7 shows, the combined average travel time needed to reach a hospital across all respondents is one hour, a 65 percent reduction from before roads rehabilitation. When the travel time by district is examined, it shows that the travel time to hospitals decreased the most in Mvomero. 20 Figure 7: Average Travel Time in Minutes to Hospital (Emergency) District [N] Before Rehabilitation After Rehabilitation Kilombero [N=24] 318 147 Mvomero [N=29] 134 26 Kiteto [N=30] 103 47 Kongwa [N=27] 140 38 Source: Evaluation questionnaire for focus group discussants. N = 110 According to community members in Kilombero and Mvomero districts improvement of the FARIL road has reduced the health risks to pregnant women, since they can now access the district hospital in a shorter time (2 of 4 FGDs with community members). “After rehabilitation, we are now able to take the pregnant women to health centers where they can undergo surgery.” [Community member, Mvomero] The rehabilitated roads have also enabled transportation of health equipment and drugs to primary health care facilities – the dispensaries (2 of 4 FGDs with community members). In an FGD, one community member stated that prior to rehabilitation medicines use to be brought to the near health center and then carried into the village by people on their heads; after road rehabilitation, medicines are brought directly by car to the village dispensary. Access to Drinking Water Roads rehabilitation has given communities greater access to drinking water (3 of 4 FGDs with community members). In Iduo, Kongwa, villagers previously walked long distances to access drinking water from the neighbor village of Chamkoroma. Now, the amount of time spent has been cut in half thanks to the availability of motorcycles. In Nyandira, Mvomero, rehabilitated roads first helped the community to have easy access to clean water from streams and rivers using motorcycles, and later enabled the government to install a tap water system in peoples’ houses. On Namwawala-Mofu Road in Kilombero, roads rehabilitation made it easier for villages to access clean water in Ihenga. “After road rehabilitation, it was a lot easier for us to access clean water from our water streams and rivers, by having motorcycles that carry gallons of water.” [Community member, Mvomero] “Before road rehabilitation, time spent fetching drinking water in Chamkoroma was 5 hours. Now, they use just 2 hours to reach the place.” [Community member, Kongwa] With support from USAID, Ihenga, Mofu and Ikwambe communities in Kilombero now have access to drinking water from the construction of deep wells. This was made possible by the rehabilitated Namwawala-Mofu Road that provided access to building material and allowed heavy equipment to reach the villages for the first time (KII with village leader). In Nyandira, Mvomero, the government installed a tap water system in households following roads rehabilitation (FGD with community members). In Kongwa, people from Iduo are reaching Suguta to access drinking water from the natural water stream using Suguta-Kibaigwa Road (KII with village leader). In Kiteto district, after road rehabilitation, several projects for water provision were initiated, including a deep water well in Chapakazi accessible from Engusero-Njiapanda Road (KII with village leader). Well in Mofu, Kilombero 21 Access to Other Social Services Religion, culture, and tradition are among key social activities reported by community members to benefit from roads rehabilitation (2 of 4 KIIs with village leaders). After rehabilitation, accessing religious services at churches and mosques has become easier, as well as attending weddings and funerals. “The majority of Mofu people are Catholics and going to church was difficult before because of the road condition. Religious services are now easier to access after the road rehabilitation.” [Village leader, Kilombero] “It’s easier to get to different ceremonies passing through the road. It’s easier to go to funerals, weddings, and other ceremonies. It’s even easier to bury a person who has died. It used to take a long time to get somewhere.” [Village leader, Mvomero] Communities also noted the improved access community meetings after roads rehabilitation. As Figure 8 shows, the travel time to community meetings has decreased by 22 percent. Figure 8: Travel Time to Community Meetings District [N] Before Rehabilitation After Rehabilitation Kilombero [N=20] 78 67 Mvomero [N=25] 31 20 Kiteto [N=20] 152 118 Kongwa [N=22] 87 53 Source: Evaluation questionnaire for focus group discussants. N = 87 CONCLUSIONS Overall, the perceived effect of the rehabilitated roads on social services is that it increased access to schools (primary and secondary), as well as to hospitals and water. It has also decreased travel time to primary and secondary schools, the hospital and community meetings. Beneficiaries cite the availability of school materials and increased supervision by school extension officers. Similarly, they cite the availability of medical supplies. In three out of four districts visited, roads rehabilitation has enabled communities to have access to clean drinking water. EQ 3: HOW HAS TARGETED ROADS’ REHABILITATION AFFECTED TRANSPORT COSTS TO MARKETS OF SPECIFIC AGRICULTURAL PRODUCTS SUCH AS MAIZE, RICE, AND HORTICULTURAL PRODUCTS? 4.3.1 Sub-question 3a. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? FINDINGS Demand for Transportation Services As farms are producing more (see EQ 2A), transporters are receiving more requests for transportation work (4 of 9 KIIs with transporters). As noted by a RUA member in Kongwa district “Previously, the transportation of goods to the market was done by buyers who would use heavy duty trucks 22 and tractors to transport goods. After road rehabilitation, both farmers and buyers of agriculture goods are transporting goods to market.” “After road rehabilitation, transportation became affordable and accessible to all because the road was in very good condition. The demand of transport service is higher when the road is passable.” [Transporter, Kilombero] “The demand has now increased, previously we could not pass the cargoes using this road, and we used the Mvomero road.” [Transporter, Mvomero] As Figure 9 shows, transporting goods by foot and ox￾cart has become less common, while cars and trucks are utilized most “after” rehabilitation. Morogoro Market, Morogoro Town Figure 9: Transportation Methods Used to Access Markets to Sell Produce Source: Evaluation questionnaire for focus group discussants. N = 117 The increase in vehicles on the road has eased options to pick up produce in three ways: (i) customers can directly pick up produce at the farms (1of 4 KIIs with RUA members, 1 of 4 FGDs with producers, and 1 KII with CDM Smith); (ii) produce can be picked up at centralized collection points (1 of 4 KIIs with RUA members); or transporters can directly pick up produce at farms rather than at collection points (1 of 4 FGDs with producers). “Before road rehabilitation, we used to carry our products and walk to market. After road rehabilitation, our agricultural products are directly accessible at the farm or the nearest collection center along Engusero-Orkine Road, like the one in Ngipa.” [RUA member, Kiteto] Transportation Prices Before road rehabilitation, a few transporters on each road could charge a high fare, but as new transporters have entered these markets after road rehabilitation, competition has increased and transport prices have been driven down (5 of 12 KIIs with transporters). This decrease in price is also due to the greater variety of vehicles than can now pass on the roads -- motorcycles, small trucks, and heavy duty vehicles that can carry large loads at a time – as seen in Figure 9 above. Collection point in Kibaigwa, Kongwa 23 “Previously we could transport goods from the farm to Morogoro for 9,000 TZS per bag for a trip, but now we charge 5,000 TZS per bag for a trip.” [Transporter, Mvomero] “There is difference in transportation price. Before rehabilitation, transportation cost of one bag of maize to Kibaigwa was 5,000 TZS but now it ranges between 2,500 and 3,000 TZS.” [Transporter, Kiteto] A TARURA officer stated that “Even for any small scale farmer, the transportation costs for the crops have been reduced after roads rehabilitation.” (1 of 2 KIIs with TARURA regional coordinators) Operating Costs for Transporters Certain roads (Wamidakawa-Dihombo Road, Mvomero; Suguta-Kibaigwa Road, Kongwa; and Engusero-Orkine Road, Kiteto) offer shortcuts for common routes to markets, decreasing the distance traveled and fuel consumed by transporter vehicles (2 of 6 KIIs with transporters and 1 of 3 KIIs with processors). For more information on operating costs, please see EQ 4. “Despite the fact that we are using the same transportation mode of trucks (Fuso and Canter) as before, we now use short rehabilitated route unlike before when we used a longer route and more fuel.” [Transporter, Kiteto] “Suguta-Kibaigwa Road is a shortcut. Before one had to go through Mlali to be able to reach Kibaigwa, but after the road got rehabilitated we access Kibaigwa straight.” [Transporter, Kongwa] Additionally, in a KII, one transporter said “Before rehabilitation, someone was forced to hire the truck alone. But now two or more transporters can combine to transport in bulk so they pay less.” (1 of 3 KIIs with transporters). As seen in Table 11, the average travel time to markets to sell produce has fallen by more than half across all four districts. The maximum decrease is seen in Kongwa where the travel time drops to 29 percent of the time prior to road rehabilitation. With more vehicles on the road and a decrease in travel time, crop loss has reduced (1 in 4 FGDs with producers, 1 of 2 KIIs with TARURA). Table 11: Average Travel Time to Markets to Sell Produce (Minutes) District [N] Average travel time to market to sell produce before rehabilitation Average travel time to market to see produce after rehabilitation Kilombero [N=16] 411 195 Mvomero [N=21] 106 37 Kiteto [N=19] 172 81 Kongwa [N=20] 333 99 “We lost our crops, especially cabbage, before the roads rehabilitation. After production, we would gather our cabbages in the farms to wait for transport which took very long, days, to come, and led to rot of the cabbages. During the rainy season, it was even worse because we lost most of our crops. After the road rehabilitation, there is no crop loss because transport comes on time and even directly to us farmers to buy our cabbages.” [Producer, Mvomero] “There was a great loss of crops before that the rehabilitation of the road because after harvest, we had nowhere take our crops.” [TARURA] CONCLUSIONS Road rehabilitation has increased the demand for transport services and competition among existing transporters. Producers and processors have become transporters themselves, carrying their goods to market and selling them at better prices. Shortcuts offered by certain roads have led to a decrease in operating costs for transporters, and with additional vehicles and competition, transportation costs to market and crop losses have decreased. 4.3.2 Sub-question 3b. How do beneficiary communities perceive their access to markets have changed, if at all, since the rehabilitation of the road? In order to measure how beneficiary communities perceive their access to markets has changed since the rehabilitation of their roads, the ET conducted focus groups with community members and RUA members, and interviewed village leaders and TARURA council managers. All groups were 24 asked to reflect on how often they go to markets, travel time to markets, whether rehabilitation gave them access to new or different markets, and the types of goods and sellers present. FINDINGS Access to a Range of Markets Communities in all four districts gained access to new markets after roads rehabilitation (3 FGDs with RUAs, 3 FGDs with community, 4 KIIs with TARURA council managers, 4 KIIs with village leaders). As several stakeholders observed: “Before roads rehabilitation, we had an open market here in the village once a week. That market here is still once per week, but we also can go to Engusero and Kibaigwa markets.” [male village leader, Kiteto] “At Idandu, they have started a public open market, so it has become a business center for residents. There are some residents who sell goods from the farms and buy clothes and other items from other sellers to then sell to other residents…At Ihenga, the villagers have established a fast growing center. The residents have started shops where they sell farming implements and inputs.” [TARURA council manager] Travel Time to Markets To evaluate changes in the travel time to markets, the ET asked respondents to reflect on the time taken to travel to markets before and after roads rehabilitation. Community members in all four districts perceived that their travel time to markets had been reduced with roads rehabilitation (4 FGDs with community members, 1 FGD with RUAs, 5 KIIs with village leaders, 2 KIIs with TARURA council managers). One male village leader in Kilombero noted a particularly large decrease in travel time for his community of Mofu: “Before roads rehabilitation, we could only get some consumer goods at Ifakara, which took a whole day. But now, we can use just three hours to reach Ifakara.” The ET also asked community members, RUA members, and producers to report their travel time to market to buy agricultural produce and agricultural inputs. As shown in Figure 10, travel times to buy produce and agricultural inputs have decreased by an average of 47 percent and 55 percent, respectively. Figure 10: Travel Time to Markets to Buy Specific Goods (Minutes) Source: Evaluation questionnaire for focus group discussants, N = 64 for produce; N= 71 for agricultural inputs. When comparing changes in travel time to buy produce and agricultural inputs across the four districts (Table 12), decreases are seen across the board.12 However, the largest decrease in travel time is seen in Kilombero. 12 Note small N-values. 125 55 0 20 40 60 80 100 120 140 Before After Agricultural Inputs 171 91 0 20 40 60 80 100 120 140 160 180 Before After Produce 25 Table 12: Travel Time to Markets to Buy Specific Goods (By District) Produce Agricultural Inputs District Before rehabilitation (mins) After rehabilitation (mins) N Before rehabilitation (mins) After rehabilitation (mins) N Kilombero 379 153 11 155 98 11 Mvomero 53 19 17 82 16 23 Kiteto 115 47 10 122 53 16 Kongwa 191 129 26 165 88 21 Source: Evaluation questionnaire for focus group discussants. Availability of Goods In all districts, stakeholders said that the number of sellers in their markets increased after roads rehabilitation (2 FGDs with RUAs, 3 FGDs with community, 5 KIIs with village leaders, 4 KIIs with TARURA council managers). As one FGD of community members in Kongwa told the evaluation team: “Yes, before only natives found selling and buying in markets, but now people from different places are coming. For example, people from Arusha, Dodoma Town, Mpwapwa, Sagara, Chamkoroma are coming to Iduo market.” Stakeholders in all districts also reported that a greater variety of goods were available at their markets after roads rehabilitation (4 FGDs with RUAs, 4 FGDs with community, 2 KIIs with TARURA council managers, 3 KIIs with village leaders). Consumer goods related to cooking (e.g., flour, sugar, salt, cooking oil, home utensils) are now sold at Kilombero (1 FGD with community, 1 FGD with RUAs) and Kiteto markets (1 FGD with community, 1 KII with TARURA council manager). Agricultural inputs (e.g., fertilizer and pesticides) were reported as available in Kilombero’s new markets (1 KII with TARURA council manager, 1 FGD with community, 1 FGD with RUAs). Construction materials (e.g., timber, cement, and corrugated roofing) were said to be available in Kiteto (1 FGD with RUAs) and Kongwa (1 FGD with community). Solar-powered products were said to be easily found at markets in Kilombero (1 FGD with RUAs), Mvomero (1 FGD with RUAs), and Kiteto (1 FGD with community) after roads rehabilitation. CONCLUSIONS Beneficiary communities perceive that their access to markets has improved after road rehabilitation because they can access new markets and buy more quickly everyday items such as agricultural produce and inputs. Road rehabilitation has also brought new sellers to village markets, greatly expanding the variety of goods available for purchase and allowing for a variety of quality of life improvements. Engusero Market, Kiteto 26 EQ 4: WHAT EFFECTS, IF ANY, HAVE ROAD REHABILITATION HAD ON OPERATION COSTS OF VEHICLES? In order to measure changes in operating costs of vehicles, transporters were asked to reflect on fuel efficiency, vehicle life, and maintenance of vehicles. The evaluation team supplemented this data by asking other focus group discussants and key informants if they owned vehicles. Those who owned vehicles were also asked the same questions as transporters. Mgudeni-Mhelule Road, Kilombero FINDINGS Fuel Efficiency Transporters in all districts studied (1 KII in Kilombero, 3 KIIs in Mvomero, 2 KIIs in Kiteto, and 2 KIIs in Kongwa) reported that vehicles experienced improved fuel efficiency after roads rehabilitation. The exact improvement in fuel efficiency given by each transporter varied based on vehicle type and material of the road traveled before and after rehabilitation (e.g., earth, gravel, or paved road),13 but one male transporter in Kongwa experienced up to a 50 percent improvement in fuel efficiency. As he noted, “We have improved vehicle fuel consumption. I used 10 liters to travel from Kibaigwa to Suguta before rehabilitation, but now I use just 5 liters.” Another transporter noted, “Before rehabilitation, vehicles spent a lot of time on the road and used more fuel. Drivers had to drive slowly. But after rehabilitation, fuel efficiency of our vehicles has increased because they take less time to reach destinations by traveling at a bit higher speed.” [Male transporter, Kilombero] Life of Vehicles Transporters interviewed in all districts noted that vehicles last longer after roads rehabilitation (2 KIIs in Kilombero, 1 KII in Mvomero, 2 KIIs in Kiteto, and 3 KIIs in Kongwa) and they go longer periods of time before needing maintenance (3 KIIs in Kilombero, 2 KIIs in Mvomero, 3 KIIs in Kiteto, and 3 KIIs in Kongwa). Two transporters revealed that prior to roads rehabilitation, they had to seek repairs after every trip (1 KII Kilombero and 1 KII in Mvomero). As the transporter from Mvomero told the ET, “Before road rehabilitation, I used to repair my Canter after each trip. The minimum repair was to replace springs, and shock-ups (which cost Tshs. 120,000/=). Now, I can do service of my car once a month.” One producer in Kongwa told the ET he started driving again after the road was rehabilitated, having originally taken a break due to vehicle wear. He stated, “Before rehabilitation, vehicles were getting frequent breakdowns to the extent that I had to stop travelling. It was so disturbing. After rehabilitation, vehicles stay longer without breakdowns and this is because the road is in good condition.” In all districts visited, transporters reported that roads rehabilitation reduced their overall maintenance costs as the frequency of repairs decreased after roads rehabilitation (3 KIIs in Kilombero, 3 KIIs in Mvomero, 2 KIIs in Kiteto, 3 KIIs in Kongwa). Transporters in Kilombero (2 KIIs) and Kiteto (1 KII) mentioned that access to mechanics became easier after roads rehabilitation, while those in Kongwa (1 KII) and Kilombero (2 KIIS) noted that it became easier to access spare parts. Despite the positive improvements for drivers, one focus group of producers in Kiteto noted a potential negative effect on local mechanics, “There are fewer technicians because there are no frequent breakdowns of the vehicles, so there is unemployment.” [Producers, Kiteto]. CONCLUSIONS Roads rehabilitation has reduced vehicle operating costs, especially in the areas of fuel efficiency and maintenance, as vehicles spend less time on the roads and less time being repaired. The reduction in 13 In KIIs, 5 of the 12 transporters (2 KIIs in Mvomero, 1 KII in Kiteto, and 2 KIIs in Kongwa) offered estimates for the amount of fuel used to travel the same route before and after roads rehabilitation. These amounts varied from 15 percent to 50 percent. 27 transporters’ operational costs due to improved road condition has consequently lowered the price of passenger transport (see EQ2) and the price charged to producers in order to transport their goods to (see EQ3). EQ 5: HOW DID THE PROJECT ENHANCE THE CAPACITY OF DISTRICT ENGINEERS TO PLAN, DEVELOP/REHABILITATE AND MAINTAIN ROADS? HOW HAVE EFFORTS IN PHASE 1 BEEN SUSTAINED? IF NOT, WHAT CONSTRAINTS WERE NOT OVERCOME TO ACHIEVE THESE RESULTS? 4.5.1 Sub-question 5a. How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? In order to measure the project’s success in enhancing the capacity of district engineers to plan, develop/rehabilitate, and maintain roads, the ET reviewed several documents containing CDM Smith’s assessment of district engineers/TARURA council managers’ capacity in planning, design, procurement, contracts administration, and maintenance areas, as well as the actions taken and training provided by CDM Smith.14 To supplement this information, the ET interviewed key informants from CDM Smith, district engineers/TARURA council managers who received the capacity building interventions, TARURA Regional Coordinators, and USAID/Tanzania staff.15 Questions covered the role of district engineers throughout each of the phases (planning, design, rehabilitation, and maintenance), as well as challenges of working with them, and training performed by CDM Smith. FINDINGS Capacity Building in the Planning Phase As part of the RRDP activities, district engineers were responsible for prioritizing the roads to be rehabilitated. These initial roads prioritization efforts were driven by politics and not objective, technical criteria, according to interviews with both CDM Smith (2 KIIs) and USAID (1 KII) and the Trainings Curriculum document provided by CDM Smith. Both CDM Smith informants reported to the ET that they responded to this need by developing a prioritization tool that would help district engineers use technical criteria to prioritize roads. As one CDM Smith informant told the ET: “District engineers used to just rank the roads depending on how they think a road is more important than another. Political influence was also there. If somebody who is maybe a member of parliament comes from that area, his road will be ranked higher than the other roads, maybe if he is a good lobbyist. Our tool eliminates that kind of thing.” USAID confirmed CDM Smith’s development of the prioritization tool to the evaluation team, stating, “CDM Smith played a major role in educating councilors on the prioritization tool.”16 When questioned about the implications of transferring the management of the rural roads to TARURA, the two TARURA Regional Coordinators and two District Executive Directors interviewed said they believed that TARURA was further helping to remove politics from the prioritization process. As one District Executive Director noted, “The establishment of TARURA actually is a government move to make council managers more independent, to make sure that they can do their jobs independently without fear of any political implications or repercussions.” Capacity Building in the Design Phase 14 The ET reviewed several documents provided by CDM Smith regarding the capacity building/training provided and found them to be insufficiently detailed. See Annex X of Lists of Documents Reviewed for this evaluation. Some reports indicate an annex for course/training evaluation, but they are only placeholders and empty. As such, the KIIs with CDM Smith were used to gather additional information on their assessment of TARURA council managers and actions taken to build their capacity in key project areas. 15 All CDM Smith, TARURA Regional Coordinators, district engineers/TARURA council managers, District Executive Directors, and USAID informants were male. 16 The prioritization tool was used in IRRIP 1 under the guidance of CDM Smith. 28 After the planning phase, district engineers were responsible for producing technical drawings and making budget estimates for the rehabilitated roads (1 KII with CDM Smith). According to CDM Smith and USAID, these drawings were unsatisfactory and failed to follow the GoT’s standards for road design (2 KIIs with male CDM Smith informants and 1 KII with a USAID informant). As one CDM Smith informant said, “They had access to existing standards of road design prepared by the Government of Tanzania, but they did not know how to use them.” He also noted that some initial drawings were, “Cut and pasted from other documents and not always on point.” The USAID informant noted that, “District engineers had minimum skills needed to determine the inputs for costing and for other technical requirements in the preparation of tender document as per project standards.” In a KII with the ET, a CDM Smith informant noted a reason for this technical gap: “District engineers typically supervise maintenance on roads that are already constructed, or they take over from maintenance, so they don't have to do design because the roads are already there.” Another observed that, “District engineers were used to using estimates provided by the Road Fund Board. These estimates are uniform across the country, regardless of area, regardless of road condition…but the big problem is that those rates are for maintenance, not construction or rehabilitation of the roads, so they were too low.” To address these gaps, CDM Smith provided training and hands-on mentorship to district engineers in technical drawing and budgeting (2 KIIs with CDM Smith, 1 KII with USAID). In a KII with the ET, USAID reported that CDM Smith built the capacity of engineers in: (i) developing technical inputs for solicitations, (ii) ensuring adherence to procurement regulations, and (iii) estimating the expected budget and forecasting for the implementation of the road. Capacity Building in the Rehabilitation Phase Contract management was one of the largest gaps initially identified in CDM Smith’s capacity assessment documentation provided to the ET. It is also mentioned in the “FARIL Roads Implementation Training Curriculum.” To attempt to remedy this gap, CDM Smith provided six days of classroom training on contract administration to district officials (according to 2 KIIs with CDM Smith, 1 KII with USAID). However, CDM Smith (2 KIIs) reported that council managers’ lack of basic contract management knowledge hindered their ability to grasp training concepts and they tended to delegate attendance at trainings to their subordinates. As a result, CDM Smith and USAID noted that district engineers’ contract management skills continued to lack throughout the project period: “District engineers had a lack of experience in construction management. They have done supervision of construction works, but not thorough supervision. Perhaps because of the setup of local government authorities, you may find out that the district engineer who was supposed to be supervising the road was always busy attending official issues in the office…They don't have enough time to concentrate on supervising works. So this leads to inexperience.” [CDM Smith] “District engineers had limited skills in contract management and local contractors had their own ways of doing things which were different than what was stated in the contract, so CDM Smith coached district engineers on effective supervision of the contracts. Together with district engineers, CDM Smith provided supportive supervision to contractors.” [USAID] CDM Smith provided supervision and coached district engineers on better contract management throughout the project period, including providing templates for reporting (1 KII with USAID, 2 KIIs with CDM Smith). Despite this on-the-job assistance, problems persisted, such as when district engineers failed to inspect the newly rehabilitated roads for defects until the very end of the contractor liability period, making it impossible to have issues corrected (1 KII with CDM Smith). 29 Capacity Building in the Maintenance Phase RUA, Kilombero In the maintenance phase, CDM Smith developed the framework for setting up community groups or “Road User Associations” that would provide routine maintenance to rehabilitated roads (2 KIIs with CDM Smith). The impetus for this action was a study that highlighted the risk that existing contractors would not bid on routine maintenance contracts because of their low value (1 KII with CDM Smith). As such, CDM Smith was tasked with forming community groups with members who would be vested in the routine maintenance and upkeep of rehabilitated roads (1 KII with CDM Smith). As a first step, CDM Smith and each district’s Council Manager and District Community Development Officer identified villages along the rehabilitated roads that would host RUA groups (1 KII with CDM Smith). District officials went to each selected village to introduce the initiative to local leadership and ask for their help in selecting three people – two people from the community (one man and one woman) and one member from the village council – to be trained as “paraprofessionals” in labor-based routine maintenance (2 KIIs with CDM Smith).17 Labor-based routine maintenance training took place in two parts. First, CDM Smith provided a training of trainers with district engineers, district community development officers, and technicians. This training was led by a labor-based technology engineer (2 KIIs with CDM Smith) and lasted one week (2 KIIs with CDM Smith, 4 KIIs with TARURA council managers). After being trained, the district officials provided 4-5 days of training to paraprofessionals, including 2 days of classroom training and 2-3 days of practical training on site (1 KII with CDM Smith, 9 KIIs with paraprofessionals). This training was challenged by the fact that many participants traveled from far away, making some people tired and causing others to leave early because they were breastfeeding (1 KII with CDM Smith, TARURA council manager). CDM Smith reported that district engineers were limited in their effectiveness as instructors because they were also new to labor-based technology and had experience only with machine-based maintenance methods (1 KII with CDM Smith). In addition, one CDM Smith informant reported in a KII: “In some places, the district's technical people were not skilled enough to provide the training to paraprofessionals.” After being trained, paraprofessionals returned to their villages and formed the RUAs, though they differed in the selection method for members. In some villages, the paraprofessionals were asked to nominate people (Iduo, Kongwa: 1 KII with paraprofessional, 1 KII with village leader), in others community members were asked to volunteer (Orkine, Kiteto: 2 KIIs with paraprofessionals, 1 KII with RUA leader, 1 KII with village leader), and finally some villages converted existing cooperative groups into RUAs (Nyandira, Mvomero: 2 KIIs with paraprofessionals, 1 KII with village leader). RUAs then voted on leadership positions such as chairperson, secretary, and treasurer (3 KIIs with RUA leaders).18 Table 13 shows the gender breakdown of RUA leaders, separated by district.19 Overall, the paraprofessionals selected from the community are more male, while other leaders are generally balanced in gender. For paraprofessionals selected from the village council, men outnumber women in 3 of the 4 districts analyzed. In Kongwa and Mvomero districts, only one paraprofessional from the village council is female. 17 The report “Task Force Guidelines” noted as Deliverable 1: Guidelines for registration, recognition and procurement of road user community organizations for involvement in road routine maintenance was prepared by CDM Smith to guide the development of RUAs. 18 The qualitative protocols for RUA Leader in Kilombero did not include the question: “How did you become RUA leader?” 19 CDM Smith could not provide the evaluation team with records of all RUA members by gender and age. 30 Table 13: Gender of RUA Leaders by District Female Male Total Kilombero Chairperson 3 6 9 Paraprofessional (Village Council) 4 5 9 Paraprofessional (Community) 8 10 18 Secretary 4 5 9 Treasurer 6 3 9 Kilombero Total 25 29 54 Kiteto Chairperson 0 13 13 Paraprofessional (Village Council) 5 9 14 Paraprofessional (Community) 11 14 25 Secretary 2 10 12 Secretary/Paraprofessional 1 0 1 Treasurer 11 2 13 Kiteto Total 30 48 78 Kongwa Chairperson 4 6 10 Paraprofessional (Village Council) 1 9 10 Paraprofessional (Community) 9 11 20 Secretary 4 6 10 Treasurer 4 6 10 Kongwa Total 22 38 60 Mvomero Chairperson 1 5 6 Paraprofessional (Village Council) 1 5 6 Paraprofessional (Community) 5 7 12 Secretary 2 4 6 Treasurer 4 2 6 Mvomero Total 13 23 36 Grand Total 90 138 228 Source: “RUA Contact Information,” provided to evaluation team by CDM Smith. CONCLUSIONS The project effectively enhanced the capacity of district engineers to plan roads by producing and implementing a prioritization tool and process to objectively select roads for rehabilitation. This was viewed by all stakeholders questioned as a positive step to reduce political influence. The project also improved the capacity of district engineers in technical drawing and budgeting through training and hands-on mentorship. The project’s capacity building activities were least effective for district engineers’ contract management skills, potentially because of district engineers’ limited knowledge and experience with contract management and their failure to attend trainings held by CDM Smith. 20 The project enhanced district engineers’ capacity to maintain roads by creating RUAs that they can engage to conduct routine maintenance on FARIL roads. 4.5.2 Sub-question 5b. How have efforts in Phase 1 been sustained? FINDINGS Maintenance of FARIL Roads None of the rehabilitated FARIL roads visited by the ET had undertaken any routine maintenance since their construction, a period of more than three years (4 KIIs with TARURA council managers, 2 KIIs with CDM Smith). As one CDM Smith informant summarized in a KII: “Over the last three years, no routine maintenance has been done by the LGAs.” Instead, he said district engineers used routine maintenance funds for other purposes and only did periodic maintenance on the rehabilitated roads: 20 IRRIP 2 has a large focus on contract management, including work by CDM Smith to develop a contract management manual and templates. Effective utilization of this manual should be a priority in future roads rehabilitation projects. 31 “We saw that once a road was finished, the LGA, the councilor, the DED, whoever, they think that the road does not need any more money for two or three years. So they don't set aside funds for maintenance when the road is in very good condition because they just received funds for it. They set the money aside to fund other roads which they think need more money. They feel it’s a waste of funds maintaining good roads.” [CDM Smith] The first routine maintenance contract with a RUA was completed in early 2018 (1 KII with CDM Smith, 1 KII with TARURA council manager). The main cause for delay was that, “RUAs were not allowed to be registered as contractors at the district level and therefore could not start working” (CDM Smith). CDM Smith and USAID explained that at the start of the project they believed that the RUAs would be eligible for contract after being registered as community groups at the district level (1 KII with USAID, 2 KIIs with CDM Smith). However, meetings with numerous government organizations revealed a 1997 regulation that required all groups doing road maintenance to be registered through the Contractors Registration Board (CRB) at the national level (1 KII with CDM Smith). RUAs began the registration process in October 2017 and it was completed in February 2018 (1 KII with CDM Smith). Registration costs each RUA a one-time fee of 105,000 TZS, with an annual renewal fee of 75,000 TZS (1 KII with CDM Smith). Though all RUAs are now registered with CRB, only nine out of 38 RUAs have maintenance contracts with TARURA (See Table 14). Further, only two districts (Kongwa and Kilombero) have budgeted in a way that will allow them to partner with all of the registered RUAs on their roads (1 KII with CDM Smith). One reason behind this could be that district engineers/TARURA council managers were unfamiliar with labor-based technology prior to this project and many are still much more comfortable with machines than RUAs, according to CDM Smith (2 KIIs). One TARURA council manager told the ET that his district’s roads required maintenance beyond what RUAs could provide: “For the past period we failed to contract them as the roads we heavily affected to the point that those groups could not be able to fix, as some sections required structure maintenance, some required excavator let’s say, so it was difficult to assign those activities to these groups as they can’t afford to hire excavator as these were activities that required machines, not labor-based.” [Male, TARURA] In Kongwa and Kiteto, locations where the RUAs are not yet contracted, RUA members are losing hope that work will ever come (3 KIIs with paraprofessionals, 2 KIIs with RUA leaders) and groups are experiencing dropouts (2 KIIs with RUA leaders): “There are a few people that left the group and we’ve replaced them. They saw the project was late so they left.” [Male RUA Leader, Kiteto] “For the RUA to be sustainable, we just need to go to work. Otherwise, RUA members will go do their own work and leave the RUA. I always tell the RUA members to remain calm every day so they don’t leave.” [Male paraprofessional, Kongwa] Table 14 summarizes the condition of each of the FARIL roads during the evaluation team’s fieldwork period in June 2018. Table 14: Condition of FARIL Roads (June 2018) District Road Length Type Contracted RUAs Quality (June 2016) Kilombero Mgudeni-Mhelule 10.0 km Earth 2/3a Goodb Namwawala-Mofu 17.3 km Earth 4/4 Fair Mpanga-Ngalimila 10.0 km Gravel 2/2 Poor Mvomero Langali-Nyandira 5.2 km Paved 1/2 Good Wamidakawa-Dihombo 19.4 km Gravel 0/4 Good Kiteto Engusero-Orkine 19.4 km Gravel 0/3 Good Engusero-Njiapanda 25.5 km Gravel 0/8 Good Olboroti-Matui 32.0 km Gravel 0/3 Fair 32 District Road Length Type Contracted RUAs Quality (June 2016) Kongwa Suguta-Kibaigwa 25.8 km Gravel 0/6 Good Pandambili-Njoge 15.2 km Gravel 0/4 Good Source: Data collected from interviews with TARURA Council Managers. Note: a All RUAs on Mgudeni-Mhelule Road were provided with requests for quotation in order to be contracted, but Kiswanya Village did not sign the contract because the timing conflicted with their farming. The other two RUAs, Mgudeni and Mhelule, were given additional work to cover the routine maintenance need. b Good: Entire road is passable throughout the year, may need spot maintenance in certain areas; Fair: Road is passable for most of the year, medium level of deterioration; Poor: The road is not passable, high level of deterioration RUA Sustainability To measure RUA sustainability, RUAs, RUA leaders, and paraprofessionals in Mvomero, Kiteto, and Kongwa were asked what strategies they have to ensure they will remain sustainable. 21 They revealed that RUAs have regular meetings (1 FGD with RUAs, 3 KIIs with RUA leaders, and 2 KIIs with paraprofessionals) and require entrance fees -- typically 5,000 TZS -- and regular dues from their members, typically 1,000 TZS per month (2 FGDs with RUAs, 2 KIIs with RUA leaders, 1 KII with paraprofessional, 1 KII with village leader). To add to their bank accounts, RUAs are partaking in various income-generating activities in their communities (3 FGDs with RUAs, 5 KIIs with paraprofessionals, and 1 KII with a RUA leader): “At Engusero, we get involved with other community works that require manpower. When the village government needs us, they use us. For example, during the rainy season, the Engusero dam was damaged and the government asked us to go repair. We now have a contract with village government to support labor work during the construction of the village market and also to collect stones and other building material needed to rehabilitate school building.” [RUA member, Kiteto] “The RUA helps farmers and transporters carry their goods and we get paid for this. We take 25% of that amount and put it in the RUA account. The rest is kept for families.” [Female paraprofessional, Mvomero] “We haven’t started work officially, but we do some work on our own for money. We did not have the equipment to go work on the road, so we worked on the farms and gained 60,000 TZS as a group. We opened a bank account and saved some of the money we’ve made there.” [Female RUA Leader, Kongwa] CONCLUSIONS Phase 1 efforts have not been sustained because LGAs failed to provide routine maintenance on the rehabilitated roads for a period of more than three years. The registration process for RUAs was unexpectedly long, yet despite all being registered, only one in four RUAs has been contracted. Contracting with RUAs is essential to the long-term sustainability of the rehabilitated roads. A few RUAs are contributing to their own sustainability by partaking in other income-generating activities and adding their earnings to their group bank accounts. However, since RUA members include individuals from vulnerable groups, there is a risk that RUAs not contracted soon could have difficulty holding on to their members. 21 Pilot qualitative protocols in Kilombero did not include this question: What strategies do RUAs have to ensure they will remain sustainable? 33 4.5.3 Sub-question 5c. If not, what constraints were not overcome to achieve these results? The ET identified several constraints that hampered project effectiveness, including a lack of enforcement mechanisms regarding allocation of funding from the Roads Fund Board, issues regarding RUAs, and other external constraints outside of the project’s control. FINDINGS Enforcement Mechanisms All revenue used for rural road works in Tanzania comes from the Roads Fund Board, which is funded by fuel levies and road revenues (1 KII with CDM Smith). This money is passed to TARURA, who passes the money directly to districts. Though the Roads Fund Board is viewed as a reliable source of money, it cannot provide enough funds to maintain all of the rural roads and districts are expected to set aside their own revenues to supplement funds from the Road Fund Board. In practice, this does not happen (4 KIIs with TARURA Council Managers, 1 KII with TARURA Council Manager, 1 KII with District Executive Director, 2 KIIs with CDM Smith): “The government has asked that when each district council makes their budget, they allocate their own source of funding which will be submitted to the council manager for road maintenance. But we haven't received any information that a council has allocated their own source of funding for road maintenance. It is not compulsory.” [TARURA Regional Coordinator] “I have never heard of a district having a separate line item for road maintenance from their own sources. They just depend on the Road Fund Board for 100% of maintenance. And in development projects, they depend on the donors for improvement of roads and upgrading them.” [CDM Smith] Maintenance funds from the Roads Fund Board are often used by districts to expand their road networks instead (1 KII with TARURA council manager, 1 KII with CDM Smith). Though regulations exist to prevent this, there is no enforcement mechanism and districts do not face financial penalties.22 CDM Smith explained efforts going forward in this area in a KII with the evaluation team: “The core goal is the allocation of funds from TARURA to routine maintenance. Previously, the money used for routine maintenance was used for spot improvement or rehabilitation. So roads were not being maintained properly. TARURA is working with them to make sure that council managers are setting aside money to do routine maintenance. That is a nationwide policy.” Issues Regarding RUAs The ET identified several constraints to project effectiveness regarding RUAs in the areas of reference materials and equipment, membership, payment, ownership by RUA members, and retraining of underperforming RUAs. Reference Materials & Equipment 22 This assessment is based upon the ET’s review of “The Road and Fuel Tolls Regulations, 2016” and “Roads Fund Board Clients’ Service Charter” and KII with CDM Smith. 34 After reviewing the training manual used to train district engineers and paraprofessionals in labor￾based technology,23 the ET questioned CDM Smith as to whether paraprofessionals were given any physical copies of the training manual. CDM Smith reported that they did not give out copies of the manual, but rather provided paraprofessionals with a pictorial brochure24 outlining the 11 routine maintenance tasks25 (1 KII with CDM Smith).26 Paraprofessionals returned their brochures to CDM Smith at the conclusion of training and did not receive them again until their first day of work as a RUA (1 KII with CDM Smith). On reviewing this pictorial brochure, the ET felt that while it showed pictures of the 11 maintenance activities, it lacked detailed information needed for paraprofessionals to ensure quality control of tasks under their supervision. To gather another perspective, the ET also questioned paraprofessionals in Engusero, Kiteto, and Iduo, Kongwa, directly about whether they received any training materials or documents during the training that show how to do maintenance on the roads.27 In KIIs, all paraprofessionals interviewed told the ET they took notes during the classroom trainings, but not during the practical sessions, nor did they have manuals showing how to do each task (3 KIIs with paraprofessionals). 28 As one male paraprofessional in Kongwa noted: “We received some papers that included a map of the road and a full list of equipment needed to conduct maintenance. We wrote our own notes about how to do the maintenance.” To understand any limitations regarding tools and equipment, the ET reviewed CDM Smith’s list of tools required for a community-based organization to register with the CRB to undertake routine maintenance,29 and questioned RUAs and paraprofessionals about equipment they received from district engineers. While contracted RUAs30 had received some safety gear (e.g., rain boots, safety vets, and helmets) and hand tools, all paraprofessionals interviewed in those RUAs noted that they needed more equipment to be effective (4 KIIs with paraprofessionals): “We don’t have enough equipment for everyone. It’s shared, and it’s not enough. For slashers, we have three and there are 30 people. The shovels are three, while the wheelbarrow is just one.” [Female paraprofessional, Kilombero] “We received trolleys and slashers, but the tools are not enough. We know how to make culverts, but we don’t have enough tools to make culverts.” [Male paraprofessional, Mvomero] 23 “Labor Based Roadworks Manual,” provided to the ET by CDM Smith. 24 “Mwongozo Wa Utunzaji Barabara Kwa Mafundi Wasaidizi Wa Barabara Wa Vijiji,” provided to the ET by CDM Smith. 25 The 11 maintenance tasks include: clearing and removal of obstructions; desilting and clearing of culvert barrel, inlet, and outlet; culvert repair of head/wing walls and apron; desilting and clearing of mitre and water catchment drains; desilting and clearing of roadside drains; repairing of scour checks and side drain erosion; repairing shoulder erosion and planting grass; filling potholes and ruts in carriage way by gravel; grubbing the edge of the carriageway and camber reshaping; grass cutting; and bush clearing. 26 The evaluation team received this information during a follow up call with CDM Smith after field work had completed and thus was not able to confirm the distribution of the brochure with contracted RUAs in Kilombero and Mvomero. 27 The question “Did you receive any training materials or documents during the training that show how to do routine maintenance on the roads?” was added to instruments after field work had already been completed in Kilombero and Mvomero. 28 One paraprofessional in Engusero, Kiteto, brought her training documents and notebook to the KII with the ET. The ET reviewed her documents and handwritten notes. The training handouts contained background information and lists of equipment to be used, but provided no guidance on how to accomplish routine maintenance tasks. Since her RUA had not yet been contracted, she did not have the pictorial brochure in her possession. 29 “The Task Force Guidelines,” provided to the ET by CDM Smith. 30 The ET conducted qualitative research with contracted RUAs in Mofu, Kilombero, and Nyandira, Mvomero. RUA, Kilombero 35 The ET assessed the limited number of tools in each RUA to be in direct contradiction to CDM Smith’s list of tools required for a community-based organization to register with the CRB to undertake routine maintenance. 31 RUAs that have yet to be contracted32 have no equipment at all except for the safety gear given to paraprofessionals (5 KIIs with paraprofessionals, 3 KIIs with RUA leaders, 1 KII with village leader, and 3 FGDs with RUAs). “Our RUA was not given any equipment, only the safety equipment paraprofessionals got at training. I always use my safety vest to put in the middle of the road to show that people are working here because we don’t have traffic cones. We use our own hoes.” [Female paraprofessional, Kiteto] “Paraprofessionals got rainboots, helmets, and vests. No one else in the RUA has received equipment. We have not received any tools.” [Female paraprofessional, Kongwa] Membership While there are women members in the RUAs,33 pressure from spouses may have prevented some women from joining the RUAs in Kilombero. One CDM Smith informant noted in a KII that: “In Ihenga, Kilombero, the Sukuma men said they were not ready for their wives to be part of the RUA groups.” A female paraprofessional in Kilombero corroborated this finding: “If someone is married, their husband will be jealous and they won’t allow them to go for training.” However, these were the only two mentions of barriers from spouses and they were limited to Kilombero District. The same CDM Smith informant also noted that youth in the project areas were not always interested in working for the RUAs, but he was the only stakeholder to mention barriers to youth recruitment: “I think sometimes a majority of the youth do not want to do manual work, they feel it is tedious. On Namwawala-Mofu Road in Kilombero, the limitation of youth is high because youth don’t want to be involved in farming or road works.” In Kiteto, RUA members felt that the selection process for paraprofessionals was not transparent and wanted it to be done in a village meeting open to everyone (1 FGDs with RUAs). While it potentially created problems within the RUA, this problem was unique to this one location. Payment Contracted RUAs in Mvomero and Kilombero are paid 8,000 TZS per day worked, due upon the completion of each task (2 KIIs with TARURA Council Managers, 3 KIIs with paraprofessionals, 1 KII with RUA leader, and 1 FGD with RUAs). This daily rate was individually set by each council manager with CDM Smith’s guidance34 and is primarily based on the market rate for the routine maintenance tasks RUAs will do, not the opportunity costs to RUA members who could be engaged in other activities (1 KII with CDM Smith). A CDM Smith informant said in a KII that RUA groups have the ability to request a higher daily rate when negotiating their contracts, but attempts thus far have been unsuccessful. Various stakeholders in areas with contracted RUAs brought up RUA’s low daily rate as a challenge (4 KIIs with paraprofessionals, 1 KII with village leader, 1 FGD with RUAs) and noted that payments need to be made sooner (1 KII with village leader, 1 FGD with RUAs) As one FGD of RUA members told the evaluation team: “We have been working for more than three weeks and have yet to be paid, waiting for the check to be written and money to be transferred in our account. A majority of RUA members do not have anything to eat after the whole day of working in the road.” [RUA, Kilombero] 31 “Task Force Guidelines” developed by CDM Smith indicate the following number of minimum tools required for a CBO group to work on routine road maintenance: hoe (10-15), shovel (10-15), panga (10-15), slasher (10-15), rake (10-15), wheel barrow (5), axe (10-15), pick axes (10), and hand compactor (2). 32 RUAs in Dakawa (Mvomero), Engusero and Orkine (Kiteto), and Iduo (Kongwa) had not been contracted at the time of this evaluation. 33 As noted in EQ5a, CDM Smith could not provide the ET with rosters of RUA members by gender. 34 In a follow up call, CDM Smith told the ET that the minimum RUA daily rate is the national minimum wage of 8,000 TZS, while the ceiling depends on the total amount of Road Fund Board money allocated to routine maintenance in the district. 36 Other External Constraints A final category of constraints relates to changes by the GoT that occurred during the project period. As noted previously in this report, TARURA was established in July 2017 and management of all rural roadworks was transferred under the new institution. To understand how this change affected (and could continue to affect) project success, the evaluation team questioned District Executive Directors, CDM Smith, and USAID about implications of the new structure. At the time of interview, these stakeholders felt that TARURA was understaffed (1 KII with DED, 1 KII with USAID, and 1 KII with CDM Smith): “TARURA is a newly established institution. It is understaffed, it does not have enough resources. We, the local government, are understaffed in different departments and the infrastructure department is one of them. They were understaffed even before so now when we are talking about dividing staff between us and TARURA, it makes the situation even worse. But actually the government has promised to bring more staff for us and also for TARURA. We are looking forward to receiving them.” [District Executive Director] “I am anticipating a lack of human resources at the regional level of TARURA to monitor all the contracts of the various districts under their supervision. What if there are ten contracts in one district and you have five districts in your region, then that's fifty contracts to manage in a financial year. How can that be managed by a few people at the regional level? This will be a big challenge.” [CDM Smith] CONCLUSIONS The main constraints to project success include the lack of an enforcement mechanism to require that funds be spent on routine maintenance, the lack of reference materials and equipment provided to RUAs in order to execute routine maintenance tasks, issues of payment to RUAs, the lack of ownership by RUA members in certain areas, and factors related to the establishment of TARURA. The lack of reference materials and equipment among RUAs is likely the easiest solved. Raising the daily rate for RUA work to make it more competitive could help to increase engagement and support in communities. Constraints regarding TARURA, both at the headquarters and district level, will likely persist. UNEXPECTED RESULTS OF PROJECT ACTIVITIES After the pilot testing period, the ET chose to include the following question in all FGDs and KIIs with beneficiaries to ensure we fully captured their opinions regarding project activities: “Were there any results of roads rehabilitation that you did not expect?” The ET then grouped the most common responses into positive and negative results below, supplemented by findings from the rest of the study. FINDINGS 4.6.1 Unexpected Positive Results Employment & Income-Generating Activities Stakeholders in all districts perceive roads rehabilitation as having increased employment for youth (2 FGDs with community, 1 KII with TARURA council manager, 5 KIIs with village leaders). Youth are reportedly finding work carrying goods or people via motorcycle (1 FGD with community, 2 KIIs with village leaders), and working as porters (2 KIIs with village leaders). Stakeholders in Kilombero, Mvomero, and Kongwa also perceived rehabilitation to have increased women’s employment and women (3 FGDs with community). Women have started restaurants and beverage shops along the roads (3 FGDs with community), and others are engaged in selling their produce (1 FGD with community). As one FGD with community told the ET: “Women during road rehabilitation had an opportunity of cooking and selling food in the site camp. After rehabilitation food vending business has reduced, it is more undertaken during the open markets days, however there are a few cafes which continue working in the village centers.” [Community, Kilombero] 37 The ET also sought to measure the project’s effect on community members who held more formal employment. To do so, the ET asked community members, village leaders, and council whether they perceived any changes in travel time to work before and after roads rehabilitation. Stakeholders in all four districts reported that the travel time to work had shortened (4 KIIs with village leaders, 3 FGDs with community). Various stakeholders also brought up that the rehabilitation had allowed social workers to live in rural villages for the first time (1 KII with CDM Smith, 1 KII with TARURA council manager, 1 KII with village leader, 1 FGD with producers, and 1 FGD with community). New Businesses To measure the effect of roads rehabilitation on business, the ET asked stakeholders to reflect on whether roads rehabilitation had an effect on business in their community. Stakeholders in Kilombero reported that after rehabilitation, people established new restaurants and cafes (1 FGD with community, 1 KII with TARURA council manager), hotels (1 KII with TARURA council manager, 1 KII with village leader), and an outlet for mobile money (1 KII with village leader). In Kiteto, stakeholders noted new restaurants and cafes (1 FGD with community, 1 KII with village leader), wholesale shops (1 FGD with community, 1 KII with village leader), and a petrol station (1 FGD with community, 1 KII with village leader, 1 KII with TARURA council manager) came after rehabilitation. In Kongwa, stakeholders reported the presence of wholesale shops (1 FGD with community) and restaurants and cafes (1 KII with village leader) after rehabilitation. Stakeholders in Mvomero did not report as much new business activity but did note an increase in shops (1 FGD with community, 1 KII with TARURA council manager) and restaurants (1 FGD with community). Quality of Life Improvements When questioned about unexpected results, many stakeholders spoke of quality of life improvements. After roads rehabilitation, villages in each district have connected to electricity for the first time (2 FGDs with community, 1 FGD with producers, 1 KII with a village leader, 1 KII with a TARURA council manager, 1 KII with a processor). As one FGD with community members in Kiteto reported: “Before road rehabilitation we only used to see electrical wires and solar powers from our neighbor’s towns, but now we have them here.” Water wells have been dug in Mofu, Kilombero, (2 KIIs with village leaders, 1 KII with council manager) and taps have been installed in homes in Nyandira, Mvomero (1 FGD with community). Stakeholders in Kilombero (1 KII with village leader) and Mvomero (1 FGD with producers) reported that mobile towers had been built, increasing villagers’ communication with the outside world. The quality of homes and other village buildings was noted to have improved in Kongwa (1 FGD with community) and Mvomero (1 FGD with producers) as trucks have been able to reach villages with cement and other construction materials. New Farming Methods In addition to construction projects, two FGDs of producers said that the rehabilitated roads have made it easier for development projects to introduce improved farming methods to communities: “There are more development projects coming into our village providing awareness on commercial farming because they have seen that transportation to the market is facilitated.” [Producers, Kongwa] “We got free education from the Farmers Association (MVIWATA) on farming especially cleanliness before getting in a farm which was very helpful to us.” [Producers, Mvomero] Road Safety In Kiteto, road signs that arrived after the road rehabilitation are perceived to have increased safety (1 FGD with producers, 1 KII with a village leader). Some of these road signs show crosswalks that warn drivers to slow down for people or animals that might be crossing, while others warn drivers of upcoming culverts, bridges, and other structures. A female village leader in Kiteto explained the importance of these road signs: “Now, we have road signs showing crosswalks for people or animals, showing bridges. Before, we did not have them. It’s important because a large percent of people here are livestock keepers and the road is used by everybody. The signs really help those who pass in cars to know to go slowly because of people or animals. It helps the driver know to slow down at culverts.” 38 4.6.2 Unexpected Negative Results Accidents Accidents were the most mentioned unexpected negative effect of roads rehabilitation among stakeholders (2 FGDs with community, 1 FGD with producers, 1 KII with village leader, 1 KII with paraprofessional, 1 KII with TARURA council manager, and 1 KII with processor). While accidents are named as a problem in all four districts, the problem appears largest in Nyandira, Mvomero, where the rehabilitated road is viewed as too narrow. According to USAID (1 KII), the narrow width of Langali-Nyandira Road was an attempt to reduce the speed of travel for motorcycles and vehicles, but community members requested more speed control measures (1 FGD with community, 1 KII with village leader). Outsiders Roads rehabilitation is perceived to have allowed outsiders into villages, raising safety concerns (1 FGD with community, 1 KII with transporters) and fears of cultural erosion (1 FGD with RUAs. Community members are concerned that new people are bringing HIV/AIDs (1 FGD with community) and prostitution (1 FGD with RUAs) and one male transporter noted an increase in car robberies along Wamidakawa-Dihombo Road: “The diversity use of road has made it to attract robbers. Since road rehabilitation we have witness several incidence of car robbers at Rice Mill area.” RECOMMENDATIONS The ET met as a team to discuss the findings and conclusions and developed the following recommendations. While it was not possible to share the recommendations with key stakeholders, the evaluation team’s recommendations include a great deal of stakeholder feedback. As part of the KIIs and FGDs, the ET received feedback on project activities and recommendations for improvement from beneficiaries and implementing partners. Additionally, the two Tanzanian nationals on the ET (one of which is a roads expert) also provided “ground-truthing” of the recommendations in the Tanzanian context. Below the ET suggests a matrix that distinguishes which entity or organization could be responsible for implementing each recommendation, as well as the immediate and future priorities for each executor. The ET recognizes that USAID cannot take the lead in all suggested recommendations, but could explore suggesting this to TARURA/GoT as deemed appropriate. Some of the more difficult recommendations to implement are listed as suggestions for the future. Recommendations for USAID/Implementing Partners Immediate Priorities 1. Ensure RUAs receive an appropriate amount of safety equipment and tools to execute contracts effectively. (with support from LGAs and TARURA) All RUA members need their own safety gear (i.e., helmets, vests, rain boots) and enough tools for everyone to be active at the same time as per the CRB regulations. If the expectation is that RUA members split up and work on multiple tasks simultaneously, ensure this is clear to them and outline how to best achieve the work in the allotted time. 39 Recommendations for USAID/Implementing Partners 2. Continue to build contract management capacity among council managers and ensure CDM Smith’s manual is distributed and utilized. CDM Smith identified contract management as a major weakness for council managers during Phase 1 and it has been a large focus of Phase 2. This is critical, especially as RUAs are beginning to work. RUA members’ skill levels will be far below those of the contractors council managers are used to working with, so they will need to be much more hands on than they have been in the past. Given that TARURA is already understaffed at the regional level and will likely continue to be so into the future, support from above may be limited. The ET recommends that TARURA distribute the contract management manual created by CDM Smith as part of Phase 2 and enforce its utilization. Additionally, the ET recommends that an assessment be completed to inform the extent to which capacity building has been achieved after Phase 2. Suggestions for the Future 1. Require quotas for youth membership in RUAs. (in conjunction with TARURA) Most RUA members are farmers, but youth are moving away from farming work, out of their home villages, and seeking employment elsewhere. This effect is made greater by the rehabilitated roads which give youth expanded access to other job opportunities. This will affect the membership base of RUAs which will also have implications on sustainability. As a result, it is recommended to set a quota for youth membership, perhaps 20-30 percent, similar to the quota for women (50 percent). 2. If future projects will require council managers to do hands-on road design, provide additional training and mentorship in this area. Though CDM Smith worked with council managers on road design, it was their first time producing original technical drawings and they are still very inexperienced. If this will not be a priority into the future, consider directing resources instead toward abuilding capacity in contract management (as discussed above). 3. Extend support to RRDP to improve the bottlenecks caused by poor quality access roads in order to realize the maximum potential of rehabilitated roads. (in conjunction with TARURA) Access roads leading to all FARIL roads were reported to be in poor condition, reducing the efficiency of movement along rehabilitated roads. Stakeholders requested that USAID install culverts to address these bottlenecks. USAID could tackle this challenge by either rehabilitating the access roads or by requiring that TARURA improve access roads in order to obtain future support from USAID. 4. Undertake data collection prior to roads rehabilitation in order to establish a baseline. (in conjunction with TARURA) Undertake data collection prior to roads rehabilitation in order to establish a baseline: The prioritization tools developed in Phase 1 include useful data such as traffic counts, roads usage metrics, population affected by the road etc. If these same tools are used in Phase 2, future evaluators will have access to a wealth of baseline data. Prioritization data can also be supplemented by collecting additional quantitative data in the form of baseline household and business questionnaires along each selected road. 40 Recommendations for TARURA/Other GoT Immediate Priorities 1. Time routine maintenance work so that it does not conflict with harvest season. Though routine maintenance must be done within a set time period each year, TARURA can offer some flexibility in scheduling the work so that it occurs at a time when members are most able to work. This will help 3with RUA sustainability if members’ other livelihood activities does not be get derailed during the short maintenance season. Open negotiation with RUA leaders can assist TARURA in timing the contracts effectively. 2. Ensure consistent work for the RUAs to cover the cost of RUAs’ registration and annual fees. Currently RUA members are incurring the one-time registration cost of 105,000 TZS and 75,000 TZS in annual fees to keep their registrations active, and do not have sufficient work to cover this cost. This creates undue burden on RUA members and also creates a free rider problem as other community members do not have to contribute. 3. Consider awarding bonuses and social recognition to high-achieving RUAs and their communities. While monetary awards might be the best incentives for RUA members in the most economically-strapped project areas, social recognition can increase the standing of RUAs in the community and help increase demand for joining the RUA when needed. This may also encourage public support for RUA members and their work. 4. Pay RUAs more frequently so they can attend to their basic needs. PPRA guidelines require that RUA members come from vulnerable groups including women, the elderly, youth, and people with disabilities. These types of people need to be paid daily, but what is happening under the current system is that RUAs are only paid after each task is fully complete. In order to engage these vulnerable groups, RUA payments must be made daily as is standard with other day labor work in Tanzania. To determine the best way forward, the ET recommends that TARURA look to other road development projects, including those done by TASAF I and II and TANROADS, for existing models of payment to vulnerable groups. 5. Create an enforcement mechanism to ensure that routine maintenance money from the Road Fund Board is spent on its intended purpose. (in conjunction with GoT) Council managers have been using routine maintenance funds to finance other road projects, starving the rehabilitated roads of routine maintenance. One way to counter this would be for the Road Fund Board to withhold periodic maintenance funds until council managers demonstrate that routine maintenance activities have been completed on their roads. This could also help to encourage the quick uptake of RUAs. 41 Recommendations for TARURA/Other GoT Suggestions for the Future 1. Simplify the initial training and also offer paraprofessionals regular opportunities to refine their skills and learn from each other. Paraprofessionals were unified in their request for more training. By holding regular workshops with paraprofessionals from across the district, TARURA can tackle common problems and introduce new skills without having to work with each group individually. This can also serve as a place for paraprofessionals to learn from each other, sharing challenges and solutions from their own communities. The same type of workshop could be instituted for RUA leaders in order to improve administrative issues. 2. Base the RUA member daily rate on supply and demand of people to work in the RUA, not on the minimum wage or typical market rate for routine maintenance tasks. For community members to want to work in the RUA, the road work must pay better than other income-generating options that are more reliable and offer year-round work. Useful insights on which to base daily rates can be gained from speaking to CDCOs about RUA recruitment efforts. It is likely that a lower rate can be offered in places where people were idle and eager to join, but a higher rate will be needed in areas that experienced difficulty recruiting RUA members. A community’s daily rate for RUAs will likely be correlated with its proximity to a market center and corresponding employment opportunities. 3. Increase community consultation during the design and planning phases in order to improve the accuracy of budgeting. (with support from LGAs) The FARIL roads have begun to deteriorate, brought about in part by the lack of attention to physical details during road design and planning phases, as well as poor budgeting for maintenance required in each specific environment. Future USAID rehabilitation efforts should incorporate effective consultation of various stakeholders during the design phase to reduce unforeseen and unanticipated damage. The project should also advocate for community consultation during Road Surface Assessments done by TARURA when planning and budgeting for road maintenance funds. 4. Pay RUA members, RUA leaders, and paraprofessionals according to their increasing levels of responsibility. RUA leaders and paraprofessionals do more than the rest of the members and their pay should be higher. However, increasing their pay should be done in a purposeful way. Prior to increasing an individual’s pay, TARURA should ensure that the responsibilities of his or her role are well-understood, as well as what will happen if he or she fails meet expectations. It is also important for the responsibilities and qualifications of RUA leaders and paraprofessionals to be shared with the community. Unlike the RUA leaders, paraprofessionals were selected by the government and thus there is some mystique about why certain people were chosen or not chosen for this role. To encourage buy-in, share the criteria and qualifications of those selected. 5. Educate the public on what routine maintenance should look like and empower them to hold district officials responsible. (with support from USAID/IPs) This effort could be accomplished by placing a visual symbol along each road signaling the quality of routine maintenance (e.g., a colored flag symbolizing the quality of routine maintenance on the road). This type of visual cue is easy to explain to community members and does not require any prior knowledge of road engineering. While the visual alone may be enough to cause community members to question their officials, it will be even more effective if paired with a consequence to the village for a poor rating. 42 Recommendations for TARURA/Other GoT 6. Consider ways to mitigate predictable negative consequences of roads rehabilitation. Solutions could include, for example, putting up more road signs to decrease speeding and traffic accidents or establishing checkpoints to restrict vehicles exceeding road weight limits. 43 ANNEXES 44 ANNEX I: EVALUATION STATEMENT OF WORK STATEMENT OF WORK Qualitative Performance Evaluation OF Rural Road Development Project (RRDP) I. PURPOSE OF THE EVALUATION The short-term performance evaluation, primarily based on qualitative information, will examine socio-economic effects of the rehabilitation of a set of roads in the Kilombero, Kiteto, Kongwa, and Mvomero Districts. A lack of baseline and monitoring data in the social sectors compels the evaluation team to rely on data gathering in affected communities through rapid techniques. The evaluation results will serve to determine a first approximation of effects from this type of intervention, enlighten USAID personnel on how communities perceived changes following the completion of rehabilitated roads, and help establish a list of sectors where changes have been identified by beneficiary communities. Furthermore, the lessons learned during the implementation of this qualitative evaluation will be used to develop a more robust performance evaluation of the second phase of the rural road development project. II. SUMMARY INFORMATION Table 1: Summary information on roads activities (Phase 1) under this qualitative evaluation Activity Name Implementing Partner Agreement Number Amount (USD) Life of Activity Region Mission DO Rehabilitation of three district roads totaling 37.3 km Kilombero LGA 621-IL-14- 00004 1,408,123 7/4/14 – 2/28/16 Morogoro DO2/IR2.1 Rehabilitation of two roads totaling 24.6 km Mvomero LGA 621-IL-14- 00007 973,276 7/4/14 – 2/28/16 Morogoro DO2/IR2.1 Rehabilitation of two roads totaling 41.0 km Kongwa LGA 621-IL-14- 00008 1,069,878 7/4/14 – 2/28/16 Dodoma DO2/IR2.1 Rehabilitation of three roads totaling 76.8 km Kiteto LGA 621-IL-14- 00009 2,060,667 7/4/14 – 2/28/16 Manyara DO2/IR2.1 III. BACKGROUND In recent years, Tanzania has devoted resources to improve its trunk roads, particularly those linking upcountry regions with the ports and neighboring countries to form a regional transportation network. Notable transport routes are the central and northern corridors. Despite the substantial investment in trunk roads, rural roads (consisting primarily of district collector and feeder roads) have not received sufficient attention. It is estimated that 70 percent of the marketing costs for commodities emanates from costly and delayed transport. It is also estimated that between 20 percent and 40 percent of agricultural produce is lost during post-harvest operations, which can be attributed to the delay in transport of produce from farms to processing and storage facilities. Rural roads are critical to market access and some studies have demonstrated how higher poverty rates correlate with increased transportation distance to key arterial roads. Tanzania has been designated a priority country for the U.S. Government’s (USG) Feed the Future (FTF) Initiative, which aims to address the root causes of global hunger by sustainably increasing agricultural productivity to meet the demand for food, supporting and facilitating access to markets, and increasing incomes for the rural poor so they can meet their food and other needs. 45 Additionally, the Partnership for Growth Constraints Analysis (2011) was used as a basis for the U.S. Government and Government of Tanzania’s (GOT) Joint Country Action Plan (2012-2016) which identified inadequate rural roads as one of the two key binding constraints to private investment and economic growth – the other being the supply of electric power. As such, USAID agreed to support GOT’s efforts to address that major binding constraint through investments in rural roads to increase access to markets. A. Description of the theory of change USAID is implementing roads rehabilitation activities through government-to-government (G2G) agreements, initially (Phase 1) directly with district governments (called local government authorities or LGAs) and then (Phase 2) through the Roads Fund Board which is the national funding agency for district roads works. Improving the network of rural roads in FTF target areas is central to increasing market access for agricultural production and facilitating the growth of rural commerce as well as improving the delivery of important social services such as health and education. If the condition of rural roads is improved and capacity of local government authorities involved in managing rural roads infrastructure is increased, then farmers will have greater access to agricultural inputs and markets, rural commerce will be increased, and access to and delivery of social services will be improved in a more sustainable infrastructure sector. See Figure 1: Components of the Logical Framework Figure 1 Components of the Original Logical Framework See annotated maps in Annex 1. 46 B. Summary of the roads activity to be evaluated The Irrigation and Rural Roads Infrastructure Project (IRRIP) is an FTF activity implemented by CDM Smith that provides technical assistance and capacity-building to the GOT to carry out improvements on up to 1,500 km of rural roads – subject to the availability of future funding. Rural roads in this context are defined as district collector and feeder roads being 4.5 meters or 6 meters in width and falling under the responsibility of LGAs. IRRIP also provides USAID with quality assurance for the road rehabilitation/construction which is undertaken through government-to￾government (G2G) agreements. The rural road program has been divided into what is informally termed as Phase 1 having individual G2G agreements with each LGA as outlined in Table 1, and Phase 2 which is one G2G agreement with the Roads Fund Board for an ongoing activity titled Rural Roads Development Project (RRDP). The IRRIP activity directly assisted the LGAs in implementing Phase 1 in regards to determining an initial set of road works that they can design, procure, and supervise themselves. In Phase 1, USAID signed fixed amount reimbursement implementation letters with Kilombero and Mvomero LGAs in Morogoro region, and in Kiteto and Kongwa LGAs in Manyara and Dodoma regions, respectively. These four agreements totaled $5,515,946 to cover 179.7 km of road rehabilitation and upgrades which were completed in January 2016. Phase 2, under RRDP, will support those same four LGAs to rehabilitate 304 km of roads with construction anticipated to start in April 2017 for two of the districts with the other two starting construction by June 2017. Three components of road rehabilitation are included in both Phase 1 and 2 as follows: 1) Design: Works include the preparation of road designs including all drawings, specifications and cost estimates to upgrade basic earth roads to all-weather passable gravel roads. 2) Supervision: Works include the supervision and oversight of construction contractors’ mobilization, implementation, quality control, material testing, and completion. 3) Construction: Works include the supply and installation of all equipment and materials necessary for completion of the road improvement. C. Summary of the activity monitoring, evaluation, and learning plan There are a very limited number of documents on which to build the contextual aspects of the intervention, therefore most of the evaluative qualitative information will emerge from the evaluation team’s field work. Existing documents include: a) USAID-Tanzania. 2014. Country Development Cooperation Strategy. October 3, 2014 - October 3, 2019. Tanzania's Socio-Economic Transformation toward Middle Income Status by 2025 Advanced. Empowering Youth and Women. Dar es Salaam: USAID-Tanzania. b) United Republic of Tanzania. Ministry of Finance and USAID-Tanzania. 2016. Cost Reimbursement Implementation Letter Number 621-IL-15-0001 Under Strategic Objective Agreement Number 621-0012 with the Government of Tanzania Roads Fund Board. Dar es Salaam: Ministry of Finance. c) USAID and Feed the Future. 2016. Irrigation and Rural Roads Infrastructure Project. Washington: USAID. d) USAID-Tanzania. 2014. Fixed Amount Reimbursement Implementation Letter Number 621- IL-14-00004 under Strategic Objective Agreement Number 621-0012 with the Kilombero District Council. Dar es Salaam: USAID-Tanzania. e) USAID-Tanzania. 2014. Fixed Amount Reimbursement Implementation Letter Number 621- IL-14-00009 Under Strategic Objective Agreement Number 621-00012 with the Kiteto District Council. Dar es Salaam: USAID-Tanzania. 47 f) USAID-Tanzania. 2014. Fixed Amount Reimbursement Implementation Letter Number 621- IL-14-00008 Under Strategic Objective Agreement Number 621-00012 with the Kongwa District Council. Dar es Salaam: USAID-Tanzania. g) USAID-Tanzania. 2014. Fixed Amount Reimbursement Implementation Letter Number 621- IL-14-00007 Under Strategic Objective Agreement Number 621-00012 with the Mvomero District Council. Dar es Salaam: USAID-Tanzania. h) USAID. 2014. Rural Roads Prioritization Report. Kilombero District. Technical Assistance to Support the Development of Irrigation and Rural Roads Infrastructure Project (IRRIP2). Washington: USAID. i) USAID. 2014. Rural Roads Prioritization Report. Kiteto District. Technical Assistance to Support the Development of Irrigation and Rural Roads Infrastructure Project (IRRIP2). Washington: USAID. j) USAID. 2014. Rural Roads Prioritization Report. Kongwa District. Technical Assistance to Support the Development of Irrigation and Rural Roads Infrastructure Project (IRRIP2). Washington: USAID. k) USAID. 2014. Rural Roads Prioritization Report. Mvomero District. Technical Assistance to Support the Development of Irrigation and Rural Roads Infrastructure Project (IRRIP2). Washington: USAID. l) Cost Reimbursement Implementation Letter with the Government of Tanzania Roads Fund Board. IV. EVALUATION QUESTIONS The evaluation team will collect and analyze the qualitative information in order to answer the following questions: 1) What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four targeted districts? 2) How has targeted roads’ rehabilitation affected prices of a number of goods and services such as agricultural inputs, consumer products, and passenger transport costs? 3) How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? 4) What effects, if any, have road rehabilitation had on operation costs of vehicles? 5) How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? If not, what constraints were not overcome to achieve these results? V. EVALUATION DESIGN AND METHODOLOGY As there is no baseline data, this performance evaluation will adopt a qualitative approach with intensive use of information gathering methods such as key informant interviews, group meetings, and, if feasible, focus group discussions. Any secondary statistical data – census results, demographic and health survey (DHS), etc. – will also be collected and analyzed, at a minimum for its contextual value. Given the lack of information and data, it is not expected that these methods will yield statistically rigorous results. However, the evaluator must compile a body of narratives and recurrent themes which should help draw a clearer picture of effects rural roads have on communities. The results of this qualitative evaluation will inform future performance and impact evaluations in the sector. 48 Table 2: Evaluation Design Questions Suggested Data Sources Suggested Data Collection Methods Data Analysis Methods 1. What has been the estimated geographical and demographic reach of the10 rehabilitated roads totaling 179.7 km in the four targeted districts? Population housing census 2002 and 2012; census cartography; central and regional mapping administrations; DHS Desk review; key informant interview; group community meetings Population projections; qualitative analysis of information may include also quantitative analysis 2. How has targeted roads’ rehabilitation affected prices of a number of goods and services such as agricultural inputs, consumer products, and passenger transport costs? Local authorities; agricultural and horticultural producers (inputs); shop owners (retail) community (consumers) Key informant interview Community group meetings Qualitative analysis thematic recurrence analysis 3. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? Local authorities; individual transporters; association of transporters; community leaders; farmers and farmer groups and millers. Key informant interview Qualitative analysis thematic recurrence analysis 4. What effects, if any, have road rehabilitation had on operation costs of vehicles? Local authorities; individual transporters; association of transporters; transporter records. Key informant interviews Qualitative analysis 5. How did the project enhance the capacity of district engineers to plan, develop/ rehabilitate and maintain roads? If not, what constraints were not overcome to achieve these results. Local government authorities; district engineers. Key informant interviews Qualitative analysis VI. DELIVERABLES AND REPORTING REQUIREMENTS 1. Evaluation Work Plan and Design: Within three weeks of the award of the contract, a draft work plan of no more than 15 pages for the evaluation shall be completed by the lead evaluator and presented to USAID. The work plan will include: (1) the anticipated schedule and logistical arrangements; and (2) a list of the members of the evaluation team, delineated by roles and responsibilities; (3) a detailed evaluation design matrix that links the evaluation questions in the SOW to data sources, methods, and the data analysis plan; (4) draft questionnaires and other data collection instruments or their main features; (5) the list of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include calculations and a justification of sample size, plans as to how the sampling frame will be developed, and the sampling methodology); and (6) known limitations to the evaluation design. 2. In-Briefing: Within one week of the award of the contract, the local evaluation team will have an in-briefing with the technical and program offices’ teams for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the Statement of Work (SOW), if necessary. 3. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided below in Section IX: Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after 49 consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the technical and program offices’ teams will have ten business days in which to review and comment on the initial draft, after which point USAID will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report five business days hence, and again the technical and program offices’ teams will review and send comments on this final draft report within five business days of its submission. 4. Final Evaluation Report: The evaluation team will be asked to take no more than three business days to respond/incorporate the final comments from the technical and program offices’ teams. The evaluation team leader will then submit the final report to USAID. Additionally, all project data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. VII. EVALUATION TEAM COMPOSITION The suggested team for this qualitative evaluation would include the following key personnel: 1. Team Leader (TL): He/she would coordinate team members and ensure relationships with USAID, local government authorities, beneficiary communities and their leaders. Although not exclusive to the TL, production of the reports would also be under his/her guidance; quality control of analyses and report writing would be a central element of the TL’s responsibility. The TL will require significant knowledge and experience in evaluation, as well as understanding of issues linked to the sector. 2. Community Development Expert (CDE): He/she would be the qualitative information analyst with extensive experience in evaluation. He/she would design the evaluation tools for most interviews and group meetings, especially at the community level. These tools would gather information on changes (positive or negative) resulting from road rehabilitations. The CDE would ideally be Tanzanian and will need to demonstrate experiences in qualitative analysis and community level information gathering techniques. 3. Sector Expert (SE): He/she would participate in all aspects of the evaluation but would especially focus on the more technical aspects such as those relating to roads’ construction/rehabilitation and their impacts on the economic environment of communities. The SE should be able to demonstrate experience in the infrastructure sector as well as evaluation methodologies. This key personnel position is optional. Recruitment of Tanzanian nationals as evaluators/enumerators is highly encouraged. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest. The evaluation team shall demonstrate familiarity with USAID’s evaluation policies and guidance included in the USAID Automated Directive System (ADS) in Chapter 201. VIII. EVALUATION SCHEDULE Table 3: Estimated LOE in days by activity for the evaluation team Task Level of Effort (LOE) in Days Team Lead CDE SE Total Document review/desk review/work planning 2 1 1 4 Preparations for travel and organizing data collection 2 1 1 4 In-brief, Evaluation Design 1 1 2 50 Task Level of Effort (LOE) in Days Team Lead CDE SE Total Preparations for data collection (scheduling) 2 1 1 4 Data collection days 20 20 20 60 Data analysis 5 5 5 15 Debriefing 1 1 1 3 Draft final report and debrief to USAID 3 2 2 7 Final report 2 1 1 4 Total: 38 32 33 103 Figure 2: Gantt chart of illustrative evaluation schedule IX. FINAL REPORT FORMAT The evaluation final report should include an abstract, executive summary, background of the local context and the activities being evaluated, the evaluation purpose and main evaluation questions, the methodology or methodologies, the limitations to the evaluation, findings, conclusions, lessons learned and recommendations, which would then be used to guide subsequent performance and impact evaluations. For more detail, see ADS 201mah, USAID Evaluation Report Requirements and How-To Note: Preparing Evaluation Reports available at the USAID Learning Lab (https://usaidlearninglab.org/library/how-note-preparing-evaluation-reports). Additionally, an optional evaluation report template is also available at the USAID Learning Lab (https://usaidlearninglab.org/library/evaluation-report-template). 51 The executive summary should be two to five pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.). The annexes to the report shall include: ● The qualitative evaluation SOW; ● Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; ● All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; ● All sources of information, properly identified and listed; ● Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of; ● Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; and ● Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. X. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.35 ● Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ● Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ● The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ● Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. 35 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. 52 ● Evaluation methodology should be explained in detail and sources of information properly identified. ● Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ● Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ● If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. XI. OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided in machine-readable, non￾proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the required elements of the SOW of the contract/agreement in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, need to be agreed upon in writing by USAID. Any revisions should be updated in the SOW that is included as an annex to the Evaluation Report. XII. LIST OF ANNEXES Annex 1: Annotated maps of rehabilitated roads sites. ANNEX II: DATA COLLECTION INSTRUMENTS Instruments FGD Protocol Questions – Community Opening 1. Tell us your name, where you live, what transport you took to get here, and how long it took you to get here. Roads Usage 2. How do you use roads? How do your neighbors use roads? 3. Do you use roads more or less since the roads rehabilitation? 4. Do you make more or less trips now compared to before the roads rehabilitation? 5. Was there any change in the settlements along the road? Did it increase or decrease? 6. Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? Prices 7. What effect has roads rehabilitation had on prices of maize, rice, and horticultural products for consumption? 8. Has roads rehabilitation changed the price of passenger transport? a. Are there more options for passenger transport available now than before roads rehabilitation? 9. Has roads rehabilitation changed the price of any consumer goods? a. Has roads rehabilitation made consumer goods more available? Access 10. Has the roads rehabilitation affected the quality of life for your family? Schools 11. Has roads rehabilitation had an effect on travel time to access schools? 12. Has roads rehabilitation given your community access to different schools? a. Which schools? b. Has it affected your child’s attendance in school? Health 13. Has roads rehabilitation had an effect on travel time to access health services? 14. Has roads rehabilitation given your community access to different health services? a. Which health services? [Probe about referral health services] Water 15. Has roads rehabilitation had an effect on travel time to access drinking water? 16. Has roads rehabilitation given your community access to different sources of drinking water? a. Which sources of drinking water? Work 17. Has roads rehabilitation had an effect on travel time to work? 18. Has roads rehabilitation given your community access to better job opportunities? Markets 53 19. What modes of transportation do you use to get to markets? 20. How often do you go to markets? a. Is this more or less than before roads rehabilitation? 21. Has roads rehabilitation had an effect on travel time to markets? 22. Has roads rehabilitation given your community access to different markets? a. Which markets? 23. What problems do you encounter in traveling to markets? 24. Has roads rehabilitation affected the types of goods available at markets? 25. Has roads rehabilitation changed who sells at markets? 26. Has roads rehabilitation had an effect on businesses in your community? 27. Has roads rehabilitation had an effect on travel time to access anything else? Unintended impacts 28. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 29. What is the quality of road now compared to what it was at the time of project signoff? Closing 30. Is there any additional comment you would like to make? FGD Protocol Questions – Producers Opening 1. Tell us your name, where you live, what transport you took to get here, and how long it took you to get here. Roads Usage 2. Do you use roads more or less since the roads rehabilitation? 3. Do you make more or less trips now compared to before the roads rehabilitation? 4. Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? Marketing Agricultural Products 5. Have the prices of any agricultural inputs changed after roads rehabilitation? a. Has the availability of agricultural inputs changed after roads rehabilitation? 6. Have you changed your production at all after roads rehabilitation – diversifying products, greater amount of the same products, etc.? 7. Have your sales increased after roads rehabilitation? [Probe on quantity sold and prices of maize, rice, and horticultural products] 8. Has roads rehabilitation affected the transportation of agricultural goods? a. In which way? 9. Is the way you transport agricultural products different now than before the roads rehabilitation? 10. What has been the effect on crop losses since the roads rehabilitation? a. [Probe regarding travel time to warehouses, markets] 11. Have you changed where you sell your agricultural products since roads rehabilitation? Vehicle Operating Costs 12. If you have a vehicle, have there been any changes on the cost of operating your vehicle? [Probe life of your vehicle and vehicle repair] Unintended impacts 54 13. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 14. What is the quality of road now compared to what it was at the time of project signoff? Closing 15. Is there any additional comment you would like to make? FGD Protocol Questions – Road Users Associations (RUAs) Opening 1. Tell us your name, where you live, what transport you took to get here, and how long it took you to get here. Access to market 2. How often do you go to market? Is this more or less than before roads rehabilitation? Did you encounter any problems in travelling to the market? If yes, please explain. 3. Did the road rehabilitated affect the types of goods available at markets? 4. Did the road rehabilitation change who sells at markets? 5. Has the road rehabilitated made it easier for you to access new markets? If yes, please explain. Selection of Process and Capacity Building 6. Please, describe how you became members of RUA? What was the criteria used in forming the RUA? When were you selected? 7. How do you feel being a member of RUAs? 8. What is your role in ensuring effective maintenance of the road rehabilitated? 9. What was done to build your capacity? a. What do you know about RUAs Paraprofessionals? b. What was the process for selection of paraprofessionals, what is your opinion regarding the process used to select paraprofessionals? 10. What resources (equipment) are made available to RUA to support your work? 11. What interaction do you have with national, regional or local organizations that support RUAs work? 12. What is the current maintenance schedule for the rehabilitated roads? a. What challenges do you face in performing your role in road maintenance? b. What are the operating costs? 13. What supportive mechanisms do you have to ensure effective implementation of your roles? [Probe on their understanding of routine maintenance allocation, support supervision from district engineers] 14. What strategies do you use as a RUA to ensure your group will remain sustainable? Unintended impacts 15. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 16. What is the quality of road now compared to what it was at the time of project signoff? Closing 17. Is there any additional comment you would like to make? KII Protocol Questions – Council Manager and Other District Officials/Village Leaders Opening 1. Please describe briefly your role in the RRDP project. 55 Impact 2. Was there any change in the settlements along the road? Did it increase or decrease? 3. Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? Access 4. Has roads rehabilitation changed the price of passenger transport? Schools 5. Has roads rehabilitation had an effect on travel time to access schools? 6. Has roads rehabilitation given your community access to different schools? c. Which schools? d. Has it affected your child’s attendance in school? Health 7. Has roads rehabilitation had an effect on travel time to access health services? 8. Has roads rehabilitation given your community access to different health services? b. Which health services? [Probe about referral health services] Water 9. Has roads rehabilitation had an effect on travel time to access drinking water? 10. Has roads rehabilitation given your community access to different sources of drinking water? a. Which sources of drinking water? Work 11. Has roads rehabilitation had an effect on travel time to work? 12. Has roads rehabilitation given your community access to better job opportunities? Markets 13. What modes of transportation do you use to get to markets? 14. How often do you go to markets? b. Is this more or less than before roads rehabilitation? 15. Has roads rehabilitation had an effect on travel time to markets? 16. Has roads rehabilitation given your community access to different markets? b. Which markets? 17. What problems do you encounter in traveling to markets? 18. Has roads rehabilitation affected the types of goods available at markets? 19. Has roads rehabilitation changed who sells at markets? 20. Has roads rehabilitation had an effect on businesses in your community? 21. Has roads rehabilitation had an effect on travel time to access anything else? Capacity Building 22. DISTRICT ONLY: What was the role of district engineers in planning and rehabilitating the roads? 23. What was done to ensure that rehabilitated roads would be maintained beyond the project period? a. What was the criteria used in forming the RUA? b. How are the members selected? c. What was done to build their capacity? 24. What challenges were encountered during the RUA training process? 25. What resources (equipment) were made available to the RUAs after project completion? 56 26. What supportive mechanisms do RUAs have to ensure effective implementation? [Probe on their understanding of routine maintenance allocation, support supervision from district engineers] 27. What strategies do RUAs use to ensure their groups will remain sustainable? 28. What is the current maintenance schedule for the rehabilitated roads? a. What challenges have been faced in maintaining the rehabilitated roads? b. What are the operating costs? 29. DISTRICT LEVEL ONLY: Does the LGA have a separate line item for road maintenance in its operating budget? Unintended impacts 30. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 31. What is the quality of road now compared to what it was at the time of project signoff? Closing 32. Is there any additional comment you would like to make? KII Protocol Questions – Paraprofessionals Opening 1. Tell us your name, and your role in the Road User Association. [Probe on main duties if not mentioned, and find out whether this person is also on the village committee] Access to market 2. How often do you go to market? Is this more or less than before roads rehabilitation? What problems do you encounter while travelling to market? 3. Did the road rehabilitated affect the types of goods available at markets? If yes, how? 4. Did the road rehabilitation change who sells at markets? If yes, please explain. 5. Has the road rehabilitated made it easier for you to access new markets? Selection of Paraprofessionals 6. What was the criteria used in selecting members forming your RUA? 7. How did you get selected to be a paraprofessional? 8. How do you feel being a paraprofessional? Capacity Building 9. Have you attended any road maintenance training organized by district officials? a. When was the training? (date) b. How long was it? c. Can you remember what the training was about? d. How well did the training give you the skills you need to do your job as a paraprofessional? e. What could have been done differently to improve the training? 10. What resources (equipment) was made available to the roads user associations to support your work? 11. What supportive mechanisms do you have to ensure effective implementation of your roles? [Probe on their understanding of routine maintenance allocation, support supervision from district engineers] 12. What strategies do you use as a RUA to ensure your group will remain sustainable? 13. What is the current maintenance schedule for the rehabilitated roads? a. What challenges have been faced in maintaining the rehabilitated roads? b. What are the operating costs of the RUA? 57 Unintended impacts 14. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 15. What is the quality of road now compared to what it was at the time of project signoff? Closing 16. Is there any additional comment you would like to make? KII Protocol Questions – Processors Opening 1. What type of work do you? 2. How long have you lived in this area? Roads Usage 3. Do you use roads more or less since the roads rehabilitation? 4. Do you make more or less trips now compared to before the roads rehabilitation? 5. Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? Marketing Agricultural Products 6. Have your sales increased after roads rehabilitation? [Probe on workload, price of services] 7. Did the people who bring you goods to process change from before roads rehabilitation to after roads rehabilitation? 8. Has roads rehabilitation changed the prices you pay for any inputs you need for processing? [Probe on fuel, spares, technicians, etc.] 9. Has roads rehabilitation affected the transportation of agricultural goods? a. In which way? 10. Are there more agricultural transporters now than before roads rehabilitation? 11. What has been the effect on crop losses since the roads rehabilitation? a. [Probe regarding travel time to warehouses, markets] 12. Have you expanded your business model or changed the services you offer since roads rehabilitation? Vehicle Operation Costs 13. If you have a vehicle, did roads rehabilitation affect the following in any way: a. Operating costs of your vehicle? b. Life of your vehicle? c. Fuel efficiency (if applicable)? d. Vehicle repair? Unintended impacts 14. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 15. What is the quality of road now compared to what it was at the time of project signoff? Closing 16. Is there any additional comment you would like to make? KII Protocol Questions – TARURA Regional Coordinator 58 Opening 1. Please describe briefly your role and responsibilities at TARURA. 2. What have been the implications of rural and urban roads getting transferred to TARURA (on July 1 2017) that were previously the responsibility of LGAs? What is the current break up of responsibility with respect to the construction of new roads, rehabilitation of existing roads, and maintenance of existing roads between TARURA and LGAs? Process & Challenges: We have several questions regarding the IRRIP roads, or phase 1 roads that were rehabilitated with funding from USAID under the RRDP project [Interviewers please print and show maps of the relevant FARIL roads to the TARURA staff] 3. To our knowledge previously a performance contract was signed between PO-RALG and LGAs to account for funding from the Road Fund for the ongoing and periodic maintenance of rehabilitated roads. What process is currently in place now that the roads have been transferred to TARURA? 4. How do you work with council managers in maintaining these rehabilitated roads? 5. How do you work with private contractors in maintaining the rehabilitated roads? 6. How do you work with RUAs in maintaining the rehabilitated roads? a. To your knowledge how many RUAs are there? b. How many RUAs do you have contracts with? c. What are the challenges of contracting with a RUA? 7. How do you choose who to award a contract to for maintenance on rehabilitated roads? 8. What is the current maintenance schedule for the rehabilitated roads? And to your knowledge who has maintained the 10 RRDP phase 1 roads thus far? a. What is your perception of the quality of the maintenance done so far? b. What is your perception of the quality of the maintenance done by private contractors? c. What do you do to check the quality of maintenance done on rehabilitated roads? d. What challenges are you facing in supervising the maintenance of the rehabilitated roads? Resources 9. Is funding from the national Roads Fund for the maintenance of the RRDP roads a reliable source? Are there instances where the Road Fund is unable to provide the 1 mill TZS for regular maintenance and the 25 mill TZS for period maintenance for each kilometer of road in the country? a. If yes, how are decisions made on which roads will get funding and which will not? 10. What other funding sources are available for the maintenance of RRDP roads? Sustainability 11. What supportive mechanisms do RUAs have to ensure effective implementation of their roles? [Probe on their understanding of routine maintenance allocation, support supervision from district engineers] 12. What strategies do RUAs use to ensure they will remain sustainable? 13. What is the quality of the roads now compared to what they were at the time of project sign off? Closing 14. Is there any additional comment you would like to make? KII Protocol Questions – Transporters Opening 1. What type of work do you? [Probe on whether they are middlemen or are hired just to take produce to markets] 59 2. How long have you lived in this area? Roads Usage 3. Do you use roads more or less since the roads rehabilitation? 4. Do you make more or less trips now compared to before the roads rehabilitation? 5. Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? Transportation 6. What has been the effect of the roads rehabilitation on the demand for your transport services? 7. Did roads rehabilitation affect the prices you charge to transport goods to the market? 8. Is the way you transport agricultural products different now than before the roads rehabilitation? 9. Are there more transporters now than before the roads rehabilitation? 10. What has been the effect on crop losses since the roads rehabilitation? a. [Probe regarding travel time to warehouse, markets] 11. Have you changed where you take agricultural products to or from, since the roads rehabilitation? 12. What modes of transportation do you use to get to markets? 13. How has roads rehabilitation affected the travel time to markets? 14. How often do you go to markets? Is this more or less than before roads rehabilitation? 15. What problems do you encounter in traveling to markets? 16. Has roads rehabilitation affected the types of goods available at markets? 17. Has roads rehabilitation changed who sells at markets? 18. Did roads rehabilitation affect the following in any way: e. Operating costs of your vehicle? f. Life of your vehicle? g. Fuel efficiency (if applicable)? h. Vehicle repair? Unintended impacts 19. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 20. What is the quality of road now compared to what it was at the time of project signoff? Closing 21. Is there any additional comment you would like to make? KII Protocol Questions – USAID 1. What mechanisms did you have in place to ensure that the preparation of road designs (including all drawings, specifications and cost estimates) to upgrade basic earth roads to all-weather passable gravel roads was conducted according to standards? a. What challenges, if any, did you encounter in overseeing those activities? 2. Please, can you describe the mechanisms used to ensure that the supervision and implementation of the roads rehabilitated were done according to standards and with compliance to terms and conditions of signed FARILs? a. Probe on monitoring and reporting systems used b. What challenges, if any, did you encounter in overseen those activities? 3. What is your opinion on the effectiveness of District Council and the Implementing Partner in undertaking their responsibilities? 4. In your opinion what challenges do the LGAs face in covering the operations costs and maintenance of the FARIL roads? 5. What is your opinion on the roles and responsibilities between the LGA, TARURA and the RUAs regarding rural roads? 60 6. How did you ensure value for money on roads rehabilitated under FARILs? 7. What mechanisms were established to ensure sustainability of the roads rehabilitated? 8. What lessons have you learned and adopted in continuing roads rehabilitation under RRDP Phase 1 and 2? 9. Is there any additional comment you would like to make? KII Protocol Questions – RUA Leader Opening 1. Tell us your name, and your role in the Road User Association. [Probe on main duties if not mentioned] Access to market 2. How often do you go to market? Is this more or less than before roads rehabilitation? What problems do you encounter while travelling to market? 3. Did the road rehabilitated affect the types of goods available at markets? If yes, how? 4. Did the road rehabilitation change who sells at markets? If yes, please explain. 5. Has the road rehabilitated made it easier for you to access new markets? Selection of RUA Leader 6. What was the criteria used in selecting members forming your RUA? 7. How did you become RUA leader? 8. How do you feel being a RUA leader? Capacity building 9. What resources (equipment) were made available to the roads user associations to support your work? 10. How does the RUA ensure effective management of resources given? 11. What supportive mechanisms do you have to ensure effective implementation of your role? 12. What challenges do you face as RUA leader? 13. What strategies do you use to ensure your RUA will remain sustainable? 14. Have there been any dropouts since the establishment of the RUA? 15. What is the current maintenance schedule for the rehabilitated roads? c. What challenges have been faced in maintaining the rehabilitated roads? d. What are the operating costs of the RUA? Unintended impacts 16. Were there any results of the road rehabilitation that you did not expect? [Probe good expectations and bad ones] 17. What is the quality of road now compared to what it was at the time of project signoff? Closing 18. Is there any additional comment you would like to make? Quantitative Questionnaire We have a few questions regarding the road rehabilitation that took place near you. Gender: Male_________________ Female________________________ Age: 18-35______________36-49______________50-65___________________66+_______________________ District:_________________________Village:____________________________ 61 How many years have you lived in this area?__________________________________________________ Occupation Types Yes No Teacher Health worker Farmer Processor Vendors Transporter No job Other (please specify) Marital status: Single_____ _______Married_____________Divorced_____________Widow/er:______________________ If farmer please list all the items you produce:_________________________________________________________________ If transporter please tell us which vehicle you use for transporting:_______________________________________________ What do you use roads for? Please select all that apply. a. Work b. Getting agricultural inputs c. Selling agricultural products d. Going to the market e. Health services f. Schools g. Other (specify) _____ What is the current main source of household income per year from agriculture …………………………………. What is the current main source of household income per year from non-agriculture -------------------------------------------- ------- Please provide an estimate of your annual household income from all sources during the following periods. Type of Household Income Before road rehabilitation During road rehabilitation After road rehabilitation Agricultural Non-agricultural Please indicate: The quality of: Before road rehabilitatio During road rehab After road rehabilitation Good Fair Poor Good Fair Poor Good Fair Poor 1. Roads within your village 2. Roads to regional towns or markets 3. Buses, minivans, and any other available transportation services Please indicate cost of transportation from your village to the nearest town center: Mode of Transport Before road rehabilitation During road rehabilitation After road rehabilitation Cycle Bajaj Boda boda (motorcycle) Dala dala (minibus) Truck Car What mode of transportation are you currently using to access the following and how long does it take you to get there? Access to: Before Road Rehabilitation After Road Rehabilitation 62 Mode of transport Travel time (minutes) Mode of transport Travel time (minutes) Markets to buy produce Markets to sell produce Hospital (emergency) Community meeting Primary school Secondary school Buying agricultural inputs Please indicate the prices of the following goods: Goods Before Road Rehabilitation After Road Rehabilitation Seeds per kg Fertilizer per kg Pesticides per liter/kg/btl Maize per kg Rice per kg Sugar per kg Tomatoes per bucket Beans per kg/bucket 63 RRDP Evaluation Design Matrix – April 2018 Sub-questions/ Themes Themes Protocol Questions Data Sources (* denotes KII; ** denotes FGD) Road Assoc** Community** Producers** Processors** Transporters* District Engineer* District Manager* USAID* CDM Smith* EQ1: What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four target districts? What is the population size affected by the road rehabilitation • Usage of roads for commercial purposes • Usage of roads for non￾commercial purposes Frequency of road usage • Vehicle load • How do you use roads? How do your neighbors use roads? • Do you use roads more or less since the roads rehabilitation? • Do you make trips now compared to before the roads rehabilitation? • Do you think there are more or less vehicles on the road now compared to before roads rehabilitation? - X X X X X X X X X X X X X X X X X EQ2: How has targeted roads rehabilitation affected prices of a number of goods and services, such as agricultural inputs, consumer products, and passenger transport costs? 2A. What are the perceived effects of the road rehabilitation on prices of goods and services, agricultural inputs, consumer products, and passenger transport costs? • Fertilizer prices • Seed prices • Agrochemical prices • Availability of agricultural inputs • Costs of food • Availability of food at markets • Availability of consumer goods • Consumer product prices • More drivers/trucks • Diversity of transportation options • Passenger transport costs • Have the prices of any agricultural inputs changed after roads rehabilitation? • Has the availability of agricultural inputs changed after roads rehabilitation? • Have you changed your production at all after roads rehabilitation -- diversifying products, greater amount of the same products? • Have your sales increased after roads rehabilitation? • What effect has the road rehabilitation had on the prices of maize, rice, and horticultural products for consumption? • Has roads rehabilitation changed the price of passenger transport? • Are there more options for passenger transport available now than before the roads rehabilitation? • Has roads rehabilitation changed the price of any consumer goods? • Has roads rehabilitation made consumer goods more available? • What has been the effect of road rehabilitation on the prices of goods that you process? • Has the road rehabilitation affected the transportation of processed goods? In which way? - X X X X X X X X X X X X X X - X X - - 64 Sub-questions/ Themes Themes Protocol Questions Data Sources (* denotes KII; ** denotes FGD) Road Assoc** Community** Producers** Processors** Transporters* District Engineer* District Manager* USAID* CDM Smith* 2B. What are the perceived effects of beneficiary communities on access to social services, such as schools, health services, water, etc.? • Availability of transportation options • Travel time to schools • Access to better schools • Travel time to health services • Access to better health services • Travel time to water • Travel time to other social services • Travel time to work • Access to better job opportunities • Has the roads rehabilitation affected the quality of life for your family? • Has road rehabilitation had an effect on travel time to schools? • Has roads rehabilitation given your community access to different schools? Which schools? • Has roads rehabilitation had an effect on travel time to health services? • Has roads rehabilitation given your community access to health services? Which health services? • Has roads rehabilitation had an effect on travel time to collect water? • Has roads rehabilitation had an effect on travel time to any other social services? • Has roads rehabilitation given your community access to other better quality social services? • Has roads rehabilitation had an effect on travel time to work? • Has roads rehabilitation given your community access to better job opportunities? • Has the roads rehabilitation had an effect on businesses in your community? - X X X X X X X X X X X - - - - X X X X X X X X X - - EQ3. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? 3A. How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? • Transport costs of maize • Transport costs of rice • Transport cost of horticultural products • Competition among transporters • Crop loss • Travel time to markets • Travel time to warehouses • Less reliance on head￾loading • Traders coming to farmers directly • What has been the effect of the roads rehabilitation on the demand for your transport services? • How has road rehabilitation changed the price of transporting agricultural products? • Is the way you transport agricultural products different now than before the roads rehabilitation? • Are there more transporters now than before the roads rehabilitation? • What has been the effect on crop losses since the roads rehabilitation? • Have you changed where you sell your agricultural products since the roads rehabilitation? - - - - X X X X X X - - - - 65 Sub-questions/ Themes Themes Protocol Questions Data Sources (* denotes KII; ** denotes FGD) Road Assoc** Community** Producers** Processors** Transporters* District Engineer* District Manager* USAID* CDM Smith* 3B. How do beneficiary communities perceive their access to markets have changed, if at all, since the rehabilitation of the road? • Travel time to markets • Problems getting to markets • Frequency of market visits • Transportation used to get to markets • Diversity of goods at markets • Access to new markets • What modes of transportation do you use to get to markets? • How has roads rehabilitation affected the travel time to markets? • How often do you go to markets? Is this more or less than before roads rehabilitation? • What problems do you encounter in traveling to markets? • Has roads rehabilitation affected the types of goods available at markets? • Has roads rehabilitation changed who sells at markets? • Has roads rehabilitation made it easier for you to access new markets? • Has what you buy at markets changed since roads rehabilitation? X X X X X X X X X X X X X X X X - - X X X X X X X X - - - - EQ4. What effects, if any, have road rehabilitation had on operation costs of vehicles? What have been the effects of the roads rehabilitation on consumers, producers and transporters? • Vehicle life • Repairs/maintenance • Fuel efficiency • Accidents • Damage to freight • Able to offer lower prices • Did the roads rehabilitation affect the following in any way: o Operating costs of your vehicles? o Life of your vehicle? o Fuel efficiency? o Vehicle repair? • Did the road rehabilitation affect the prices you charge to transport goods to the market? - X X X X X X - - - EQ5. How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in Phase 1 been sustained? If not, what constraints were not overcome to achieve these results? 66 Sub-questions/ Themes Themes Protocol Questions Data Sources (* denotes KII; ** denotes FGD) Road Assoc** Community** Producers** Processors** Transporters* District Engineer* District Manager* USAID* CDM Smith* - • Training – Duration • Training – Quality • Protocols put in place • Oversight • Ongoing resources/contacts • Funding • Challenges • What was the role of district engineers in planning and rehabilitating the roads? • What was done to ensure that rehabilitated roads would be maintained beyond the project period? • What challenges were encountered during the training process? • What resources were made available to the roads associations after project completion? • What is the current maintenance schedule for the rehabilitated roads? o What challenges have been faced in maintaining the rehabilitated roads? o What are the operating costs? • Does the LGA have a separate line item for road maintenance in its operating budget? • What is the quality of the road now compared to what it was at the time of project sign off? X X X X X - - - - X X X X X X X X X X X X X X X X X X X X X X X 67 Coding of Documents Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 7 Planning Multiple DROMAS is a road management system (RMS), intended to store and process road and bridge inventories, traffic counts, contract details, indeed any related data that can be used for road network planning. It is an asset management tool. Use of RMSs has been actively encouraged by the World Bank; similar systems have been developed in several East and Central African countries. DROMAS has largely fallen into disuse: of the four districts participating in the FTF program, only the Mvomero District Engineer (DE) had a copy on his laptop. He appeared to be making limited use of the package. All the other DEs had been trained in its use but had made little or no use of it since initial installation. The reasons for the decline of DROMAS are unclear. Problems with the code, lack of testing at all levels and access to the PMO-RALG server have all been mentioned. The residual benefit of DROMAS appears to be the traffic count and road condition survey forms, which remain in use. A revised web-based RMS is currently under development by PMO￾RALG, supported by a DFID consultancy. It is expected to hold information relevant to prioritization and so could generate reports that would facilitate prioritization, but it is unclear at present whether it will actually include a prioritization tool. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2012 - 2012-1122 Road Prioritization.pdf 68 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 5 Planning Multiple Eleven roads were short-listed using the following criteria: (i) current demand for transport along the road, using evidence from moving observer traffic counts during field visits and any available manual classified counts, (ii) relevance to agricultural development (essentially judged by whether a road was essential for access to an irrigation scheme to be improved/developed under FTF, or by an indication by district agricultural staff that an area had high potential or that developments were planned) and (iii) considerations of access, which in practice meant the presence of a World Bank bottleneck. #1-~ demo reach of the 10 rehabilitated roads (179.7km) in the 4 targeted districts? 2012 - 2012-1122 RoadPrioritizatio n.pdf 1 Training Multiple Absence of software to plan and report road maintenance activities, and absences of priorization tool were noted as gaps during planning stage. IRRIP developed the priorization tool (via Excel), and offered 1- week DROMAS classroom and practical training and on-the-job mentoring on use of prioritization tool. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Training curiculum.docx 69 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 1 Training Multiple Low understanding of design steps, poor contract drawing skills, unrealistic road work cost estimation (no unit rates build up), and non-designing of cross drainage structures (apparently due to non-use of topo maps and discharge estimates software) were noted as gaps during design stage. IRRIP developed a step-by-step guide (not shared with engineers) and the curriculum included training on road condition, materials investigations and testings and interpretation of test results. On job mentoring provided on how to use TRRL East African model to calculate discharge and proper use of existing design manuals like: Specifications, Geometric design manual, Materials testing manuals, Pavement design manuals and unit rates building up. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Training curiculum.docx 70 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 1-2 Training Multiple No experience in LGAs procuring road consultants before and poorly prepared and opaque evaluation reports were noted as gaps during the procurement stage. As a result, roads needed to be designed and supervised by consultants. IRRIP included classroom and on-the-job training about procuring consultants and contractors to procurement officers, treasurers, and technical staff. Gaps noted with regard to contract admin include low capacity to interpret contract clauses, manage implementation schedules, perform quality control, and understand the roles of each party, and contract closure. Also challenges in interpreting test results. IRRIP included 6- day classroom training on contract admin and on-the-job mentoring on the importance of conducting and interpreting tests to achieve design quality. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Training curiculum.docx 2 Training Multiple Lack of Road User Associations, low engagement of RUAs by LGAs, lack of skills in RUAs and communities to execute maintenance tasks, and lack of awareness by LGAs and communities about routine maintenance operations were identified as gaps. IRRIP provided guidelines on formation, registration, and procurement for RUAs, training of trainers training to LGA technical staff, and training of paraprofessionals on civil works by use of LBTs. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Training curiculum.docx 71 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 2-3 Maintenance Multiple Detailed information about how to constitute and register the RUA and selection of members. Required criteria are interest, age 18 and over, and residence in the local gov district road side village; preferred criteria include female [50+% of selected candiates should be female], training in village civil road works [paraprofessionals], poverty [50+% of selected candiates should be from poor households], leadership skills, and basic reading/writing/math skills. Involvement of youth, elders, and persons with disabilites are also encouraged. Final criteria should be clearly stated and agreed upon to keep the process transparent and escape political influence. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 3 Maintenance Multiple Information on the formation of routine road maintenance group in the village should be disseminated as widely as possible before members of the routine maintenance group are selected to form the group for registration. Methods such as flyers and posters can be used, and local leaders and organizations need to be informed. Dissemination of information regarding the routine road maintenance employment opportunities should include details of the activities to be carried out, expected working hours, and remuneration levels and means of payment. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 72 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 3 Maintenance Multiple It is important to make an effort to inform vulnerable groups—especially women and the poor-who generally have less access to common means of communication and information. By ensuring that they are informed, their chances of being able to participate in the selection process and of obtaining employment through membership in the group are increased significantly. It is important to consider the reality of the target groups, indicating clearly that vulnerable groups are also advised to apply into road maintenance group membership workers, that the required experience and skills are within range of most persons (e.g., by stating that experience in agriculture is considered sufficient), and clearly indicating that part time and flexible working hours and working days are allowed to increase participation by women, who may be generally unable to participate in full-time employment due to their numerous other responsibilities. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 73 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 4 Maintenance Multiple Qualities of individuals selected as village paraprofessionals are ethical conduct; clear communication with others; willingness learn new skills and knowledge and to be involved in continuing para-professional development activities; meet policy, legal, and regulartory requirements and protect road user interests; recognition of foreseeable social, economic, cultural, and environmental effects of activities, and regard for sustainability of road users' benefits and asset preservation; local knowledge specific to road routine maintance jurisdiction; reading/writing/math skills; trusted by the community; willing to organize others to manage road routine maintenance as a group; premanent resident of the village and problem solver; adult. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 74 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 9 Maintenance Multiple Role of the RUA in terms of carrying out the road routine maintenance work is to: Inspect the road, identify and remove all obstructions (e.g. dead animals, minor anthills, land slide, fallen trees, large stones, etc.); Clear side drains of all vegetation, slit and debris including disposal. Repair / replace scour checks. Repair eroded ditches. Clear mitre drains of all vegetation, slit, debris and pounded water, and dispose of appropriately. Unblock all silted culverts including inlets and outlets. Clear all stream channels of debris and vegetation to ease the flow of water through bridges and culverts. Clear catch water drains of all slit, debris and vegetation and dispose of appropriately. Fill potholes and minor gullies that have developed on the carriageway using approved material. Do grubbing to reinstate road camber and shoulder cross fall where they exist. Reinstate eroded shoulders and ditch slopes using appropriate material. Cut/weed all vegetation off the carriageway and shoulders. Cut grass to specified levels. Clear bush/shrub and move the resultant debris. Specifics of each task are outlined later on, in section 8 (p.13). #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 75 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 11 Maintenance Multiple Local Government Authority shall on request provide to the CBO hand tools such as: Shovels Slashers, Hoes, Wheel Barrow, Machetes, Pangs, and Rakes etc. As detailed in Annex 2.2 hereto. otherwise CBO should provide his/her own tools. The tools shall be provided based on context specific best options available for hire purchase/acquisition arrangements and tools management alternatives ways in use by the District-LGA agreement with CBO. The tools will become the property of the CBO once the cost of the tools is fully recovered by the council, in case hire acquire modality is applicable. The hand tools whether on hire purchase or owned by the CBO shall be inspected at each time of payment to ensure that work does not suffer due to lack of appropriate tools. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? - - Task Force Guidelines.pdf 1 Training Multiple 18 individuals (2 short of the 20 targeted) participated in the Procurement of Consultancy Services course held 23-27 of November 2015. Kilombero, Mvomero, and the National Irrigation Commission each had 4 people, while Kiteto and Kongwa had 3 each. Most were engineers, supplies officers, and heads of PMU. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Oct-Dec Consultancy Procurement Course Participants.docx 76 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 7 Training Multiple Procurement of Consultancy Services course provided printed material for the training but also future reading and reference after the course. Course included a PPT presentation cover 41% of the duration, while case studies and group exercises made up the remainder. Case studies focused on real life problem solving encountered in construction project (e.g., preparation of expression of interet evaluation report, terms of references, selection and employment of consultants procedures). #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Oct-Dec Consultancy Procurement Course Report.doc 8 Training Multiple Procurement of Consultancy Services course was evaluated by participants at the end. They rated every dimention at 100%, with the notable exception of duration of the course, which was at 53%. Certificates of attendance were provided to each participant at the end. The course anticipates that those trained will transfer the knowledge gained to their respective staff members by applying it in procurement of consultancy services. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Oct-Dec Consultancy Procurement Course Report.doc 8 Training Multiple Procurement of Consultancy Services course covered the following key components: An Overview of Public Procurement Act No. 7 of 2011; Public Procurement Regulations of 2013 on Selection and Employment of Consultants; Principles of Selection of Consultants; Preparation and Evaluation of Expression of Interest (EOI); Application of Guidelines for Preparation, Submission and Evaluation of Proposals; Overview on Preparation of Terms of Reference for Consultancy Assignment; and Case studies on evaluation of Expression of Interest and Consultancy Proposal #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Oct-Dec Consultancy Procurement Course Report.doc 77 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 10 Training Multiple Key comments and recommendations about the Procurement of Consultancy Services course were: b) The course has helped to acquire enough knowledge for start of consultancy services activities. h) The training was well organized and the approach was good thus became understandable (participatory, group exercises). A general recommendation was made about the NCC continuing to build capacity via other courses, such as one on AUTOCAD or evaluation and its critics. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Oct-Dec Consultancy Procurement Course Report.doc 1 Training Multiple DROMAS course, held 26-30 June 2017, was attended by 12 participants, 3 (always a district engineer, an engineer, and a technician) tied to each of the 4 districts in which project activities occurred. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Course Participants.docx 2 Training Multiple The DROMAS course is to support the following: 1) Each LGA has their road network and its asset (Culverts, Drift and Bridges) structure in the system (DROMAS 2), 2) The location of all roads and associated structure located within each LGA are shown on the Dromas map, 3) All roads inventory and road condition are reported in the system for future developments decisions, 4) All performance agreement which are engaged between LGA and IDU and contracts that are engaged between LGA and Contractors to be available on the system than can be easily monitored. 5) Quarterly Progress report are available on time. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 78 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 2 Training Multiple The course "Training on the use of District Road Management System version 2 (DROMAS 2), GPS and Quantum GIS training (QGIS)" was designed as 5-day, 5 module course covering the range of topics towards 1) Enabling participants to be able to correct road inventory and road condition and enter this data into the Dromas 2 system, 2) Enabling participants to be able to report performance agreement and Contracts progress on the system, and 3) Enabling participants to be able to use GPS devices in collecting road inventory and road condition and how to process those data on Quantum GIS. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 79 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 4 Training Multiple Module 1 - Intro to DROMAS included an overview and background on DROMAS, and how to install and configure user access. Module 2 - Road network (database) covered updating road network, road inventory and condition data, traffic data, and road network photographs. Module 3 - Annual road maintenance Plannning (ARMP) covered entry of budgetary item rates, selecting roads to be maintained and maintenance activities, and viewing the District summary. Module 4 - Contract management covered creation of contract and entry of contract details and bill of quantities data, updating progress milestones and physical progress, updating variation, updating interim payment certificate and received funds, and viewing quarterly progress reports. Finally, Module 5 - Intro do GPS/GIS/QGIS covered an overview of these technologies, training on using GPS for road surveying, processing the surveyed roads in QGIS, and uploading the surveyed roads on DROMAS 2 map. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 5-7 Training Multiple All participants were provided with a Training Manuals for DROMAS 2, a GPS device and Quantum GIS. A variety of learning delivery and assessment methodologies were employed, in an attempt to enable the course to be participatory and interactive and to allow participants to effectively get the knowledge intended for the training. Participants entered 5 contracts and associated details and surveyed 10 roads of a total length of approximately 21km. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 80 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 9-15 Training Multiple The training was evaluated on the basis of a survey. Results are outlined but some highlights are:10/12 participants who answered agreed (7) or strongly agreed (3) that the trainer was knowledgable about the topic. 11 /12 agreed (6) or strongly agreed (5) that the trainer was able to explain and illustrate concepts, 1 disagreed. 10 out of 12 participants agreed (7) or strongly agreed (3) that the trainer was able to answer questions completely. 11/12 thought the topics covered were relevant to them, 12/12 thought participation and interaction were encouraged, Participants also generally thought content was organized and easy to follow (9/12), that topics covered during training were relevant to them (11/12), and that training experience will be useful to their work (8/12). Everyone also agreed or strongly agreed that they would recommend this training to colleagues. 10/12 however would like to have longer training. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 16 Training Multiple Several participants from different councils were concerned that the DROMAS system is online, as they have poor internet. QGIS was also viewed as a challenge to use even after training. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2017 April-June DROMAS Training Report.docx 81 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 8-9 Training Multiple The 5-day course on Construction Contract Administration was held in Morogoro between March 2-6 2015 by the National Construction Council. It targeted practitioners in the construction industry and 8 engineers were sponsored by IRRIP (1 for each district, 3 from the zonal irrigation office, and 1 from TANROADS Morogoro). It is anticipated that those who attended the course will transfer the knowledge gained to their respective staff members by applying such knowledge in their day to day management of the construction projects #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Jan-March ConstrContract Admin_CourseR eport.doc 8 Training Multiple The course on Construction Contract Administration was comprised of the following key components: Introduction to PPA 2011 and Regulations of 2013, Management of Pre-Contract Phase, Rights and Obligations of the Parties, Risk Management in Construction Contracts, Time, Quality & Cost Control, Finishing the Contract, Management of Claims, Introduction to Dispute Resolution Techniques and Case Studies. Extensive details on the subtopics covered under each component are between pages 10 and 50. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Jan-March ConstrContract Admin_CourseR eport.doc 53 Training Multiple Participants skills evaluation summary is missing. #5-How did project enhance capacity of district engineers to plan, develop/rehabilitat e and maintain roads? Were phase 1 efforts sustained? 2015 Jan-March ConstrContract Admin_CourseR eport.doc 82 Page # Associated project stage Relates to… Text Related Eval Question Reporting year Reporting quarter (if applicable) Document filename 7 Procurement Kilombero The balance of FARIL funding was used to grade 6km of road connecting the village providing gravel to the improved road (and to purchase Acrow Panel bridge parts to accomplish the ongoing construction of Mofu Bridge along Namwawala-Mofu road [Activity C]) #1-~ demo reach of the 10 rehabilitated roads (179.7km) in the 4 targeted districts? 2016 Jan-March Kilombero USAID report.pdf 83 Illustrative Quotes EQ1: What has been the estimated demographic reach of the 10 rehabilitated roads totaling 179.7 km in the four target districts? Estimated Demographic Reach of the FARIL Roads “Before rehabilitation, Orkine was a sub-village with the very scattered and countable houses. After road rehabilitation, the place has turned into a village with many new houses continuing to be built.” [Village leader, Kiteto] “Along Namwawala-Mofu Road, people have even reached a place where they have invaded forests, cleared sites, and begun building permanent houses after road rehabilitation.” [Community member, Kilombero] “Before road rehabilitation, Suguta village was home to only natives. After road rehabilitation, even sellers of agricultural pesticides have decided to settle in Suguta and sell agricultural inputs while engaging in farming.” [Community member, Kongwa] “The houses have increased and villagers in Nyandira are now constructing new modern houses and guest houses with electricity and tap water inside.”[Village leader, Mvomero] Use of the Rehabilitated Roads “Now everyone uses road as it is easier and passable throughout the year and we also have an assurance of reaching where we plan to visit.” [Community member, Kilombero] “We use Wamidakawa-Dihombo road more frequently because it is a shortcut to go to Turiani and easily connects to many other villages on the route to Handeni in Tanga Region.” [Community member and transporter, Mvomero] “Apart from agricultural, we also do fishing activity at Ikwambe area within Mofu ward and also in the neighboring ward of Mbingu were there is a natural fish pond. The Namwawala – Mofu Road has helped me to sell my products more efficiently. Unlike before road rehabilitation where we could only fish for home consumption, now we are doing it for business and have many customers in Mofu, Ihenga, and even at Namwawala.” [Community member, Kilombero] “Just after rehabilitation of Engusero-Orkine Road, during harvesting time Food Agricultural Organization (FAO) made a camp in Engusero village to buy agricultural products; the road has real help us to transporting our agricultural products to a FAO Market, at Engusero.” [Producer, Kiteto] “Before rehabilitation, transportation was hard. Cars were only carrying goods, not passengers. Even the transportation of goods was difficult. Now, there are many cars for passengers and people can move freely. Many cars are also coming to carry goods throughout the years and more increasing during the harvesting period.” [Village leader, Kilombero] “There more vehicles now, before road rehabilitation bicycles was a common vehicle used in Mofu, and when it rains the road to Namwawala cuts off and no vehicles could pass. The village of Mofu was an island with no access to other services. For transportation, only one Land Rover was used in the section of Ihenga – Namwawala. After road rehabilitation trucks, motorcycles, minibuses, even bajaji are doing transportation from Mofu. Youth and elders are able to use bicycle to reach to neighboring village, children use bicycles going to schools, and more trucks which transporting agriculture produce.” [Village leader, Kilombero] “There are more vehicles, even trucks come to carry our agricultural products, unlike before where we had to carry our load of agricultural product to Ihenga where our buyers would have access to transport.” [Producer, Kilombero] EQ2: How has targeted roads rehabilitation affected prices of a number of goods and services, such as agricultural inputs, consumer products, and passenger transport costs? Sub-question 2A: What are the perceived effects of the road rehabilitation on prices of goods and services, agricultural inputs, consumer products, and passenger transport costs? Availability of Agricultural Inputs “Availability of agricultural inputs has increased since rehabilitation. After rehabilitation, transportation has been simplified and traders are bringing agricultural inputs to the village more easily.” [Producers, Kiteto] “At Ihenga, the villagers have established a fast-growing center. The residents, after selling their crops, have also started shops where they sell farming implements and inputs. For those that are not able to go to town to purchase such farm items, they can buy them there at the village.” [Village leader, Kilombero] “The increased use of tractors for farming is because of road rehabilitation. Now, more businesspeople come with their tractors and work with us preparing farms.” [Producer, Mvomero] 84 “Agricultural inputs are more available in the village and there are more shops selling them, but also our road which has been rehabilitated has provided better accessibility to other inputs in the nearby towns. Before, even some basic agricultural inputs such as hoes, rakes, and machetes, let alone improved seeds or fertilizer, were not available in the village.” [Producer, Kilombero] “They are not available here, but with good roads, I can just move easily to Ifakara to buy spares for my machines.” [Processor, Kilombero] “Shops selling spare parts like belts are many at Kibaigwa. Before, there was only one shop at Kibaigwa selling spare parts.” [Processor, Kongwa] “Before, they were no local technicians for repair, also no availability of spare parts. Nowadays we get some spare parts which are regularly used in the village center, and those which not available can easily accessed from Ifakara District Council.” [Processor, Kilombero] Changed Production in the Market “Before rehabilitation, we used to produce very little of beans, radishes, and beets, just enough for our own use. But after road rehabilitation, production of these agricultural products is very high and also the demand became high as well.” [Producer, Mvomero] “Production has increased because of the accessibility of markets. Producers are easily accessing the new market at Engusero and selling their crops at competitive prices, unlike before road rehabilitation when they had no means of transporting their crops to the near-by markets.” [Village leader, Kiteto] “Before rehabilitation, we were not transporting bananas for sale. We used to buy and sell them to each other and some got rotten. But after road rehabilitation, transportation is very easy and affordable as there are so many vehicles now, so we transport bananas on time and sales have increased as well as the customers.” [Producer, Kilombero] “Production has changed a lot. For example, sunflower producers have greatly increased their production because sunflower oil can easily be extracted and transported compared to before rehabilitation when transportation was a great challenge.” [Community member, Kiteto] “Before roads rehabilitation, we used to wake up at 3:00 am to queue up to find customers and when we were late we’d have to go back home with our crops. But after road rehabilitation, we go to the markets anytime and we sell because customers are so many now.” [Producer, Kilombero] “Yes, our sales have definitely increased after road rehabilitation because nowadays we get customers from Kigali in Rwanda, Mwanza, and Shinyanga. They buy peas from us and they travel direct to our farmers, so we do get a lot of profit and have a wide market to sell our agricultural productions. [Producer, Mvomero] Yes, sales have increased a lot because we now produce more rosemary and customers come direct to us. We hardly even go to the market to sell.” [Producer, Mvomero] “The increase in the amount of products that we can collect in short period of time, reduction of transportation cost for crops, better access to better products from farms, and reduction of crop loss during transportation made profits increase.” [Processor, Kongwa] “Sales have been increasing. It has helped us to produce more, so we make more money... Before the road was rehabilitated, I was producing 100 gallons of sunflower oil every 24 hours. Now, I produce 180 gallons in the same amount of time.” [Processor, Kongwa] “Previously, producers used to sell at their villages, but now the Dihombo people bring the commodities to the warehouses here at Dakawa. They sell here or transport their products to Morogoro and Dar es Salaam.” [RUA member, Mvomero] “Farmers now sell their goods in Mlali at the market opposite from Mzumbe University. They also go to Morogoro.” [Village leader, Mvomero] “My business has grown and expanded as we now have more customers.” [Processor, Kilombero] “Compared to before roads rehabilitation, now many people are coming to my shop from outside this area. They come from different villages like Sokoine, Kambala, and other areas in Dakawa. Before, customers did not come from those places.” [Processor, Mvomero] “After roads rehabilitation, public servants such as nurses and teachers use their free time to bring sunflower seeds to the factory for production and then take sunflower oil to sell in the villages where they live. The scenario wasn’t there before road rehabilitation.” [Processor, Kongwa] 85 “I am now focusing on processing agricultural goods, packaging them, and selling the processed products in further markets…That expansion has been possible due to availability of quality products which, once processed, meet the Tanzania Food and Drugs Authority standards for further markets.” [Processor, Kongwa] “When the road is in good condition, I can use my small truck to distribute processed goods and also to collect agricultural products for my factory directly from farms. Before, I was just relying on middlemen who came and sold agricultural products in the international market of Kibaigwa.” [Processor, Kongwa] Cost of Transportation “Before road rehabilitation, the fare paid to be transported to Ifakara was from 10,000 TZS to 15,000 TZS depending on the road condition. We can now hire a motorcycle for 25,000 TZS and get there much faster. We also use Noah as public transportation for a fare of 7,000 TZS to Ifakara.” [Community member, Kilombero] “A trip from Langali to Nyandira cost 5,000 TZS before roads rehabilitation, but now is 1,000 TZS only.” [Community member, Mvomero] Sub-question 2B: What are the perceived effects of beneficiary communities on access to social services, such as schools, health services, water, etc.? Schools “Before rehabilitation, the secondary school was 5km from Dakawa Centre and 12km from Kambala and it was difficult to get to. During rainy season, water crossed the road when the Divue River flooded. Now, all areas can easily access the school.” [Village leader, Kongwa] “The road has helped a lot related to schools at district centers because a lot of students are coming from their homes to school. The villages are a great distance from the center, so when the students reach school they have used a lot of time. But the road has helped a great deal.” [Village leader, Kilombero] “The secondary school is at Iduo and there is also Pandambili Secondary School which is just close to the beginning of the road. So, for those who are coming from distant villages, they may use motorcycle or they can easily get support due to the increased of traffic on the road.” [Village leader, Kongwa] “Before road rehabilitation, the nearby secondary school was at Mgeta. Students could walk for about 3 hours to access school at Mgeta. After rehabilitation other ward secondary schools were built, including the Langali secondary school, Tchenzema secondary school, and Kikeo secondary school which only requires our children to walk for 20 minutes to access it. Construction of these secondary schools was impossible with the previous road condition.” [Community member, Mvomero] “Kibaigwa secondary school is now accessible, and people also reach other special schools at Kibaigwa.” [Village leader, Kongwa] “After road rehabilitation, some community members decided to enroll their children in Kongwa Primary School which is a private school. Before, no one would think of doing that. It takes their children only one and a half hours to reach Kongwa, unlike before road rehabilitation where it took three hours minimum to reach Kongwa.” [Community member, Kongwa] “Our children now use bicycles to reach school, this helps them to attend school.” [Community member, Kilombero] “Before the roads rehabilitation, it was normal for secondary school to be close for one or two weeks each year during rainy season, because most of students were not able to go to school. After rehabilitation, the school is now open throughout a year.” [Village leader, Kilombero] “In Mofu, not only secondary school of Misheni, which wasn’t reachable for the community of Mwaya and Kidimu sub-villages, but also Mofu and Nganyangira primary schools were not easily accessible and during rainy season pupils will have to stay home and not to go to school. Now it is reachable, even in rainy season.” [Community member, Kilombero] “Engusero-Orkine Road has helped teachers arrive on time to school. Travel time from Engusero to Orkine is only 30 minutes now, whereas before it took 1-2 hours.” [Village leader, Kiteto] “Before road rehabilitation, few teachers in our primary schools were natives of these villages. Now there is an increase of teachers and newcomers are also teaching in our schools. Before road rehabilitation, our school headmasters complained that teachers who were posted in our schools reported once and never came back.” [Community member, Kilombero] “There are some remote areas where teachers cannot live. As they were located far from their working station, they in most cases missed work regularly before roads rehabilitation. But nowadays they can stay as far as Kibaigwa and be able to use motorcycle to go to work.” [Village leader, Kongwa] “Having a secondary school at Engusero with qualified teachers and regular supervision done by extension officers has increased the pass rate of our children to join secondary education. Before, only two pupils would be chosen to 86 join secondary school. Now, up to 50 students from our primary school are selected to join secondary school at Engusero.” [Community member, Kiteto] Health “Before, it was hard to reach health services. We had only a missionary hospital and to reach it was difficult. Only God helped us. Now we have another hospital and it takes just 10-15 minutes by bicycle.” [Village leader, Kilombero] “Health centers are at Songambele and Engusero. Before, we used 6 hours walking to Engusero and 4 hours to Songambele. After road rehabilitation the travel time to access those health centers have reduced to 30 min and 20 min by motorcycle to Engusero and Songambele, respectively.” [Community member, Kiteto] “The road has made possible for us to access Ifakara District Hospital for further health services which now take only 1.5 hours by motorcycle, which we normally hire for emergency purposes, and 2 hours by car. Before, access was impossible due to travel means and time to reach district hospital which was 5 to 6 hours and only available during the dry season. That situation put in danger most of the pregnant women who need further support during delivery.” [Community member, Kilombero] “For referrals, people go to Kongwa first, and others goes to Dodoma and often use ambulances that pass on Suguta-Kibaigwa Road now.” [Village leader, Kongwa] “Now, our big hospital is working very well, and if it happens there is an emergency, maybe a mother or sick person has to be brought to the big hospital, a family member hires a car and just gets rushed to the hospital. Before road rehabilitation, they had to walk carrying sick people to hospital.” [Village leader, Mvomero] “We access referral health services at Kiteto District Hospital. Before road rehabilitation, one could use a day to reach the referral hospital at Kiteto, but after road rehabilitation, two hours of traveling by car is sufficient.” [Community member, Kiteto] “Wamidakawa-Dihombo Road has supporting access to the new district hospital at Mvomero, and further referral to Morogoro General Hospital.” [Village leader, Mvomero] “Before road rehabilitation, Mwaya Dispensary had no facilities. We were going to Misheni Dispensary to access health services. After road rehabilitation, the government is now able to distribute drugs and health equipment, and we access primary health services in our dispensary at Mwaya.” [Community member, Kilombero] “Health services are now easily accessible. Before, we could walk for 2 hours to Tchenzema and Kibuko to access the dispensary. But now, the dispensary is around here and we only walk for 3 to 30 minutes to the dispensary, depending on where you come from.” [Community member, Mvomero] “We have a mobile clinic that comes from Kiteto.” [Village leader, Kiteto] “Before rehabilitation, medicines used to be brought up to a nearby village health center and people from our village would carry them on their heads from there to our dispensary. But since roads rehabilitation, the medicines are brought by car direct to our dispensary.” [Community member, Mvomero] “A big specialized hospital for skin diseases is 200m up in the village along our road. Before, it was difficult to reach and not easy for drugs and medical equipment to reach to the hospital. Electricity was not available, which was needed for the hospital to function fully. Now, all medical facilities are easily transported and community member are fully using that health facility.” [Village leader, Mvomero] Drinking Water “Roads rehabilitation has helped us at the beginning to reach drinking water in Ihenga. And later, thanks to the support of USAID, we have big deep water wells in our village and at Ikwambe Dispensary. Before the roads rehabilitation, support of water provision was only in Ihenga.” [Village leader, Kilombero] “The road has helped us get drinking water from a natural water stream located at Suguta Village which is a one end of Suguta-Kibaigwa Road.” [Village leader, Kongwa] “The roads have helped in facilitating transportation of cement and pipes for construction of drinking water taps. And as for now, we can easily access drinking water in our homes, we have the water taps at our homes.” [Community member, Mvomero] “After road rehabilitation, there are now many projects concerning water provision that are ongoing, including the construction of a water well that has been established last year around Chapakazi area, along Engusero- -Njiapanda Road.” [Village leader, Kiteto] “We have access to drinking water from a stream in Chamkoroma Ward. Youth from Iduo are transporting water from Chamkoroma and selling it to Iduo for 500 TZS per 20 liters.” [Community member, Kongwa] 87 EQ3: How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? Sub-question 3A: How has targeted roads’ rehabilitation affected transport costs to markets of specific agricultural products such as maize, rice, and horticultural products? “The number of transporters has increased a lot. Most of the transporters are people from Kibaigwa, but some are also coming from outside.” [Processor, Kongwa] “Most of the people in the area are farmers, peasants, who depends on selling what they produce. They used to take crops from the farms to the market centers, whether on head or bicycle. But after rehabilitation, even light vehicles, goods vehicles, go straight to the farms to collect the produced crops.” [CDM Smith] “Earlier, we carried the sacks by ourselves with our heads to the markets. But after road rehabilitation, customers come to buy direct from us in the farms because of easy transportation.” [Producer, Mvomero] “Before road rehabilitation, we carried our children and our crops on our backs to take them to Langali for sale. After road rehabilitation, transportation is easy so we just call for the transporters to carry our crops or the customers to come to us directly at the farm to buy the crops.” [Producer, Mvomero] “Here in Ihenga, we have some collection points, mainly for cereals, which are sometimes also used as sell points. Buyers can easily collect the needed quantity of agriculture goods in a short time.” [Processor, Kilombero] “Before rehabilitation we used to get big profit because there were very few vehicles on the road so we charged high fare but after rehabilitation it has affected our business because there are so many vehicles on the road so the fare is competitive now.” [Transporter, Mvomero] “If you compare the transport costs before the road was redone, the prices have lowered because competition has increased among transporters. They can no longer charge a high price because they are one of a very few.” [Processor, Kongwa] “Transportation prices for major crops such as sunflower, maize and millet have decreased. The cost for transporting a bag of cereal before was 2,000 TZS, while now it costs 1,000 TZS.” [Processor, Kongwa] “I use the road more frequently because Wamidakawa-Dihombo Road is a shortcut to Dihombo, Turiani, and many other villages on the route to Tanga. The tar road via Magori is far, so after rehabilitation, Wamidakawa-Dihombo Road is more used and passable throughout the year. Before rehabilitation, it was just passable during dry season.” [Processor, Mvomero] “Before rehabilitation there were a lot of plums fruits that were scattered in the farm and rotten due to no market. But after road rehabilitation, plum customers are so many to the extent that you will never now find a plum fruit scattered anywhere.” [Producer, Mvomero] “Before rehabilitation, we were not transporting bananas for sale; we bought and sold them from each other and still they were getting rotten. But after road rehabilitation, transportation is very easy and affordable as there so many vehicles now than before, so we transport our bananas on time and sales have really increased, as well as the customers.” [Producer, Mvomero] “Now even if you have one ton of produce to transport, you can use kirukuu. Before rehabilitation, you had to find other 15 friends to be able to transport it all to the market.” [Transporter, Kongwa] Sub-question 3B: How do beneficiary communities perceive their access to markets have changed, if at all, since the rehabilitation of the road? Access to a Range of Markets “People in Olboroti could not easily reach Matui for the market before, but now they can go easily.” [RUA, Kiteto] “There are not new markets, but the markets already in existence have become fuller. These markets have more items, different items, and more people.” [Village leader, Kongwa] “Before road rehabilitation, our market was only once per week on Thursdays. After rehabilitation, the market is every day and customers are many.” [Producer, Mvomero] Travel Time to Markets “We now use a very short time to the market place, depending on where a person is coming. For example, people in Magana sub-village used 15 to 20 minutes by walking to here in Nyandira center and now we use only 5min by motorcycle.” [Community member, Mvomero] “Before road rehabilitation, we spent so much time on the road. We use to spend five hours travelling to reach Kibaigwa by tractor while transporting agricultural products, but now we use only two and a half hours to reach 88 Kibaigwa by motorcycle and we mainly use “FUSO” trucks to transport our agricultural goods to Kibaigwa.” [RUA member, Kongwa] “Our travel time to market has reduced. Before road rehabilitation, the travel time from Kilashi Farms to market was about 2 hours, but after road rehabilitation we use shortcuts to access Engusero-Orkine Road, and only use 30 minutes. [Community member, Kiteto] “We go to market more than before road rehabilitation, because the road is easily passable, and we use less time and cost compared to before road rehabilitation.” [RUA member, Kiteto] “I go to market three times a day. It is more than before roads rehabilitation.” [RUA member, Kilombero] “There is an open market once per week, but people can go to market at any time they want now. They make many more trips than before the rehabilitation.” [Village leader, Mvomero] “Now, I go to market in Ifakara less because I can get most of my needs on gulio (open market) days.” We did not have an open market here before. [Community member, Kilombero] “Not very often as before because most of goods for consumption are available here, and we used to go to Kibaigwa once a month to buy things that were not available here.” [Producer, Kongwa] Availability of Goods “More products are readily available at the fair price. For example, rice was sold at 3,000 TZS per kg before, but now Orkine villagers can buy at just 2,400 TZS per kg because the roads are passable.” [Producer, Kiteto] “We have different items available in the market now. We have Chinese vegetables, cauliflower, cabbage (both green and red), beets, beans, other vegetables, and pork meat. We also have clothes.” [Village leader, Mvomero] “Non-agricultural goods such as construction materials, clothes, and shoes were only accessible at Kibaigwa before roads rehabilitation.” [Community member, Kongwa] “Yes, goods and products such as kettles, home utensils, clothes, crafts made by our colleagues, solar power, and agricultural inputs like pesticides and fertilizer were not available in the village markets before.” [Community member, Kilombero] “We never had formal market before. At the few selling points used before road rehabilitation, only people from our village came to sell some goods, and the majority of it was food. During the dry season, some villagers would make an effort to travel to Kibaya, Kiteto, and come back with other goods for selling including cloth, sugar, oil etc. Now, during market days, business people from Engusero, Matui, and Kiteto, come to us with different products throughout the year.” [Community member, Kiteto] “There are some traders who would previously never go to our outdoor markets, like Njoge, due to road condition. But after rehabilitation, there are so many traders going there for the markets.” [TARURA] EQ4: What effects, if any, have road rehabilitation had on operation costs of vehicles? Fuel Efficiency “From Orkine to Matui to Olboroti, 30 liters of fuel was used before rehabilitation. After rehabilitation, only 20 liters are used and we make more trips than before.” [Male transporter, Kiteto] “There is a big difference in fuel efficiency. We use less fuel after rehabilitation. From Engusero to Njiapanda used to use up to 10 liters, but now it only uses 5 liters.” [Male processor, Kiteto] “We have improved vehicle fuel consumption. 10 liters were used to travel from Kibaigwa to Suguta before rehabilitation, but now just 5 liters can go the same distance.” [Male transporter, Kongwa] “I now use less fuel compared to before rehabilitation. Before, I was using 20 liters of fuel to go from Dakawa to Mgongola farms. After road rehabilitation, I use 15-17 liters to travel the same distance.” [Male transporter, Mvomero] Maintenance of Vehicles “We have less operational cost because of the availability of mechanics and spare parts at the village unlike before when both was accessed at Ifakara.” [Male transporter, Kilombero] “Vehicle repair has become cheaper and access to technicians is easier now compared to before.” [Male processor, Kiteto] “It used to cost a lot to repair these vehicles. Prior to the rehabilitation, if you went to the farm and came back, the operational cost was high and damage was high. Now, it is not as costly as before.” [Male processor, Kongwa] “Before rehabilitation, our motorcycles had a lot of challenges and need for regular spare parts like tires and spokes. After rehabilitation, the motorcycles stay longer on the road with no regular repairs.” [Producer, Mvomero] 89 EQ5: how did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? How have efforts in phase 1 been sustained? If not, what constraints were not overcome to achieve these results? Sub-question 5A: How did the project enhance the capacity of district engineers to plan, develop/rehabilitate and maintain roads? Capacity Building in the Planning Phase “We have gone through capacity building as engineers and TARURA, we have learned about prioritization, there is a tool that we are using in prioritization.” [TARURA Regional Coordinator] Capacity Building in the Design Phase “In the designing stage, the major challenge was the limited qualification and experience of district engineers. Engineers at LGAs had not enough technical skills to design the road as per required project standards. CDM Smith supported the process by coaching district engineers in this area.” [USAID] Capacity Building in the Rehabilitation Phase “Because they had not worked with consultants in supervision, district engineers lacked proper knowledge on how to use conditions of contract to manage the contract on site. They had documents, but they failed to follow all those conditions. For example, perhaps a contractor delayed work for a month or two for some reason and he asked for an extension of construction period for a reason and that reason does not follow anywhere in the contract, but they grant him an extension. There are things like that.” [CDM Smith] “After gravel roads are finished, they enter into a 6 month defect liability period where the contractor is supposed to attend to any defects which appear in the road during that period. We found that in some areas, district engineers couldn't manage this period. They waited until the end of the period to go visit sites and inspect for any defects, which we think is wrong. They need to visit sites more frequently, maybe monthly or every two months, to know the regular changes. Because if you wait until the end of the period, you cannot bring in the contractor to fix any defects.” [CDM Smith] “We received a reporting format from CDM Smith. We have benefited in how to write reports – monthly progress reports. Before, we never used to do these things.” [TARURA] Capacity Building in Maintenance Phase “According to the legal framework, the minimum number of RUA members should be 15 to 30. So we opted for that range depending on the willingness of communities in the villages. The composition of gender, age, and socio￾economic aspects, that was also in the PPRA guidelines. It requires the following: must include women (provided the women are above 18 years old), youth 18 to 35 years old, people with disabilities, and elders 40 or above. Those are the specific compositions required by law. They say that those people must make up 70% of the group and 30% can be filled in with others.” [CDM Smith] “After the training, each of the three paraprofessionals created a list of nine people whom they wanted to join the RUA. We gave the paraprofessionals the same criteria we used to select them. I advised the process. Then there was a meeting where the 27 people agreed to be in the RUA.” [Village leader, Kongwa] “The RUA here started as an agricultural cooperative group in 1997. In 2018, we got registered as a RUA.” [Paraprofessional, Mvomero] “On July 11, 2017, we held a meeting in the village. Among the paraprofessionals, I was chosen to speak in public. We were trained by the district to keep the payment part secret. We wanted to see people volunteer themselves to be in the RUA before they knew about the payment. We were trying to choose 27 other people, to make a total of 30. It was difficult, people were pulling back because they love money and wanted to be paid. In the end, the exercise went very well.” [Paraprofessional, Kiteto] Sub-question 5B: How have efforts in Phase 1 been sustained? Maintenance of FARIL Roads “For some engineers, labor based technology was new because most of them are used to machine based technology. When they want to maintain a road, they bring in machines to do the maintenance for them. From what they studied in school to what they do today, they have no experience with labor based technology. It was like we had to teach them new ideas.” [CDM Smith] “Procurement of RUAs has been challenged because not all districts have budgeted to use the RUAs in their areas. Kiteto did not budget to use RUAs in 2017 or 2018, Kilombero budgeted for all, Kongwa budgeted for all, and Mvomero budgeted for one of the two roads, which is two of six RUAs. We are going to track the budget for 90 procurement of these groups but also to enforce now within the coming financial year to ensure that they put the budget for the groups that are already in their area.” [CDM Smith] “I have received a lot of complaints from other members who feel they are being deceived. They say: ‘It has been a whole year. Where is the project? Where is the work? If there is no project, please let us know.’ People think we as paraprofessionals are keeping secrets from them.” [Paraprofessional, Kongwa] “Last year, the RUA was stuck in the registration process. Now they have the certificate and are registered.” [Village leader, Kilombero] “The registration process took longer than we expected because we were not aware that the groups need to be registered with the Contractors Registration Board (CRB)...Most of the time was taken because people didn’t know how to register the RUAs.” [USAID] “The RUA initiative has faced challenges and been slow to get established. The key element was that the RUAs were not allowed to be registered at the district level and therefore could not start working until registered nationally.” [CDM Smith] “Nationally, we have the CRB which does registration for all contractors. Their registration requirements are quite different from what we provided in the project, so these RUA groups, these paraprofessionals, did not really need to requirement to get registered as labor based contractors. We registered them as community based groups and not contractors, not companies. They are non-profits, they are just there to do service, they are service oriented so we registered them as groups at the district level. There was a challenge of getting them registered by CRB which has higher and more complicated requirements for getting groups registered. The RUA now fall under one class, I think labor-based class 3, so they can get registered. Before it was a big challenge. [CDM Smith] RUA Sustainability “As a group, we are trying not to depend on the road itself and contribute extra money to our fund. We do different things to add money. We might go help people build houses and get money that way.” [Paraprofessional, Mvomero] “As a group, we cannot remain jobless. We wrote a letter to the ward office asking to be doing cleanliness in the village. We listed some equipment we would need for us to start working. We submitted our letter to the Public Health Extension Officer who liked our idea and suggested that we write another letter and expand our service to the ward level. We are working on writing a letter to the District Environmental Officer.” [RUA member, Mvomero] “We are planning on making bricks for sale so that we can increase funds in our account.” [RUA member, Kiteto] Sub-question 5C: If not, what constraints were not overcome to achieve these results? Enforcement Mechanisms “We are solely relying on the Road Fund Board for maintenance. The councils are supposed to contribute but none of them has ever been able, claiming their revenues are small.” [TARURA council manager] “The amount of resources provided by the Road Fund is sufficient only for “good” roads. The problem is that resources for periodic maintenance are being used for the construction of new roads.” [CDM Smith] “LGAs cite their ability to use own source revenues for road maintenance, but this is not really the case. The FARIL mechanism lead to savings due to an appreciating USD to TZS exchange rate. Thus, savings from the FARIL have sometimes been used for maintenance.” [CDM Smith] Issues with RUAs: Reference Materials & Equipment “We do not have enough equipment, but we agreed to start with what we have. Maybe in the long run we will be adding to it. We need a big car in order to get the sand needed to fill holes in the road.” [Paraprofessional, Kilombero] Issues with RUAs: Membership “We developed about 13 criteria for selecting the two people to be trained at the village. We selected them with a village councilor. The first criteria was that we'd ensure those people stayed in the village and be able to disseminate information. The common one is the age, it should be eighteen and above, but also resident in the village, a woman and a man, those are the common ones and then there are others interested to understand social relations, commitment to the work to be done in the village, literacy in reading and writing, and others associated with making them interested and committed to work for the community. “[CDM Smith] “Always, the leader is chosen by God. They saw that I was accountable, I persevere, and I educate people, and I speak in public well.” [Paraprofessional, Kiteto] “I always am volunteering so they chose me to be a paraprofessional. They also thought I had experience because I had been watching the engineers to see what they do.” [Paraprofessional, Kiteto] 91 “Paraprofessionals were chosen because they are honest and people of integrity.” [RUA member, Kilombero] “Paraprofessionals were chosen because they have ability to represent others, not be scared to stand before people, and give views that benefit of people” [RUA member, Kiteto] “The selection of paraprofessionals should be done in the village meeting for all people to understand criteria used for selection, and therefore increase acceptance of the people selected.” [RUA member, Kiteto] “In the future, the selection process of paraprofessionals should be done through the village meetings for all villagers to have equal opportunity to participate.” [RUA member, Kilombero] Issues with RUAs: Payment “The problem is money. The payment to RUAs is very little but the job they do is very big compared to the work done. They only get 8000 TZS per day. They should get maybe more than 10,000 TZS. Maybe 15,000 TZS.” [Paraprofessional, Kilombero] “We have contracts now. The contract will help us have money to live. We receive a small amount of money as payment, but it should be more. We receive 8000 per day, but we should get 10,000 TZS.” [Paraprofessional, Mvomero] “The daily rate per workload is low: The pay that we get do not sustain our needs, we work more than amount of money we receive.” [RUA, Kilombero] “Payment to the RUAs is a challenge because it’s small. The RUAs do the work just for the sake of themselves.” [Village leader, Mvomero] “Some contracted RUAs wanted more than 10,000 TZS per day, but they ended up settling for 8,000 TZS.” [CDM Smith] “The payments to RUAs should be done on time. It seems that RUAs get their payments late. They are doing this work with all their heart and we cannot demoralize them.” [Village leader, Kilombero] Issues with RUAs: Ownership by RUA Members “I feel that the RUA will help us have income and sustain our lives.” [RUA member, Kiteto] “I feel good to be in a group that helps me to gain income.” [RUA member, Kiteto] “I feel good to work and earn anything from this group.” [RUA member, Kongwa] “These people should bring that project soon so that we can earn income…I beg them to start this project soon to save us from this tragedy.” [RUA member, Kongwa] “I was told to join a group and that it will help me to meet my basic needs.” [RUA member, Kiteto] “I feel good being a RUA because it helps us put our village environment in a good condition.” [RUA member, Mvomero] “Though we earn some income out of it, I was motivated to preserve our roads so that they may give us access to markets for our agricultural products.” [RUA member, Kiteto] Issues with RUAs: Other External Constraints “The challenge is manpower, as with any other new structure established in the country. It will take time for TARURA to have all human resources in place.” [USAID] Unexpected Results Positive: Employment & Income Generating Activities “Villagers of Mjimwema and Ndurugumi are now easily going to Kibaigwa to work in the Kibaigwa International Market and in factories.” [Community member, Kongwa] “Engusero-Orkine Road has had a large impact for social workers – teachers, agricultural extension officers, community development extension officer – who stay in Engusero and work in our village as they find it easier to work here now compared to before road rehabilitation. And to community members, mainly youth who are small entrepreneurs and often have to travel to Engusero for work, it has been possible after road rehabilitation.” [Village leader, Kiteto] “There are workers from Kibaya, Kiteto, who go to work in villages along FARIL roads and also some government employees from those villages who are coming to work here under the DED or Council Manager-TARURA. Roads’ good condition have improved driving speed and using USAID roads which are now passable throughout the year, gives assurance of going to work all year through and lessen time spend traveling to work.” [Village leader, Kiteto] 92 “There are lots of transportation options to get to Kambala village which making it easier to go to the irrigation farm. Before, people used to ride bicycles and have to carry them on their backs until they got to a place where they could ride.” [Village leader, Mvomero] “It has helped travel time to work. Before road rehabilitation, 1.3 hours was been spent to walk from Chapakazi to Lemelo farms. Now we walk for 30 minutes only.” [Community member, Kiteto] “Before road rehabilitation, we lived at our farms throughout farming season. After road rehabilitation, we go to farm in the morning and come back in the evening every day using motorcycles.” [Community member, Kiteto] “More motorcycles have increased, our youths have secured jobs.” [Community member, Kiteto] “Now, our youth can have self-employment, such as taking fish from the center and supplying them to people throughout the road. They also can go to nearby villages using this road.” [Village leader, Kilombero] “It has created self-employment. We have an open market here so youth in motorcycle “bodaboda” now carry people to the open market and make money that way. Other people can go by bicycle and pick up chickens, then bring them to this village to sell.” [Village leader, Kiteto] “There are lots of job opportunities now. Through motorcycle, youth are transporting people and goods, including gathered stones for building houses. They stack them near the road so businessmen can buy them and transport them by car.” [Village leader, Mvomero] “Youth are getting new types of work. Most of them carry different goods including sugar cane, maize, and fruits on the road from Iduo to Suguta and to Kibaigwa, though Suguta-Kibaigwa Road.” [Village leader, Kongwa] “During road rehabilitation, women had an opportunity of cooking and selling food in the construction site camps. After rehabilitation, the food vending business has reduced and it is more undertaken during the open markets days. However, there are a few cafes which continue working in the village centers.” [Community member, Kilombero] “Now different people including women are in business. They buy from the farmers and sell at the market place and transport as well.” [Community member, Mvomero] “In Mofu economy, we depend on agriculture, fishing, and livestock keeping such as cows. Now, our youth can have self-employment, such as taking fish from the fish market in Mbingu, and supplying them to people in the village and to nearby villages using this road.” [Village leader, Kilombero]. Positive: New Business “Wholesale shops for drinks including beers and sodas have been established in the village, and businessmen who own shops in Gejungae and Lemelo sub-villages access those products from here.” [Community member, Kiteto] “When you pass along Namwawala-Mofu Road, you will see some shops selling various goods together with mobile money transfers. There are rest houses, especially in Namwawala, for the people who have engaged in agriculture but do not live here always. All these are new establishments, developed after road rehabilitation.” [Village leader, Kilombero] “I’ve noticed new shops with agricultural inputs like seeds and fertilizer, and drinks.” [Village leader, Kiteto] “There are new shops in the community. People have opened restaurants along the roadside.” [Village leader, Kongwa] “Matui has becoming a small town with a number of big businesses including a petrol station, wholesale shops, and a transit center to facilitate the transportation of maize…All these were not here before.” [Village leader, Kiteto] Positive: Quality of Life Improvements “In Nyandira, there has been electricity since 2016 or 2017. Previously, we asked Rural Energy Agency why there was no electricity and they said they could not afford to go there because their trucks could not go there.” [TARURA] “Unexpected results are the availability of social service and consumable goods such as electricity, water, and ambulance for referral transportation. Cement and other construction material are available.” [Community member, Kongwa] “We have started to think about village electricity. They think it can be possible to bring it without challenges because of the road. TANESCO has already surveyed here, and starting in July they will start electric poles. They will connect this village to electricity. This is possible because of this road.” [Village leader, Kilombero] “Before road rehabilitation, there was no electricity but after rehabilitation there is electricity which makes us have television set. We can watch videos now. We have a communication tower are in our village now, we have good communication network. And we are even able to build good houses, we moved from the huts that we used to live.” [Producer, Mvomero] Positive: Road Safety 93 “Now we have road signs along the road. For example, warning of a bridge. We also have corner road signs.” [Producers, Kiteto] Negative: Accidents “The problem is the speed of motorcycles. Motorcycles move very freely now, but the road is narrow so if one comes face to face with the car, accidents happen.” [Village leader, Mvomero] “Road accidents due to increasingly heavy trucks for transportation of agricultural products which makes it hard for alternation with the other road users such as motorcyclists.” [Community member, Mvomero] “Accidents, mainly from the driver of motorcycles speeding.”[Processor, Kilombero] Negative: Outsiders “A possibility of spread of diseases like HIV/AIDS due to interactions of people, many outsiders come in our village.” [Community member, Mvomero] The diversity use of road has made it to attract robbers. Since road rehabilitation we have witness several incidence of car robbers at Rice Mill area.” [Transporter, Mvomero] We have started to see prostitutes in our village something that has never been a practice before road rehabilitation.” [RUA member, Kiteto] “Interaction with people who are coming from different parts of the country with different cultures have started to erode our culture.” [RUA member, Kiteto] 94 ANNEX III: SOURCES OF INFORMATION Key Informants Interviewed Name Title District Gender Richard Mwakasitu Senior Roads Engineer, CDM Smith n/a M Lameck Methusela Rural Roads Engineer, CDM Smith n/a M G.S. Mpinzile TARURA Regional Coordinator n/a ? Mohamed Mkwata TARURA Regional Coordinator n/a M Boniphace Marwa IRR Infrastructure Engineer (Feed the Future), USAID n/a M Florent Kyombo* District Executive Director Mvomero M William Lameck Council Manager Mvomero M Blandia Marijani* Acting District Agricultural Irrigation and Cooperative Officer Mvomero F Foya Hozeniel* District Agricultural Extension Officer Mvomero F Jibril Mandauri* Statistics Officer Mvomero M Elijaneth Mganga* Community Development Officer Mvomero F Fabiana Antony Paraprofessional Mvomero F Method J. Msimbe Paraprofessional Mvomero M Peter Joseph Zeyhoe Ward Councilor Mvomero M Hamis Daudi Processor Mvomero M Ally Juma Gojigo Representative of the RUA Chairperson Mvomero F Abdallah Mgaya Village Chairperson Mvomero M Bakari Hassan Processor Mvomero M Joseph Mvula Transporter Mvomero M Salum Mohamed Transporter Mvomero M Shani Haji Transporter Mvomero M Salum Bwaya Council Manager Kongwa M Vendelin J. Mbeya Ward Executive Officer Kongwa M Helena John Chinyari Paraprofessional Kongwa F Mathias P. Ngiga Paraprofessional Kongwa M Maria Temaumj'i RUA Chairperson Kongwa F Hezron Hassam Kibona Acting Manager, Chambo Oil Mills Kongwa M Mohamed Hasani Processor Kongwa M Mohamed Mwaja Manager, Mwaja Oil Mills Ltd. Kongwa M Robert Mhando Transporter Kongwa M Timoth Mahava Transporter Kongwa M Charles Mpuza Transporter Kongwa M Joseph Kalendi Processor Kongwa M Salim Ally Manyundo Processor Kongwa M Braison Edward Manager, Komfood Product, Ltd. Kongwa M Edwin Magiri Council Manager Kiteto M Beatrice Rumbeti* District Planning Officer Kiteto F Edgar Kavenuke* District Statistician Kiteto M Elias Sumilla Paul* Civil Engineer, Kiteto Kiteto M Joseph Mwaleba* District Community Development Officer Kiteto M Robert Urassa* District Agricutural Irrigation and Coperative Officer Kiteto M Julieth Mussa Mlugu Paraprofessional Kiteto F Seif Omari Seif Paraprofessional Kiteto M 95 Name Title District Gender Dickson Ngombe Katambo RUA Leader Kiteto M Neema J. Temba Village Executive Officer Kiteto F Murajabu A. Ngwada Paraprofessional Kiteto M Hassan Iddi Mtumbelo Processor Kiteto M Dennis Londo* District Executive Director Kilombero M Richard Anyeylwisye Council Manager Kilombero M Ashura A. Kalufya* District Economist and Statistician Kilombero M Athuman Maundu* District Agricultural Irrigation and Cooperative Officer Kilombero M Christopher Kafunga* District Economist Kilombero M Joseph Minofu* District Road Technician Kilombero M Loyce Mnyenyelwa* District Community Development Officer Kilombero M Aluwia Lyawatwa Paraprofessional Kilombero F Anna Mgendela Paraprofessional Kilombero F Reinfred S. Magungu Mofu Ward Councilor Kilombero M Eric Edward RUA Leader Kilombero M John Masunga Processor Kilombero M Maganga Daudi Processor Kilombero M Bright A. Mwinyi Acting Ward Executive Officer Kilombero M Mussa Ally Akachapa Transporter Kilombero M Madata Thobias Transporter Kilombero M Vicent Clavery Transporter Kilombero M Haruna Shabani Processor Kilombero F -Total 65 *Provided information to the evaluation team but did not participate in a formal KII Focus Group Participants FGD Participants by Gender Type of Stakeholder Gender Location Title District Male Female Total Participants Ihenga RUA Kilombero 7 3 10 Dakawa RUA Mvomero 3 7 10 Orkine RUA Kiteto 4 4 8 Engusero RUA Kiteto 5 5 10 Iduo RUA Kongwa 5 4 9 Mofu Community Kilombero 5 5 10 Nyandira Community Mvomero 6 9 15 Orkine Community Kiteto 6 5 11 Iduo Community Kongwa 6 4 10 Mofu Producers Kilombero 5 5 10 Nyandira Producers Mvomero 10 7 17 Orkine Producers Kiteto 5 4 9 Iduo Producers Kongwa 5 5 10 TOTAL 72 67 139 96 List of Documents Reviewed File Name "Titles" 2012-1122 Roadprioritization.Pdf Working Paper On Rural Road Prioritization Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Keith F. Williams, P.E., Chief Of Party, 22 November 2012 Cdcs Tanzania Empowering Women And Youth: Tanzania’s Socio-Economic Transformation Toward Middle Income Status By 2025 Advanced Country Development Cooperation Strategy October 3, 2014 – October 3, 2019 Cm Manual Final - Complete Irrigation And Rural Roads Infrastructure Projects Technical Assistance: IRRIP2 United States Agency For International Development 19 January 2018 Version 00 Constrcontractadmin_Coursereport.Doc Course On Construction Contract Administration Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Lameck Methusela, Roads Engineer Keith F. Williams, P.E., Chief Of Party 13 March 2015 Consultancy Procurement Course Participants.Docx 1.1.1 Procurement Of Consultancy Services Course Offered By NCC 23rd To 27th November 2015 Consultancy Procurement Course Report.Doc Course On Procurement Of Consultancy Services Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Lameck Methusela, Roads Engineer Richard Mwakasitu, Roads Component Leader Keith F. Williams, P.E., Chief Of Party 10 December 2015 Distribution Of Roads, Villages And Ruas Distribution Of Road Network, Villages And Road User Associations In The Four LGAS District Leaders Contact Information Irrigation And Rural Road Infrastructure Project Contact Information For Ded,Cdo/Deco,De/Cm And Weo In Kiteto,Kongwa,Mvomero And Mvomero District 4 May 2018 Dromas Course Participants.Docx Dromas Course 26th To 30th June 2017 Dromas Training Report.Docx Training Report; Report On Training Programme Conducted From 26th To 30th June, 2017 In Dodoma Global Positioning Systems (Gps), Geographical Information Systems (Gis,) Quantum (Gis), Data Collection And District Management Version Two (Dromas 2) Engusero Njiapanda Drawings Pdf￾Rev2014jul30 Kiteto District Council For Rehabilitation Works Along Engusero – Matui – Njiapanda Road 25.5 Km In Kiteto District Bidding Document Volume 2 Of 3 Drawings August, 2014 Evaluationreport-Engusero-Olkine Bid Evaluation Report And Recommendation For Award Of Contract For Rehabilitation Works Along Engusero - Olkine Road 19.4km In Kiteteo District 5 September 2014 Fomu Ya Mkataba Kati Mkandarasi Jamii Na Tarura-1 Fomu Ya Makubaliano Ya Mkataba 97 File Name "Titles" Invitation For Quotation Works April 07 Form No 1-.3 Mwaliko Sanifu Wa Kotesheni Kwa Ajili Ya Ununuzi Wa Kazi Ndogo Ndogo Za Ujenzi Mamlaka Ya Udhibiti Wa Ununuzi Katika Sekta Ya Umma. S.L.P. 49, Dar Es Salaam. Desemba, 2009 Irrip Adopted Paraproffessional Training Exract Swahili Manual For Routine Maintenance Jamhuri Ya Muungano Wa Tanzania Tawala Za Mikoa Na Serikali Na Mitaa Mradi Wa Uboreshaji Barabara Vijijini Mwongozo Wa Utunzaji Barabara Kwa Mafundi Wasaidizi Wa Barabara Wa Vijiji Irrip Rural Roads Design Guide (Rev 30 Nov) Irrigation And Rural Road Infrastructure Project Rural Roads Design Guide For District Engineers Kilombero Prioritization Report Rural Roads Prioritization Report Kilombero District Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Richard Mwakasitu, Roads Component Leader Keith F. Williams, P.E., Chief Of Party Kilombero Usaid Report.Pdf Irrigation And Rural Roads Infrastructure Project (IRRIP2) Final Report For: Rehabilitation Works Along Mgudeni – Kanyenja 10km Road Mpanga – Ngarimila 10km Road And Namwawala – Mofu 17.3km Road 26 February, 2016 Kiteto Prioritization Report Rural Roads Prioritization Report Kiteto District Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Richard Mwakasitu, Roads Component Leader Keith F. Williams, P.E., Chief Of Party Kongwa Prioritization Report Rural Roads Prioritization Report Kongwa District Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Richard Mwakasitu, Roads Component Leader Keith F. Williams, P.E., Chief Of Party Lbt Routine Maintenance Operations Labour Based Roadworks - Maintanance Operations - Routine Maintanance Lbt Routine Maintenance Tools & Tasks Swahili Jamhuri Ya Muungano Wa Tanzania Tawala Za Mikoa Na Serikali Na Mitaa Mradi Wa Uboreshaji Barabara Vijijini Mwongozo Wa Utunzaji Barabara Kwa Mafundi Wasaidizi Wa Barabara Wa Vijiji Mvomero Prioritization Report Rural Roads Prioritization Report Mvomero District Technical Assistance To Support The Development Of Irrigation And Rural Roads Infrastructure Project (IRRIP2) Richard Mwakasitu, Roads Component Leader Keith F. Williams, P.E., Chief Of Party Mwongozo Wa Usajili Wa Vikundi Na Mkataba Na Tarura (1) Kiambatanisho Cha Taarifa Ya Kikosi Kazi Namba 1 Mradi Wa Miundombinu Ya Umwagiliaji Na Barabara Vijijini Mwongozo Wa Usajili, Utambuzi Na Manunuzi Ya Huduma Za Matunzo Na Matengenezo Ya Barabara Kwa Kushirikisha Vikundi Vya Kijamii Vya Watumia Barabara Vijijini Machi 2017 Roads And Fuel Tolls Regulations 2016 The Road And Fuel Tolls Regulations, 2016. Government Notice No. 80 Published On 19/02/2016 98 File Name "Titles" Roads Fund Board Client Service Charter Roads Fund Board Clients’ Service Charter Rua Contact Information Mradi Wa Uendelezaji Miundombinu Ya Umwagiliaji Na Barabara Vijijini – Irrip Orodha Ya Viongozi Pamoja Na Mafundi Wasaidizi Wa Vikundi Vya Matengenezo Ya Barabara Wilaya Ya Kongwa Task Force Guidelines Irrigation And Rural Roads Infrastructure Project (IRRIP2) Task Force Report Appendix No. 1 Delivarable 1: Guidelines For Registration, Recognition And Procurement Of Road User Community Organizations For Involvement In Road Routine Maintenance Training Curiculum.Docx Faril Roads Implementation Training Curiculum Unit Rates Build Up Ifakara Mlimba Road Works - Unit Rate Analysis ( Tshs.) Vol2c Labour Based Roadworks - Maintenance Operations 99 ANNEX IV: DISCLOSURE OF CONFLICTS OF INTEREST DISCLOSURE OF CONFLICT OF INTEREST FOR USAID EVALUATION TEAM MEMBERS Name Ritu Nayyar-Stone Title Principal Research Scientist Organization NORC at the University of Chicago Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Rural Roads Development Program (Phase 1); CDM Smith; 621-IL-14-00004; 621-IL-14-00007; 621-IL-14- 00008; 621-IL-14-00009 I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. -- I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date September 26, 2018 100 101 DISCLOSURE OF CONFLICT OF INTEREST FOR USAID EVALUATION TEAM MEMBERS Name Bahati Tenga Title Subject Matter Expert Organization -- Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Rural Roads Development Program (Phase 1); CDM Smith; 621-IL-14-00004; 621-IL-14-00007; 621-IL-14-00008; 621-IL￾14-00009 I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. -- I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 25th September 2018 102 DISCLOSURE OF CONFLICT OF INTEREST FOR USAID EVALUATION TEAM MEMBERS Name Zoe Grotophorst Title Principal Research Analyst Organization NORC Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Rural Roads Development Program (Phase 1); CDM Smith; 621-IL-14-00004; 621-IL-14-00007; 621-IL-14-00008; 621-IL￾14-00009 I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. -- I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date September 24, 2018 103 ANNEX V: EVALUATION TEAM BIOGRAPHY Dr. Ritu Nayyar-Stone served as the Team Leader for this project. Dr. Nayyar-Stone is an economist and Principal Research Scientist at NORC at the University of Chicago’s International Programs department, with over 20 years of experience in 24 countries across the world. As a public finance economist, Dr. Nayyar-Stone’s expertize includes the design and econometric analysis of both quantitative and qualitative data collected in developing countries. Her qualitative data experience (focus group discussions and key-informant interviews) includes information obtained for a performance evaluation for a governance and economic management support project for USAID in Liberia; evaluation of two family strengthening projects in Burundi and Moldova funded by USAID’s Displaced Children and Orphan’s Fund; Georgia’s agribusiness and road rehabilitation impact evaluation for MCC; and performance evaluation of a capacity building project for local thinks in Asia, Africa and Latin America for Global Development Network. Besides serving as NORC’s Project Director for the Data for Development task order, Dr. Nayyar-Stone is currently Chief of Party for a multi-year performance and impact evaluation of the USAID funded Literacy Achievement and Retention Activity in Uganda examining improvements in reading and school safety due to a decrease in school related gender based violence. Ms. Zoe Grotophorst served as the Evaluation Specialist for this project. Ms. Grotophorst is a Principal Research Analyst with NORC at the University of Chicago with more than 7 years of experience conducting quantitative and qualitative research for international and domestic government, corporate, and non-profit organizations. As a polling and strategic communications expert, she has extensive experience planning, execution, and interpretation of surveys, in-depth interviews, and focus group discussions, and is a trained focus group facilitator. Her research has focused on democracy and governance and institutional development, electoral reforms, housing, women’s empowerment, and migration, among other issues. Prior to joining NORC, Ms. Grotophorst was a Senior Analyst for Lake Research Partners, providing strategic guidance for governments, political parties, candidates, foundations, and non-profit organizations from 2016-2018. She has worked on surveys and qualitative projects in a variety of countries, focusing primarily on citizens’ perceptions and experiences with government institutions. Ms. Bahati Tenga served as the Local STTA Researcher and Roads SME for this project. Ms. Tenga is a socio-development specialist with over 14 years of professional experience in community engagement and monitoring and evaluation of roads and transportation projects in Tanzania. During her work as a consultant with ST Associates and as a freelance consultant, Ms. Tenga conducted Environment as well as Social and Impact Assessments (SIAs) for many of the country’s roads projects. Examples of Ms. Tenga’s research include the following projects: Impact Assessment of the Gender Integration into the M&E of the Village Travel and Transport Programme (VTTP) under the National Rural Transportation Initiative (GRTI), (2004); the Impact Assessment, Strategy and Participatory Methodology of Rural Infrastructure (2004); and the Environmental and Social Impact Assessments of Iringa – Dodoma Road (2010), Chalinze – Segera – Tanga Road (2006), Iringa – Mafinga Road (2007), and Tanga – Horohoro Road (2006).These assessments involved collecting and analyzing qualitative data from a variety of populations. Ms. Tenga has extensive experience working with local institutions as well as government and local NGOs to improve engagement in development projects. Mr. Daud Siwalaze served as the M&E Specialist for this project. Mr. Siwalaze has more than 7 years of experience working in this area and serves as the M&E specialist for the Data for Development task order. In addition to his expertise in this area, Mr. Siwalaze has experience with training and community mobilization, local and institutional capacity building, and government advocacy. Mr. Siwalaze’s work has also involved quantitative and qualitative instrument design, research, and analysis for donor-funded projects in mainland Tanzania. In one of his roles as an M&E Specialist at Technoserve, Mr. Siwalaze worked to conduct various surveys and lead analysis and dissemination of data related to the Strengthening Rural Youth Development through Enterprise (STRYDE) program in Mbeya. As an M&E specialist with DAI LLC GLobal, Mr. Siwalaze provided technical guidance for the Impact Evaluation of Land Tenure Assistance conducted by MSI and NORC in Iringa. He also participated in the data collection for the evaluation as an independent local consultant, having designed surveys and questionnaires on project indicators for data collection purposes 104 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 105