This publication was produced for review by the United States Agency for International Development. It was prepared independently for the E3 Analytics and Evaluation Project by Management Systems International, A Tetra Tech Company; and NORC at the University of Chicago. Photo Credit: Jacob Patterson-Stein PHASE II BASELINE AND PHASE 1 MIDLINE REPORT IMPACT EVALUATION OF THE FEED THE FUTURE TANZANIA LAND TENURE ASSISTANCE ACTIVITY MARCH 2018 PHASE II BASELINE AND PHASE I MIDLINE REPORT IMPACT EVALUATION OF THE FEED THE FUTURE TANZANIA LAND TENURE ASSISTANCE ACTIVITY March 28, 2018 Contracted under AID-OAA-M-13-00017 E3 Analytics and Evaluation Project Prepared by: Dr. Lauren Persha (Principal Investigator, NORC at the University of Chicago) Jacob Patterson-Stein (Evaluation Coordinator, Management Systems International) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY I CONTENTS ACRONYMS III ACKNOWLEDGEMENTS IV EXECUTIVE SUMMARY V 1 INTRODUCTION 1 2 LTA ACTIVITY BACKGROUND 1 2.1 TANZANIAN LAND CONTEXT 1 2.2 LTA ACTIVITY DESCRIPTION 2 2.3 DEVELOPMENT HYPOTHESIS 2 2.4 PROJECT IMPLEMENTATION STATUS 2 3 EVALUATION BACKGROUND PURPOSE, AUDIENCES, AND USES 4 3.1 PURPOSE 4 3.2 AUDIENCE 4 3.3 INTENDED USE 4 4 EVALUATION DESIGN 5 4.1 THEORY OF CHANGE 5 4.2 EVALUATION QUESTIONS 5 4.3 EVALUATION DESIGN 6 4.3.1 RANDOM SELECTION 8 4.4 BASELINE DATA COLLECTION 9 4.4.1 HOUSEHOLD SAMPLE SELECTION 11 4.5 SURVEY INSTRUMENT 12 4.6 CHALLENGES ENCOUNTERED DURING DATA COLLECTION 14 5 BASELINE FINDINGS 14 5.1 HOUSEHOLD CHARACTERISTICS 15 5.2 TENURE SECURITY AND LAND DISPUTES 17 5.3 LANDHOLDINGS, INVESTMENT, AND ENVIRONMENT 23 5.3.1 FOOD INSECURITY 25 5.3.2 SOCIAL AND EMPOWERMENT OUTCOMES 26 5.4 BALANCE AND POWER 31 5.4.1 TESTING FOR BALANCE ACROSS TREATMENT AND COMPARISON GROUPS 31 5.4.2 REVISITING POWER ASSUMPTIONS WITH PHASE II BASELINE DATA 32 5.5 CONCLUSION AND DISCUSSION 36 5.5.1 NEXT STEPS 37 6 PHASE I MIDLINE 38 6.1 IMPLEMENTATION BACKGROUND 39 6.1.1 CHANGES TO IMPLEMENTATION TIMING AND IMPLICATIONS FOR THE IE 39 6.2 MIDLINE RESULTS 40 6.2.1 SUMMARY STATISTICS BETWEEN BASELINE AND MIDLINE 42 6.2.2 LANDHOLDINGS, INVESTMENT, AND ENVIRONMENT 44 6.3 ESTIMATION APPROACH 52 6.4 DISCUSSION OF RESULTS 54 ANNEX A: EVALUATION STATEMENT OF WORK 58 ANNEX B: PHASE II BASELINE SURVEY INSTRUMENT 69 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY II ANNEX C: MIDLINE SURVEY INSTRUMENT 123 ANNEX D: MEMO EXPLAINING RISKS TO RCT DESIGN FROM CHANGING EVALUATION TIMELINE 146 ANNEX E: COMPARISON OF PHASE I AND PHASE II BASELINE DATA 150 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY III ACRONYMS CCRO Certificate of Customary Right of Occupancy CI Confidence Interval DAI Development Alternatives, Inc. DDL Development Data Library (USAID) DLO District Land Office E3 Bureau for Economic Growth, Education, and Environment (USAID) FTF Feed the Future GOT Government of Tanzania GSES Generalized Self-Efficacy Scale HH Household ICC Intra-Cluster Correlation IE Impact Evaluation KII Key Informant Interview LTA Land Tenure Assistance LU Office of Land and Urban (USAID/E3) MAST Mobile Application to Secure Tenure MDES Minimum Detectable Effect Size MSI Management Systems International NGO Non-Governmental Organization PLC Office of Planning, Learning, and Coordination (USAID/E3) RCT Randomized Controlled Trial RSA Research Solutions Africa SAGCOT Southern Agricultural Growth Corridor of Tanzania SOW Statement of Work VLUP Village Land Use Plan USAID United States Agency for International Development PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY IV ACKNOWLEDGEMENTS The evaluation team thanks the families and village leaders in Iringa District who shared their time to participate in this study. The Iringa District Land Office and the Tanzanian Ministry of Lands, Housing, and Human Settlements both deserve praise for their assistance, cooperation, and patience with the evaluation process and team members. Special thanks to Research Solutions Africa for conducting the household surveys on an updated schedule, and for quickly responding to data￾related questions. The team is grateful to the LTA activity staff for their continued collaboration, cooperation, and communication. The team also acknowledges Gerald Usika for his quality oversight during data collection, his translation support, and general guidance as the local coordinator for this evaluation. Finally, the evaluation team appreciates the technical review and support for this report from Management Systems International’s Amanda Janczak, Jeremy Gans, Irene Velez, Carolyn Fonseca, and Kevin Warr. The team also thanks Ioana Bouvier (USAID/E3/LU), Sarah Lowery (USAID/E3/LU), Harold Carey (USAID/Tanzania), and Bhavani Pathak (USAID/E3/PLC) for their continued support of this study. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY V EXECUTIVE SUMMARY This report corresponds to the impact evaluation (IE) of the Feed the Future Tanzania Land Tenure Assistance (LTA) activity commissioned by the Office of Land and Urban in the United States Agency for International Development’s Bureau for Economic Growth, Education, and Environment (USAID/E3). The evaluation uses a two-phase randomized controlled trial design to rigorously test how mobile mapping and facilitation of land tenure certification affect income, women’s empowerment, dispute prevalence, and other factors related to land use and tenure security in Iringa District, Tanzania. This document provides findings from the Phase II baseline for the IE, which includes a snapshot of key demographics, household characteristics, and outcome variables. The report also covers the Phase I midline and provides comparisons between the Phase I midline and baseline data. The document further provides a robust overview of key metrics for households in rural Iringa, and investigates whether changes have occurred between the two Phase I data collection rounds. LTA ACTIVITY DESCRIPTION Tanzania presents a dynamic land tenure context. All land in Tanzania is owned by the state and held in trust by the president, but individuals residing on or using designated “Village Land” have the right to obtain formal documentation of their use rights in the form of a Certificate of Customary Right of Occupancy (CCRO).1 However, insufficient capacity of district land offices (DLOs) that issue CCROs, a lack of funds to pay CCRO fees, unfamiliarity with formal land laws, and other factors have resulted in few villagers obtaining formal documentation for their plots. Increasingly, the Government of Tanzania (GOT) and the donor community recognize that improving the security of land rights is essential to protecting the rights of smallholders, reducing disputes and tensions, and maximizing the economic potential of the region. USAID/Tanzania awarded the four-year, $6 million LTA activity to DAI in December 2015. The activity seeks to clarify and document land ownership, support local land use planning efforts, and increase local understanding of land use and land rights in Tanzania. The LTA activity assists villages and the local DLO in Iringa and Mbeya districts in completing the land use planning process and delivering CCROs in select villages. It also provides education on land laws, CCROs, and land management. The LTA activity is using the Mobile Application to Secure Tenure (MAST), an app that facilitates the mapping and CCRO process. The LTA activity is being implemented in 36 villages: six that were chosen for initial implementation, and an additional 30 in Iringa District, Tanzania as part of the IE. EVALUATION QUESTIONS Table 1 shows five questions addressed by the LTA IE that the evaluation team developed and finalized in collaboration with USAID. They are derived from the LTA’s theory of change. 1 For more on Tanzania’s land ownership system, see the USAID Country Profile, “Land Tenure and Property Rights: Tanzania,” at https://www.land-links.org/wp-content/uploads/2016/09/USAID_Land_Tenure_Tanzania_Country_Profile.pdf. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY VI TABLE 1: THEMATIC AREAS OF INVESTIGATION AND EVALUATION QUESTIONS Thematic Area Evaluation Questions 1. Tenure security and land management 1. In what ways and to what extent do landholders who have received formal land documentation through the assistance of LTA perceive their land rights to be more secure? 2. Land disputes 2. To what extent are landholders who have received formal land documentation through the assistance of LTA less likely to experience land disputes? 2.1 What kinds of disputes (if any) are affected and what are the mechanisms by which LTA affects them? 3. Investment and land use 3. To what extent do landholders who have received formal land documentation through the assistance of LTA change their investment and land use decisions in a manner that reflects strengthened incentives resulting from increased tenure security? 3.1 What (if any) are the specific decisions that are affected and how does LTA influence them? 4. Empowerment 4. To what extent do the LTA outreach and communication activities, as well as mapping, verification, and the formal registration of land, lead to a greater sense of empowerment on the part of women, youth, and pastoralists? 4.1 What (if any) are the specific aspects of empowerment that are affected and how does LTA influence them? 5. Economic and environmental outcomes 5. To what extent do the LTA interventions to strengthen land tenure lead to increased agricultural productivity, household income, and wealth, as well as more environmentally sustainable land-use practices and associated environmental benefits? 5.1 Which (if any) of these outcomes are affected and how does LTA influence them? EVALUATION DESIGN The LTA IE uses a cluster randomized design whereby villages are randomly assigned to receive the LTA activity or serve as control villages. The IE will measure LTA’s impacts on activity beneficiaries in 30 randomly selected villages in Iringa District. Project implementation in the 30 villages is planned to take place in two phases: beginning in 2017 in an initial set of 15 randomly selected villages, followed by a second set of 15 randomly chosen villages beginning in mid-2018. Ideally, all 30 villages will be selected at the outset, with a single baseline collected prior to implementation. However, in response to concerns raised by DAI, village selection was designed to take place in two stages at the start of each implementation phase. DAI’s concerns stemmed from potential shifts in village administrative and geographic boundaries during the implementation period, a common occurrence in Tanzania as village populations grow. In August 2017, due to concerns about achieving activity goals, the second phase of implementation was brought forward by approximately six months, as shown in Table 2. This has implications for midline data collection and findings, since behavior changes and other outcomes of interest are less likely to occur at scale over the revised six-month timeframe between survey rounds. TABLE 2: PROPOSED AND REVISED LTA ACTIVITY IE PHASES Proposed Implementation Start Revised Implementation Start Control Treatment April 2017 April 2017 15 randomly selected villages do not receive LTA 15 randomly selected villages receive LTA April 2018 October 2017 15 randomly selected villages do not receive LTA 15 randomly selected villages receive LTA PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY VII The evaluation conducted a household panel survey of a random sample of respondents in each village prior to each implementation phase. An initial survey round in March-April 2017 served as the Phase I baseline, and a second survey round in September-Oct 2017 served as the midterm data collection for Phase I villages and the baseline for Phase II villages. Phase II baseline data collection consisted of two household surveys administered to 1,320 respondents across 807 households in 32 villages2 in Iringa District: • The “Head of Household Survey” was administered to the identified head of household. • The “Wives’ Survey” was administered to the primary spouse/partner of the head of household. In addition to reporting on Phase II baseline data, this report also includes analysis of the Phase I midline. The midline survey redeployed an amended version of the Phase I/Phase II baseline survey and targeted respondents from the Phase I baseline. Of the original 1,179 respondents across 755 households in the Phase I baseline, the midline survey re-interviewed 907 respondents across 610 households, or around 81 percent of the original sample. PHASE II BASELINE KEY FINDINGS TENURE SECURITY AND LAND DISPUTES • Phase II baseline results indicate low familiarity with land laws, as only nine percent of the treatment group and seven percent of the comparison group reported some level of knowledge. • Almost half the treatment group household head sample (n = 182) reported an expectation that the incidence of disputes will improve over the next 12 months; the comparison group reported similar findings (n = 215). • Overall, disputes were generally reported as inconsequential, but for those who did report dispute concerns, grazing disputes were perceived to be most problematic across both assignment groups. • There is a slight clustering of disputes within villages that reported more than one dispute. In most cases, this clustering can be found where two respondents each reported one dispute within close proximity (i.e., the dispute is with a nearby neighbor). • For the Phase II baseline, 11 percent of treatment group respondents and 14 percent of comparison group respondents possessed land-related documentation at baseline. LANDHOLDINGS, USE, AND INVESTMENT • Households in both the treatment and comparison groups reported owning or renting about the same number of parcels, with a median of two parcels for both assignment groups. Only 59 (4.92 percent) and 49 (4.75 percent) respondents in the treatment and comparison groups, respectively, reported more than three parcels. • Few respondents in the treatment group reported making any investments in their land. Low levels of building, soil conservation, and terracing are expected, given the capital-intensive nature of these activities. Less than 20 percent of respondents in both assignment groups reported tree-planting activity. 2 Two buffer villages were randomly assigned to treatment and control as part of Phase I and Phase II data collection. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY VIII SOCIAL AND EMPOWERMENT OUTCOMES The evaluation team examined food security, self-efficacy, and decision making as part of the social and empowerment outcomes for this evaluation. Key findings from baseline data collection included: • Within the treatment group, 39 percent of female headed households (n = 35) and 23 percent of male headed households (n = 64) reported facing food insecurity over the previous 12 months. For the same period in the comparison group, 33 percent of female headed households (n = 35) and 17 percent (n = 54) of male headed households reported food insecurity in the previous year. • When asked whether any household members went to sleep hungry, almost 11 percent of treatment respondents and only five percent (n = 21) of comparison households said this happened sometimes or often (n = 40). • Household heads in the sample reported making a majority of the decisions on parcel use. In contrast, primary female spouses reported joint decision making most frequently on parcel use decisions. PHASE II BASELINE CONCLUSIONS • Household characteristics: The baseline dataset includes 615 LTA beneficiary respondents, of whom 244 are primary female spouses. Around 40 percent of both of the assignment group samples is comprised of primary spouses. • Tenure security and land disputes: The data show substantial perceived tenure insecurity around fallowing, but low dispute incidence and little familiarity with land laws. • Landholdings, use, and investment: Most households use multiple land parcels, with a wide variety in the size of the landholding. There was very little investment reported in either assignment group, but almost 20 percent of respondents in the treatment group reported planting non-fruit trees in the previous 12 months. • Social and empowerment outcomes: About 27 percent of the treatment group and 21 percent of the comparison group reported facing food insecurity. Household heads generally reported that they were responsible for most parcel use decisions, while primary spouses reported that parcel use decisions were jointly made, on average. This is perhaps due to various cognitive or social-emotional biases and will need to be examined more thoroughly at endline and through qualitative work. • As expected, given the randomized design, no major differences were observed between the treatment and control groups that would raise concerns for the IE. • The IE is expected to have sufficient statistical power to accurately measure the impacts of LTA on a broad range of outcomes. However, the fact that implementation is limited to 30 villages may mean that the IE is not able to reliably detect impacts for a limited number of the anticipated outcomes, such as the environmental outcomes. PHASE I MIDLINE The Phase I midline data included 610 households and 907 respondents. This sample includes about 81 percent of the Phase I baseline sample (n = 755 households and 1,179 respondents). CHANGES BETWEEN PHASE I BASELINE AND MIDLINE The overall sample for the two Phase I survey rounds shows some changes to sample means over time, in the following key areas: PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY IX LAND RIGHTS AND TENURE SECURITY • While 16 percent (n = 60) of treatment group households surveyed at baseline said they possessed land-related documentation, at midline this had risen to 43 percent (n=125), a statistically significant change (p<0.01). • Willingness to pay for CCROs fell in both the treatment and comparison groups between survey rounds by 18,881 shillings and 5,187 shillings, respectively. This difference was statistically significant for the treatment group (p<0.01). • When asked about land disputes in the coming year, there was an 11 percent increase in treatment group respondents who expected that problems with land disputes will improve (151 to 168) (p<0.01). The comparison group saw the opposite between survey rounds: a nine percent decrease in respondents who expected that land disputes will improve in the next 12 months (149 to 135) (p<0.10). LAND DISPUTES • The incidence of land disputes did not change for the treatment group between survey rounds; however, the percentage of the comparison sample who reported experiencing a dispute over the same period fell from 10 percent (n = 38) to around six percent (n=17) (p<0.10). LANDHOLDINGS, INVESTMENT, AND ENVIRONMENT • Given the short period between baseline and midline, the evaluation team does not expect major changes in landholdings and investment behavior beyond what may be seasonally driven. • The treatment group reported an increase in building construction investments from around 21 percent (n= 81) of the treatment sample to 47 percent of the sample (n=224); this likely reflects seasonal variation between survey rounds. The comparison group reported similar increases in building activity. HOUSEHOLD INVESTMENT AND WIVES’ SURVEY • The average number of treatment group respondents reporting that they would be able to obtain a loan if needed increased from 51 to 61 percent (p<0.01). There was no statistically significant change in this indicator for the comparison group over the same period. • The percentage of respondents in the treatment sample who were aware of women’s groups grew from 57 percent (n = 111) to 73 percent (n = 104) (p<0.01). There was also an increase in this measure, from 53 percent (n = 107) to 69 percent (n = 111) (p<0.01), in the comparison group. INFERENTIAL ANALYSIS FOR PHASE I MIDLINE The evaluation team conducted preliminary inferential analysis to assess the causal impact of the LTA activity on select outcomes of interest. Anticipated changes to outcomes at the Phase I midline are likely to be smaller than expected at the time of the evaluation design, given the change in the implementation timeline and earlier collection of the midline data for Phase I. The inferential analysis can account for confounding factors that may drive part of the change in means between survey rounds. The evaluation team used a fixed effects difference-in-difference (DID) panel regression specification to test for the impact of the LTA activity on outcomes under each of the thematic outcome categories (tenure security and land management; land disputes; investment and land use; and empowerment). There were three main findings from this analysis: PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY X • Household possession of land-related documentation: Controlling for household head gender, age, education level, and distance to Iringa, there is, on average, a 29.8 percent increase in the likelihood of a household having land document at midline, for households in the treatment group relative to those in the comparison group. The magnitude of impact is relatively large, and the statistical significance is robust to alternative model specifications. • Total household landholdings: Results suggest that, on average at midline, total landholdings by treatment group households has increased by 0.67 ha relative to comparison group households. However, the magnitude of impact is small, the results are only marginally significant (p<.10), and they are not robust to alternative model specifications. • Land related decision-making power exclusively by the male household head: For treatment group households, and controlling for household and village factors, results suggest that there has been an 11.4 percent decrease in the likelihood of a land-related decision solely by the male household head (p<0.05). PHASE II BASELINE CONCLUSIONS At this early midline stage, evidence suggests that LTA implementation may be leading to positive impacts on some key intermediate outcomes across three of the four outcome categories assessed. Under the LTA theory of change, continuation of such impacts over the activity lifetime is expected to lead to significant improvements in longer terms outcomes, such as increased agricultural productivity and household income. The midline analysis did not find statistically significant impacts for many of the outcomes assessed at this stage. However, this may not be surprising given that the analyses measure impacts for activities that have been underway for only six months. The generally low proportion and lack of change on household familiarity with land laws for the treatment group may indicate that project messaging on this has not yet taken hold. Additionally, households that have only recently obtained their CCROs and begun to understand their potential benefit for securing their landholdings may not yet have experienced a lower expropriation risk, or changed their land investment behavior accordingly. Overall, the midline results (1) indicate that achievement of some of the anticipated LTA impacts appears to be underway, (2) confirm the validity of the IE design and sample power, (3) highlight the role endline qualitative data are likely to play in helping to explain impacts at endline, and (4) confirm the utility of measuring longer term outcomes as planned at endline. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 1 1 INTRODUCTION This baseline and midline report corresponds with the impact evaluation (IE) of the Feed the Future (FTF) Tanzania Land Tenure Assistance (LTA) activity commissioned by the Office of Land and Urban in the United States Agency for International Development’s Bureau for Economic Growth, Education, and Environment (USAID/E3/LU). The E3 Analytics and Evaluation Project3 designed and is implementing the evaluation. The evaluation uses a randomized controlled trial (RCT) design to test how mobile mapping and facilitation of land tenure certification affect income, women’s empowerment, dispute prevalence, and other factors related to land use and tenure security in Iringa District, Tanzania. Data collection for this evaluation occurred during two phases to account for potential contextual challenges. Annex A provides USAID’s statement of work (SOW) for the evaluation. This report provides findings from Phase II baseline data collection for the IE – which includes a snapshot of key demographics, household characteristics, and outcome variables – and Phase I midline follow-up data collection. The document: • Describes and summarizes findings for the Phase II baseline, • Assesses the balance between treatment and control groups for the Phase II baseline, • Compares the Phase I and Phase II baseline villages across treatment and comparison groups, and • Summarizes and analyzes data for the Phase I baseline and Phase I midline. 2 LTA ACTIVITY BACKGROUND 2.1 TANZANIAN LAND CONTEXT The Tanzanian land rights system is based on public ownership of land. All land is owned by the state and held in trust by the president. The majority of land in Tanzania is designated as Village Land, which is governed by the 1999 Village Land Act. The act recognizes the rights of villages to hold and administer land according to customary law. Individuals who use or occupy Village Land have the right to obtain formal documentation of their use rights via a Certificate of Customary Right of Occupancy (CCRO), issued by local government.4 In practice, most villagers do not have CCROs for their plots and lack formal documentation of their land rights (Pederson 2010). In many villages, the land use demarcation and mapping required to issue the documents has not yet been completed. Moreover, the district land offices (DLOs) responsible for issuing CCROs frequently lack the capacity to do so, and rural land users are often unaware of their land rights under the law. Meanwhile, multiple factors contribute to increasing pressure on land, particularly in the Southern Agricultural Growth Corridor of Tanzania (SAGCOT) region. The confluence of climate change, population growth, and the regular migration of pastoralist communities to the region causes tensions over land and give rise to many types of disputes at various levels (Mwamfupe 2015). Large￾scale agricultural investments are increasing in the area, leading to insecurity on the part of smallholders, due to weak land rights protection and limited bargaining power (Deininger 2011). Recognition is increasing on the part of the Government of Tanzania (GOT) and the donor community that improving the security of land rights is essential to protect the rights of smallholders, reduce disputes and tensions, and maximize the economic potential of the region. 3 Management Systems International (MSI) implements the E3 Analytics and Evaluation Project in partnership with Development and Training Services, a Palladium company, and NORC at the University of Chicago. 4 For more on Tanzania’s land tenure system, see USAID Country Profile, “Land Tenure and Property Rights: Tanzania,” at https://www.land-links.org/wp-content/uploads/2016/09/USAID_Land_Tenure_Tanzania_Country_Profile.pdf. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 2 2.2 LTA ACTIVITY DESCRIPTION The LTA activity, which is a part of the United States Government’s Feed the Future (FTF) initiative, is implemented through a four-year, $6 million contract awarded by USAID/Tanzania to DAI in December 2015. The LTA activity will clarify and document land ownership, support local land use planning efforts, and increase local understanding of land use and land rights in Tanzania. The interventions under the LTA activity aim to increase land tenure security and lay the groundwork for sustainable agricultural investment for both smallholder farmers and commercial investors throughout the SAGCOT and in the value chains of focus for Tanzania’s FTF program. The LTA activity comprises two larger activities (1 and 2) and two smaller activities (3 and 4), described below. Local sustainability is a critical component of the overall activity. The goal of the LTA is to empower district and village land institutions in targeted districts to carry forward the capacity development and land administration process independently, with little or no outside financial support, once the activity concludes. The LTA activity works within the current land management bureaucracy, but helps facilitate formal land certification and education through the following activities: 1. Assist villages and district administrations in completing the land use planning process and delivering CCROs in select villages within two districts (Iringa and Mbeya). 2. Educate and develop the capacity of village land governance institutions and individual villagers to complete the land use planning and CCRO process; effectively manage land resources; respect the land rights of women, youth, and pastoralists; and build agriculture￾related business skills. 3. Educate and develop the capacity of district-level land governance institutions in the Mbeya Region to complete the land use planning and CCRO process; effectively manage land resources; respect the land rights of women, youth, and pastoralists; and build agriculture￾related business skills. 4. Develop capacity to use the Mobile Application to Secure Tenure (MAST) application throughout the SAGCOT and, nationally, to assist with tenure certification. 2.3 DEVELOPMENT HYPOTHESIS USAID envisions that if the LTA activity provides clarification and documentation of land ownership, supports land use planning efforts, and increases local understanding of land use and land rights, then this will lead to increased agricultural investment, reduced land tenure risk, and more empowered people and local institutions. The LTA activity components work in tandem to promote inclusive agricultural development, food security and investment, and institutional capacity. Figure 1 illustrates the causal linkages that USAID envisions for translating results under each of the activities into the LTA activity’s intended intermediate and final outcomes. 2.4 PROJECT IMPLEMENTATION STATUS DAI started implementing LTA in late 2016 in six pilot villages in Iringa District (these results are not included in the IE). Full-scale implementation in 15 Phase I villages began following baseline data collection for the IE in April 2017. A new DAI chief of party took over the LTA activity in early 2017, which resulted in some adjustments to the implementation and evaluation approach. Phase II implementation was originally planned to occur approximately 12 months after Phase I, but due to concerns regarding target achievement, the originally agreed schedule was amended to begin Phase II six months earlier than planned. LTA implementation has occurred in all Phase I villages, and, as of the first draft of this report, most of the Phase II villages. Annex D provides more information on the change in timeline. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 3 FIGURE 1: THEORY OF CHANGE FOR THE LTA ACTIVITY PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 4 3 EVALUATION BACKGROUND PURPOSE, AUDIENCES, AND USES This IE comes at an opportune time, as USAID and the GOT are already investing elsewhere in land tenure programming while recognizing that additional research is needed to strengthen the evidence base of how land rights clarification and documentation affects investment, the incidence of disputes, women’s empowerment, and tenure security. While USAID and implementers from international development organizations and non-governmental organizations (NGOs) have been exploring different approaches for documenting land ownership and sustainable land investment, few rigorous evaluations have measured the impact of more formal approaches and outcomes from customary tenure systems. 3.1 PURPOSE The purpose of this IE is to provide USAID with evidence on the impacts of its investment in the LTA activity, and to contribute to research on the impacts of land mapping, registration, and formalization in rural customary land tenure settings in Tanzania. The results of this evaluation will be made widely available to assess lessons learned and, as applicable, encourage replication within or beyond Tanzania. As such, this evaluation will apply USAID’s Evaluation Policy guidance with respect to using the most rigorous evaluation design and methods possible to demonstrate accountability for achieving results. The evaluation is also designed to capture practical lessons from USAID’s experience with increasing sustainable agricultural investment by securing land tenure through first￾time registration. 3.2 AUDIENCE The evaluation is aimed at several audiences. The findings are expected to be of value, from an accountability and learning standpoint, to USAID, specifically USAID/E3/LU and the USAID/E3 Office of Global Climate Change, as well as the Tanzania Mission. Findings and lessons learned from this evaluation will also be of interest to the GOT and donor community active in the sector. Both aim to scale CCRO delivery rapidly across Tanzania. DAI and other practitioners in the land tenure sector working to document customary land rights will also find the evaluation useful. Finally, the evaluation will also be relevant to donors such as those involved with the Land Tenure Support Program, a large-scale effort funded jointly by the United Kingdom’s Department for International Development, the Swedish International Development Cooperation Agency, and the Danish International Development Agency, as well as implementers and scholars generally interested in its important contribution to the evidence base on land tenure interventions. 3.3 INTENDED USE This evaluation will inform the design of future donor and government activities that aim to improve tenure security and generate economic benefits by strengthening land rights. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 5 4 EVALUATION DESIGN 4.1 THEORY OF CHANGE Figure 1 illustrates the causal linkages that USAID envisions for translating results under each of the activities5 into LTA’s intended intermediate and final outcomes. By contributing to the issuing of CCROs to land users, as well as education on the land laws and capacity-building components, the LTA activity will contribute to improved tenure security and reduced incidence of land disputes. These outcomes will, in turn, spur increased investment in agriculture, as land users change their behavior in response to stronger incentives brought about by improved security. It is expected that women, youth, and pastoralists who receive a CCRO will experience a greater sense of empowerment. Empowerment should also result more broadly from LTA outreach and education on land laws which protect the rights of women, youth, and pastoralists. Developing Village Land Use Plans (VLUPs), as well as some of the trainings for village and district officials, will improve the capacity of village and government institutions to manage land resources. This includes identifying and maintaining protected areas, establishing or strengthening the management of communal forest areas or woodlots, limiting excessive expansion of areas under cultivation, and implementing other environmental management practices or sustainable land uses within villages. Finally, activities under LTA to raise awareness about MAST and build capacity to use it within the GOT and donor community should result in greater uptake of the MAST technology in future land mapping and registration projects. This would encourage to more transparent, participatory, and efficient processes to issue CCROs. The IE is limited to measuring LTA’s impacts on the direct beneficiaries of the activity through the issuing of CCROs and LTA’s outreach and education component (i.e., the first two “activity” boxes in Figure 1). Assessing the extent to which other efforts to issue CCROs have taken up the MAST technology would require different data sources and methods, and would likely require a longer timeframe as well. Thus, the last benefit stream in Figure 1 will be beyond the scope of this IE. 4.2 EVALUATION QUESTIONS The LTA IE addresses five questions derived from the theory of change, shown in Table 3. The evaluation team developed and finalized these questions in collaboration with USAID.6 5 Figure 1 shows only three activities, since Activity 3 is specific to Mbeya District and this IE focuses solely on LTA activities in Iringa District. This theory of change diagram has been updated since the SOW shown in Annex A, with USAID’s approval. 6 The evaluation questions outlined in this section have been revised since the SOW provided in Annex A was prepared. These changes have been approved by USAID as part of the evaluation design proposal. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 6 TABLE 3:THEMATIC AREAS OF INVESTIGATION AND EVALUATION QUESTIONS Thematic Area Evaluation Questions 1. Tenure security and land management 1. In what ways and to what extent do landholders who have received formal land documentation through the assistance of LTA perceive their land rights to be more secure? 2. Land disputes 2. To what extent are landholders who have received formal land documentation through the assistance of LTA less likely to experience land disputes? 2.1 What kinds of disputes (if any) are affected and what are the mechanisms by which LTA affects them? 3. Investment and land use 3. To what extent do landholders who have received formal land documentation through the assistance of LTA change their investment and land use decisions in a manner that reflects strengthened incentives resulting from increased tenure security? 3.1 What (if any) are the specific decisions that are affected and how does LTA influence them? 4. Empowerment 4. To what extent do the LTA outreach and communication activities, as well as mapping, verification, and the formal registration of land, lead to a greater sense of empowerment on the part of women, youth, and pastoralists? 4.1 What (if any) are the specific aspects of empowerment that are affected and how does LTA influence them? 5. Economic and environmental outcomes7 5. To what extent do the LTA interventions to strengthen land tenure lead to increased agricultural productivity, household income, and wealth, as well as more environmentally sustainable land-use practices and associated environmental benefits? 5.1 Which (if any) of these outcomes are affected and how does LTA influence them? 4.3 EVALUATION DESIGN The goal of an IE is to generate objective, scientifically valid evidence of the causal impact of an intervention. The central methodological consideration for an IE is its approach to establishing causality. The challenge in this regard arises because, for most interventions, the outcomes of interest are affected by a range of factors in addition to the intervention itself. For example, in the present context, one would expect beneficiaries of the LTA activity to experience increases in agricultural earnings as a result of their participation in the activity. To separate the impact of the intervention from the influence of other factors, IEs establish the causal impact of the intervention on an outcome for a beneficiary population by considering what would have happened to that beneficiary population over the same period of time in the absence of the intervention. To represent what would have happened, IEs use a control group to represent the counterfactual, i.e., the hypothetical outcomes for the beneficiaries in the absence of the activity. An important methodological consideration for IEs is the approach to selecting the control group. The LTA IE uses a clustered RCT design to assign treatment and construct the control group. Prior to activity implementation in the areas of focus for the IE, a set of villages was randomly assigned to either a treatment group that will receive the LTA intervention, or a control group that will not participate in the activity. Such randomized experimental designs are widely considered to be the most methodologically rigorous IE approach. They provide a more convincing demonstration of causality 7 The economic and environmental outcomes covered in Evaluation Question 5 are expected to unfold over a longer period, hence, the full impact of LTA on these outcomes may not be observable over the timeframe of the evaluation. Thus, the endline analysis will provide a preliminary indication of these impacts, while a more comprehensive assessment would require an additional round of data collection. The evaluation team and USAID will explore the possibility of further data collection pending the endline findings. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 7 than alternative designs that utilize non-random approaches to select a comparison group. An RCT minimizes the potential for selection bias — which occurs when underlying differences between treatment and comparison groups lead to differences in outcomes — by assigning the intervention in a systematically random way. The IE will measure LTA’s impacts on activity beneficiaries in 30 randomly selected villages8 in Iringa District. Implementation in these 30 villages will occur in two phases: an initial set of 15 randomly chosen villages beginning in 2017, then a second set of 15 randomly chosen villages beginning in mid￾2018. Ideally, all 30 villages would be selected at the outset with a single baseline collected prior to implementation. However, in response to concerns raised by DAI, selection of the villages was designed to take place in two stages prior to the beginning of the two phases of implementation. These original concerns stemmed from the fact that the context of the LTA activity may change over time as village administrative and geographic boundaries shift, an increasingly common occurrence as a village’s population grows. Village subdivision or boundary changes presented implementation challenges. This is because the LTA activity relies on specific satellite imagery and has limited resources to work through VLUPs, sensitization, and other activities without repeating processes for newly created villages. These would be required should a village subdivide. These challenges could also affect the evaluation team’s estimation strategy if changes occur in the local context, since any adjustments will require adding some kind of control or weights, and likely reduce analytical precision. Under the original design, a list of potential LTA activity villages developed in 2016 would not be appropriate later, as a village on the list may merge with another, or split into two villages. Criteria that once made a village suitable for the LTA activity in 2016 thus may no longer apply in later years. To address these potential challenges, the evaluation team proposed a phase-in RCT design in which implementation and evaluation activities would take place gradually and in tandem over the course of two years. The approach to village selection has been discussed in detail and agreed upon by DAI, USAID, the GOT, and evaluation team. As a first step in this process, the Iringa DLO prepared a master list of 75 villages suggested for potential LTA activity implementation according to its own priorities. From this list, the evaluation team randomly selected 37 candidate villages to allow for 15 Phase 1 treatment villages, 15 Phase 1 control villages, and up to seven villages to be eliminated for implementation reasons prior to randomized assignment.9 After identifying potential villages, it was necessary to assess the suitability of these villages for LTA implementation. Villages may not be appropriate for implementation for a variety of reasons, such as the presence of other certification outreach programs, inaccessibility, or impending village subdivision. To address these issues, the evaluation team, DAI, and the Iringa DLO collaborated on field reconnaissance in September 2016 to gather information to assess the suitability of each of the 37 candidate villages for implementation. From the remaining Phase 1 candidate villages, the evaluation team randomly assigned 15 villages to the Phase 1 treatment, and 15 to the Phase 1 control group. Two of the remaining villages were designated as “reserve” villages and candidates for implementation if implementation could not take place in the originally designated treatment villages. Phase II villages were originally slated for selection prior to spring 2018 using a similar process. However, after Phase I implementation, both DAI and USAID raised concerns about the activity’s ability to achieve implementation goals under a phase-in approach. In addition, LTA activity staff found that the data collected during field reconnaissance contained critical inaccuracies; for example, 8 The number of villages in the study is determined by the size of the activity. In 2016, LTA begin implementing in a preliminary set of non-randomly selected villages in Iringa, and is also implemented in a set of five test villages in Mbeya. These villages are not included in the IE and were not selected from the list of potential IE villages. The selected 30 villages were chosen randomly after accounting for key factors such as whether the village planned on subdividing, accessibility during the rainy season, and the presence of villagers capable of running the MAST application. 9 To improve balance, the initial 37 villages were selected by stratifying by constituency and blocking on whether the village had a VLUP, geographic location (constituency and ward), and the number of parcels in the village. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 8 villages that reported having a VLUP turned out not to have one or to have one that was expired. Table 4 shows the original phase-in schedule, while Table 5 shows the revised schedule. TABLE 4: ORIGINAL PHASE-IN DESIGN OF THE LTA IE Implementation Start Control Treatment April 2017 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA April 2018 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA TABLE 5: REVISED PHASE-IN DESIGN OF THE LTA IE Implementation Start Control Treatment April 2017 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA October 2017 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA Prior to Phase II, the Phase 1 treatment, control, and reserve villages, as well as any villages that were unsuitable for implementation, were removed from the original “master list” of 75 villages compiled by the DLO. The remaining villages were reviewed in coordination with the GOT and DAI to determine if any should be removed from consideration due to circumstances such as changing administrative boundaries or new land tenure programs. To the greatest extent possible, the evaluation team sought to adhere to the original list and remove villages only when necessary. For the remaining villages on the DLO list, the evaluation team and DAI decided not to repeat the field reconnaissance process to assess suitability for implementation. Rather, DAI and the DLO reviewed the list and assessed whether there were any obvious issues with including the remaining villages based on recent field work. As in Phase 1, the evaluation team then randomly assigned 15 to treatment, 15 to control, and up to five remaining villages as reserve. Annex D explains some of the challenges this posed to the original evaluation design. 4.3.1 RANDOM SELECTION The randomization procedure for the Phase II baseline was slightly different from the approach used to randomize treatment for Phase I villages. Prior to Phase I, the evaluation team conducted a field reconnaissance trip in coordination with DAI to collect data on each village that could potentially be assigned to treatment. Given that data collection for the Phase II baseline took place earlier than expected and that the new DAI chief of party decided there was sufficient information about the potential Phase II villages, no additional pre-selection data collection was done. This introduces some divergence in design fidelity across the two phases, but it is not considered a major limitation. In Phase I, randomization was based on data collected during field reconnaissance using a stratified random sampling approach. For Phase II, the evaluation team took a similar approach using data from the DLO to group villages into pairs based on the following strata: • Constituency • Ward • Population size • Number of CCROs already issued in the village prior to the LTA intervention • VLUP status Villages were paired based on their similarity on these five criteria prioritized in the order shown above (e.g., ward takes precedence over similar VLUP status). From here, villages were randomly PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 9 assigned within their paired grouping to either the treatment or comparison group. This approach helps improve the comparability of the villages across assignment groups. However, it is still important to test for statistical balance across the groups. This is because stratification only occurred across these five categories, some villages only had partial data (e.g., VLUP status was missing), and there are variables that may affect the outcomes of interest, but were not included in the DLO data, such as the presence of other interventions in the village. The IE is designed to include 30 villages in Phase II. The evaluation team randomly assigned 16 villages to treatment and 16 to control, with the inclusion of two “buffer villages” to allow for adaptation in the implementation approach should DAI face challenges. The evaluation team also collected data from randomly selected buffer villages during Phase I. Section 6.3 addresses differences between the Phase I and Phase II baselines. The evaluation team found no major differences between the two phases, despite the alternation to the assignment procedure. 4.4 BASELINE DATA COLLECTION The evaluation team conducted baseline data collection for Phase II villages, and the midline data collection for Phase I villages, in September and October 2017. Research Solutions Africa (RSA), a Kenyan survey research firm with an office in Dar es Salaam, conducted these data collection rounds.10 The RSA survey team included 35 enumerators, seven team leaders, and an overall survey supervisor working with a local coordinator from the evaluation team. For the Phase II baseline, the sampling frame consisted of households within the 32 Phase II villages across 19 wards in Iringa District. The evaluation team did not tell enumerators, field supervisors, or associated staff which villages were assigned to receive LTA interventions and which would serve as control villages. The target sample was 25 households per village, and one to two respondents per households, depending on availability. Figure 2 shows the number of survey respondents in each village. 10 RSA conducted the Phase I baseline. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 10 FIGURE 2: 1,320 PHASE-II SURVEY RESPONDENTS IN 32 VILLAGES IN IRINGA DISTRICT PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 11 4.4.1 HOUSEHOLD SAMPLE SELECTION Seven field teams, each consisting of four enumerators and a field supervisor, conducted the household surveys. When possible, enumerators worked in pairs, with one enumerator interviewing the male head of household and another the primary wife or spouse of household. When both male and female respondents were available, enumerators sought to interview female respondents outside the earshot of male respondents, such as inside the home, while the husband was interviewed outside of the home. However, in some cases, only one member of the household was home due to farming or market activities. In those cases, the team surveyed only one household member. The survey team used systematic random selection to find respondents. After arriving in a village, the team followed these steps: 1. Met with the village leader, usually the village chairman. With guidance from the village leader, the teams would split up, each taking a direction and starting a random walk from an appropriate point (e.g., from the nearest intersection in the village or at the village meeting place). 2. Each enumerator pair applied a skipping interval based on the percentage of target households for the village to the total village population, with a minimum skipping interval of 10. Once a team reached a target household, it would then walk to, at a minimum, the 10th household after the one it just visited. 3. Informed consent was required for all household interviews. If a respondent refused to be interviewed or decided that they did not want to continue midway through the interview, the enumerator would then move on to the next household based on the skipping interval. Prior to the start of data collection, the evaluation coordinator and local coordinator, along with RSA’s field supervisor and six enumerators, implemented a pretest for the baseline survey in Kihanga, Kidilo, Ulata, and Isaka villages in Iringa District. The goal of the pretest was to refine the relevance, sequencing, and wording of survey questions, as well as ensure that the mobile platform accommodated the correct skip patterns and logic checks in the survey. The pretest villages were purposively selected based on their omission from the evaluation field reconnaissance process in 2016 and random assignment ahead of data collection to avoid potentially pre-testing the survey on respondents who may receive the survey again at a later date, which could bias their responses. The evaluation coordinators and RSA field supervisor met with the head land officer at the Iringa DLO to explain the evaluation process, share results from Phase I, and maintain local support for the overall evaluation. DLO personnel were helpful in obtaining village leader contact information throughout the data collection process. In each pretest village, the survey team identified target households using a systematic random sampling approach, with the applicable skipping interval per village ranging from two to four. In each identified household, the team interviewed the male and female household heads, as appropriate, and simultaneously, if possible. The pretest team completed 54 interviews and went through 53 iterations of the survey instrument and daily updates to the mobile platform, Dooblo Survey to Go.11 RSA scripted English and Swahili versions of the questionnaire using the mobile platform to ensure translation accuracy and track changes to the software. Phase II baseline data collection activities took place from October 16 through November 17, 2017. In addition to the local coordinator and RSA field supervisor, each group of five enumerators was led by an enumerator team leader who was responsible for team oversight, communicating with village leaders, and conducting sit-in checks, call-backs, and back checks to ensure that enumerators were properly conducting the survey. The field supervisor managed enumerator assignments, held daily check-ins with enumerator team leaders, and undertook random data quality checks. The use of electronic data collection allowed RSA to submit raw data to the evaluation coordinator as an additional level of oversight. The evaluation coordinator checked variation in duration, assessed the 11 See www.dooblo.net. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 12 distribution of interview types by team and enumerator, and assessed missing and “don’t know” responses to ensure survey implementation fidelity. Each participant provided verbal informed consent after being read a statement about the purpose of the evaluation and the content of the survey. The survey team assured participants that their involvement was voluntary and could be withdrawn at any point. Enumerators further assured respondents that their answers would be kept confidential and all data would be anonymized prior to any publication or use. The survey team made follow-up visits to households in the following situations: • When there was no one in the household at the time of initial (first and second) visits. • When there were no adult household members/target respondents at the time of the visit. • When the target respondent(s) were busy at the time of the initial visit, and requested that enumerators come back at a later time. • When the enumerators were not able to complete either one or all of the household interviews during their previous visit, but it was still possible for them to return at a later time. During the Phase II baseline, and excluding the pre-test, there were 533 sit-ins (40 percent of surveys) by the survey firm field team leaders, field coordinator and project manager, as well as 220 back checks (17 percent of surveys) and 61 call backs (5 percent of surveys) to ensure data accuracy and quality.12 4.5 SURVEY INSTRUMENT Baseline data collection for Phase II consisted of two main household interview surveys: 1. The “Head of Household Survey” was given to the individual who identified as the head of household when enumerators presented themselves at the house for data collection. This survey lasted around 75 minutes. 2. The “Wives’ Survey” was given to the primary spouse/partner of the head of household. This survey lasted around 40 minutes. The survey team collected data via mobile devices. Both surveys included questions on disputes, self￾efficacy, loans, decision-making, and familiarity with land laws. The head of household survey also included a sketch map portion to use as a reference for follow-up interviews. The wives’ survey included a time-use component that asked respondents to describe their activities in the previous 24 hours. All surveys were geo-coded for additional quality assurance and to facilitate follow-up data collection rounds. Annex B provides the survey questionnaire the evaluation team developed, and Table 6 shows the questionnaire’s 13 modules. Most questions are based on validated questions from the Tanzanian National Panel Survey questionnaires. 12 Sit-ins include a field supervisor being present for the entirety of the interview; back checks consist of supervisors randomly re-interviewing respondents on select survey items to ensure accuracy; and call backs were conducted by contacting respondents via mobile phone to ask about select survey items and to ensure accuracy. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 13 TABLE 6: SURVEY QUESTIONNAIRE MODULES Modules Indicators I. Household Roster and Information • Age, schooling, marital status • Household size, number of adults and children • Economic activity II. Agricultural Organization, Services • Farmer cooperative involvement • NGO activity involvement III. Landholdings and Characteristics • Parcels owned and rented, parcel size, documentation status • Parcel acquisition method, inheritance, planning • Topography and physical characteristics of parcels • Irrigation, fallowing, and parcel improvements IV. Agricultural Production—Annual Crops • Parcels cultivated, crops grown by parcel, tools used • Seeds planted, amount paid for seeds • Use of inputs (e.g., fertilizer), cost of inputs, use of hired labor • Amount harvested, quantity sold, income from sales V. Agricultural Production—Perennial Crops • Parcels cultivated, crops grown by parcel • Use of intercropping • Trees planted, planned use for trees • Amount harvested, quantity sold, income from sales VI. Perception of Land Rights • Expropriation • Land tenure security • Knowledge of land laws, LTA, and CCROS VII. Land Disputes • Dispute incidence • Nature of disputes • Dispute resolution VIII. Non-Agricultural Income, Consumption, and Assets • Asset inventory • Livestock inventory • Household construction materials • Formal, non-farm employment IX. Household Savings, Borrowing, and Shocks • Borrowing amount and lender • Household shocks X. Food Security • Incidence of food insecurity in the past 12 months XI. Self-Efficacy • Ability to make decisions, confidence, problem solving XII. “Wives’ Survey” • Demographic information, education level • Expropriation in the event of husband’s death • Income activities, decision-making, disputes • Borrowing • Self-efficacy • Familiarity with land laws, LTA, and CCROs • Time allocation XIII. Sketch Map • Respondent-drawn map showing parcels, terrain, and crop allocation Figure 3 shows the final sample sizes that resulted from the sampling process. In all but two villages, the enumerators were able to visit the planned number of households. The remoteness of the study area villages and the fact that many household members were unavailable at certain parts of the day due to farming activities meant that enumerators often made follow-up visits to the selected households. The evaluation team set out to interview both male and female representatives in each household, but this was not always possible. Phase II data collection visited 807 households, 57 more than the 750 planned as part of the evaluation design. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 14 FIGURE 3: SURVEY RESPONDENTS BY ASSIGNMENT AND RESPONDENT TYPE 4.6 CHALLENGES ENCOUNTERED DURING DATA COLLECTION Baseline data collection occurred with limited interruption. RSA faced several scripting issues during the pretest period, and there were several scripting anomalies that took multiple days to resolve. There were a few technical issues related to uploading data given the remote location of a few of the villages; however, these issues posed more of a problem for conducting pretest quality assurance than for the full survey deployment. Each survey was geotagged to allow for additional data quality oversight and to help with respondent tracking for the next phase of data collection. During pretesting and baseline data collection, finding respondents who were home proved challenging in three of the villages where farming parcels are located far from where people actually live. 5 BASELINE FINDINGS This section presents baseline findings on key demographics, household characteristics, and outcome variables between the two assignment groups in Phase II. The findings provide a snapshot of the characteristics, conditions, and outcomes that the IE will measure in the study area. A brief discussion of the differences in summary statistics between Phase I and Phase II is presented following this section. It is worth noting that, in the Phase I baseline report, the evaluation team presented results only for the treatment group across respondent types. The team made the assumption that comparison group results should be similar, given adequate balance and appropriate randomization. This document reports summary baseline findings across assignment types, since analysis will proceed across these groups (and, secondarily, across respondent types within each of these groups). The Balance and Power section examines statistical differences between the treatment and control groups, and confirms sufficient study power within the context of inference and effect estimation for the evaluation. Both treatment and comparison groups are included in this Phase II baseline report to help readers compare the midline Phase I results and the results reported below. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 15 5.1 HOUSEHOLD CHARACTERISTICS Table 7 shows general characteristics of respondents by assignment type. There are a total of 615 respondents in the treatment group and 705 in the comparison group, with 371 and 417 heads of household respondents in each respective group (see Figure 3). In general, key characteristics between assignment groups overlap. The two assignment groups were very similar. Seventy-seven percent (n=472) of the treatment group and 75 percent (n=528) of the comparison group reported primary education, while 20 percent (n=143) of the comparison group and 18 percent (n=114) of the treatment group household heads (n=67) reported no schooling. TABLE 7: BASIC HOUSEHOLD CHARACTERISTICS BY ASSIGNMENT Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Age 615 44.49 14.42 19 97 705 46.12 15.45 18 102 Cooperative membership (y/n) 371 0.24 0.43 0 1 417 0.22 0.42 0 1 Education Level* 615 0.86 0.46 0 2 705 0.85 0.48 0 3 Miles to Iringa Town 570 30.98 15.23 7.92 59.21 657 27.59 12.8 3.18 59.21 Number of HH Members 371 4.48 2.49 1 26 417 4.37 2.16 1 17 *0 = No schooling, 1= Primary, 2 = Form, 3 = University Distance to markets is one of many factors that can affect the outcomes of interest. The evaluation team used the baseline data to calculate travel distance via road utilizing the Google Developer API. Using the API, the evaluation team wrote a script to cycle through each survey geo-stamp and determine the Google maps’ driving distance and drive time. It is important to note that officially mapped roads into many areas of Iringa are, at best, estimates. For this reason, some driving directions are unable to be calculated, resulting in slightly lower sample sizes for these statistics. Villages in both assignment groups are located, on average, around 30 miles to Iringa Town, the main economic hub of Iringa District and the home of the Iringa DLO. The evaluation team also investigated the age distribution of respondents. As Figure 4 shows, the reported ages were somewhat similar across assignment groups and respondent types. However, it is worth highlighting the variation across the assignment groups to better understand potential imbalance, since age may affect the ability of LTA activity participants to benefit. The median age for male household heads in the treatment group is 44, with 90 as the highest age reported; for female household heads, the median is higher at 56.5, with 84 as the highest age reported. Similarly, the comparison group’s male household head median age was 45, with the oldest respondent reporting an age of 100; for female household heads, the median age was 58, with the oldest respondent reporting her age as 102. Survey methodologists have long known that self-reported age can be a fraught metric.13 Despite well-known challenges in self-reported metrics, it is important to understand whether villagers in both treatment and comparison groups generally report the same distribution of ages or other self￾reported measures that may affect the outcomes. As shown above, this does appear to be the case, but with only minor variation between the two assignment groups. 13 S. Denic, F. Khatib, and H. Saadi, “Quality of Age Data in Patients from Developing Countries,” Journal of Public Health 26 (2004): 168–71. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 16 FIGURE 4: AGE BY ASSIGNMENT GROUP AND RESPONDENT TYPE PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 17 TABLE 8: LAND RIGHTS AND TENURE SECURITY VARIABLES Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Compared to one year ago, do you think the possibility that someone could try to take one of your parcels has increased? (1=Y, 0=N) 371 0.06 0.24 0 1 417 0.05 0.22 0 1 Do you have familiarity with land laws (1=Y, 0=N) 615 0.09 0.28 0 1 705 0.07 0.25 0 1 Expropriation possible in the next five years (1=Y, 0=N) 371 0.07 0.25 0 1 417 0.04 0.2 0 1 Heard of CCROs (1=Y, 0=N) 615 0.60 0.49 0 1 705 0.61 0.49 0 1 Is there a risk that someone will take over one of your plots if you leave it fallow? (1=Y, 0=N) 371 0.47 0.5 0 1 417 0.46 0.5 0 1 Is there community perception of expropriation risk (1=Y, 0=N) 371 0.11 0.31 0 1 417 0.10 0.29 0 1 Possess land￾related documentation (1=Y, 0=N) 615 0.11 0.31 0 1 705 0.14 0.34 0 1 Willingness to pay for a CCRO (in TZS) 370 29,751 60,285 0 500,000 428 25,028 33,010 0 270,000 5.2 TENURE SECURITY AND LAND DISPUTES Table 8 presents baseline data related to land rights and tenure security. An increase in the risk of expropriation over the past year was generally perceived to be low, at six and five percent of respondents in the treatment and comparison groups, respectively. However, almost 50 percent of respondents in both assignment groups expressed concerns regarding expropriation due to fallowing. Baseline results also indicate low familiarity with land laws (at nine and seven percent of respondents in treatment and comparison groups, respectively), while the percentage of respondents who had heard of CCROs was close to 60 percent for both assignment groups. Ninety-five respondents in the comparison group and 62 in the treatment group reported having some kind of land documentation. The baseline survey asked respondents to choose CCRO, Granted Right of Occupancy, Inheritance Letter, or Other Government Document, or to specify some other type of non-government documentation. In the treatment group, 41.3 percent (n=19) of respondents who reported having land documentation said they had a CCRO (this was 3.1 percent of the overall treatment group sample), while 37 percent (n = 17) said they had a Granted Right of Occupancy document (3.8 percent of the overall treatment sample). In the comparison group, 39.5 percent (n =32) of those who reported having some kind of documentation reported having a CCRO (4.5 percent of the overall comparison group sample), while 33.3 percent (n = 27) of those who reported documentation said they had “Other Government Documentation,” and 19.8 percent PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 18 (n = 16) said they had a Granted Right of Occupancy. One respondent in the treatment group and two respondents in the comparison group reported having a letter of inheritance. Respondents who reported having some kind of land documentation reported their willingness to pay (WTP) for CCROs at around 25,000 Tanzanian shillings, on average, in both assignment groups.14 In contrast, respondents in the treatment group who reported not having any land related documentation reported an average WTP for a CCRO of 30,426 (~13.69 USD), while comparison group respondents with no documentation reported an average WTP of 24,820 shillings (~11.17 USD) (Figure 5). The difference in WTP between those with and without documentation may reflect greater awareness of the actual market costs of CCROs by respondents who have already gone through the process of obtaining the documentation and, thus, have first-hand experience with its associated cost in shillings. As shown in Figure 5, documentation status relative to WTP has a right skewed distribution, but those without documentation have a longer tail. That means there are more respondents in the sample without documentation who report a WTP that is much higher than its typical actual costs, compared to those with documentation who report a smaller WTP range. FIGURE 5: WILLINGNESS TO PAY FOR CCROS BY ASSIGNMENT AND LAND DOCUMENTATION STATUS 14 One Tanzanian shilling is approximately 0.00045 US dollars, so 25,000 shillings is around $11.25. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 19 TABLE 9: DISPUTE-RELATED SUMMARY STATISTICS Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Border dispute risk in the next five years (1=Y, 0=N) 371 0.15 0.36 0 1 417 0.17 0.37 0 1 Expect disputes to improve, get worse, or not change in the upcoming 12 months* 371 0.37 0.69 -1 1 417 0.41 0.68 -1 1 Improvement, worsening, or no change in disputes over the past 12 months* 371 0.3 0.7 -1 1 417 0.34 0.69 -1 1 Improvement, worsening, or no change in risk of boundary dispute compared to a year ago* 371 -0.29 0.6 -1 1 417 -0.41 0.59 -1 1 Number of disputes in the past 12 months 28 1.21 0.79 1 5 24 1.17 0.38 1 2 Family disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.4 0.63 0 2 417 0.51 0.71 0 2 Grazing disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.66 0.82 0 2 417 0.69 0.83 0 2 Investor disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.09 0.32 0 2 417 0.22 0.54 0 2 Neighbor disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.43 0.62 0 2 417 0.55 0.68 0 2 Non-Family land disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.32 0.58 0 2 417 0.32 0.56 0 2 Rental disputes: severity (0=Not a problem, 1 = A small problem, 2 = A big problem) 371 0.18 0.42 0 2 417 0.28 0.55 0 2 *0 = no change, 1 = improve, -1 = get worse PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 20 Disputes were reported by about seven percent of the total household head sample. As shown in Table 9, there was an average of a little over one dispute for respondents in both assignment groups. In the treatment group, almost half the sample (n = 182) reported that they expect the incidence of disputes to improve over the next 12 months; the comparison group reported similar findings (n = 215). Overall, disputes were generally reported as inconsequential. There are a few areas where the data show that risk of dispute or perceived dispute severity is problematic for one or both assignment groups. Grazing disputes were perceived to be most problematic across both assignment groups, followed by neighbor disputes. There were only four villages in the sample in which respondents did not report any disputes. Of the villages that reported disputes, 15 reported more than one dispute, and, as Table 10 shows, seven reported three or more disputes. About 55 percent (n = 34) of total disputes were reported in treatment villages. Most respondents reported only one dispute, if any. In Kisanga, for example, four respondents reported disputes that occurred in the previous 12 months. Of those, one reported five disputes, while the other three each reported one. TABLE 10: VILLAGES WITH THREE OR MORE DISPUTES As seen in Figure 6, there is a slight clustering of disputes within villages that reported more than one dispute. In most cases, this clustering can be found where two respondents each reported one dispute within close proximity (i.e., the dispute is with a nearby neighbor). Additional analysis also suggests that there is little clustering among respondents who reported multiple disputes; multi￾dispute respondents did not appear to be clustered with respondents who reported only one dispute. When asked about the severity of specific dispute types, around 50 percent of all respondents in both the treatment and comparison groups felt there was no problem with disputes of various types. There are a few notable exceptions, however, as seen in Figure 7. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 21 FIGURE 6: MAP OF INDIVIDUAL DISPUTES BY ASSIGNMENT GROUP PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 22 FIGURE 7: PERCEIVED DISPUTE SEVERITY BY ASSIGNMENT GROUP In both the treatment and comparison groups, grazing, neighbor, and family disputes were viewed as more problematic, while investor disputes were least frequently seen as a problem. In the treatment group, 22.4 percent of respondents (n=83) viewed grazing disputes as a big problem, while 23.3 percent (n=97) of the comparison group said the same. For neighbor-based disputes, 29 percent of the treatment group said these were a small problem (n=108), while seven percent (n=26) said they were a big problem. There were similar findings in the comparison group. Thirty-four percent (n=143) and 10.6 percent (n=44) reported neighbor disputes as small and big problems, respectively. About 25 percent of both the treatment and comparison groups reported family disputes as a small problem; however, while only 7.5 percent (n=28) of the treatment group reported family disputes as a big problem, 12.7 percent (n=53) of the comparison group reported the same. With respect to grazing disputes, respondents who reported a greater number of disputes also tended to report that such types of disputes were more problematic (greater severity) (Figure 8). Notably, this is not the case for family and neighbor disputes, the other two disputes types for which respondents reported problems. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 23 FIGURE 8: GRAZING DISPUTE SEVERITY AMONG RESPONENTS WHO REPORTED EXPERIENCING DISPUTES 5.3 LANDHOLDINGS, INVESTMENT, AND ENVIRONMENT Respondents were asked about the number of parcels owned or rented, as well as the parcel size. All size values were converted to hectares, but self-reported quantities, whether parcel size or number of trees are somewhat noisy estimates.15 Households in both the treatment and comparison assignment groups reported owning or renting about the same number of parcels, with a median of two parcels for both assignment groups. As shown in Figure 9, there are a few outliers that push the mean number of parcels higher for the treatment group. Only 59 and 49 respondents reported more than three parcels in the treatment and comparison groups, respectively. In both the treatment and comparison groups, female headed households reported slightly fewer parcels owned: 2.0 parcels for female headed households to 2.6 parcels for male headed households in the treatment group, and 2.0 parcels for female headed households to 2.4 for male headed households in the comparison group. Similarly, average parcel size differed by gender of household head. In the treatment group, male headed households reported an average of 4.3 hectares to 3.9 hectares for female headed households. In the comparison group, male headed households reported an average of 3.7 hectares to 2.8 hectares for female headed households. 15 There are many definitions for noise in a data set. In this instance, we refer mainly to outliers and misrepresentations of self-reported characteristics, whether deliberate or not, which result in a large range of responses that likely differ from the true value. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 24 FIGURE 9: NUMBER OF PARCELS BY ASSIGNMENT GROUP The evaluation team examined five main investment categories. As Table 11 shows, household investments in fencing their parcels were low. This is not surprising given the context of rural Iringa, where fencing is rarely used by smallholder farmers. Low levels of building, soil conservation, and terracing are expected given the capital intensiveness of these activities. Less than 20 percent of respondents in both assignment groups reported tree-planting activity. The treatment group reported slightly higher average planting activity of non-fruit trees. Male and female headed households in the treatment group reported somewhat similar rates of non-fruit tree planting at 20.6 percent and 15.6 percent, respectively. In the comparison group, 14.7 percent of male headed households reported planting non-fruit trees, while only 5.7 percent of female headed households reported the same. It is important to note here that the self-reported responses on the number of non-fruit trees planted are highly noisy. Tree planting is a metric that the evaluation team will continue to investigate once follow-up data are collected during endline. As mentioned previously, a discussion of balance occurs later in this report. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 25 TABLE 11: SUMMARY STATISTICS OF LAND ENVIRONMENT, USE,AND INVESTMENT Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Parcel size (in hectares for all parcels owned) 371 4.22 8.08 0.1 74.66 417 3.42 5.32 0.1 49.37 Number of parcels owned or rented 371 2.45 1.37 1 11 417 2.31 1.18 1 8 Non-fruit trees planted (1=Y, 0=N) 371 0.19 0.40 0 1 417 0.12 0.33 0 1 Fruit trees planted (1=Y, 0=N) 371 0.16 0.37 0 1 417 0.11 0.32 0 1 Household invested in: (1=Y, 0=N): Buildings 371 0.23 0.42 0 1 417 0.17 0.38 0 1 Fencing 371 0.05 0.22 0 1 417 0.06 0.24 0 1 Soil conservation 371 0.26 0.44 0 1 417 0.21 0.41 0 1 Terracing 371 0.20 0.4 0 1 417 0.17 0.38 0 1 Wells 371 0.05 0.21 0 1 417 0.06 0.23 0 1 5.3.1 FOOD INSECURITY The baseline survey examined food insecurity, and tried to capture respondents’ experience of food insecurity in the previous 12 months through multiple questions designed to better understand anxieties and perceptions around this issue. As Table 12 shows, under one third of the sample in both assignment groups has faced food insecurity. Within the treatment group, 39 percent of female headed households (n = 35) and 23 percent of male headed households (n = 64) reported facing food insecurity over the previous 12 months. For the same time period in the comparison group, 33 percent of female headed households (n = 35) and 17 percent of male headed households (n = 54) reported food insecurity in the previous year. The number of days respondents faced food insecurity varied widely, as shown in Table 12. The 95 percent confidence intervals for the treatment and comparison groups are somewhat similar, however, at 41.9-69.4 and 44.6-77.0 days, respectively. TABLE 12: SUMMARY STATISTICS OF FOOD INSECURITY Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Have you been faced with a situation when you did not have enough food to feed the household in the past 12 months? (1=Y, 0=N) 371 0.27 0.44 0 1 417 0.21 0.41 0 1 For how long did you face this situation? (in days) 99 55.62 68.95 1 365 89 58.81 67.51 1 365 While the responses on general food insecurity are similar for treatment and comparison groups, there are some differences regarding the nature of this insecurity. When asked if lack of food in their house was caused by lack of resources for obtaining that food, 3.5 percent (n = 13) of treatment households and about 1.7 percent (n = 7) of comparison households said this was often (more than 10 times) the case. As to whether any household members went to sleep hungry, almost 11 percent of treatment respondents, but only five percent (n = 21) of comparison households, said this happened sometimes or often (n = 40). As shown in Figure 10, less than 50 percent (n = 180) of the treatment group said they have never been unable to eat their preferred foods due to lack of resources. This is the highest food insecure measure captured in the baseline data. In contrast, 58 percent (n = 243) of the comparison group said the same. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 26 The differences between households on these food insecurity measures are important to consider because they may affect estimates of the LTA activity’s outcomes. The treatment group in Phase II appears to be slightly more food insecure than the comparison group. Although this difference does not have a strong enough magnitude to create imbalance between the assignment groups, the evaluation team may investigate these differences further during qualitative data collection. 5.3.2 SOCIAL AND EMPOWERMENT OUTCOMES The LTA activity provides tenure certification as well as education on Tanzania’s land laws and land management. Given the LTA activity’s theory of change, we would expect changes in beneficiary land status to affect self-perception, social capital within local networks, and standing within the village community. These factors may all affect whether and how people invest or make use of LTA’s inputs, from the tangible (e.g., CCROs) to the intangible (e.g., widespread understanding of women’s right to land). To gain some insight into how sampled households view themselves and their level of self-confidence, the evaluation team employed the Generalized Self-Efficacy Scale (GSES).16 The GSES has been employed in 25 countries and multiple contexts to assess how respondents view their “capability to deal with certain life stressors.” All respondent types (i.e., male heads of household, female heads of household, and wives) were asked the 10 self-efficacy questions from the GSES (see Figure 11). The evaluation team will use follow-up surveys, as well as qualitative data collection at endline, to determine whether and how self-efficacy changes as land tenure is formalized and households go through the mapping and certification process. The evaluation team converted the self-efficacy responses to numeric values, with 4 equal to “Exactly true” (i.e. high self-efficacy), and 1 equal to “Not at all true,” or a score suggesting lower self-efficacy. The treatment group had an average score of 3.05, and the comparison group an average score of 3.12 across all GSES items. These scores suggest generally high self-efficacy. The evaluation team further investigated how these numeric scores varied by respondent type across the assignment groups. Female household heads in the comparison group reported the lowest self-efficacy, with an average score of 2.78 (n = 105), while the same respondent type in the treatment group had an average score of 2.80 (n = 90). The highest score belonged to the male household heads in each assignment group, with treatment respondents averaging 3.21 (n = 281) and comparison respondents averaging 3.27 (n = 312). Baseline data collection also included questions related to decision-making power. The survey asked heads of household about decision making for each of the reported parcels. Because so few respondents report more than four parcels, this report presents the frequencies for decision making reported by treatment and comparison group household heads for the first four parcels (Tables 13 and 14). Household heads in the sample reported making most of the decisions on parcel use. The only exception was three respondents who noted decision making on their eighth parcel. For the treatment group, decision making was split between “self” and “both self and spouse together” (n = 2), while the sole respondent in the comparison group reported joint decision making. The respondent in the treatment group who reported 11 parcels noted that, for parcels 9 and 10, joint decision making occurred as well. 16 For more, see Aleksandra Luszczynska, Urte Scholz, and Ralf Schwarzer, “The General Self-Efficacy Scale: Multicultural Validation Studies,” The Journal of Psychology 139, no. 5 (2005): 439-457. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 27 FIGURE 10: FOOD INSECURITY ACROSS TREATMENT AND COMPARISON GROUPS PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 28 FIGURE 11: SELF-EFFICACY BY TREATMENT GROUP PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 29 TABLE 13: HOUSEHOLD HEAD DECISION MAKING FOR PARCELS 1 AND 2 TABLE 14: HOUSEHOLD HEAD DECISION MAKING FOR PARCELS 3 AND 4 Decision making on use of parcel 1 Decision making on use of parcel 2 Response Treatment Comparison Response Treatment Comparison n % n % n % n % Self 277 74.66 299 71.7 Self 195 67.24 214 68.15 Spouse 9 2.43 5 1.2 Spouse 8 2.76 6 1.91 Both self and spouse together 78 21.02 104 24.94 Both self and spouse together 80 27.59 88 28.03 Other male household member 3 0.81 3 0.72 Other male household member 2 0.69 1 0.32 Other female household member 2 0.54 3 0.72 Other female household member 2 0.69 2 0.64 Other, specify 2 0.54 3 0.72 Other, specify 3 1.03 3 0.96 Decision making on use of parcel 3 Decision making on use of parcel 4 Response Treatment Comparison Response Treatment Comparison n % n % n % n % Self 94 69.12 98 67.12 Self 38 64.41 31 63.27 Spouse 3 2.21 5 3.42 Spouse 1 1.69 0 0 Both self and spouse together 36 26.47 39 26.71 Both self and spouse together 19 32.2 16 32.65 Other male household member 1 0.74 0 0 Other male household member 1 1.69 1 2.04 Other female household member 1 0.74 2 1.37 Other female household member 0 0 0 0 Other, specify 1 0.74 2 1.37 Other, specify 0 0 1 2.04 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 30 5.3.2.1 WIVES’ SURVEY AND DECISION MAKING The evaluation team surveyed primary female spouses on many of the topics included in the household head survey, as well as on knowledge of and attendance at village meetings. More than half of the respondents in both the treatment and comparison group wives’ samples were aware of women’s groups in their village or nearby. The number of meetings respondents said they attended was similar across assignment groups, but slightly more respondents in the comparison group (65 percent, n = 186) said they were comfortable speaking in meetings or group settings. Table 15 shows select summary statistics of the wives’ survey that directly capture LTA activity inputs, such as the establishment of women’s groups and introducing beneficiaries to the details of Tanzania’s land laws. TABLE 15: SUMMARY STATISTICS OF WIVES’ MEETINGS AND LAND LAWS Primary spouses in both assignment groups reported similar levels of familiarity with land laws. Of the 70 treatment and 71 comparison spouse respondents who said they were familiar with the land laws, only 10 percent (n = 7) and 18 percent (n = 13), respectively, also reported having some kind of documentation for their parcels. This suggests that there is at least some perceived basic understanding among spousal respondents regarding the land laws. However, as shown in Table 15, this group is still in the minority of the overall sample. The wives’ survey for the Phase II baseline also asked about decision making generally within the household. In contrast to head of household respondents, primary spouses reported joint decision making more frequently. As Table 16 shows, only about a quarter of spouses reported being the lead decision maker on income use. Slightly more respondents, about 28 percent (n = 68) and 31 percent (n = 88) in the treatment and comparison groups, respectively, noted being the lead decision maker on parcel use across all parcels. For each question related to decision making in the wives’ survey, joint decision making was reported by both assignment groups as the most frequent approach, with the exception of wage-labor decisions. Only four respondents in both groups confirmed making wage-labor decisions. All treatment group respondents (100 percent) reported joint decision making related to wage-labor, while three (i.e., 75 percent) of the comparison respondents noted making those decisions themselves. Only one respondent reported joint decision making on the subject. Variable Treatment Comparison n Mean SD Min Max n Mean SD Min Max Are there women's groups in the village or surrounding area? (1=Y, 0=N) 244 0.66 0.48 0 1 288 0.62 0.48 0 1 Number of group meetings attended in the past six months 244 2.77 2.74 0 24 288 2.75 2.89 0 24 Do you feel comfortable speaking in meetings? (1=Y, 0=N) 244 0.58 0.49 0 1 288 0.65 0.48 0 1 Did you or anyone else in the household borrow money in the past year? (1=Y, 0=N) 244 0.19 0.39 0 1 288 0.24 0.43 0 1 Familiarity with land laws 244 0.29 0.45 0 1 288 0.25 0.43 0 1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 31 TABLE 16: FREQUENCIES OFWIVES’ DECISION MAKING 5.4 BALANCE AND POWER In addition to providing the descriptive statistics presented in this document, the baseline data can also be used to test some of the statistical assumptions related to the evaluation methodology. This section investigates two such assumptions. First, balance tests are used to assess and confirm the comparability of the treatment and control groups. Second, the power calculations presented in the evaluation design proposal are revisited using updated parameters from the baseline data to assess statistical power, given the actual sample size and other sample parameters. 5.4.1 TESTING FOR BALANCE ACROSS TREATMENT AND COMPARISON GROUPS Baseline data offer a snapshot of the pre-intervention context, and can be used to both test assumptions of the evaluation design, and ensure that randomization occurred as intended. An important consideration is to assess the balance between the treatment and control groups at baseline. If substantial differences in their characteristics exist, the control group may not be a valid representation of the counterfactual. While randomization in both assignment and survey respondent selection should theoretically increase the probability of balance between the treatment and comparison groups, it is important to test this assumption once data are collected to confirm the fidelity of the randomization procedure. Researchers often use t-tests or regressions using treatment indicator variables to assess balance. However, no conceptual justification exists for using the statistical significance of such tests as a criterion for assessing balance. 17 As in Phase I, the evaluation team used a normalized differences approach to assess balance between assignment groups. This method calculates a statistic based on the difference between the treatment and control group means, divided by the square root of one￾half the sum of the treatment and control group variances. An absolute value greater than one for this statistic raises concerns, while an absolute value of 0.25 or less indicates particularly strong 17 Douglas Altman, “Comparability of Randomised Groups,” Journal of the Royal Statistical Society: Series D (The Statistician) 34, 1 (1985): 125-136; K. Imai, G. King, and E.A. Stuart, “Misunderstandings among Experimentalists and Observationalists in Causal Inference.” Journal of the Royal Statistical Society Series A 171, 2 (2008): 481–502; P. Austin, “Using the Standardized Difference to Compare the Prevalence of a Binary Variable between Two Groups in Observational Research,” Communications in Statistics: Simulation and Computation 38, 6 (2009): 1228-1234. Decision-making on general parcel use Response Treatment Comparison n % n % Self 68 27.87 88 30.56 Spouse 47 19.26 40 13.89 Both self and spouse together 115 47.13 151 52.43 Other male household member 3 1.23 0 0 Other female household member 1 0.41 0 0 Other, specify 10 4.1 9 3.12 Decision-making on income use Response Treatment Comparison n % n % Self 63 25.82 72 25.00 Spouse 29 11.89 36 12.5 Both self and spouse together 138 56.56 169 58.68 Other male household member 0 0 0 0 Other female household member 0 0 0 0 Other, specify 14 5.74 11 3.82 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 32 balance.18 Normalized differences also help assess whether any potential imbalance can be addressed in the analysis phase. Table 17 shows the results of the normalized differences for 23 variables across six thematic areas. The evaluation team chose these variables to reflect a broad range of the outcome categories and covariates that the IE analysis will use; these include household demographic characteristics, several measures of perceived tenure security, outcomes related to land disputes, women’s empowerment, household wealth and economic outcomes, and several types of land related investment. In no cases are large differences between the treatment and control group sample means observed. As the last column illustrates, the normalized difference statistic falls below 0.25 for all of the variables, meeting the Imbens and Rubin standard for good balance. The evaluation team concludes, with a high level of confidence, that the treatment and control groups are well balanced, as would be expected given the randomized assignment between the two groups. As mentioned earlier, the statistic on driving miles to Iringa Town should be taken as a general metric of distance to the economic center, rather than a respondent’s actual access to markets, since road coverage estimation is inexact in much of rural Iringa. Given the higher standardized difference for this metric, 0.24, the evaluation team will determine if weighting or other specification parameters should be included during analysis. 5.4.2 REVISITING POWER ASSUMPTIONS WITH PHASE II BASELINE DATA The baseline data allow the evaluation team to revisit the power calculations presented in the evaluation design proposal to improve their accuracy and to reassess the expected statistical precision of the IE. In many IEs, power calculations are used to determine the minimum sample size required for the desired level of statistical power. In the case of the LTA IE, however, the sample size is constrained by the fact that LTA implementation is limited to 30 villages. In the case of this IE, the number of villages (i.e., clusters) is limited to 30 as this is the scope of the intervention. Thus, the focus of the power calculations is to determine the anticipated minimum detectable effect size for the different outcomes that will be possible rather than the required sample to achieve a given power. To revisit the power calculation assumptions, the evaluation team used the underlying values of 80 percent power and a 0.05 statistical significance level. Power is the probability of detecting an effect where one actually exists (i.e., a true positive); a value of 80 percent or higher is generally considered sufficient. There are several key metrics for determining power for the village-level cluster RCT design used in this evaluation. 18 See Guido Imbens and Donald Rubin, Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction (Cambridge: Cambridge University Press, 2015). PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 33 TABLE 17: NORMALIZED DIFFERENCE BALANCE TESTS FOR SELECTED VARIABLES Variable Treatment Control Normalized N Mean SD N Mean SD diff. stat. Demographics Female headed households, % 615 0.15 0.35 705 0.15 0.36 -0.01 Household head age 371 48.03 14.39 417 49.86 15.9 -0.12 Farmer cooperative membership 371 0.24 0.43 417 0.22 0.42 0.05 Annual household farm income (self-reported) 371 644406 4127285 417 555773 2195375 0.03 Education level for household heads and primary wives (0 = None, 1 = Primary, 2= Form, 3 = University) 615 0.86 0.46 705 0.85 0.48 0.03 Driving distance in miles to Iringa Town 570 30.98 15.23 657 27.59 12.8 0.24 Household size 371 4.48 2.49 417 4.37 2.16 0.05 Perceived tenure security Expropriation in next five yrs. is possible, % 371 0.07 0.25 417 0.04 0.2 0.11 Most/all in village worried about losing land, % 371 0.11 0.31 417 0.1 0.29 0.03 Has documentation for at least one parcel, % 615 0.11 0.31 705 0.14 0.34 -0.10 Land disputes Experienced land dispute in past year, % 371 0.08 0.26 417 0.06 0.23 0.07 Believe land disputes increased in past year, % 371 0.14 0.35 417 0.12 0.33 0.05 Believe land disputes will increase in next year, % 371 0.12 0.33 417 0.11 0.31 0.04 Assets and economic outcomes Size of total landholdings, acres 371 10.42 19.97 417 8.45 13.15 0.12 HH did not have enough to eat in past yr., % 371 0.27 0.44 417 0.21 0.41 0.13 Land-related Investment: % of HHs making each land-related investment on at least one parcel Wells/irrigation, % 371 0.05 0.21 417 0.06 0.23 -0.04 Erecting buildings, % 371 0.23 0.42 417 0.17 0.38 0.15 Erecting fencing, % 371 0.05 0.22 417 0.06 0.24 -0.05 Terracing, % 371 0.2 0.4 417 0.17 0.38 0.07 Soil conservation, % 371 0.26 0.44 417 0.21 0.41 0.12 Women’s empowerment (wives’ survey) Land use decisions made by male head of HH only, % 244 0.19 0.4 288 0.14 0.35 0.14 Attended village meetings in past yr., % 244 0.83 0.38 288 0.8 0.4 0.07 Comfortable speaking in village meetings, % 244 0.58 0.49 288 0.65 0.48 -0.14 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 34 An important parameter in the power calculations is the village intra-cluster correlation coefficient (ICC), which measures the extent to which observed variation in a variable is due to village-level differences, rather than individual differences. In the absence of similar datasets to draw on, power calculations must make assumptions about the ICCs and sample sizes. The design proposal for this IE thus presented statistical power for a range of assumptions about the ICCs. Now that the evaluation team has village-level observations about units (i.e., households) within each cluster and an actual sample size (n), the actual ICC for each outcome can be calculated and used, together with other updated sample parameters, to update the understanding of the evaluation’s power to detect policy￾relevant magnitudes of change for the outcomes of interest, given the sample parameters. The Minimum Detectable Effect Size (MDES) is another important parameter to inform overall study power. The MDES is an estimate of the smallest change in the outcome of interest that is detectable based on other sample parameters, such as the desired statistical significance (alpha) level, the ICC, the sample mean and variance, and the size of clusters. Table 18 shows the updated power calculations for the IE. The design proposal for this IE concluded that the analysis was likely to be sufficiently powered for most outcomes, but that outcomes for which the ICC was greater than 0.10 and/or for which impacts were particularly small (MDES less than 0.2), the IE would be statistically underpowered. Being underpowered means the IE would run a substantial risk of finding no impact even if LTA did, in fact, have some impact on these outcomes. It is important to note that the values shown in Table 18 should be taken as general suggestions, rather than clear thresholds for whether an impact can be satisfactorily measured. As Andrew Gelman and others have noted, power is gameable and assumption laden calculations, but “can be useful in giving a sense of the size of effects that one could reasonably expect to demonstrate with a study of given size.”19 The values in Table 18 suggest that the conclusions of the RCT design hold, while keeping the general caveats of power calculations in mind. True effects for certain outcomes may less likely be detectable, particularly if the magnitude of the activity effect is small. Given widespread misunderstanding regarding statistical power, it is worth stating that there are other reasons that may contribute to findings of no change at endline. For example, even if an outcome is indicated to be sufficiently powered, based on the calculations in Table 18, changes on the ground may not take place within the span of the study, or may be difficult to attribute to the activity (LTA, in this case) due to the magnitude of actual change being smaller than the MDES. 19 See Andrew Gelman and Jennifer Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models (Cambridge: Cambridge University Press, 2006), 442. Also, Daniel J. O'Keefe, "Brief Report: Post Hoc Power, Observed Power, A Priori Power, Retrospective Power, Prospective Power, Achieved Power: Sorting Out Appropriate Uses of Statistical Power Analyses." Communication Methods and Measures 1, no. 4 (2007): 291-299. Esther Duflo, Rachel Glennerster, and Michael Kremer. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics 4 (2007): 3895-3962. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 35 TABLE 18: POWER CALCULATIONS FOR SELECTED OUTCOME VARIABLES, ASSUMING ALPHA = 0.05 AND POWER= 0.80 Variable N Treatment Mean/SD Comparison Mean/SD ICC MDES Lower-Upper 95% CI Perceived tenure security Expropriation in next five years is possible, % 788 0.07 (0.25) 0.04 (0.2) 0.01 0.11 0.03 0.18 Most/all in village worried about losing land, % 788 0.11 (0.31) 0.10 (0.29) 0.05 0.24 0.07 0.40 Land disputes Experienced land dispute in past year, % 788 0.08 (0.26) 0.06 (0.23) 0.00 0.07 0.02 0.11 Believe land disputes increased in past five years, % 788 0.14 (0.35) 0.12 (0.33) 0.06 0.25 0.07 0.43 Believe land disputes will increase in next five yrs., % 788 0.12 (0.33) 0.11 (0.31) 0.06 0.25 0.08 0.43 Economic outcomes HH did not have enough to eat in past yr., % 788 0.27 (0.44) 0.21 (0.41) 0.07 0.28 0.08 0.48 Land-related Investment: % of HHs making each land related investment on at least one parcel Wells/irrigation, (%) 788 0.05 (0.21) 0.06 (0.23) 0.00 0.03 0.01 0.10 Erecting buildings, (%) 788 0.23 (0.42) 0.17 (0.38) 0.04 0.21 0.06 0.36 Erecting fencing, (%) 788 0.05 (0.22) 0.06 (0.24) 0.05 0.24 0.07 0.41 Terracing, % 788 0.20 (0.40) 0.17 (0.38) 0.07 0.27 0.08 0.46 Soil conservation, % 788 0.26 (0.44) 0.21 (0.41) 0.12 0.36 0.11 0.61 Women’s empowerment (wives’ survey) Land use decisions made by male head of HH only, % 532 019 (0.4) 0.14 (0.35) 0.02 0.15 0.04 0.25 Attended village meetings in past yr., % 532 0.83 (0.38) 0.80 (0.40) 0.04 0.21 0.06 0.36 Comfortable speaking in village meetings, % 532 0.50 (0.49) 0.65 (0.48) 0.03 0.19 0.06 0.33 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 36 5.5 CONCLUSION AND DISCUSSION This baseline report presented background information about the LTA activity and the IE design, summarized the Phase II baseline data collection process, investigated some of the methodological assumptions in the evaluation design proposal, and presented descriptive statistics from the Phase II baseline data. In general, the evaluation team concludes that the Phase II dataset is of sufficient quality for estimation of activity impacts. We did not encounter any major challenges with the data once the final dataset was obtained. Below are a few concluding observations regarding the baseline data: • Household characteristics: There is a large primary spouse presence in the dataset, especially compared to Phase I. As shown in Figure 3, around 40 percent of both assignment groups is comprised of primary spouses. This is due to lessons learned from Phase I and the timing of data collection (Phase I data collection took place during the height of the rainy season, while Phase II occurred during the dry season). • Tenure security and land disputes: Baseline data show that awareness of CCROs was somewhat high in both assignment groups (around 60 percent of both treatment and comparison groups had heard of them). However, only few respondents actually had some kind of land documentation. Similarly, less than 10 percent of either group reported familiarity with the land laws, which suggests that the educational portion of the LTA activity may be able to make strong gains and measurable impact in this area. Almost half the respondents in both assignment groups reported a risk of land grabbing, should they leave one or more of their parcels fallow. • Landholdings, use, and investment: Most households reported owning more than one parcel, although there was significant variability in the self-reported parcel size and female households in both the treatment and comparison groups reported fewer parcels on average. Investments in parcels were low, but soil conservation and building investments were the most common. Almost 20 percent of male and female heads of household in the treatment group reported planting non-fruit trees in the past year. • Social and empowerment outcomes: Around 27 percent of the treatment group and 21 percent of the comparison group reported facing food insecurity. Household heads generally reported that they were responsible for a majority of parcel use decisions, while primary spouses reported that parcel use decisions were jointly made, on average. This is perhaps due to various cognitive or social-emotional biases, and will need to be examined more thoroughly at endline and through qualitative work. • Balance and statistical power: No major differences were observed between the treatment and comparison groups on key variables. The data suggest that the evaluation is sufficiently powered to detect policy-relevant effect sizes for the main outcomes of interest, but there are limits to this given the time it make take for some of the impact of the LTA activity to occur. As mentioned earlier, the difference in data collection timing is important when attempting to compare the two phases of baseline data collection. Annex D contains a memo that was shared with USAID noting the challenges that this may create for future analysis. With this difference in mind, it is also desirable to examine some of the more notable differences between Phase I and Phase II. As shown in Annex E, the sample size was slightly higher for Phase II. Phase II interviewed 1,320 respondents across 807 households, while Phase I sampled a total of 1,179 respondents in 763 households. The difference is partly a result of preferential weather and travel conditions for the survey team during Phase II. However, the total number of household heads was similar across both phases, with 782 sampled in Phase I and 788 in Phase II. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 37 The graph in Figure 12 shows the change in sample size between treatment and comparison groups between each phase of the evaluation. The number of female household heads in the sample fell between Phases 1 and 2, and the number of wives interviewed increased between phases. This may be a function of the seasonal difference in timing across the two survey rounds, as more households had multiple respondents (i.e., male head and female spouse) available during the dry season when farm activities are less time intensive. FIGURE 12: PHASE 1 AND PHASE 2 SAMPLE BY RESPONDENT AND ASSIGNMENT 5.5.1 NEXT STEPS Next steps for the LTA activity IE consist of three main activities: endline planning and data collection; review and adjustments to analytical approach; and final analyses, reporting, and dissemination. In the next phase of the evaluation, the MSI/NORC team will develop the endline survey instrument based on the baseline and midline instruments used in Phase I and Phase II. The goal of the endline analyses will be to estimate the impact of the LTA activity on the outcomes of interest. The endline data collection phase aims to re-survey all Phase I and II respondents, with minimal attrition. The evaluation team will also review the data from the first two phases and assess where qualitative data collection can effectively help to fill gaps in understanding and better interpret the quantitative estimates of activity impacts. Focus group discussions and key informant interviews with stakeholders from DAI, DLO, and village leadership may help inform reasons for impacts (or lack thereof), and how and why impacts may have varied for different types of beneficiaries. Qualitative data will help the evaluation team better understand the mechanisms through which activity impacts PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 38 may or may not have occurred, as well as provide a richer understanding of how tenure certification has affected perceptions among farmers in Iringa. The evaluation team will continue exploring Phase I and II data to determine if any additional adjustments to the analytical approach are warranted. This may include applying a weighting scheme or other methods to account for variance across villages in the number of surveyed respondents and seasonal factors between Phase I and Phase II. The endline analysis will directly address and attempt to provide answers to each of the evaluation questions. While the baseline data can provide a point-in-time overview of the sample, endline analysis will bring together all of the data collection phases to actually address each evaluation question in detail. Finally, the evaluation team anticipates sharing the findings from the midline and endline reports to improve the evidence base and understanding of land tenure programming impacts within the development community. As part of this effort, the evaluation team will present the midline results at the World Bank’s 2018 Land and Poverty conference in Washington D.C. To enhance the Agency’s investment in IEs, the evaluation team will also work with USAID to identify additional opportunities to disseminate findings from this report and the prospective endline findings and conclusions. 6 PHASE I MIDLINE This section of the report presents initial results from the Phase I midline. The midline survey used an amended baseline survey instrument to re-survey respondents from the Phase I sample. The sampling approach within villages was purposive, that is, enumerators used the geo-stamps and phone numbers of Phase I baseline respondents to find and interview respondents. The midline portion of this report provides a brief overview of the current status of implementation, as well as more details about the current implementation process. Select summary statistics are presented for the Phase I baseline and midline to show raw comparisons between the survey rounds. Preliminary inferential analysis is then presented, followed by a general discussion of the data and results. As with any midline, the results in this section, whether descriptive or inferential, should be seen as preliminary. In some cases, a positive or negative change may simply be a result of temporary factors not accounted for in the data and, thus, may not be sustained through endline and beyond. For this evaluation, it is important to note that, due to a shift in the implementer’s timeline, the midline for Phase I village took place five months earlier than planned, and approximately six months after the start of implementation in those villages.20 As many of the outcomes of interest to this evaluation rest on substantive behavioral change within households, villages, and the local DLO—which will likely take time to accrue—they may not be evident at this early stage of the evaluation. The evaluation team reviewed implementation data to gain a better sense of the sequence and scale of implementation. LTA activity data and documentation provide some insight into responses that may reflect lagged effects or may differ from the evaluation survey data. Analysis at endline will help explain divergence between evaluation and implementation data, and qualitative interviews will provide plausible explanations for potential discrepancies between IP reporting and the evaluation team’s findings. For example, although only half of the treatment sample reported possessing some form of land documentation, the most recent quarterly data from DAI show that an average of 93 20 The evaluation design planned for a phase-in approach that included data collection at 12-month intervals, and the Phase I baseline took place in March and April of 2017, during the rainy season. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 39 percent of the 10,535 registered CCROs had been collected. 21 The evaluation team will continue to coordinate with LTA activity staff to better understand implementation sequencing and processes. 6.1 IMPLEMENTATION BACKGROUND The LTA activity performed demarcation and adjudication, objection and correction, printing and registration, and CCRO registration across nine villages in the treatment group between May and November of 2017. This can generally be considered the Phase I implementation period. An additional three villages received demarcation and adjudication and objection and correction services through December 2017. Figure 13 presents the stages of LTA activity implementation. As LTA activity documentation notes, each stage can require multiple teams working simultaneously and in coordination with local officials in villages and at the DLO. FIGURE 13: ILLUSTRATIVE LTA ACTIVITY VILLAGE IMPLEMENTATION PROCESS On average, 85 percent of parcels in nine Phase I villages where registration took place received CCROs. Hence, a total of 10,535 CCROs were issued through November. Of those CCROs issued, 92% (n = 9,257) have been claimed, according to LTA M&E data shared with the evaluation team. LTA has 2017 data for seven villages. These show that, of the more than 9,000 CCROs claimed, 3,781 were issued to unique claimants (i.e., individuals claiming CCROs for their parcel(s), since one person may claim multiple CCROs). There was an average of 82 days between the start of demarcation and adjudication and the issuance of CCROs in the nine villages where these activities took place.22 Moreover, the time taken between adjudication and CCRO issuance varied. For example, the process took 35 days in Mwambao village, with its 663 parcels, and 115 days in Mgama village for 2,301 parcels. LTA staff not only provided the evaluation team with data on registration and CCROs, they also helped update or facilitate the issuance of Village Land Use Plans (VLUPs). In many cases, VLUPs were either out of date or nonexistent, despite reports stating otherwise. The three villages that did not have any Phase I activities as of the end of calendar year 2017 are all awaiting VLUPs. VLUPs are required to move forward with demarcation and other processes, as they lay out how a village will manage and use the land within its boundaries. 6.1.1 CHANGES TO IMPLEMENTATION TIMING AND IMPLICATIONS FOR THE IE As noted previously, the timing for Phase I midline data collection (concurrent with Phase II baseline data collection) changed from what was originally planned in the design proposal for this evaluation. Annex D provides the formal memo the evaluation team drafted to USAID outlining the anticipated challenges associated with the change in the LTA implementation timeline and subsequent change in the timing of Phase II baseline and Phase I midline data collection. As the team noted, the change in the implementation timeline and earlier collection of Phase I midline data would likely result in smaller than expected changes in outcomes at the Phase I midline. In reviewing the Phase 1 midline findings, then, it should be kept in mind that midline data collection took place six months after baseline, rather than at the planned 12-month interval, and during the 21 Based on disaggregated data provided to the evaluation team, but reported in “Monthly Report No. 24” 1-30 November 2017 Feed the Future Land Tenure Assistance, Annex 3. 22 The nine villages are Malagosi, Mgama, Mfukulembe, Udumka, Ilandutwa, Muwimbi, Mwambao, Nyamihuu, and Ngano. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 40 dry, rather than, rainy season. Seasonal differences can affect the outcomes of interest. For instance, seasonal changes in migration patterns and resource use may heighten or lessen dispute prevalence. 23 During final analysis, the evaluation team may be able to control for seasonal variation using external sources such as rainfall data. In addition, the change in the IE data collection timeline has implications for midline questions regarding investments and parcels. 24 Although the focus of the midline findings is on changes in outcomes for treatment households, summary statistics for comparison households are presented to show general trends and provide context. 6.2 MIDLINE RESULTS Phase I midline data covered 610 households and 907 respondents (Figure 14). This sample includes roughly 81 percent of the Phase I baseline sample (n = 755 households and 1,179 respondents). The evaluation team was unable to visit one Phase I village, Makuka, due to safety concerns tied to ongoing and heated land disputes in the village. 25 After two meetings with the Makuka village chairman, the evaluation team and field coordinators decided it would not be safe to proceed with data collection. The overall household attrition rate from baseline to midline was 17 percent (excluding Makuka village). Wives had the highest rate of attrition, at 28 percent. In about 60 percent of attrition cases, respondents from the Phase I baseline could not be re-interviewed, due to the fact they were travelling or absent from the village during the survey team’s visits. Another 20 percent of attrition cases resulted from the evaluation team’s inability to locate respondents, with no explanation given by neighbors or other members of the household (i.e., attrition was not due to relocation or travel). Finally, household members reported that five percent of the baseline sample had passed away. Other causes of attrition included illness and refusal to be re-interviewed. 23 Milline J. Mbonile, "Migration and Intensification of Water Conflicts in the Pangani Basin, Tanzania," Habitat International 29, no. 1 (2005): 41-67. 24 Margareta Wandel and Gerd Holmboe-Ottesen, "Food Availability and Nutrition in a Seasonal Perspective: A Study from the Rukwa Region in Tanzania," Human Ecology 20, no. 1 (1992): 89-107. 25 Twenty-three households in Makuka, comprising seven female household heads, 16 male household heads, and 10 wives, were not included in the midline sample. LTA activity staff has reported that the situation in Makuka has calmed down, hence, it is possible these respondents will be included in the endline sample. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 41 FIGURE 14: BASELINE AND MIDLINE SAMPLE BY ASSIGNMENT GROUP PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 42 6.2.1 SUMMARY STATISTICS BETWEEN BASELINE AND MIDLINE 6.2.1.1 LAND RIGHTS AND TENURE SECURITY Phase I baseline and midline samples show changes in tenure security and land rights measures within the treatment group relative to the comparison group (Table 19). While only 16 percent (n = 100) of treatment group respondents surveyed at baseline said they possessed land-related documentation, at midline the rate was 43 percent (n=205), a statistically significant change (p<0.001). In contrast, 11 percent of comparison group respondents had land documentation at baseline, but only 12 percent at midline (this change was not statistically significant). In the treatment group, the WTP for CCROs fell from baseline to midline by an average of 18,881 shillings. While an average decline of 57 percent, the change was not statistically significant. However, the proportion of treatment group respondents familiar with CCROs increased from 52 percent to 77 percent. According to LTA documentation, the average unit cost per CCRO is $8.97 USD, or roughly 20,000 shillings.26 A more modest decrease in WTP was recorded for the comparison group (5,187 shillings on average). Also for that group, familiarity with CCROs increased by an average of three percentage points across the two survey rounds. It is possible that changes to WTP at midline for both groups reflect respondents’ increased familiarity with the document and its typical cost to obtain. In order to promote understanding of land laws and documentation, the LTA activity ran a pre￾recorded radio show on five local stations in Iringa in June 2017, and a live radio show in September 2017. Activity staff also recorded three one-minute radio spots that were played 15 times over the course of six days ahead of the live radio show.27 These activities were aired evenly in both treatment and comparison areas. It is worth examining whether CCRO awareness and valuation changed as result of the total LTA activities implemented in the treatment villages between baseline and midline. Respondents in both assignment groups were generally more positive about future land disputes and potential expropriation risks at midline. When asked about land disputes in the coming year, 11 percent more treatment group respondents from baseline to midline (151 to 168) expected an improvement in land dispute problems. The opposite occurred in the comparison group, where nine percent fewer respondents (from 149 to 135) expected improvements over the next 12 months. Both assignment groups reported less community concern about land grabbing at midline, which may suggest less anxiety regarding this issue in Iringa. Additional data collected at endline will provide more insight into this and the other tenure security metrics in Tables 19 and 20 that saw little or no change between survey rounds. 26 See Annex 3, Table 3.11 in the LTA Activity Annual Report: Year 2, 2-47. 27 For more information on these activities see the LTA FY2017 Annual Report, 7. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 43 TABLE 19: TREATMENT GROUP TENURE SECURITY AND LAND RIGHTS SUMMARY STATISTICS AT BASELINE AND MIDLINE Treatment Group Variable Baseline Midline Mean Diff. n Mean SD Min Max n Mean SD Min Max Do you have familiarity with land laws (1=Y, 0=N) 585 0.04 0.19 0 1 445 0.05 0.22 0 1 0.01 Household possesses land-related documentation (1=Y, 0=N) 585 0.16 0.37 0 1 445 0.43 0.5 0 1 0.27*** Heard of CCROs (1=Y, 0=N) 585 0.52 0.5 0 1 445 0.77 0.42 0 1 0.25*** Willingness to pay for a CCRO (in TZS) 307 33,054 51,443 0 500,000.00 344 14,173 24,621 0 200,000.00 18,881*** In general, how many people in your community are worried that someone might try to take their land against their will? (1=Y, 0=N) 389 0.16 0.37 0 1 303 0.09 0.29 0 1 -0.07** In the next 12 months, do you expect problems with land disputes will improve, stay the same, or get worse? (1 = improve, 0 = stay the same, -1 = get worse) 389 0.23 0.71 -1 1 303 0.49 0.61 -1 1 0.26*** In the next five years, do you think it's possible that someone could try to take one of your parcels from you without your permission? 389 0.09 0.29 0 1 303 0.06 0.23 0 1 -0.03* Is there a risk that someone will take over one of your plots if you leave it fallow? (1=Y, 0=N) 389 0.44 0.5 0 1 303 0.44 0.5 0 1 0 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 TABLE 20: COMPARISON GROUP TENURE SECURITY AND LAND RIGHTS SUMMARY STATISTICS AT BASELINE AND MIDLINE Comparison Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. Do you have familiarity with land laws (1=Y, 0=N) 594 0.03 0.18 0 1 462 0.03 0.17 0 1 0.00 Household possesses land-related documentation (1=Y, 0=N) 387 0.11 0.32 0 1 292 0.12 0.33 0 1 0.01 Heard of CCROs (1=Y, 0=N) 594 0.56 0.5 0 1 462 0.59 0.49 0 1 0.03 Willingness to pay for a CCRO (in TZS) 331 23,094 42,023 0 500,000 273 17,907 23,177 0 200,000 -5,187* In general, how many people in your community are worried that someone might try to take their land against their will? (1=Y, 0=N) 393 0.15 0.36 0 1 302 0.09 0.29 0 1 -0.06** In the next 12 months, do you expect problems with land disputes will improve, stay the same, or get worse? (1 = improve, 0 = stay the same, -1 = get worse) 393 0.22 0.7 -1 1 302 0.32 0.68 -1 1 0.10* In the next five years, do you think it's possible that someone could try to take one of your parcels from you without your permission? 393 0.1 0.31 0 1 302 0.1 0.3 0 1 0.00 Is there a risk that someone will take over one of your plots if you leave it fallow? (1=Y, 0=N) 393 0.46 0.5 0 1 302 0.5 0.5 0 1 0.04 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 44 6.2.1.2 LAND DISPUTES The LTA activity facilitates dispute resolution processes and informs farmers about their rights under the law. Evidence has shown that assisted dispute resolution can, paradoxically, result in an initial increase in disputes.28 One reason is that an improved dispute resolution process or greater discussion of disputes may cause respondents to increase their reporting. Generally, land disputes are fairly uncommon among both assignment groups. Although there was no change in the proportion of treatment group respondents who experienced a dispute in the past year, there was a four percent decline (from 10 to 6 percent) in the proportion of comparison group respondents who experienced a dispute (there is substantial overlap in the reporting period for baseline and midline data). From Phase I baseline to midline, there was also no change in the mean number of disputes reported by the treatment group, while the average duration of disputes in the comparison group rose from 0.5 to 1.9 months (p< 0.10) (Table 21). The number of people in the treatment and comparison groups reporting disputes declined between data collection rounds. Additional data from the endline investigation will explore whether this is a general trend in Iringa; for example, both assignment groups reported an increase in the average duration of disputes, even though fewer people from both groups reported disputes. 6.2.2 LANDHOLDINGS, INVESTMENT, AND ENVIRONMENT Given the short period between baseline and midline, major changes would not be expected in landholdings and investment behavior beyond what may be seasonally driven. Both assignment groups reported an increase in the number of parcels owned and mean parcel size between survey rounds (Table 22 and Figure 15). The approach to asking about investments changed slightly between survey rounds. At baseline, respondents were asked about specific investments based on their response to previous survey items. At midline, the evaluation team posed the issue to all heads of household as a simple yes/no question. As shown in Tables 22 and 23, the proportion of respondents in both assignment groups who reported investment activity was similar across survey rounds. Both groups experienced a similar increase in investment in buildings. This may reflect the change in survey approach across the two rounds. The uptick in building investment may also reflect the survey’s seasonal variation in timing. It is easier to construct buildings during the dry season, and recall may be more accurate for activities that were done closer to the survey date. This type of recall bias and difference in seasonal timing of the baseline and midline may have led to the drop in fruit and non-fruit tree planting activity reported by both assignment groups at midline. The dry season (midline timing) is not as conducive for tree planting as the rainy season (baseline timing).29 28 See Christopher Blattman, Alexandra Hartman, and Robert Blair, "How to Promote Order and Property Rights Under Weak Rule of Law? An Experiment in Changing Dispute Resolution Behavior through Community Education," American Political Science Review 108, 4 (2014): 1-21. 29 In both survey rounds, respondents were asked about the preceding 12 months. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 45 TABLE 21: SUMMARY STATISTICS OF TREATMENT AND COMPARISON GROUP DISPUTES BY BASELINE AND MIDLINE Treatment Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff Experienced a dispute in the past year (1=Yes, 0 = No) 389 0.08 0.27 0 1 303 0.08 0.27 0 1 0 How long did the dispute last? (in months) 389 0.61 2.44 0 12 303 1.53 11.32 0 148 0.92 Number of disputes in the past 12 months 30 1.03 0.18 1 2 23 1.09 0.29 1 2 0.06 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 Comparison Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. Experienced a dispute in the past year (1=Yes, 0 = No) 393 0.1 0.3 0 1 302 0.06 0.23 0 1 -0.04* How long did the dispute last? (in months) 393 0.54 2.07 0 12 302 1.95 16.52 0 204 1.41* Number of disputes in the past 12 months 38 1.13 0.34 1 2 17 1.06 0.24 1 2 -0.07 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 TABLE 22: SUMMARY STATISTICS OF TREATMENT GROUP LANDHOLDING BY SURVEY ROUND Treatment Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. Parcel size (in hectares for all parcels owned) 389 2.57 5.37 0 86.6 303 3.84 6.72 0 55.4 1 1.27*** Number of parcels owned or rented 389 1.98 1.07 1 8 303 2.5 1.55 1 14 0.52*** Fruit trees planted (1=Y, 0=N) 389 0.44 0.5 0 1 303 0.39 0.49 0 1 -0.05 Non-fruit trees planted (1=Y, 0=N) 389 0.24 0.43 0 1 303 0.16 0.37 0 1 -0.08** Household invested in: (1=Y, 0=N): Buildings 389 0.21 0.41 0 1 303 0.47 0.5 0 1 0.26*** Fencing 389 0.05 0.21 0 1 303 0.03 0.18 0 1 -0.02 Soil conservation 389 0.3 0.46 0 1 303 0.34 0.48 0 1 0.04 Terracing 389 0.21 0.41 0 1 303 0.25 0.43 0 1 0.04 Wells 389 0.04 0.2 0 1 303 0.05 0.21 0 1 0.01 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 46 FIGURE 15: NUMBER OF PARCELS BY ASSIGNMENT GROUP AND SURVEY ROUND PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 47 TABLE 23: SUMMARY STATISTICS OF COMPARISON GROUP LANDHOLDING BY SURVEY ROUND Comparison Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. Parcel size (in hectares for all parcels owned) 393 2.32 3.3 0.1 36.83 302 2.8 3.54 0.1 24.28 0.48* Number of parcels owned or rented 393 2.02 0.98 1 6 302 2.47 1.25 1 8 0.45*** Fruit trees planted (1=Y, 0=N) 393 0.51 0.5 0 1 302 0.43 0.5 0 1 -0.08** Non-fruit trees planted (1=Y, 0=N) 393 0.25 0.43 0 1 302 0.12 0.32 0 1 -0.13*** Household invested in: (1=Y, 0=N): Buildings 393 0.18 0.39 0 1 302 0.47 0.5 0 1 0.29*** Fencing 393 0.05 0.22 0 1 302 0.07 0.25 0 1 0.02* Soil conservation 393 0.31 0.46 0 1 302 0.37 0.48 0 1 0.06 Terracing 393 0.21 0.41 0 1 302 0.24 0.43 0 1 0.03 Wells 393 0.03 0.18 0 1 302 0.03 0.18 0 1 0 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 48 6.2.2.1 HOUSEHOLD INVESTMENT AND WIVES’ SURVEY As noted in the LTA theory of change and in the evaluation design proposal, secure tenure may lead to increased access to finance, as farmers have formal documentation for collateral, and an improved ability to buy, sell, or rent land. The midline survey asked respondents the same questions related to borrowing activity as the baseline survey. As shown in Table 24, there was no change in the percentage of the treatment group sample that borrowed money between baseline and midline. However, the percentage of comparison group heads of household who reported borrowing funds decreased over this period, from 12 percent at baseline (n=49) to 8 percent at midline (n=23) (p<0.01). Nine of the 15 treatment villages went through the full LTA implementation process and received CCROs by midline. However, it may take longer than six months for a household to go from obtaining land documentation to utilizing it for increased financial activity. The evaluation team found limited initial changes for wives in the treatment sample. The exception was their awareness of the presence of women’s groups (Table 25). The percentage of respondents who were aware of women’s groups increased from 57 percent (n = 111) to 73 percent (n = 104), a statistically significant increase (p<0.01). Awareness on this measure also increased for wives in the comparison group (Table 26). Though the change was slightly smaller (53 percent, n = 107 to 69 percent, n = 111), it, too, was statistically significant (p<0.01). The general increase in awareness of women’s groups will be explored during endline data collection and qualitative interviews to determine if it is due to LTA’s broadcasts or other outreach in Iringa. Women’s tenure security is often compromised by land grabbing from family members, after the death of a spouse, for example. There was no statistically significant change in the percentage of respondents in either group who reported that it was likely that family members could take their land without permission. Seventy-eight percent (n = 111) of the treatment group sample and 66.4 percent (n = 107) of the comparison group sample reported this at midline (Table 27). Women in the treatment group reported an increase in joint decision making on farming. There was a statistically significant increase in joint decision making for food crops in the treatment group (p<0.05) from 72 percent (n = 134) to 81 percent (n = 112). During the same period, there was a slight decline in joint decision making regarding food crops in the comparison group. A statistically significant change occurred (p<0.05) in joint decision making on parcel use for the treatment group. This measure increased in both percentage and numerical terms from 37 percent (n = 73) to 67 percent (n = 96) between survey rounds. Notably, in both assignment groups, a small percentage of wives reported being the lead decision maker on parcel use and parcel income, as shown at the bottom of Table 27. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 49 TABLE 24: SUMMARY STATISTICS ON BORROWING BY ASSIGNMENT AND SURVEY ROUND Treatment Group Variable Baseline Midline Mean Diff. n Mean SD Min Max n Mean SD Min Max Borrowed money in the past six months (1=Yes, 0=No) 389 0.11 0.32 0 1 303 0.12 0.32 0 1 0.01 Approximate total borrowed in past year (in TZS) 44 791,886 2,172,814 20,000 10,000,000 36 276,306 296,940 5,000.00 1,500,000 -515580 Would household be able to obtain a loan if needed (1=Yes, 0=No) 389 0.51 0.5 0 1 303 0.61 0.49 0 1 0.1*** Comparison Group Variable Baseline Midline n Mean SD Min Max n Mean SD Min Max Borrowed money in the past six months (1=Yes, 0=No) 393 0.12 0.33 0 1 302 0.08 0.27 0 1 -0.04** Approximate total borrowed in past year (in TZS) 49 488,860 1,030,412 3,000.00 5,000,000 23 586,522 1,648,693 1 8,000,000 97662 Would household be able to obtain a loan if needed (1=Yes, 0=No) 393 0.51 0.5 0 1 302 0.56 0.5 0 1 0.05 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 TABLE 25: SUMMARY STATISTICS OF TREATMENT GROUP WIVES BY SURVEY ROUND Treatment Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. How many group/village meetings have you attended in the past six months? 196 2.51 2.56 0 17 143 2.57 1.91 0 10 0.06 Are there women's group in your village or the surrounding area? (1=Yes, 0=No) 196 0.57 0.5 0 1 143 0.73 0.45 0 1 0.16*** Number of women's groups attended 111 2.21 5.67 0 52 104 2.25 4.14 0 24 0.04 Do you feel comfortable speaking at village meetings or in group settings (1=Yes, 0=No) 196 0.59 0.49 0 1 143 0.58 0.5 0 1 -0.01 Has your household borrowed money in the past 12 months (1=Yes, 0=No) 196 0.19 0.39 0 1 142 0.19 0.39 0 1 0 Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 50 TABLE 26: SUMMARY STATISTICS OF COMPARISON GROUP WIVES BY SURVEY ROUND Comparison Group Variable Baseline Midline Mean n Mean SD Min Max n Mean SD Min Max Diff. How many group/village meetings have you attended in the past six months? 201 2.3 2.42 0 17 161 2.34 2.12 0 15 -0.08 Are there women's group in your village or the surrounding area? (1=Yes, 0=No) 201 0.53 0.5 0 1 162 0.69 0.47 0 1 0.19*** Number of women's groups attended 107 2.5 5.81 0 26 111 3 6.65 0 38 -2.81 Do you feel comfortable speaking at village meetings or in group settings (1=Yes, 0=No) 201 0.59 0.49 0 1 162 0.67 0.47 0 1 0.18 Has your household borrowed money in the past 12 months (1=Yes, 0=No) 201 0.22 0.42 0 1 162 0.14 0.35 0 1 -0.28** Statistical significance is denoted by the following: *** p<0.01, ** p<0.05, * p<0.1 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 51 TABLE 27: FREQUENCIES OFTENURE SECURITY AND DECISION MAKING Likelihood of land seizure from family members Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Don’t know 6 3.1 3 2.1 Don’t know 9 4.5 1 0.6 Likely 10 5.1 10 7.0 Likely 20 10.0 8 5.0 Neutral 17 8.7 10 7.0 Neutral 14 7.0 15 9.3 Somewhat unlikely 16 8.2 4 2.8 Somewhat unlikely 14 7.0 16 9.9 Very Likely 8 4.1 5 3.5 Very Likely 10 5.0 14 8.7 Very unlikely 139 70.9 111 77.6 Very unlikely 134 66.7 107 66.5 Food crop farming decisions Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Self 12 6.4 2 1.5 Self 7 3.6 4 2.5 Spouse 41 21.9 24 17.4 Spouse 33 17.0 33 20.9 Both spouse and self (joint decision making) 134 71.7 112 81.2 Both spouse and self (joint decision making) 154 79.4 121 76.6 Cash crop farming decisions Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Self 0 0 1 3.6 Self 0 0 1 3.5 Spouse 13 31.7 5 17.9 Spouse 6 17.1 5 17.2 Both spouse and self (joint decision making) 28 68.3 22 78.6 Both spouse and self (joint decision making) 29 82.9 23 79.3 How confident are you that you would receive a fair hearing if you had a land dispute? Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Not confident 23 11.7 11 7.7 Not confident 15 7.5 16 9.9 Somewhat confident 60 30.6 42 29.4 Somewhat confident 65 32.3 52 32.3 Unsure 23 11.7 11 7.7 Unsure 15 7.5 10 6.2 Very confident 83 42.4 78 54.6 Very confident 99 49.3 80 49.7 Very unconfident 7 3.6 1 1.0 Very unconfident 7 3.5 3 1.9 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 52 6.3 ESTIMATION APPROACH As noted in the overview to section 6, the analysis of outcomes at midline is primarily designed to provide updated information on the implementation process and a preliminary understanding of potential change in impacts at this early stage of activity implementation. At midline, we focus on assessing a select sub-set of outcomes for which it is reasonable to anticipate potential change at this stage of implementation. The midline analyses focus on select outcomes for four of the five outcome families/thematic areas30 on which the evaluation questions are focused: tenure security and land management; land disputes; investment and land use; and empowerment. As noted, midline data collection only six months after the start of implementation narrows down this list considerably, and reduces the likelihood that investment and other economic outcomes will have accrued at scale at this early stage in the project. However, we retain at midline some key outcomes under this theme to provide a benchmark understanding of change prior to endline. We do not include food security variables at midline due to the seasonal difference in data collection between baseline and midline. The evaluation team conducted midline analysis of impacts for the following 10 outcomes: 1. Tenure security and land management: a. Familiarity with land laws b. Household possesses land-related documentation c. Expropriation risk d. Fallowing risk 30 These are tenure security and land management; land disputes; investment and land use; empowerment; and economic and environmental outcomes. Who primarily decides how to use this household's parcels? Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Both spouse and self (joint decision making) 73 37.2 96 67.6 Both spouse and self (joint decision making) 97 48.3 100 61.7 Spouse 108 55.1 40 28.2 Spouse 90 44.8 52 32.1 Other household member 15 7.7 0 0 Other household member 14 7.0 0 0 Self 0 0 6 4.2 Self 0 0 10 6.2 Who primarily decides how to use income from this household's parcels? Treatment Group Comparison Group Response Baseline Midline Response Baseline Midline n % n % n % n % Both spouse and self (joint decision making) 100 51.0 97 68.3 Both spouse and self (joint decision making) 115 57.2 111 68.5 Spouse 80 40.8 38 26.8 Spouse 73 36.3 43 26.5 Other household member 16 8.2 1 1.0 Other household member 13 6.5 0 0 Self 0 0 6 4.2 Self 0 0 8 4.9 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 53 2. Land disputes: a. Incidence of land disputes in past year (noting this is a low frequency event at baseline and midline) b. Duration of land disputes (in months) 3. Investment and land use: a. Total land holding by household (in ha) b. Credit access by household31 (over the past six months) c. Incidence of tree planting on farms (fruit and non-fruit trees) 4. Empowerment: a. Land-related decision-making power exclusively by male household head We use the fixed effects difference-in-difference (DID) panel regression specification below to test for the impact of the LTA activity on each of the above outcomes. The model includes a set of pre￾treatment covariates to control for potential differences in the treatment and control groups, and village-level fixed effects that control for time-invariant unobserved factors. The treatment effect is estimated by a regression coefficient on a dummy variable that interacts time and treatment. For continuous outcome variables at the household level, the panel regression models take the following form:32 𝑌𝑖𝑡 = 𝛾0 + 𝛾1𝑋𝑖𝑡 + 𝛾3𝛿𝑡 + 𝛽(𝛿𝑡 ∗ 𝑇𝑖 ) + 𝛾𝑖 + 𝜀𝑖𝑡 (1) Where: Yit is the outcome of interest for household i at time t, Xit is a vector of covariates, δt is a dummy variable equal to 1 at the midline, T is a dummy variable equal to 1 for members of the treatment group, γi is a vector of village-level fixed effects εit is a random error term, and γ and β are parameters to be estimated. We use robust standard errors clustered at the village level. The estimate of LTA impact is given by β, which reflects the Average Treatment Effect. Under standard assumptions, β provides an unbiased estimate of the causal impact of the LTA activity on the outcome Y. We also include a set of individual, household, or village level control variables measured at baseline to further improve the precision of the outcome estimates. These are: Gender of household head; Household head age; Head education level; and Village distance to Iringa Town (driving distance in kilometers). Alternative Specification: For added robustness, we run the alternative specification below. 𝑌𝑖𝑗 = 𝛽0 + 𝛽1𝑇𝑖𝑗 + 𝛽2𝑋𝑖𝑗 + 𝑌𝑖𝑗 𝜊 + 𝛾𝑖 + 𝜀𝑖𝑡 (2) Where 𝑌𝑖𝑗 is the outcome measured for household i in village j measured at midline; 𝑇𝑖𝑗 is a dummy which indicates treatment status; 𝑋𝑖𝑗 is a vector of co-variates as listed above; 𝑌𝑖𝑗 𝜊 j is the value of the outcome as measured at baseline; 𝛾𝑖 is household fixed effects; and 𝜀𝑖𝑡 is the error term. Robust 31 Note this excludes informal lending by friends, neighbors, or families, but includes all formal sources of credit, such as from banks and micro-finance institutions, as well as informal lending from community savings and loans groups. 32 Note that logit models are used for binary outcomes. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 54 standard errors clustered at the village level are also used. Under this analysis of co-variance (ANCOVA) approach, the main control variable is the baseline value of the outcome variable. Limitations of Midline Analyses • Timing of midline data collection. The collection of the midline data only six months after the start of implementation, rather than one year into implementation as initially planned by the evaluation team, is likely to result in smaller observable impacts, and fewer significant outcomes, at this stage. This is because at the timing of the current midline, there has been less time for such impacts to accrue at scale for LTA beneficiaries. The evaluation team aimed to mitigate this issue by focusing on a select number of intermediate outcomes for the midline analyses. In addition, the seasonal timing of midline data collection could have some implications for reliability of measurements for some indicators across the two survey rounds. However, the indicators used for the midline analyses are not considered by the evaluation team to be highly susceptible to this seasonal difference, with the potential exception of the incidence of tree planting on farms. At endline, the evaluation team aims to replicate the seasonal timing of the baseline sample. • Limited observations. This evaluation is designed to examine impacts across 30 treatment villages where the LTA activity is implemented. The evaluation team anticipates this to be a sufficient sample size to detect impacts on outcomes of interest, particularly tenure security, investment, and empowerment, based on power calculations conducted to date. However, a larger number of village clusters would generally be preferable for cluster￾randomized designs. The small number of villages for this IE presents some risks for the ability to make causal linkages of the LTA activity to certain outcomes and impacts further down the causal chain. For example, sustainable land clearing practices will ideally lead to lower greenhouse gas emissions, but it is doubtful that the effects of this can be measured within the timeframe and from the limited number of villages under study for this IE. The evaluation team will address this issue by measuring more proximate outcomes and indicators that are highly correlated with impacts that take longer to accrue. Since the Phase I midline is conducted on half the total evaluation sample, constraints due to limited sample size are even more salient at this stage. 6.4 DISCUSSION OF RESULTS This section provides a summary of key findings from the Phase I midline analysis of LTA impacts on select tenure security, land disputes, land use and investment, and empowerment outcomes. Table 28 presents impact estimates for each of the ten outcomes measured at midline, while Figure 16 enables a comparison of the magnitude and statistical significance of each outcome assessed. The results suggest statistically significant and positive impacts for the following three indicators: Household possession of land-related documentation: Results suggest that holding household head gender, age, education level, and village distance to Iringa constant, there is, on average, a 29.8 percent increase in the likelihood of a household having land documentation at midline, for households in the treatment group relative to those in the comparison group. The magnitude of impact is relatively large, and the statistical significance is robust to alternative model specifications. This finding is not necessarily surprising, since LTA has been actively working to issue CCROs to households in activity villages, and LTA M&E data confirm that the activity has been fairly successful in achieving wide scale issuance of CCROs to households. The measure provides an overall estimate for the household, rather than one disaggregated by respondent type, but inclusion of a gender covariate for the household head in the estimation model was not significant. The results provide useful confirmation that the project has been successful in increasing land documentation among project beneficiaries, a key intermediate outcome in the causal chain to improved tenure security impacts. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 55 TABLE 28: FIXED-EFFECTS REGRESSION RESULTS Variable Estimate (log odds) P-value 95% CI Total n Treatment n Comparison n Familiarity with land laws 0.63 0.22 -0.37 1.66 1,807 882 925 Household possesses land-related documentation 0.29*** 0.00 0.21 0.39 1,198 588 610 Expropriation risk -0.94** 0.04 -1.96 0.08 1,198 588 610 Fallowing risk 0.00 0.98 -0.48 0.49 1,807 882 925 Incidence of land disputes in past year 0.14 0.38 -0.18 0.47 95 46 49 Duration of land disputes (in months) -0.03 0.96 -1.56 1.48 95 46 49 Total land holding by household (in ha) 0.67* 0.06 -0.02 1.30 1,198 588 610 Credit access by household (over past six months) 0.91 0.03 0.11 1.74 1,198 588 610 Incidence of tree planting on farms (fruit and non-fruit trees) 0.80** 0.02 0.15 1.45 1,545 758 787 Land-related decision-making power exclusively by male household head -1.10** 0.03 -2.15 0.03 610 294 316 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 56 FIGURE 16: STATISTICALLY SIGNIFICANT (MINIMUM P<0.10) OUTCOMES PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 57 Household total landholdings: Results suggest that, on average, total landholdings by treatment group households has increased by 0.67 ha relative to comparison group households at midline. However, the magnitude of impact is fairly small and the results are only marginally significant (p<.10). The statistical significance of the land holding results are variable under alternative model specifications, while there is currently little supporting evidence in the midline data to explain if or why households in LTA villages are using their increased familiarity with land laws and possession of CCROs to expand their landholdings. It is also possible that the mapping of individual parcels conducted by LTA provides households in the treatment group with an updated understanding of their actual plot size, and that respondents underestimate their actual acreage at baseline, prior to obtaining that knowledge. Given a range of potential explanations, this intermediate finding should be interpreted with caution. It will be investigated further at endline, when the combination of three time points of panel survey data collection and qualitative data collection will enable a stronger understanding of the validity and reasons for this trend. Land-related decision-making power exclusively by the male household head: Results suggest that, holding the same household and village factors constant as above, there has been an 11.4 percent decrease in the likelihood of a land-related decision solely by the male household head, for treatment group households. The magnitude of impact is somewhat smaller across alternative model specifications, but the significance of the effect remains. This finding suggests that LTA activities designed to inform women of land rights and to encourage their management and decision making regarding land they use appear to have begun to take hold. Thus, at this early midline stage, LTA implementation may be having positive impacts on some of the key intermediate outcomes, across three of the four outcome categories assessed at this stage. Under the LTA theory of change, continuation of such impacts over the activity’s lifetime is expected to lead to significant improvements in longer terms outcomes, such as increased agricultural productivity and household income. The midline analysis did not find statistically significant impacts for the remaining outcomes assessed at this stage. However, this may not be surprising, given that the analyses measure impacts for activities that have only been underway for six months. The generally low proportion and lack of change in household familiarity with land laws for the treatment group may indicate that project messaging on this has not yet taken hold. In addition, households that have only recently obtained their CCROs and begun to understand their potential benefit for securing their landholdings may not yet have experienced a lower expropriation risk, or changed their land investment behavior accordingly. Overall, the midline results (1) indicate that achievement of some of the anticipated LTA impacts appears to be underway, (2) confirm the validity of the IE design and sample power, (3) highlight the role that endline qualitative data collection is likely to play in helping to explain impacts at endline, and (4) re-confirm the utility of measuring longer term outcomes as planned at endline. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 58 ANNEX A: EVALUATION STATEMENT OF WORK Impact Evaluation of the Feed the Future Tanzania Land Tenure Assistance Activity This Statement of Work is for an impact evaluation commissioned by the United States Agency for International Development (USAID) that will examine the Feed the Future Tanzania Land Tenure Assistance (LTA) Activity. 1. Project Information LTA is a four-year activity awarded by USAID/Tanzania to DAI in 2015 and is a part of the Feed the Future (FTF) initiative. The LTA activity seeks to clarify and document land ownership, support land use planning efforts, and increase local understanding of land use and land rights in Tanzania. It is envisioned that the interventions carried out under LTA will reduce land tenure-related risks and lay the groundwork for sustainable agricultural investment for both smallholder farmers and commercial investors throughout the Southern Agricultural Growth Corridor of Tanzania (SAGCOT) and in the value chains of focus for Tanzania’s FTF program. The LTA activity was designed in line with the Government of Tanzania’s (GOT) land tenure objectives to safeguard USAID’s ongoing agricultural and economic growth investments and to protect the interests of the private sector and local communities. The activity seeks to achieve these goals by: 1. Assisting villages in completing the land use planning process and delivering Certificates of Customary Right of Occupancy (CCROs) through the use of open source mobile technology developed under USAID’s Mobile Application to Secure Tenure (MAST) pilot activity; 2. Developing the capacity of village and district land governance institutions, and individual villagers, to complete the land use planning and CCRO process, effectively manage land resources, respect women’s land rights, and build agriculture-related business skills through education and awareness-raising activities; and 3. Raising awareness of the MAST technology within the GOT, civil society, academia, and the private sector, with the goal of increasing uptake of the technology on a national level. LTA is comprised of two larger activities (1 and 2) and two smaller activities (3 and 4), described below. Local sustainability is a critical component of the overall LTA activity. The goal of LTA is to empower district and village land institutions in targeted districts to carry forward the capacity development and land administration process independently (and with little or no outside financial support) once the activity concludes. • Activity 1: Assist villages and district administrations in completing the land use planning process and delivering CCROs in select villages within two districts (Iringa and Mbeya). • Activity 2: Educate and develop the capacity of village land governance institutions and individual villagers to complete the land use planning and CCRO process, effectively manage land resources, respect the land rights of women, youth, and pastoralists, and build agriculture￾related business skills. • Activity 3: Educate and develop the capacity of district-level land governance institutions in the Mbeya District to complete the land use planning and CCRO process; effectively manage land PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 59 resources; respect the land rights of women, youth, and pastoralists; and build agriculture￾related business skills. • Activity 4: Develop capacity to use the MAST application throughout the SAGCOT and nationally. DAI plans to implement LTA in five to six test villages over the summer of 2016. These initial villages are likely to be in Iringa District, due to Ministry preferences, but may be in Mbeya District as part of the LTA’s capacity development activities. Full rollout of LTA is expected to occur in early 2017 in Iringa District, with at least 30 villages selected to receive the interventions. 2. Development Hypothesis USAID envisions that if the LTA activity clarifies and documents land ownership, supports land use planning efforts, and increases local understanding of land use and land rights, then this will lead to increased agricultural investment, reduced land tenure risk, and more empowered people and local institutions. The LTA activity components work in tandem to promote inclusive agricultural development, food security and investment, and institutional capacity. This section provides a preliminary version of the development hypotheses and causal linkages that the evaluation will consider, which will be refined and further elaborated in the Evaluation Design Proposal. Figure 1 illustrates the causal linkages that USAID envisions for translating results under each of the activities33 into the LTA activity’s intended intermediate and final outcomes and that this evaluation will be expected to examine. In this Theory of Change diagram, the proliferation of CCROs leads to increased investment and reduced disputes through improved perception of tenure security. As illustrated in the diagram, the possible hypotheses for examination within the LTA activity could include: 1. If villages and district administrations receive assistance for completing the land use planning process and delivering CCROs to formalize land rights, then disputes over land tenure will decline and crop yields will improve. 2. If village land governance institutions and individual villages are educated and trained on the land use planning and CCRO process, including on respecting the land rights of women, youth, and pastoralists, then women, youth, and pastoralists will experience an increase in titling, improvement in skills, and have better representation in their villages. 3. If the LTA activity develops capacity to use the MAST application throughout the SAGCOT and nationally, then communities and institutions at all levels will be able to sustainably certify land tenure, which will promote agricultural commercial activity and investment. 33 Only three activities are shown in the Theory of Change diagram, since Activity 3 is specific to Mbeya District, and this evaluation will largely focus on Iringa District. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 60 FIGURE 1: THEORY OF CHANGE FOR THE LTA ACTIVITY PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 61 3. Existing Performance Information Sources The LTA activity is currently in its start-up phase and is developing an inception report that will outline its approach to implementation. There have been similar, albeit smaller scale, land rights interventions in Tanzania that utilize mobile technology,34 but these have not been rigorously evaluated. The evaluation team has received limited documentation on the LTA activity’s implementation plans to date, but USAID and DAI have committed to share all implementation reports, results frameworks, and survey materials as they become available. USAID has already provided the evaluation team with the following documents and data related to the LTA activity: • Scope of Work for the LTA Request for Task Order Proposals (RFTOP) • USAID/Tanzania letter to the Ministry of Lands, Housing, and Human Settlement Development • Iringa Village Data • Iringa District Map with potential selection sites The following additional documents have not yet been provided to the evaluation team but will be shared as the evaluation progresses: • DAI proposal for LTA RFTOP • Results framework from DAI for LTA • All future quarterly and annual project management and progress reports prepared by DAI for LTA • Copies or detailed descriptions of content of land tenure campaigns • Documents pertaining to the certification, selection, and implementation of tenure projects • Annual USAID/Tanzania LTRM Survey materials, including M&E data, sampling plans, and survey instruments In addition to information provided by USAID and DAI, the evaluation team may need to access other types of secondary data, including administrative information on the relevant Tanzanian municipalities from a variety of sources, including Government of Tanzania (GOT) statistical agencies. The evaluation team will work with USAID and DAI as needed to obtain relevant introductions and permissions to access any such data that are needed. 4. Evaluation Purpose, Audience, and Intended Use Purpose The purpose of this impact evaluation is to provide USAID with an evidence base on the impacts of its investment in the LTA activity and also to build the evidence base on the impacts of land mapping, registration, and formalization in rural customary land tenure settings in Tanzania. The results of this evaluation will be made widely available to encourage replication within or beyond Tanzania, as applicable. As such, this evaluation will apply USAID’s Evaluation Policy guidance with respect to using the most rigorous evaluation design and methods possible to demonstrate accountability for achieving results. The evaluation is also designed to capture practical lessons from USAID’s experience with 34 Mobile technology refers to MAST, which uses open source code and readily available mobile technologies (e.g., GPS/GNSS￾enabled smart phones and tablets) coupled with broadly participatory crowd-sourced data collection methods. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 62 regard to increasing sustainable agricultural investment by securing land tenure through first-time registration. Audience The evaluation is aimed at several audiences. First, the findings are expected to be of value from an accountability and learning standpoint to USAID. Secondly, findings and lessons learned from this evaluation will also be of interest to the GOT, which aims to scale CCRO delivery rapidly across the country, and to DAI and other practitioners in the land tenure sector working to document customary land rights. Finally, the evaluation will be of interest to donors, implementers, and scholars more generally by making an important contribution to the evidence base on land tenure interventions. Intended Use This evaluation will be used to inform the design of future donor and government activities that aim to improve tenure security and generate economic benefits by strengthening land rights. One such activity is the upcoming Land Tenure Support Program, a large-scale effort jointly funded by DfID, SIDA, and DANIDA. 5. Evaluation Questions The evaluation will address a specific set of evaluation questions that will be developed and finalized in close collaboration between USAID/E3/Land, USAID/Tanzania, the evaluation team, DAI, and other stakeholders as appropriate. This SOW will be updated following final agreement on the evaluation questions. In general, the evaluation questions are expected to focus on the impact of the LTA activity on four types of outcomes: 1. Investment: by improving tenure security and reducing disputes, LTA is also anticipated to stimulate small-scale agricultural investment. Stronger land rights increase landholders’ confidence that they will be able to reap the benefits of investments in their land that pay off over time. Such investments may include small-scale irrigation technology, soil conservation measures, or switching to perennial crops such as coffee, cashews, or fruit trees. The existing evidence on the relationship between land rights and these kinds of investments shows considerable variation in the levels and types of impacts that are observed; a summary and meta-analysis of the evidence from West Africa is provided by Fenske (2011). 2. Perceived tenure security: an important outcome associated with LTA is the extent to which beneficiaries perceive the activity as having strengthened their land rights. In practice, this means that LTA should reduce beneficiaries’ concerns that their land could be expropriated, or that they could face costly disputes related to their land. Measuring the activity’s impact on these kinds of perceptions requires careful attention to the context, so that survey questions can be structured around the particular issues and concerns that beneficiaries face. A number of previous impact evaluations commissioned by USAID/E3/Land have considered these issues, and the impact evaluation of LTA will draw on these experiences in developing its approach to measuring tenure security. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 63 3. Incidence of land-related disputes or disputes: in addition to changing perceptions, another outcome that the evaluation may consider is the actual incidence of disputes and disputes over land. As above, careful attention to context is needed in designing the approach to measuring these outcomes. While reducing land dispute is an important outcome, a potential challenge with measuring impacts on dispute is that interventions such as those under LTA can actually increase the incidence of land disputes in the short run. For example, disputes may arise in the course of establishing boundaries, or latent disagreements about land rights may rise to the surface in the course of establishing formal claims. Such disputes were observed for the first MAST pilot site, with several reported cases of border disputes, intra-family disputes over ramifications for inheritance, as well as former residents returning to try to reassert old claims when they learned that land registration was occurring. In course of finalizing the evaluation questions, the evaluation team should assess the potential for the evaluation to accurately measure these kinds of outcomes within the anticipated timeframe for the evaluation. 4. Empowerment: the evaluation will also consider outcomes related to empowerment. Empowerment is often considered from the standpoint of potentially vulnerable sub-groups such as women, youth, or the poor, and can also be conceptualized more generally. A World Bank study by Alsop and Heinsohn (2005) defines empowerment broadly as “as a person’s capacity to make effective choices; that is, as the capacity to transform choices into desired actions and outcomes,” and presents a framework for measuring different dimensions of empowerment. In the context of LTA, strengthening land rights in expected to act on empowerment by improving security of assets that are critical to people’s lives in the household, community, and economy. For the impact evaluation of LTA, empowerment outcomes are of particular interest in the context of gender. A recent paper by Allendorf (2007), for example, found that land rights are closely linked to women’s empowerment in Nepal. In addition, USAID has funded the development of the Women’s Empowerment in Agriculture Index, which is widely used to measure women’s empowerment in FTF activities. The Index includes a battery of survey questions and methods to measure various dimensions of empowerment, and could be incorporated directly into the household surveys for the LTA impact evaluation. The types of outcomes described above reflect changes in behaviors and attitudes that are expected to be measurable over a relatively short timeframe (approximately one to two years following the conclusion of implementation). LTA is also anticipated to potentially impact a broader set of economic outcomes in the longer term, as the benefits of these changes in behaviors and attitudes are realized over time. These include frequency of land transactions, access to credit, agricultural productivity, and ultimately improvements to household income, consumption, and food security. In light of the limited evidence base on the impact of land tenure interventions - particularly in a randomized controlled trial (RCT) setting – the evaluation may also examine these longer-term outcomes. One approach would be for the evaluation to include an initial round of follow-up data collection and analysis focused on the four intermediate outcomes above, with a second follow-up at a later date to measure longer term impacts. This would allow the evaluation to generate useful findings within one to two years of implementation, while still taking full advantage of the learning potential of a RCT to investigate broader economic outcomes. 6. Gender Considerations In line with USAID’s Gender Equality and Female Empowerment Policy and Automated Directives System 203.3.1.5, the evaluation will consider gender-specific and differential effects of LTA. The PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 64 evaluation team will disaggregate access and participation data by gender at multiple points along the Theory of Change diagram to analyze the potential influence these effects have on activities and outcomes. Data collected through surveys conducted under this evaluation will be gender-disaggregated to identify gender differences with respect to benefits and outcomes, as well as lessons learned from female title holders and farmers. The evaluation team will conduct further inquiry on gender themes as they emerge during data analysis. 7. Evaluation Methods Impact Evaluation Design Impact evaluations identify activity impact by comparing outcomes between activity beneficiaries to those of a control or comparison group of non-beneficiaries. The control or comparison group is intended to represent the counterfactual, or what would have happened in the absence of the LTA intervention. As per the USAID Evaluation Policy, impact evaluations using experimental designs – whereby units are randomly assigned to treatment and control groups – provide the most rigorous evidence of activity impact, and this will be the preferred approach for the LTA impact evaluation. Where randomized assignment is not feasible, quasi-experimental impact evaluation designs can be employed as an alternative. The evaluation team responding to this SOW will work with USAID/E3/Land, USAID/Tanzania, and DAI staff to develop a design that suits the objectives, timing, and constraints of the LTA evaluation. The evaluation team will produce an Evaluation Design Proposal to be approved by USAID/E3/Land prior to site selection or randomization taking place. It is expected that the evaluation questions will be answered using an experimental or, if necessary, quasi-experimental design, and that a mixed-method approach may be suitable to answer the evaluation questions. Data Collection Methods A range of methodologies can be used in impact evaluations, and the most appropriate approach in any particular case depends on a variety of factors including the goals of the evaluation, the outcomes to be measured, the nature of the activity being examined and its implementation approach, and the resources and timeframe available for the evaluation. USAID anticipates that data collection for this evaluation will involve the use of household-level surveys that cover all of the villages targeted for LTA. This is likely to include a baseline survey that would be conducted before major LTA interventions commence. The survey would collect information on basic demographics, household and individual characteristics, and the outcomes of interest that the evaluation will measure. The evaluation team responding to this SOW shall provide further details on data collection methods and the specific survey methodology in the Evaluation Design Proposal, including proposing specific data collection methods on a question-by-question basis. Pending further discussion with USAID and DAI, data collection for this evaluation may also include collecting village-level information about potential activity sites that can be used to determine which villages may be eligible to participate in the activity. 8. Data Analysis Methods In its Evaluation Design Proposal, the evaluation team responding to this SOW should propose specific data analysis methods on a question-by-question basis, including the appropriate mix of methods PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 65 necessary to estimate the impact LTA has on the primary outcomes of interest. Potential data analysis methods include difference-in-difference and multivariate regressions. The Evaluation Design Proposal should also explain what statistical tests will be conducted on data collected to address all evaluation questions, how qualitative data will be analyzed, and whether that analysis will allow the evaluation team to transform some data obtained from qualitative into quantitative form. The Evaluation Design Proposal should also indicate and justify the evaluation team’s proposed sequencing of quantitative and qualitative data collection. For example, if key informant qualitative interviews are conducted during the endline data collection process, these lines of data may be collected and analyzed in parallel and only synthesized once data from all other sources are available. 9. Strengths and Limitations The strengths and limitations of the LTA impact evaluation will depend on the final design proposed by the evaluation team in consultation with USAID and DAI. The final design should reflect a rigorous approach to answering the evaluation questions and contribute to the global knowledge on land tenure. One key contribution of this evaluation is that it is expected to specifically test the impact of LTA on women, youth, and pastoralists, which is a great contribution to the evidence base on land tenure and investment. Sample size, activity reach, and implementation fidelity could all create internal validity limitations for this evaluation. Ensuring that the sample size achieves sufficient statistical power will be critical for identifying impact and answering the evaluation questions. In addition, ensuring that randomization is done properly and random assignment, if applied, is systematic will improve the internal validity of the evaluation but must be done in a transparent manner. Indirect contamination across treatment arms and control groups is always a possibility, which is why it is important for the evaluation team and the implementation team to coordinate from the outset. 10. Evaluation Deliverables It is anticipated that the evaluation team responding to this SOW will be responsible for the deliverables listed in Table 1. A final list of proposed deliverables and due dates will be included in the Evaluation Design Proposal for USAID’s approval. TABLE 1: EVALUATION DELIVERABLES Deliverable Estimated Due Date 1. Concept Paper, describing design and methodological options to answer the evaluation questions TBD in consultation with USAID 2. Draft Evaluation Design Proposal TBD in consultation with USAID 3. Final Evaluation Design Proposal, including data collection and analysis methods, evaluation instruments, team composition, and proposed timeline TBD in consultation with USAID 4. Baseline Report o/a 60 days following completion of baseline data collection 5. Fully cleaned, redacted, and documented baseline data submitted to DDL o/a 90 days following completion of baseline data collection 6. Draft Evaluation Report o/a 60 days following completion of endline data collection PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 66 Deliverable Estimated Due Date 7. Final Evaluation Report o/a 21 days following receipt of USAID comments on Draft Evaluation Report 8. Fully cleaned, redacted, and documented endline data submitted to DDL o/a 90 days following completion of endline data collection All documents and reports will be provided electronically to USAID no later than the dates indicated in the approved Evaluation Design Proposal. The format of the evaluation report should follow USAID guidelines set forth in the USAID Evaluation Report Template. 11. Team Composition The Evaluation Design Proposal should describe the specific composition and qualifications of the team members who will be carrying out this evaluation, including CVs for core team members. General qualifications and roles anticipated for the primary positions on the core evaluation team are listed below. Local survey research firm(s) with experience in the conduct of household surveys at the village level and/or qualitative data collection may also support the evaluation team, as necessary. Principal Investigator The Principal Investigator for this impact evaluation will hold a Ph.D. in a relevant economic development field. S/he will have previous experience with land tenure programs and will have previously served as a team leader for one or more impact evaluation(s). Familiarity with a range of impact evaluation designs and with USAID evaluation guidance will be sought for this position. Experience in publishing evaluation research in peer-reviewed journals is desirable, as is experience working in East Africa. A demonstrated ability to gather and integrate both quantitative and qualitative findings to answer evaluation questions is expected. Demonstrated experience managing multinational teams and producing highly readable reports for USAID and its developing country partner audiences on a timely basis is expected. This individual will be primarily responsible for the quality of the evaluation design and its execution, particularly with respect to the evidence obtained on questions involving causality and the attribution of outcomes to USAID’s intervention. This is not anticipated to be a full-time position. Evaluation Specialist The Evaluation Specialist should have a graduate degree in a relevant social science field and may be a Tanzanian national. The individual will have sufficient previous experience with evaluations and other types of studies involving sample surveys to be actively engaged in efforts to oversee and ensure the quality of multiple rounds of household surveys, that data codebooks are clearly written, and that all study data prepared by local survey research firms can be properly transferred to USAID. Gender analysis experience is also desirable. This is not anticipated to be a full-time position. 12. USAID Participation The desirability of USAID participation in evaluation activities such as field reconnaissance will be considered in consultation with USAID and the evaluation team, and any specific roles and responsibilities of USAID staff will be described in the Evaluation Design Proposal. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 67 13. Scheduling and Logistics Figure 2 provides a preliminary timeframe for impact evaluation activities, which will be updated and refined by the evaluation team in its Evaluation Design Proposal. It is anticipated that implementation of LTA will occur at the start of FY17. Figure 2: Preliminary Timeline for LTA Impact Evaluation The evaluation team will be responsible for procuring all logistical needs such as work space, transportation, printing, translation, and any other forms of communication. USAID will offer some assistance in providing introductions to partners and key stakeholders as needed, and will ensure the provision of data and supporting documents as possible. 14. Reporting Requirements The format of the evaluation report should follow USAID guidelines set forth in the USAID Evaluation Report Template (http://usaidlearninglab.org/library/evaluation-report-template) and the How-To Note on Preparing Evaluation Reports (http://usaidlearninglab.org/library/evaluation-report-template). The final version of the evaluation report will be submitted to USAID and it is anticipated that it will not exceed 30 pages, excluding references and annexes. All members of the evaluation team will be provided with USAID’s mandatory statement of the evaluation standards they are expected to meet, shown in the following text box, along with USAID’s dispute of interest statement that they should sign before field work starts. Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Concept Paper Scoping Trip Evaluation Design Proposal Survey Pre-Test Enumerator Training Baseline Data Collection Baseline Data Analysis and Report Oral Presentation of Baseline Findings LTA Program Implementation (100%) Endline Data Collection and Analysis Endline Report Draft Final Report Oral Presentation(s) Final Report FY 21 Implementation Period for LTA Project FY 20 Tasks FY 16 FY 17 FY 18 FY 19 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 68 15. Budget The evaluation team responding to this SOW will propose a notional budget for this evaluation, including cost implications of the methodological options proposed. A full detailed budget will then be prepared for USAID’s approval. USAID EVALUATION POLICY, APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 69 ANNEX B: PHASE II BASELINE SURVEY INSTRUMENT A. Introduction and Consent Greetings! My name is._____ I am from Research Solutions Africa (RSA) and is currently undertaking a survey on behalf of MSI/NORC, a contractor with the United States Agency for International Development, in conjunction with the Iringa District Land Office to learn more about villagers in this district. We are currently visiting villages in Iringa to gain a better understanding of village land use, administration, and the local community. The answers from this questionnaire will be used to learn more about land-use and life in the village. I will not tell anyone about your answers to these questions. Only the research team will view your responses. Although we will ask for information about this village and your experience here, we will never use personal information in our documentation and will not report sensitive village information to anyone. This survey does not mean that a project or NGO will come to this village, and your answers will not affect whether any future projects come to this village. The entire survey will take about 2 hours. If you have any questions in the future, you can contact MSI via phone at XXX Are you willing to proceed with the interview? 1. Yes…. >>>(Tick category of hhd respondent and proceed as appropriate) 2. No…. >>>(Tick respondent category and Terminate interview) Category of household respondent 1. Male household head >>>Section B 2. Female household head >>>Section M 3. Head of household (for households with only one household head: widows/widowers/single parents/single￾member households, etc.) >>> Section B PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 70 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 71 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 72 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 73 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 74 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 75 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 76 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 77 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 78 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 79 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 80 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 81 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 82 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 83 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 84 Enter quantity PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 85 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 86 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 87 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 88 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 89 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 90 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 91 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 92 ; PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 93 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 94 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 95 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 96 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 97 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 98 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 99 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 100 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 101 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 102 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 103 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 104 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 105 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 106 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 107 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 108 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 109 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 110 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 111 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 112 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 113 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 114 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 115 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 116 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 117 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 118 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 119 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 120 L.ATime Allocation Now I’d like to ask you about how you spent your time during the past 24 hours. We’ll begin from yesterday morning, and continue through to this morning. This will be a detailed accounting. I’m interested in everything you do (i.e. resting, eating, personal care, work inside and outside the home, caring for children, cooking, shopping, socializing, etc.), even if it doesn’t take you much time. PLEASE RECORD A LOG OF THE ACTIVITIES FOR THE INDIVIDUAL IN THE LAST COMPLETE 24 HOURS (STARTING YESTERDAY MORNING AT 4 AM, FINISHING 3:59 AM OFTHE CURRENT DAY). THE TIME INTERVALS ARE MARKED IN 15 MIN INTERVALS AND ONE ACTIVITY CAN BE MARKED FOR EACH TIME PERIOD BY DRAWING AN X THROUGH THATACTIVITY. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 121 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 122 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 123 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 124 ANNEX C: MIDLINE SURVEY INSTRUMENT ADMINISTRATIVE INFORMATION PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 125 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 126 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 127 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 128 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 129 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 130 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 131 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 132 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 133 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 134 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 135 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 136 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 137 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 138 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 139 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 140 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 141 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 142 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 143 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 144 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 145 PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 146 ANNEX D: MEMO EXPLAINING RISKS TO RCT DESIGN FROM CHANGING EVALUATION TIMELINE Options Memorandum: Impact Evaluation of the Land Tenure Assistance Activity in Tanzania This memorandum was prepared at the request of the Office of Land and Urban in USAID’s Bureau for Economic Growth, Education, and Environment (E3/LU). It summarizes two options for E3/LU’s consideration for moving forward with the ongoing impact evaluation (IE) of the Feed the Future Tanzania Land Tenure Assistance (LTA) activity, given recent unanticipated changes in LTA activity implementation that present significant challenges for completing the IE as planned. The E3 Analytics and Evaluation Project (“the Project”) is implementing the IE. This memorandum begins with an overview of the LTA implementation changes, then summarizes the original IE design and timeline, the key methodological challenges created by the LTA implementation changes, the two options for proceeding with the IE given the LTA implementation changes, and updated estimated budget information for the IE. These two options are: • Option 1: Adhere to the original, approved IE design but have all remaining IE activities occur six months earlier than planned, and take steps to ensure that the IE sample includes a full roster of villages as per the approved design. • Option 2: Proceed with six-month accelerated IE timeline as in Option 1, but with a reduced sample of villages. LTA Implementation Changes On August 9th, USAID informed the Project team of two significant and unexpected changes in activity implementation based on recent decisions the implementation team had taken. First, LTA intends to have implementation in its next set of target villages occur approximately five months earlier than previously discussed with the Project team. The change in the LTA timeline is being proposed after the Project team completed the first round of IE baseline data collection and analysis, and despite known challenges that such changes create for the IE, which the Project team has repeatedly stressed in conversations with the LTA implementation team over the past year. Second, LTA and the Iringa District Land Office (DLO) have ruled out 8 of the remaining villages in the master list used to determine the IE sample, leaving 27 villages – which is below the minimum threshold that the IE design requires. The Approved Timeline and Evaluation Design The Project team’s approved IE design, developed in coordination with USAID and LTA in 2016, is based on a cluster randomized controlled trial approach that has IE data collection taking place prior to LTA implementation in two phases, as shown in Table 29. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 147 TABLE 29: APPROVED TWO-PHASE IE DESIGN AND LTA IMPLEMENTATION SCHEDULE Phase Implementation Year Control Treatment 1 2017-2018 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA 2 2018-2019 15 randomly chosen villages do not receive LTA 15 randomly chosen villages receive LTA The Project team completed Phase I baseline data collection in April 2017, randomly selecting 30 villages (and 2 buffer villages) from a list of 78 villages approved by LTA and the Iringa DLO. LTA, with input from the Iringa DLO, subsequently removed several villages from this list of 78 due to the potential challenges to LTA implementation, leaving 36 villages available for random assignment in Phase II. Per the approved IE design, Phase II baseline data collection – which also includes midline data collection for the Phase I households – was planned for March-April 2018, approximately one￾year after the Phase 1 baseline. LTA’s decision to accelerate activity implementation would require that IE data collection for Phase 2 occur around late October 2017. Methodological Considerations for the Options One of the most important contributions of this IE is its rigorous design, since there have been few experimental studies on the impact of land formalization to date. Thus, the Project team sought to develop options in response to these LTA implementation changes that would preserve as much of the IE’s rigor as possible. Three methodological considerations need to be kept in mind for each of the options presented: • Data Collection Timing: All IE baseline data collection in Phase II villages must occur prior to LTA implementation activities in those villages, regardless of the timeline for implementation. Otherwise, the IE will not be able to estimate LTA’s impact because it could not convincingly show that treatment villages would have been similar to the control villages had they not received the activity. Also, the Project team learned during Phase I baseline data collection that LTA started sensitization activities in two treatment villages prior to the IE baseline being conducted there. Going forward, it is critical that no additional implementation activities that involve LTA staff interacting with treatment villages take place before IE baseline data collection is completed. • Ability to Detect an Effect: The IE design uses a panel survey, with respondents interviewed at the same time of year before, during, and after LTA implementation to rigorously estimate LTA’s impact and compare it to villagers in the control group. The requirements to survey households at the same time of year and to conduct a midline survey of Phase I households are critical for the statistical power of the IE (i.e., its ability to detect an effect where one occurred). Changing the timeline for baseline data collection, and potentially reducing the number of villages included in the IE, would dramatically reduce the rigor of the IE design and increase the likelihood that the evaluation will not be able to detect any impact of the LTA interventions. While the IE can attempt to address the timeline change through statistical weighting and other approaches during analysis, any estimation of impact will be sensitive to the estimation methods beyond what was originally proposed and it is doubtful that the IE could make up for the loss of statistical power that would result from these implementation changes. • Bias: The new LTA timeline will introduce bias into the responses of household survey respondents, given the very different survey contexts. Phase I baseline took place during the rainy season in Iringa District, but if baseline data collection for Phase II takes place in late October it would be the dry season in Iringa, during which village life and activities differ. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 148 The variance in responses between rainy and dry seasons, as well as the recall bias from people answering questions about spending, harvesting, and disputes, will also present estimation challenges during analysis. The IE’s ability to control recall bias (e.g. respondents remembering with more precision their harvest amounts in October as compared to March), and even the perception of the survey at a different time of the year, are difficult to fully account for in the analysis and will likely limit the comparisons that can be made between the first and second baseline groups. Option 1: Shifted Timeline, Full Village List The first option identified by the Project team is to shift the timeline for Phase II baseline data collection from March-April 2018 to October-November 2017, as well as have the Project team and USAID work with LTA and the DLO to ensure that 30 villages are available for Phase II data collection and LTA implementation (i.e., 15 treatment villages and 15 control villages). Option 1 still presents the following challenges and risks: • Bias from time-inconsistent responses: Instead of collecting data from comparable groups at the same point in time in years one, two, and three of the study, the IE would have a full dataset of Phase I survey responses that are from a different context and limited in their comparability to Phase II. • Risk to power: The ability to detect an effect based on the number of villages dictated by the IE design assumed that a panel survey would occur over three time periods (baseline, midline, and endline). The challenge for Option 1 is that period 1 and period 2 will differ in critical ways, namely that village life during the rainy and dry seasons is driven by different activities, and the gains to power by having three comparable periods of data collection may be diminished since the data may no longer be comparable due to seasonal differences. The Project team would need to conduct additional data simulation exercises to determine exactly what effect this will have on the IE’s ability to detect an impact. While Option 1 would not overcome the potential bias from time-inconsistent responses, it could allow for the IE to detect impact for outcomes where the effect size is large. Should USAID wish to proceed with Option 1, it is critical that the following occur: • The Iringa DLO and LTA would need to agree to expand the village list for Phase II to a minimum of 32 villages (which includes two buffer villages should LTA encounter issues in the selected villages). Also, all villages must also be assigned to the treatment group at the same time; once villages have been assigned to treatment or control groups, they cannot be re-assigned nor can villages get added to the sample ex-post. • The IE would still need to conduct the midline survey of Phase I villages, since the original IE design is based on collecting data from all villages at the same time of year over three phases. Thus, Phase II data collection in October-November 2017 would need to include a midline survey of all 750 households from the Phase I baseline, as well as a baseline survey of the additional 750 Phase II households. • The IE team would need to revisit its survey instrument to ensure that reference points included in the original survey are consistent with the new timeframe (e.g., “in the past rainy season” previously referred to 2016, but respondents would likely reference the 2017 rainy season in October). While Option 1 preserves as much of the rigor of the original IE design as possible given the LTA implementation changes, the internal validity of the IE would still be diminished because of the changing period for midline data collection for Phase I, which in the original IE design helped the IE’s statistical power by increasing the number of observations and time periods of observation. PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 149 Option 2: Shifted Timeline, Diminished Village List The second option identified by the Project team is similar to Option 1 and includes the same limitations, but entails greater risk and challenges as it would only use the current list of 28 remaining villages to randomly assign to treatment and control groups. Under Option 2, in October￾November 2017 the IE would still conduct a midline survey of the Phase I villages and would survey the reduced number of villages as part of the Phase II baseline data collection. Option 2 faces the following challenges: • Risk to power: The IE would collect data on 58 instead of 60 villages, and it would require a minimum of a 21 percent change in outcomes between treatment and control under the original design.35 The IE’s ability to detect an effect cause by LTA given the reduction in villages and the time change is difficult to estimate, and the Project team would need more time for further data simulation. However, it is unlikely that the IE would be able to reliably detect LTA’s impact for outcomes that under the original IE design were already on the margins of being sufficiently statistically powered, such as women’s empowerment outcomes. The challenge with reducing the number of villages and changing the timeline is that any estimate of impact would be difficult to differentiate from random noise, become highly sensitive to variance in the data, and be highly contingent on researcher estimation techniques.36 Option 2 would, however, save time by not revisiting villages that were removed from the master list in mid-2016. Further sensitivity to implementation issues: Option 2 leaves little to no room for further LTA implementation challenges and changes. If LTA encounters an issue in one of the randomly selected treatment villages and cannot fully implement there, the probability that the IE will be able to detect an effect for even the largest impacts will be significantly lower since there will be no buffer villages from which to choose. 35 Intra-cluster correlation coefficient: 0.05. 36 This is particularly an issue with studies that have poor or compromised designs, with little clear estimation strategy. See Gelman, Andrew and Eric Loken, “The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or ‘p-hacking’ and the research hypothesis was posited ahead of time.” Department of Statistics, Columbia University (2013). PHASE II BASELINE AND PHASE I MIDLINE REPORT: IMPACT EVALUATION OF THE FTF TANZANIA LTA ACTIVITY 150 ANNEX E: COMPARISON OF PHASE I AND PHASE II BASELINE DATA Table 30 shows the overall Phase I and Phase II averages across baselines. More respondents reported disputes during Phase I (n = 68), but there were a greater number of disputes overall reported in Phase II; as mentioned previously, several Phase II respondents reported having more than two disputes. The Phase II respondents report more household members on average, as well as a higher range of household members. Notably, education level, age, and cooperative membership were similar across both phases. TABLE 30: COMPARISON OF PHASE I AND PHASE II BASELINE DATA *0 = No schooling, 1= Primary, 2 = Form, 3 = University, 4 = Diploma Variable Phase II Phase I n Mean SD Min Max n Mean SD Min Max Age 1320 45.36 14.99 18 102 1179 47.27 16.08 18 101 Number of parcels owned 788 2.38 1.27 1 11 782 2 1.02 1 8 Parcel size (in hectares for all parcels owned) 788 3.8 6.77 0.1 74.66 1179 1.64 3.85 0 86.6 Cooperative membership (y/n) 788 0.23 0.42 0 1 782 0.2 0.4 0 1 Education Level* 1320 0.85 0.47 0 3 782 0.88 0.52 0 4 Have you been faced with a situation when you did not have enough food to feed the household? (1=Yes, 0=No) 788 0.24 0.43 0 1 782 0.33 0.47 0 1 Possess land related documentation (1=Yes, 0 = No) 1320 0.12 0.33 0 1 1179 0.15 0.36 0 1 Heard of CCROs 788 0.68 0.47 0 1 782 0.58 0.49 0 1 Number of HH Members 788 4.42 2.32 1 26 782 3.95 1.95 1 12 Experienced a dispute in the past year (1=Yes, 0 = No) 788 0.07 0.25 0 1 782 0.09 0.28 0 1 Number of reported disputes 52 1.19 0.63 1 5 68 1.09 0.29 1 2 Do you have familiarity with land laws (1=Y, 0=N) 1320 0.08 0.27 0 1 1179 0.03 0.18 0 1