This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by International Solutions Group. Foreign Agricultural Service, United States Department of Agriculture Photo of a FINCA outlet in Wakala, Tanzania Tanzania Food for Progress Project FINAL EVALUATION February 2017 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Final Evaluation Report FINCA Tanzania Food for Progress The United States Department of Agriculture (USDA) Food for Progress (FFPr) project in Tanzania was implemented by FINCA International from 2012 to 2016. The project aimed to expand access to agricultural finance. Agreement Number: FCC-621-2012-044-00 Project Duration: 2012-2016 Implemented by: FINCA International Evaluation Authored by: International Solutions Group Evaluation Project Supervisor: Jason Wares Evaluation Team Leader: Christopher Root FINAL EVALUATION REPORT FINCA TANZANIA FOOD FOR PROGRESS PREPARED BY INTERNATIONAL SOLUTIONS GROUP PREPARED FOR FINCA International 23 February 2017 i Table of Contents List of Tables ................................................................................................................................ ii List of Figures .............................................................................................................................. iii Executive Summary ..................................................................................................................... iv Evaluation introduction ................................................................................................................. iv Findings ......................................................................................................................................... iv Select lessons learned and recommendations .............................................................................. vi 1. Introduction ............................................................................................................................. 1 Food for Progress Tanzania ................................................................................................................ 1 2. Evaluation Purpose, Questions, and Methodology .................................................................... 3 Purpose ............................................................................................................................................... 3 Evaluation Questions .......................................................................................................................... 3 Evaluation methodology ..................................................................................................................... 4 Document review ........................................................................................................................... 4 Qualitative research ....................................................................................................................... 4 Survey ............................................................................................................................................. 5 Limitations ...................................................................................................................................... 5 3. Findings ................................................................................................................................... 7 EQ1: Progress towards PMP targets ................................................................................................... 7 Impact results ................................................................................................................................. 7 Activity results ..............................................................................................................................12 EQ 2: Pamoja loan product ...............................................................................................................13 Background...................................................................................................................................13 Client perspective .........................................................................................................................16 EQ 3: Branchless channels ................................................................................................................16 FINCA Express ...............................................................................................................................16 FINCA Mobile ................................................................................................................................17 Use and access for both services ................................................................................................. 17 Client perspective – FINCA Express ..............................................................................................18 Client perspective – FINCA Mobile ...............................................................................................19 Other potential services ...............................................................................................................20 EQ 4: Training and technical assistance............................................................................................20 Client perspective .........................................................................................................................21 EQ 5: New branches .........................................................................................................................22 Client perspective .........................................................................................................................23 EQ 6: Client perspective ...................................................................................................................23 4. Conclusions, Lessons Learned, and Recommendations ............................................................ 24 Pamoja loan product ........................................................................................................................24 Lesson learned: ............................................................................................................................24 Lesson Learned: ............................................................................................................................25 Lesson learned: ............................................................................................................................25 Recommendation: ........................................................................................................................25 Lesson learned: ............................................................................................................................25 Recommendation: ........................................................................................................................25 Recommendation: ........................................................................................................................26 Branchless channels .........................................................................................................................26 Lesson learned: ............................................................................................................................26 Lesson learned .............................................................................................................................26 Recommendation .........................................................................................................................27 Lesson learned: ............................................................................................................................27 ii Recommendation: ........................................................................................................................27 Training and technical assistance .....................................................................................................27 Lesson learned .............................................................................................................................27 Recommendation: ........................................................................................................................28 New branches ...................................................................................................................................28 Progress towards PMP targets .........................................................................................................28 Lesson learned: ............................................................................................................................28 Recommendation: ........................................................................................................................29 Annexes..................................................................................................................................... 29 Annex I: References ................................................................................................................... 30 External references......................................................................................................................30 FINCA documents and data.........................................................................................................30 Annex II: List of Acronyms and Abbreviations ............................................................................. 31 Annex III: KIIs, FGDs and Sites Visited ......................................................................................... 32 Annex IV: Composition of Evaluation Team................................................................................. 33 Annex V: Indicator Performance Tracking Tables ........................................................................ 34 Annex VI: Methods .................................................................................................................... 36 Document review .........................................................................................................................36 Qualitative research .....................................................................................................................36 Survey ...........................................................................................................................................36 Annex VII: Survey tools .............................................................................................................. 37 Question checklist for KIIs – Clients............................................................................................ 37 Question checklist for KIIs – Staff and agents ............................................................................38 Focus Group Discussion Guide ....................................................................................................40 Client survey ................................................................................................................................43 Annex VIII: Evaluation Scope of Work ......................................................................................... 48 List of Tables Table 1: Summary of Branches Visited ................................................................................................... 5 Table 2: KII and FGD Interviewees by type and location ........................................................................ 5 Table 3: Impact Result - Increased Agricultural Productivity .................................................................. 7 Table 4: Intended use of loan ................................................................................................................. 8 Table 5: Loan success .............................................................................................................................. 8 Table 6: Impact Result - Increased use of financial services ................................................................... 8 Table 7: Borrowing, saving and use of financial institutions in the past year (%) .................................. 9 Table 8: Impact Result - Improved Farm Management and Productive Activities ..............................11 Table 9: Impact Result - Expanded Trade of Agricultural Products ......................................................11 Table 10: Impact Results - Improved Transaction Efficiency & Improved Market and Trade Infrastructure ........................................................................................................................................11 Table 11: Activity Results ......................................................................................................................12 Table 12: Income increase ....................................................................................................................15 Table 13: Percentage of transactions by service ..................................................................................17 Table 14: Time to Branch of Agent in Minutes .....................................................................................18 Table 15: How well FINCA Express meets clients’ needs......................................................................19 Table 16: How well FINCA Mobile meets clients’ needs.......................................................................20 Table 17: Training impact on farm or business .....................................................................................21 Table 18: Percent income increase associated with loan by success in achieving loan goal ...............21 Table 19: Percent of Agricultural Borrowers by Branch .......................................................................22 Table 20: Comparison of new branch clients with old branch clients ..................................................23 iii Table 21: Likelihood to recommend FINCA ..........................................................................................24 List of Figures Figure 1: Sources of loans for Tanzanian borrowers .............................................................................. 9 Figure 2: Alternative sources of loans ..................................................................................................10 Figure 3:Chicken coop facilities purchased with FINCA loan ................................................................11 Figure 4: Pamoja Loan use by type of business ....................................................................................14 Figure 5: FINCA client fishmonger and customer in Bunda...............................................................15 Figure 6: FINCA Express outlet in Tanga ...............................................................................................17 Figure 7: Client record book..................................................................................................................20 Figure 8: New FINCA Branch in Arusha ................................................................................................. 22 Figure 9: Reasons for choosing FINCA over competitors ......................................................................23 iv Executive Summary Evaluation introduction The United Stated Department of Agriculture (USDA) Food for Progress Program (FFPr) in Tanzania was implemented by FINCA from 2012 to 2016. The program aimed to expand access to agricultural finance through six components: 1. Develop a new lending product 2. Establish new bank branches 3. Establish new financial service delivery channels 4. Promote savings accounts and mobilize deposits 5. Provide agricultural loans 6. Research client satisfaction and social impact FINCA International commissioned this independent end-of-project evaluation to identify project successes and areas for improvement. Specifically, this evaluation addresses eight evaluation questions (EQs) put forth by FINCA and summarized below. 1. How well did the project do versus its performance management targets? 2. How effective was the new agricultural loan product? 3. Are the new branchless banking technologies improving access to finance? 4. How effective is client training and technical assistance? 5. Are the new branches well located and equipped to serve agricultural clients? 6. What is the perspective of clients on the new services offered through FFPr? 7. What are the lessons that can be learned from FFPr? 8. What were the most and least effective activities under FFPr? This evaluation addresses evaluation questions 1-6 individually through separate subsections. Evaluation questions 7 and 8 are addressed in conclusions, lessons learned and recommendations. These evaluation questions were answered using mixed methods, including a review of FINCA’s documentation and data, fieldwork in Tanzania to conduct key informant interviews and focus group discussions, and a 400 completed client surveys. Findings EQ1: Progress towards PMP targets Seventy-Five percent of FINCA clients say that if they had not had access to finance through FINCA, they would not have had an alternative source of finance for their investment. The FINCA loans that these clients accessed are increasing their incomes significantly, as the first bullet point below shows. This finding indicates that, through its agriculture-focused financial services, FINCA is expanding clients’ access to finance in a way that leads to increases in income. FINCA exceeded its impact result targets with few exceptions. In many cases, targets were exceeded by a significant margin. Highlights of FINCA’s impact performance include:  93% of on farm and 95% of off farm agriculture clients report that their income increased at least 5% as a direct result of borrowing from FINCA  61% of clients report improved on-farm management against a target of 50%  82,381 of savings and credit clients utilize branchless banking delivery channels against a target of 46,500 v FINCA met or exceeded nearly all targets for FFPr activity results. Highlights include:  Providing 65,600 new agricultural lending product loans against a target of 10,000  Establishing 124 point of sale banking outlets against a target of 87  124,254 registered savings accounts with total savings of $8,500,000 against a target of 35,000 accounts with a value of $2,000,000 However FINCA fell short in one area – opening new branches. FINCA opened four branches instead of the six mandated that the FFPr grant agreement mandated EQ 2: Pamoja loan product After several pilot projects testing different approaches to agricultural lending, FINCA designed the Pamoja loan product. Pamoja is a group lending product. Group lending is a lending practice whereby small groups of people borrow money collectively and encourage each group member to repay their loans. FINCA clients form a group of five to ten business people that apply for a Pamoja loan from FINCA. The group’s borrowers own diverse types of farms, off-farm agriculture businesses, and non-agricultural businesses. In fact, individual borrowers may own more than one of these three revenue sources. Building borrower-groups that have a variety of businesses successfully diversifies Pamoja loans away from the highly correlated risks that plague pure agricultural lending. Other design elements in Pamoja that are based on lessons learned from pilot projects include larger loan limits, group decision-making on loan sizes for members, and an optional three month grace period to accommodate agricultural seasonality. For the most part, these features are successful and appreciated by clients, although group decision making on loan sizes can inefficiently stymie business growth for some borrowers (see more below). One unpopular feature is the 28-day repayment instead of the client-preferred monthly repayment. FINCA is looking to address this source of client dissatisfaction. Clients overwhelmingly associate their Pamoja loan with increased income. In fact, 95% of FINCA’s clients reported through this evaluation’s survey that their income increased by at least 5 percent as a result of their loan, and nearly a quarter reported more than a 30% increase in income. EQ 3: Branchless delivery channels FINCA developed two branchless delivery channels - FINCA Express and FINCA Mobile – to expand access to finance for eligible agricultural clients. FINCA Express involves a decentralized network of agents offering FINCA banking transactions out of their small shops. FINCA Mobile is a mobile phone application that allows FINCA clients to make withdrawals and deposits electronically. FINCA Express has been extremely successful; expanding to 124 agents in two years. FINCA Express’s decentralized locations bring banking closer to clients (23 minutes closer for survey respondents) and transactions appear to happen quicker due to FINCA’s biometric, paperless technologies and shorter queues than appear at branches . FINCA Mobile has had less success than FINCA Express. FINCA Mobile use plateaued at about 10% of FINCA’s transactions in 2016 with some clients expressing concern about the fees as well as occasional delays in transfers. FINCA is working with its telecommunications counterpart to resolve this latter issue. In the evaluation’s survey, clients gave high satisfaction rankings to both branchless services. Eighty￾four percent of FINCA Express users said it met their needs “very well” or “well” while 86% of FINCA Mobile users responded the same. vi EQ 4: Training and technical assistance FINCA overhauled its training program for Pamoja to focus more on group strengthening. Clients are positive about the three-day training curriculum with 71% responding that it completely or mostly improved their farm or business management. Some clients however wished for technical support, especially for agriculture which their loan officer was unable to provide. Clients who were successful in achieving the original objective they had when they took their loan reported significantly higher incomes than those who were not successful. EQ 5: New bank branches According to the contract with USDA, FINCA was to open six new bank branches in rural areas to serve agricultural clients. FINCA built four new branches, which it located in agricultural trade hubs rather than rural areas. Two of the branches only offer the urban version of Pamoja, Pamoja Plus. These branches serve a lower number of agricultural borrowers than regular Pamoja branches. FINCA intended for these branches to be modern flagship branches to establish them as more professional than their competitors. Based on the evaluation team’s branch visits, this goal was achieved. EQ 6: Client perspective Overall the evaluation found that clients are very satisfied with FINCA; 95% said they would come back to FINCA if they needed another loan and 92% said they would recommend FINCA to someone else. Advantages for FINCA over other lenders include better location, no collateral requirement, monthly instead of weekly repayments, higher loan ceilings, and savings accounts. Select lessons learned and recommendations Pamoja loan product Lesson learned: FINCA’s innovative Pamoja loan product provides finance to many (75% of clients) borrowers who would otherwise not have access to finance. The vast majority of these borrowers experience significant income increases as a result. Lesson learned: Because the group decides loan size, the loan size of entrepreneurial borrowers may be constrained by the risk preference of less entrepreneurial group members. This constrains the growth potential of budding entrepreneurs. Recommendation: FINCA should create a fund within Pamoja to share risk with the group for vetted, high potential borrowers. Lesson learned: While progress was made in on-farm agricultural lending, the majority of Pamoja loans still went to off farm business. On-farm agricultural lending risk is still high, even when diversified with off farm agriculture. Recommendation: Value chain lending, which FINCA is already exploring, would diversify risks across more actors (farmer, buyer and FINCA). Technical assistance could further mitigate risk. Recommendation: FINCA could also decrease risk by diversifying lending across agricultural products. Research could quantify these risks and risk correlations by product and identify risk-optimized portfolio strategies. vii Branchless channels Lesson learned: FINCA Express has been highly successful and looks likely to continue to expand and bring lower cost financial transactions to more clients. Lesson learned: Over two thirds of FINCA clients not using either FINCA Express or FINCA Mobile said they had not heard of them. Additionally, some FINCA Express agents expressed a desire to learn more how to market FINCA to prospective clients. Recommendation: Add a module to the Pamoja training manual on these branchless channels and how to use them. Provide FINCA Express agents with a bi-annual orientation on FINCA products and services and how to market them. Training and technical assistance Lesson learned: Clients have positive views about training but some wish for technical assistance to help them implement their loan projects more effectively. FINCA has attempted to partner with agricultural technical assistance providers (such as NGOs) previously, but was unsuccessful in large part due to insufficient scale of overlap between partner constituents and FINCA clients. Recommendation: USDA and other donors can facilitate these partnerships by requiring partnerships in place at the proposal stage and through national convening events between finance providers and NGOs. Progress towards PMP targets Lesson learned: FINCA exceeded many of its performance targets. However, FINCA failed to collect and organize some of its required performance data. USDA requires at least 3% of funding to be allocated to M&E; however, it is not apparent if this target was met or if the minimum was sufficient. Recommendation: USDA should ensure that projects have an explicit M&E budget, including for a full-time M&E staff. 1 1. Introduction The Food for Progress Program (FFPr), managed by the United States Department of Agriculture (USDA) Foreign Agricultural Service (FAS), assists “developing countries and emerging democracies (to) modernize and strengthen their agricultural sectors1 .” It currently funds 106 projects in 39 countries. FFPr funds programs through Commodity Credit Corporation funds and by donating commodities to international development organizations. These organizations then sell the commodities and use the proceeds to implement country-level projects. Food for Progress Tanzania As is true in many of Sub-Sahara Africa’s developing countries, Tanzania’s agriculture sector is crucially important to its economy. The agriculture sector employs 67% of Tanzania’s labor force and produces approximately 31% of the country’s GDP. Many of the people employed in the sector are smallholder farmers with under 2 hectares of land. While agricultural productivity is growing, the pace is not rapid enough to significantly reduce rural poverty. Tanzania’s farmers face constraints similar to those faced by farmers throughout the developing world. These constraints include low input use leading to low yields, poor irrigation, insufficient agricultural road networks, inadequate extension services, and limited processing capacity.2 The United States Department of Agriculture (USDA) Food for Progress (FFPr) Program, implemented by FINCA Tanzania (FINCA) from September 2012 to September 2016, aimed to provide financial services to Tanzania’s agriculture sector. This was intended to increase agricultural productivity and trade, ultimately increasing individuals’ incomes. FINCA generated $8.3 million dollars in proceeds selling FFPr commodities and used these funds for its agricultural lending program in Tanzania. Improving access to finance in agriculture has long been recognized as a priority in Tanzania and was prioritized in a statement by the Bank of Tanzania in 1979. However, the percentage of credit going to the agricultural sector actually declined from nearly 14% in 2004 to just under 12% in 20123 . In 2013 the Government of Tanzania’s Agricultural Policy reiterated the country’s commitment to expanding access to finance in agriculture.4 Financing agriculture in Tanzania is a difficult and expensive undertaking. Nearly 70% of the population is rural.5 Much of this population lives dozens of miles from bank locations, connected only by dirt roads and public transportation. Additionally, banks often have collateral requirements and interest rates that are prohibitive for agricultural workers or entrepreneurs. In designing FFPr 1 United States Department of Agriculture Foreign Agricultural Services (USDA FAS). 2017. Food for Progress. Accessed at: http://www.fas.usda.gov/programs/food-progress 2 Ministry of Agriculture, Food Security and Cooperatives, The United Republic of Tanzania. 2013. National Agriculture Policy. Accessed at: http://www.fao￾ilo.org/fileadmin/user_upload/fao_ilo/pdf/ICA_MLW_and_TZ/NATIONAL_AGRICULTURAL_POLICY￾2013.pdf 3 Stein, Howard et. al. 2016. The Formal Divide: Customary Rights and the Allocation of Credit to Agriculture in Tanzania. The Journal of Development Studies, 52(9). 4 Same as footnote 2. 5 Food and Agriculture Organization (FAO). 2016. AQUASTAT – United Republic of Tanzania. Accessed at: http://www.fao.org/nr/water/aquastat/countries_regions/TZA/ 2 Tanzania, USDA and FINCA worked together to develop a strategy that would overcome the cost and logistical obstacles to financial access for Tanzania’s agricultural sector. Specifically, FFPr funds went to implement the program’s six components: 1. Develop a new lending product. FINCA was to design, pilot and scale up a new agricultural lending product with terms suitable for agriculture. After several iterations during the pilot phase, the lending product evolved into the Pamoja Loan product (Pamoja). Pamjoa is distributed through groups of 5-10 borrowers. The group consists of a mix of agricultural and non-agricultural borrowers. 2. Establish new bank branches. FINCA was to use FFPr funds to establish six new branches in rural areas.6 3. Establish new financial service delivery channels. FINCA was to establish two new financial service delivery channels to expand access to finance: point of sale service locations and mobile phone banking. These channels were branded FINCA Express and FINCA Mobile respectively. 7 4. Promote savings accounts and mobilize deposits. FINCA was to promote and mobilize savings and deposit accounts using its exiting village bank network. A significant part of this was an overhauled training program. 5. Provide agricultural loans. The bulk of the FFPr funding was designated as loan capital for agriculture and food related borrowers. This loan capital and interest payments were to be owned by FINCA and reinvested during the project and after its completion. 6. Research client satisfaction and social impact. FINCA was to research customer satisfaction both to inform and improve its operations and to measure quality of life improvements resulting from FINCA loans. FINCA was to report results to USDA Through these six components, FINCA goal was to produce the following results:  Increased agricultural productivity  Increased use of financial services for men and women, on and off the farm  Improved farm operational and financial management  Expanded trade of agricultural products  Increased market access for the sale of agricultural products  Improved market and trade infrastructure  Improved transaction efficiency Achieving these results would expand agricultural production, increase sales and income, generate employment, and improve quality of life for women and men in Tanzania’s important agricultural regions. The definitions, targets and achievement of indicators to measure project performance can be found in section 3. 6 Grant Agreement FCC-621-2012/044-00 dated 12 September 2012 and amended on 9 August 2015 7 An initial plan for ATM services was scraped with USDA approval. 3 2. Evaluation Purpose, Questions, and Methodology Purpose FINCA International selected International Solutions Group (ISG) to perform a final evaluation of its FFPr Tanzania project. The purpose of this evaluation was “to assess whether the project has achieved the expected results as outlined in the results framework”.8 The evaluation documents findings, lessons learned and recommendations to assist stakeholders in sustaining and scaling Tanzania FFPr going forward as well as improve the design and implementation of similar FFPr projects in the future. Evaluation Questions This evaluation answers the eight evaluation questions (EQs) put forth in the terms of reference. 1. To what extent have the planned objectives been achieved for the program? Specifically, what progress did the project make on achieving PMP targets? 2. Are the terms of the new Agricultural Loan appropriate and effective? How can the product be strengthened? 3. Is Branchless banking technology improving access to financial services for rural and agriculture dependent communities? How can the mobile delivery of financial services be strengthened? 4. Are the Technical Assistance training materials and delivery appropriate? Which materials and/or delivery techniques need strengthening, if any, and how? 5. Are new rural expansion offices appropriately located and equipped to serve rural and agriculture dependent populations? How can they be improved with respect to operational efficiency and effectiveness? 6. What is the perspective of beneficiaries (both FINCA clients and training participants) with regards to the services provided under the Food for Progress program? 7. What are lessons learned from this project and how can they be addressed in future, similar programs? 8. Which Food for Progress project activities were the most effective in increasing agricultural production and strengthening trade of agricultural products? Why? Which program components were the least effective? Why? Questions 1 to 6 are each addressed with separate sub-sections in the Findings section (section 3). Question 6 is also addressed as part of questions 2 to 5. Section 4 in this report documents lessons learned (question 7) and in so doing identifies the activities the evaluation team believes were most and least effective (question 8).9 8 Final Evaluation Tanzania Terms of Reference 9 Note that the evaluation questions do not explicitly address the savings component of the project and therefore savings is only indirectly addressed throughout the evaluation. 4 Evaluation methodology The evaluation was carried out between November 2016 and January 2017, with fieldwork taking place from November 28 through December 7, 2016. The evaluation was carried out using mixed methods. This includes document review, qualitative research including key informant interviews (KIIs) and focus group discussions (FGDs), and a survey of 400 of FINCA’s clients. The mixed methods approach allowed the evaluation team to verify results through multiple sources of information. Document review The evaluation team undertook an extensive document review that informed all stages of the evaluation. The team used the document review initially to learn about the project and inform data collection tools. Later the review was useful in explaining different features of program design. It was also the source of most of the data the evaluation team used to compare performance results against targets. Documents reviewed include the following: loan portfolios, bi-annual project narratives, the mid￾term evaluation, loan training manuals and guidelines, other project technical documents, and background documents on Tanzanian agriculture and finance. A full list of documents reviewed can be found in the annexes. Qualitative research The evaluation team conducted key informant interviews (KIIs) and focus group discussions (FGDs) to provide qualitative information. The team used qualitative field research to create a narrative that provided context to survey findings and detail to evaluation question answers. Because the evaluation survey featured mostly closed-ended questions to gauge satisfaction (described below), the qualitative field research focused on explorative open-ended questions. The evaluation team asked questions to expand the its understanding of the FFPr project, answer evaluation questions, verify the team’s tentative hypotheses, and confirm information gleaned from other sources. Question guides were used to lead semi-structured interviews. The guides ensured that the evaluation team asked the same key questions across similar respondent types while allowing the freedom for conversational-style interviews. These guides are included in the annexes. Qualitative field research was conducted in six locations (FINCA Tanzania headquarters in Dar es Salaam and five branch locations) over the course of the evaluation’s team 10 days in country. The five branches were selected to capture a cross section of branch features and agricultural client types. Table 1 summarizes these five bank locations. 5 TABLE 1: SUMMARY OF BRANCHES VISITED Branch Branch feature Agricultural client types Morogoro USDA funded Crop production and retail Tanga Crop and livestock production, fish retail Arusha USDA funded; Pamoja + Crop and livestock trade, manufacture and retail Mara Fishing and fish retail Bunda Cashless Fishing and fish retail The evaluation team conducted 36 KIIs with branch staff, clients and agents at these five branches. The team also carried out two focus groups with a total of 22 clients. Additionally, four FINCA Tanzania headquarter staff were interviewed. Interviewees are summarized by type in table 2 below. TABLE 2: KII AND FGD INTERVIEWEES BY TYPE AND LOCATION Branch Clients Agents Branch staff HQ staff Morogoro 13 2 8 Tanga 13 1 4 Arusha 3 1 6 Mara 4 Bunda 2 1 Dar es Salaam 4 Total (female) 31 (17) 4 (1) 23 (10) 4 (0) Survey The evaluation also includes a brief client phone survey to collect client perceptions with statistical validity. Phone surveys are a low-cost option to reach respondents with straightforward questions. Because no travel is involved in a phone survey, the evaluation used simple random sampling rather than cluster sampling. The survey team randomly selected respondents from FINCA’s client list of 27,333 active Pamoja loan clients. Enumerators then called clients on the list in order until 400 interviews were complete. This final sample size results in a margin of error of 5% with a confidence level of 95%. The questionnaire used for this survey can be found in annex 1. Limitations To make the most of the limited time available for fieldwork, the evaluation team visited five of FINCA’s 26 branches in three of the six regions in which FINCA operates. While effort was made to choose representative branches, travel times between branches combined with the limited fieldwork schedule made this objective challenging. However, the evaluation team consulted with FINCA Tanzania staff to ensure that the five branches visited were broadly representative of the types of agricultural clients FINCA serves throughout Tanzania. Interviews with clients were arranged by FINCA staff and therefore it is possible selection bias was introduced through the selection of successful clients who had previously provided positive feedback. As a counter check, the evaluation team conducted one FGD and several KIIs in remote 6 areas and with more disadvantaged clients. Another potential source of bias is interviewee time availability. Busier clients and agents are less likely to be available for interviews and may have different views of the FINCA, on average, than less busy clients. The evaluation team attempted to mitigate this issue by visiting many clients, visiting all agents at their place of work, and by financially compensating those clients who did come to meet the team for the client’s time and expenses. Phone surveys have certain, inherent limitations. Respondents’ attention is limited in phone interviews, and it is more difficult for interviewers to explain questions. Therefore the questionnaire was shorter and featured simpler questions than if interviews had been conducted face to face. This weakness was mitigated by the evaluation’s in-depth qualitative research and high-level statistically valid findings. Phone surveys also are weakened by how people choose to use their phones. Though respondents were offered phone credit for completing the survey, they did not know of this offer before they picked up the call and may not wanted to have used phone credits on an unfamiliar caller. It is also possible that clients called during working hours may have been unable to answer their phones. Also, a small portion of FINCA clients do not have cell phones and would have been excluded from the survey. 7 3. Findings EQ1: Progress towards PMP targets This section discusses findings for two types of indicators: impact and activity results performance indicators. Impact results Impact results identify how FINCA’s activities and outputs affected the performance of client businesses and improved clients’ quality of life. FINCA tracked progress toward and achievement of many of the impact results. In those cases, the evaluation team reviewed documentation and used KIIs and FGDs to verify FINCA’s records. In the case of impact results that FINCA did not track, the evaluation team attempted to estimate progress toward impact results using FINCA documents, the evaluation survey, and qualitative data collection. The FFPr grant agreement identified seven impact results that the project aimed to achieve (shown below). We discuss each result in the remainder of this sub-section.  Increased agricultural productivity  Increased use of financial services  Improved farm management  Expanded trade of agricultural products  Increase access to markets to sell agricultural products10  Improved market and trade infrastructure  Improved transaction efficiency TABLE 3: IMPACT RESULT - INCREASED AGRICULTURAL PRODUCTIVITY Indicator Target Achieved Average annual percent increase in income of all on-farm clients 5 93% over 5% increasea Number of individuals benefiting directly from USDA-funded interventions 65,000 76,210b Number of clients reporting a positive unit change in their land, animal, or selected product 3,000 >5,301c Sources: a. Evaluation survey; b. FINCA documentation; c. Mid-term evaluation FINCA did not track average annual percent increase in income for on or off farm clients. In order to estimate income increases, we asked survey respondents ““Did the loan you obtained from FINCA increase your income?” If the respondent answered yes to the question, the respondent was then asked “How much did your income increase?” Respondents were asked to select from a set of 10 Note that FINCA did not track or report on this result and we do not report on it here. 8 percentage ranges to estimate how much their income had increased; less than 5%, 6% to 10%, 11% to 20%, 21% to 30%, more than 30%. Ninety-three percent of the respondents in our survey indicated that their income increased at least 5% as a result of the loan they received from FINCA for agricultural production purposes. FINCA does not track the unit change in agricultural clients’ products. We report that the number reported in the midterm evaluation was further exceeded. Though we had neither the resources nor time to estimate the unit change in client’s products, we did address whether or not clients believe they achieved the goal they originally had in taking their loan. Measuring goal achievement is a sufficient proxy measure because clients ultimately were using the loans to make investments that would create positive unit change (see table 4). As corroborated in KIIs and FGDs, clients invest in inputs to increase crop or livestock production. Others buy inventory to sell during lean times. Still others reported using loans to expand into a poultry or livestock business. Clients may also invest in various non-agricultural business activities or personal investments; including home improvement or education. Finally, many clients report splitting single loans among multiple uses. Nearly 60% of surveyed clients responded that they were successful or very successful in achieving the goal they had set when they took a loan. These results are overwhelmingly positive given Tanzania’s volatile business conditions and low success rates of small businesses worldwide. The 41% indicating slight success may however indicate room for technical support to increase success and repayment rates. TABLE 6: IMPACT RESULT - INCREASED USE OF FINANCIAL SERVICES Indicator Target Achieved Number of farmers and others receiving financial services as a result of USDA assistance 35,000 63,833b Percent of female loan recipients as a result of USDA assistance 50% 49%a Number of agricultural producer (on-farm) clients receiving loans 5,000 >14,709c Total value of on-farm loans $1,260,000 $12,000,000b Number of off-farm clients receiving loans 30,000 38,123b Total value of off-farm loans $15,000,000 $50,000,000b Average per capita value of credit accessed by off-farm clients in the target region $250 $1,282b Number of FINCA savings clients 35,000 65,595b TABLE 4: INTENDED USE OF LOAN % Increase business capital 39.5 Business expansion 33.2 Improve quality of product 20.8 Non-business use 6.5 Source: Evaluation survey TABLE 5: LOAN SUCCESS % Very successful 15.4 Successful 43.6 Slightly successful 40.8 Unsuccessful 0.3 Source: Evaluation survey 9 Sources: a. Evaluation survey; b. FINCA documentation; c. Mid-term evaluation Though listed among impact results indicators, the “increased use of financial services” result tracks mainly output indicators. FINCA exceeded all targets in this results category. In order to contextualize this result and indeed FINCA’s work expanding access to financial services, the evaluation team sought to better understand access to finance across Tanzania, especially for people in rural areas, women and the poor. Access to Formal Financial Services FFPr and FINCA have made an important contribution to access to financial accounts. Still, there is a long way to go if Tanzania is to catch up with other developing countries in Sub-Saharan Africa (SSA). While borrowing and savings rates are similar for Tanzania and the rest of SSA, Tanzania lags behind in the share of its population with accounts at a financial institution (see table 7). About 29% of people in developing Sub-Saharan African countries have an account at a financial institution compared with just 19% in Tanzania. This percentage is even lower for those in rural areas (14.4%), women (17.1%) and the poorest 40% of the population (11.3%). TABLE 7: BORROWING, SAVING AND USE OF FINANCIAL INSTITUTIONS IN THE PAST YEAR (%)11 Tanzania SSA All Rural Women All Poorest 40% Account at a financial institution 19.0 14.4 17.1 11.3 28.9 Borrowed any money last year 55.6 55.1 54.7 59.0 54.5 Borrowed from a financial institution 6.5 6.5 6.4 4.0 6.3 Saved any money last year 59.2 58.3 56.3 53.1 59.6 Saved at a financial institution 9.0 6.8 8.2 4.3 15.9 11 Sub-Saharan Africa here only includes developing countries. FIGURE 1: SOURCES OF LOANS FOR TANZANIAN BORROWERS Family or friend, 39.9% Financial institution, 6.5% Informal lender, 6.6% Other, 2.1% Store on credit, 0.6% 10 Source: World Bank Global Financial Inclusion Database (2015) Savings rates are similar between Tanzania and the rest of Sub-Saharan Africa at about 59%, with this percentage rate slightly lower for in rural areas, among women, and among the poorest 40% of the population. Only 9% of Tanzanians use financial institutions to save compared to 16% in Sub￾Saharan Africa. As might be expected, rate is even lower in rural areas, among women, and among the poor. Nearly 56% of Tanzanians borrowed money in the previous year. However, borrowing from financial institutions is low at 6.5%, but in-line with regional averages. It is similar for rural borrowers and women, but lower for the poorest 40% of the population (4%). As figure 1 shows, the majority of borrowers get their money from a family or friend who may be unlikely to lend for business investment. The low rates of savings and loans provided by financial institutions in Tanzania suggests that FINCA’s financial services are likely to go to those with few other formal financial service alternatives. The services therefore expand access to finance. Alternate sources of finance for FINCA clients The evaluation team sought to discover the degree to which people receiving FINCA loans under the FFPr program had other options for financing. Our survey asked FINCA clients if they would have had access to another, similar source of finance if not for FINCA. Figure 2 shows that 75% of FINCA clients reported that they would not have access to finance from an alternative source if FINCA was not available. This finding is significant. It implies that FINCA’s impact is not primarily marginal improvements in loan terms over alternative sources of finance but makes capital available to those who would not otherwise be able to access credit. 10% would have self-financed and only 7% said they would have accessed finance from another financial institution. Together these results imply that only 15% have access to finance, compared to the 56% in the World Bank data in table 7. However, it’s important to note that FINCA clients are mostly using loans for productive purposes whereas the World Bank data includes personal use. The 6.6% of FINCA clients who would use another financial institution is nearly exactly the same as percentage use of financial institutions for Tanzanians in table 7 (6.5%). In KIIs and FGD, some farmers reported that they could access inputs on credit from their input dealer, and pay back the credit after harvest. However, the interest rates on these loans were more than double FINCA’s interest rates, and there was no help or recourse if crops failed or other calamity befell the farmer. This last point illustrates the benefit of group lending, which provides support to clients that mitigates downside risk. Source: World Bank Global Financial Inclusion Database (2015) FIGURE 2: ALTERNATIVE SOURCES OF LOANS Source: World Bank Global Financial Inclusion Database (2015) Unable to finance investmen t, 75.3% Self￾financed, 10.1% Other financial institution, 6.6% Other, 8.6% 11 TABLE 8: IMPACT RESULT - IMPROVED FARM MANAGEMENT AND PRODUCTIVE ACTIVITIES Indicator Target Achieved Percent of clients with improved financial management of their farm and productive activities 50% 61%a Source: Evaluation survey FINCA does not track the degree to which its clients improve the management of their farms or businesses. The evaluation team asked clients if they’d received training, and whether or not that training led to improved farm or business management. Note that training is discussed in more detail under evaluation question 4. TABLE 9: IMPACT RESULT - EXPANDED TRADE OF AGRICULTURAL PRODUCTS Indicator Target Achievement Average annual percent increase in income of all off-farm clients 5 95% over 5% increase a Number of jobs attributed to USDA assistance 3,000 >12,951c Percentage of women in rural households reporting enhanced business opportunities 50 Not Available Number of clients that report an increase in business sales 15,000 Not Available Source: a. Evaluation survey; c. Mid-term evaluation FINCA did not track indicators for this result. Nevertheless, the evaluation survey found that 95% of clients taking loans for off-farm agricultural purposes reported an income increase of over 5%. The evaluation team did interview clients who used FINCA loans to expand their business, including into new business areas. For example, one poultry farmer near Arusha used FINCA loans to expand from vegetable production to poultry and rabbit production. She used loans from FINCA along with her own savings from vegetable production to build the coops and cages to house the chickens and rabbits. She’s now looking to expand to local variety chickens which are hardier and tastier. TABLE 10: IMPACT RESULTS - IMPROVED TRANSACTION EFFICIENCY & IMPROVED MARKET AND TRADE INFRASTRUCTURE Result Indicator Target Achievement Improved Transaction Efficiency Percent of transactions that are conducted electronically 50 46 Percent of clients reporting reduced travel time and/or cost savings when conducting financial transactions 50 Not Available Improved Market and Number of savings and credit clients 46,500 82,381 FIGURE 3: CHICKEN COOP FACILITIES PURCHASED WITH FINCA LOAN 12 Trade Infrastructure utilizing branchless banking delivery channels. Source: FINCA documentation FINCA Express and FINCA Mobile both aim to improve transaction efficiency. FINCA tracks very closely the success and progress of agents contracted through FINCA Express, and the use of its FINCA Mobile application. FINCA does not track the degree to which either reduces travel or cost of conducting transactions, although we estimate that FINCA express cuts travel time and wait time. We elaborate on this in more detail in the response to evaluation question 3. Activity results USDA FAS required that FINCA conduct six activities and produce specific outputs under those activities. USDA FAS mandated an activity objective for each activity in its grant agreement with FINCA International. These can be found in the left column of table 11. FINCA met or exceeded nearly all targets related to FFPr activities and outputs. Highlights include:  Providing 65,600 new agricultural lending product loans against a target of 10,000  Establishing 124 point of sale banking outlets against a target of 87  124,254 registered savings accounts with total savings of $8,500,000 against a target of 35,000 accounts with a value of $2,000,000  152,894 loans disbursed to farmers and others compared to a target of 60,000 loans. FINCA fell short in two areas – opening new branches and conducting client satisfaction and impact research. FINCA opened four branches instead of the six that the FFPr grant agreement mandated. Additionally, while FINCA did carry out client satisfaction research It did not conduct social impact research. TABLE 11: ACTIVITY RESULTS Activity Objective Output Indicators Target Achieved Develop New Agricultural Lending Product Number of clients participating in pilot of new rural loan product 500 929c Number of clients reached through scale up of new rural loan product 4,500 27,332b Total number of loans provided under new rural lending product 10,000 65,600b Provide Agricultural Loans New loan capital made available to FINCA clients $5,000,000 $5,000,000d Number of loans dispersed to farmers and others as a result of USDA assistance 60,000 152,894b Value of agricultural and rural loans provided with USDA assistance $15,000,000 $62,300,000b Establish New Financial Service Delivery Channels Mobile banking service launched 1 1 b Number of point of sale banking outlets established 87 124d Establish New Bank Branches Number of new branches established 6 4 b Number of new staff 114 100d Promote Savings Accounts and Number of established savings accounts 35,000 124,254b Number of new and prospective clients 35,000 65,595 b 13 Mobilize Deposits participating in information sessions on savings Research Client Satisfaction and Social Impact Number of clients surveyed for satisfaction and quality of life improvement 1,500 1,991b Source: b. FINCA documentation; c. Mid-term evaluation; d. Conversation with FINCA In the remainder of this subsection, we discuss the activity results for “Promote Savings Accounts and Mobilize Deposits” and “Research Client Satisfaction and Social Impact.” The remaining activity objectives are discussed in the appropriate evaluation question findings sections that follow.12 Promote Savings Accounts and Mobilize Deposits FINCA began promotion of savings products in April 2015. FINCA designed savings products through market studies that included surveys of clients and studies of their needs. Savings accounts provide loan capital to FINCA and are a key part of FINCA’s strategy to capture market share through high quality customer service and trust-building. FINCA’s strategy and rigorous market research have clearly paid off as FINCA’s achievement greatly surpassed program targets. FINCA has three savings product: Hakika, Mkwanja, and Mipango. The three accounts are aimed at different types of savers. Hakika is low cost and intended for people saving money for a specific project, like paying school fees or building a house. Mkwanja is for people who want to build savings and carry out a lot of transactions from that account. Mipango has a high minimum opening balance, but earns a high interest rate. Research Client Satisfaction and Social Impact FINCA was to conduct an annual survey and produce an annual report. The subject matter of the survey and report was “satisfaction and quality of life improvement.” FINCA was to survey 500 clients a year for the first three years of the project. In terms of number of clients surveyed, FINCA exceeded the goal, surveying a total of 1,991 clients. FINCA conducted client satisfaction surveys to inform product design and social impact surveys in the form of its FCAT household survey. EQ 2: Pamoja loan product Background FINCA Tanzania began offering agricultural finance in 2013. FINCA initiated the process with a pilot test of a loan product available only to farmers and agricultural producers. FINCA rolled out the pilot product through three branches: Ifakara, Morogoro, and Iringa. The pilot product had the following major features:  Available to farmers who wanted either an individual or group loan.  A loan amount determined by the number and types of crop to be grown. 12 Note that “Provide Agricultural Loans” was all to Pamoja and therefore accounted for in the Pamoja evaluation question finding. 14  Repayment term of nine months with an early payment option.  Monthly interest servicing  Lump sum principal repayment at the end of the loan term.  Included crop insurance against destructive weather, a completely new loan feature in Tanzania. Initially, the agriculture lending product was successful. FINCA had disbursed $336,208 in loans by March of 2014. Shortly after though, the new product was beset by difficulties that proved the value of conducting pilot tests before a nationwide launch. Floods in Ifakara resulted in a 100% loss among pilot clients there. Additionally, a bumper crop of maize across Tanzania created a glut in the market, lowering the price FINCA’s clients could obtain for their crops, making them unable to pay back on time and sometimes at all. The loan product’s built-in insurance proved insufficient to cover for FINCA’s losses. These incidents demonstrated to FINCA the classic challenge of agricultural lending – correlated risks. That is, risks such as bad weather, market conditions or pest infestations that are likely to be highly correlated across a geographic area. Therefore, lenders are exposed to the risk that a large portion of their clients will default at the same time. The Pamoja Lending Product In response to these losses, FINCA Tanzania developed Pamoja (Swahili for “Together”) and implemented it starting in 2015. FINCA sought to incorporate the lessons they learned during the pilot test into Pamoja’s design to mitigate agricultural risk. Key to this was diversifying borrower groups by combining farms, off farm agriculture business (e.g. agricultural trade, food manufacturing, input selling) and non￾agricultural borrowers within a single group. In fact, a Pamoja client will often have agricultural and non-agricultural sources of income. FINCA loan officers base the client’s eligibility for a loan on the less risky, non-agriculture income source, while the loan itself is invested in an agricultural concern. Some clients also reported investing the loan initially in a non-agriculture business, which would then produce new profits that the client could use to invest in agricultural inputs, accessing new agricultural markets, or making other agriculture related investments. The diversified strategy is also advantageous because crop agriculture is seasonal with often a single major money-making harvest period. Therefore, it is often impossible for borrowers to make monthly loan repayments based on agricultural business revenue alone, creating a need for a second source of income. Lending to diversified client groups and clients with agriculture and non-agriculture sources of income serves as a “back door” for FINCA to agricultural lending. FIGURE 4: PAMOJA LOAN USE BY TYPE OF BUSINESS Source: Evaluation survey Agriculture on-farm, 25.2% Agriculture off-farm, 52.6% Non￾agricultural , 22.2% 15 Other significant changes made in the design of Pamoja were increasing the maximum loan size from three to five million Tanzania Shillings and requiring group members to approve their fellow group members’ loan request. Whereas previously client loan size increases were based on savings, under Pamoja they are based on the size of the previous loan re-payed with one or fewer late repayments. These changes allow full funding for higher revenue clients and an easier way to earn larger loan sizes for small and growing clients. As the group is responsible for repayment of each member’s loan in the case of a member’s default, the maximum approved by the group is a function of the composition of the group. Groups may combine members operating livelihood business with members operating growth businesses. However the loan size request of the latter is subject to the approval and therefore risk appetite of the former. This has resulted in entrepreneurial borrowers not receiving an adequately large loan to finance optimal expansion of their business. Another change made with Pamoja was adding a three￾month grace period for agriculture, and so-called “bullet-payments” where a significant portion of the loan can be repaid at the end of the payment period. During the grace period, interest accrues but payments are not due – a concession to the seasonal nature of crop revenue. However clients still stated a preference for a longer grace period to accommodate the up to six-month process from input purchase and planting to harvest where they finally realize return on their investment. Because of this gap in crop agriculture, clients with additional other businesses reported making repayments on their agricultural loan from revenue generated through these other sources. However not all potential agricultural clients have an alternative income source to finance agricultural loan repayments FIGURE 5: FINCA CLIENT FISHMONGER AND CUSTOMER IN BUNDA TABLE 12: INCOME INCREASE % <5% 5.2 5% to 10% 11.2 16 In addition to the basic Pamjoa product, FINCA developed Pamoja Plus. Pamoja Plus is FINCA’s urban group lending product. It is designed for agriculture and food related trade, manufacture and retail, and non-agriculture clients. Compared to Pamoja, the percentage of agricultural borrowers is much lower for Pamoja Plus. As of April 2016, only 7% of Pamoja Plus clients were categorized as having agriculture-related businesses compared to 41% for Pamoja. Key differences in loan terms compared to the Pamoja include a higher minimum loan size and no grace period. Client perspective Clients view Pamoja positively as indicated by both our survey and our field work. Only 1% of clients were unable to use their loan successfully to achieve their business goal and 59% were successful or very successful (see table 5). Perhaps most importantly, clients associate their loans with increased incomes. 95% of clients cited income increases of at least 5%. In KIIs and FGDs, clients expressed positive views about the larger loan sizes and transparency of the group decision making on loan sizes. One commonly cited unpopular feature of Pamoja is the 28-day repayment period. Requiring a 28 day repayment period allows FINCA to offer more repayment periods over the course of the year and keeps interest accrual consistent from period to period. However, clients said that they found it easier to remember a group meeting that was the same date of every month, and that scheduling the meeting on the same date allowed their monthly payment to coincide better with other financial flows such as payments from a milk buyer. FINCA Tanzania is aware of this common complaint and are deciding how best to address it. EQ 3: Branchless channels “Overall the world is moving to more technology and so we are too in the banking industry. We are adapting technology. It is a matter of educating customers. Using agents to adapt change and educate customers. Every bank is going to have to initiate these changes.” - A FINCA Branch Manager FINCA Tanzania developed and implemented two branchless banking channels: FINCA Express (agency banking) and FINCA Mobile.13 Both services aim to reduce the cost of financial transactions for FINCA and FINCA’s clients and allow FINCA to serve more clients, including those in more remote areas. FINCA Express FINCA Express is an arrangement through which FINCA contracts with retail outlets, or agents, that are located near clients. These arrangements allow FINCA to offer many locations for clients to 13 The ATM branchless service was deemed inappropriate and FINCAs agreement with USDA was amended in 2015 to remove this requirement. 10% to 20% 36.9 20% to 30% 22.4 >30% 24.2 Source: Evaluation survey 17 conduct transactions without having to travel to an established FINCA branch. FINCA Express agents are connected to FINCA branches and are authorized to make deposits and withdrawals for clients. The main advantage for clients in using FINCA Express is convenience. Agents are closer to clients’ businesses then FINCA branches (see table 14), and some clients did report they have to spend less time waiting in a queue at FINCA Express outlets compared to branches. The evaluation team’s field work confirmed this: there were often long lines of clients in banks while agents’ shops had only one or two clients, if any. Clients can make loan withdrawals and deposits with agents, but need to go a branch to open an account. In some cases bank staff can go to an agency and open an account for a client. Paperwork at FINCA Express locations is minimized through biometric fingerprinting to quickly identify clients and their account details. FINCA owns these biometric machines, which reduces costs and risk for agents. The criteria to become an agent includes having 4 million Tanzanian Shillings in cash (about 2,000 US dollars) and a secure storefront from which to operate. This latter criterion is a requirement of the Bank of Tanzania. These two requirements make it difficult to establish FINCA Express in very rural areas. In addition to FINCA Express, agents typically also provide services on behalf of MPesa and other Mobile Money Operators (MMOs). FINCA Mobile FINCA Mobile is a service that allows clients to use their mobile phones to make withdrawals and deposits from their FINCA account. It also offers certain types of bill payment. After withdrawing funds from FINCA to FINCA Mobile, clients can then transfer to MPesa for a fee and use MPesa to pay bills or access cash. FINCA Mobile saves time both for clients who live far from branches or for those who are busy. Despite the convenience, FINCA Mobile use has plateaued at about 10% of FINCAs transactions in 2016. This levelling-off appears to be somewhat of a disappointment given that FINCA Tanzania views FINCA Mobile as an integral tool in bringing down the cost of financial service provision. Use and access for both services As of October 2016, according to FINCA’s records, 54% of transactions were at FINCA branches, 35% with FINCA Express, and 10% with FINCA Mobile. 87% of all clients were registered to use FINCA Express and 47% for FINCA Mobile. The survey conducted for this evaluation showed FIGURE 6: FINCA EXPRESS OUTLET IN TANGA TABLE 13: SERVICE USE BY SURVEY RESPONDENTS % FINCA Express 67.3 FINCA Mobile 39.9 Both 29.7 Neither 22.4 Source: Evaluation survey 18 that 67% of respondents use FINCA Express while 40% use FINCA Mobile. Thirty percent use both and 22% use neither. The evaluation survey revealed that over 67% of those not using either branchless banking channel said that they had never heard about them. Twenty percent said the channels were not available to them. The fact that many of its own clients are unaware of its branchless services indicates that FINCA could do a better job marketing them. In fact, FINCAs own Pamoja loan training guide does not contain mention of these services. According to both some agents and FINCA staff, FINCA could also do a better job marketing these services to prospective clients as well. While there is some radio and TV advertising, a portion of the responsibility for marketing falls on staff and word of mouth. Additionally, though agents are a major source of new clients, some agents expressed that they did not feel prepared to market FINCA to potential clients who come in to use other services. TABLE 14: TIME TO BRANCH OF AGENT IN MINUTES Source: Evaluation survey Notes: * denotes significant difference at the p <0.05 level. 1. Does not use either FE or FM On average, clients in the evaluation survey were located 64 minutes away from their branch or agent. Those using agents are significantly closer than those who do not use agents or FINCA Mobile (56 vs 79 minutes). The difference implies that FINCA express cuts travel time by 23 minutes on average. Contrary to expectation, those using FINCA Mobile are actually closer to branches or agents than those who use neither FINCA Express nor FINCA Mobile at 70 versus 79 minutes (not a statistically significant difference however). Client perspective – FINCA Express FINCA Express has grown rapidly from just a few agents during its pilot phase in August 2014 to 124 agents located around the country currently. In KIIs and FGDs, clients were nearly uniformly positive about agent services. This result is corroborated by the evaluation survey results, which revealed that 51% of clients responded to the question “how well does FINCA express serve your needs” with very well and 33% responded with well. Only 3% of clients in the survey responded with “poorly” or “very poorly”. Service User Non-user1 Difference FINCA Express 55.6 (n = 270) 78.5 (n = 90) 22.9* FINCA Mobile 69.9 (n = 160) 78.5 (n = 90) 8.6 19 It is important to note that clients do not yet pay to use FINCA Express. However, given willingness to pay fees for other services provided by FINCA Express Agents including MPesa and FINCA clients’ satisfaction with FINCA Express, charging a fee does may not cost FINCA many clients. Agents are pleased the level of support they get from FINCA. Channel officers visit agents twice a week and check bank statements, records, receipts, and client privacy protection protocol. They also help agents with other issues, such as delays due to network issues, delays in reversing faulty transactions, and problems maintaining financial records. Throughout the course of the two year plus agent program, FINCA has only lost three agents – two who quit and one who was terminated. Client perspective – FINCA Mobile As noted in this reports “Activity results” sub-section, clients have been slow to adopt and use FINCA Mobile. The most common client complaint about FINCA Mobile is delays in transactions. FINCA is aware of this problem, which it estimates affects about 10% of all mobile transactions, and attributes the issue to technical difficulties with experienced by the aggregator, who manages the transactions between the phone company and FINCA. FINCA is working with the aggregator to resolve the issue. Another concern among some FINCA clients is the cost of using the service. Clients are charged twice – once by their mobile provider and once by FINCA. Despite the fact that for some remote clients a trip to town is more expensive than using FINCA Mobile, some clients still prefer to use agents. Clients may prefer agents because the cost of travel is offset by clients’ need to visit town for other errands. Regardless of their banking needs, clients travel to town once or twice a month for personal and business purposes. While in town, adding an agent visit to the ‘to-do’ list allows the client to take care of banking without the expenses associated with mobile transactions. Another factor that accounts for the limited use of FINCA Mobile is that clients can use Mpesa to make deposits to their account. Clients appear to prefer MPesa because they use MPesa for a wide range of purposes. This preference leaves withdrawals as the primary value added by FINCA Mobile for many FINCA customers. However, withdrawals are made less frequently than deposits, which further erodes FINCA Mobile’s use among clients. TABLE 15: HOW WELL FINCA EXPRESS MEETS CLIENTS’ NEEDS % Very well 50.7 Well 33.3 Neutral 13.0 Poorly 1.9 Very poorly 1.1 Source: Evaluation survey 20 Nevertheless, despite relatively low usage and some complaints about service, clients using FINCA Mobile are satisfied overall with the service. As the accompanying table shows, 51% said it met their needs “very well”, 35% said “well”. Only one respondent responded with poorly or very poorly. Other potential services FINCA is interested in expanding service offerings from both FINCA Express and FINCA Mobile as a way to generate more revenue. The survey found that among other potential services FINCA Express or FINCA Mobile could offer, clients chose bill paying (51%), insurance (21%) and TV payments (19%) as services they most would like to see offered. Twenty eight percent did not think any additional services were needed. EQ 4: Training and technical assistance FINCA overhauled the client training curriculum with the development of Pamoja to place more emphasis on group engagement. The change ushered in the centrality of the groups’ new role in deciding the loan sizes of its members. In general training is viewed as more important under Pamoja because larger loan sizes involve greater risk. However training is a significant cost which FINCA Tanzania hopes to eventually reduce through digital delivery using tablets or smartphones. New Pamoja groups receive a three-day training from FINCA. Day one of the training features group formation finalization including election of leaders and an overview of FINCA and the products and services it offers. Day two focuses on loan assessment, including verification of clients’ assets and businesses. Day three includes a review and finalization of the loan request. Training for Pamoja clients is carried out by the loan officer assigned to them. Loan officers themselves are trained by Pamoja supervisors several times a year. However, one loan officer mentioned that they felt they could use more training to prepare for training clients. This training is repeated with each new loan dispersal, though intensity is customized to an extent based on the needs of the group. Some clients however complain about excessive and unneeded follow-up training and attribute this to delays in loan disbursement. While this is not the cause of loan delays according to FINCA, they are aware of the need to streamline follow-up training. This appears to be a “low hanging fruit” opportunity for cost savings TABLE 16: HOW WELL FINCA MOBILE MEETS CLIENTS’ NEEDS % Very well 51.3 Well 35.0 Neutral 13.3 Poorly 0.6 Very poorly 0 Source: Evaluation survey FIGURE 7: CLIENT RECORD BOOK 21 In some cases clients have asked loan officers for technical assistance (TA) in agriculture, which FINCA is unable to provide. Providing technical assistance is outside of loan officer’s scope of work and usually technical expertise. However at least one branch coordinated with an agricultural Non￾Governmental Organization (NGO) to form groups with the NGO’s beneficiaries. The branch manager at this branch reported seeing higher repayment rates as a result of the partnership. FINCA has in fact explored partnerships with several NGOs in Tanzania implementing farmer capacity building projects but nothing materialized. According to FINCA this was because they had a difficult time finding an agricultural TA partner for which there was a large enough overlap between FINCA’s clients – or potential clients based on branch locations – and the potential partners’ beneficiaries. There was also concern about the professionalism of some of the available partners. Given the coordination costs, a certain scale is necessary for such a partnership to work. FINCA still believes the idea is a good one and is continues discussing partnerships with several NGOs who operate at significant scale. Client perspective Clients are positive about training and nearly all view it as having a positive impact (see table 17). A particularly popular topic is group record keeping. Some clients report using group record keeping as a guide for record keeping in their own business. Some also said the initial analysis and target setting that FINCA performs when determining loan eligibility helped clients understand and manage their own business better. As mentioned previously, some clients expressed a desire for more streamlined followup training. Two-thirds of respondents who rated themselves as having been successful or very successful in achieving their original loan objective said the loan increased their income by over 20%. On the other hand, only 18% of clients who rated themselves as unsuccessful or slightly successful achieved the same income increase. This intuitive but important finding makes a strong case for technical support to help clients better achieve their investment goals. TABLE 18: PERCENT INCOME INCREASE ASSOCIATED WITH LOAN BY SUCCESS IN ACHIEVING LOAN GOAL % for successful or very successful % for unsuccessful or slightly successful <5% 2.6 8 5% to 10% 4.3 20.9 10% to 20% 26.1 53.4 20% to 30% 31.2 10.4 >30% 35.9 7.4 Source: Evaluation survey TABLE 17: TRAINING IMPACT ON FARM OR BUSINESS % Completely improved 20.4 Mostly improved 50.6 Slightly improved 19.6 About the same 0.4 Source: Evaluation survey 22 EQ 5: New branches The location of the four new branches – in Mbeya, Arusha, Moshi and Morogoro - was based on market assessments which aimed to identify the largest concentrations of potential clients. While USDA’s grant agreement called for FINCA to locate the branches in “rural areas”, the four new locations are all urban areas, each with populations in the hundreds of thousands. FINCA took this strategy based on the high concentration of unmet demand for finance in these locations as well as their importance as agricultural processing and trade hubs. Interestingly, Morogoro is the only branch that serves an above average number of agricultural clients at 42% compared to 41% for the whole Pamoja portfolio. Though Mbeya is also a Pamoja loan branch, only 22% of its loans are to agricultural clients. Arusha and Moshi are both Pamoja Plus (the urban client version of Pamoja) branches and only 12% and 6% of their borrowers are agricultural borrowers which in total exceeds the 7% average across all Pamoja Plus branches. It is therefore not clear that opening up these two Pamoja Plus branches was consistent with the agricultural focus of the FFPr program. TABLE 19: PERCENT OF AGRICULTURAL BORROWERS BY BRANCH Loan type Branch name % agricultural borrowers Average % agricultural borrower by loan type Pamoja Morogoro 42 41 Mbeya 22 41 Pamoja Plus Arusha 12 7 Moshi 6 7 Source: FINCA loan portfolio The new branches were also intended to serve as FINCA’s flagship branches, with more modern and spacious facilities that would help establish FINCA as more professional than its competitors. From the Evaluation Team’s comparison of USDA and non-USDA branches, FINCA has achieved this modernization. The building of these new branches also coincided with the establishment of, and transition towards, FINCA Express agent-based banking and away from brick and mortar branch banking. Agent banking is a lower cost and more efficient way than branches to extend outreach to rural areas where client concentration is lower. Therefore a strategy of basing branches in more urban, high client concentration areas and agents more rurally appears sensible. The FFPr grant agreement signed in September of 2012, and its amendment signed in August of 2015 included a target of 6 new bank branches opened over the course of the program. The August 2015 amendment specifies that the bank branches were scheduled for the first three years of the program. Somewhere along the course of the program the target was changed to four new FIGURE 8: NEW FINCA BRANCH IN ARUSHA 23 branches. FINCA’s midterm evaluation states that FINCA had achieved its goal of opening four new bank branches, indicating that the revision of the target occurred sometime before December 2014. The ISG evaluation team, however, found no evidence that FINCA or USDA had revised the target. Client perspective Our evaluation survey data allows us to compare client satisfaction results for clients registered with new, USDA funded branches (n = 58) versus those with other branches (n = 342). 14 Table 19 shows that clients at new branches are indeed closer to those branches than clients at other branches. This supports FINCAs assertion that they based new branches in areas with higher concentrations of clients. Clients at new branches have similar outcomes and satisfaction with FINCA services as those at other branches. TABLE 20: COMPARISON OF NEW BRANCH CLIENTS WITH OLD BRANCH CLIENTS New branch Other branches Average client distance from branch (minutes) 53.4 65.2 % clients reporting using loan to successfully or very successfully achieve goal 58.6 58.9 % clients reporting income increase over 10 percent 93.1 91.8 % clients likely or very likely to recommend FINCA 84.5 86.3 Source: Evaluation survey EQ 6: Client perspective “To continue to be deeply concerned about its work because it saves the lives of many poor people who have nowhere else to turn” - Woman vegetable seller FINCA client near Arusha when asked what she would like to tell FINCA In the answers to the previous evaluation questions we looked at client perspectives relevant to each question. Here we look at client perspectives on FINCA Tanzania FFPr as a whole. The evaluation fieldwork and survey both asked clients who had access to other sources of finance why they chose FINCA. The primary reason survey respondents gave for choosing FINCA was that it had better location for them than competitors. The importance of location in client preferences underscores the significance of FINCA’s branchless channels in bringing finance closer to more people. 14 Only includes clients from the two Pamoja, not Pamoja + new branches: Mbeya and Morogoro. FIGURE 9: REASONS FOR CHOOSING FINCA OVER COMPETITORS Source: Evaluation survey 5.2% 5.5% 5.7% 9.2% 11.5% 14.7% 54.6% Good training Fast disbursement Monthly repayment Terms and conditions Lower interest rate More trustworthy Better location 24 In KIIs and FGDs, a critical reason clients said they chose FINCA over other formal sources of finance was that that FINCA has no collateral or land title requirement. Clients also cited ease of access afforded by FINCA Express and FINCA Mobile and an easier application process as advantages over competitors. One client reported preferring FINCA over a competitor because of FINCA’s lower interest rates, higher loan ceiling, and savings account options. Overall clients were extremely satisfied with FINCA and 99% said they would return to FINCA if they needed another loan. Ninety-two percent of clients said they would recommend FINCA to someone else. 4. Conclusions, Lessons Learned, and Recommendations Pamoja loan product After several pilot projects that tested lending exclusively to agricultural producers, FINCA designed Pamoja, a loan product that, among other innovations, reduces agricultural risk by including agricultural and non-agricultural borrowers in the same peer-lending group. Other key design features are larger loan sizes and group decision making on how large a loan each group member may take. Pamoja has been very successful with total loan disbursement amounts far exceeding original grant targets ($12 million vs $1.2 million for on-farm lending). Clients are pleased with Pamoja and its new design. Consequently, clients return for multiple rounds of loans despite their awareness of alternative sources of finance. The overwhelming majority of clients (95%) report loans increasing their incomes by over 5%. Examples of how profits are used include school fees and home improvements. Lesson learned: The changes FINCA made to group lending in designing its Pamoja product, such as group decision making regarding loan size and mixing agricultural and non￾agricultural business to diversify portfolios and reduce risk, were innovative and successful. Clients appreciate the transparency of group decision making on loan size, though there was some concern about not receiving large enough loans (see next lesson learned). Clients TABLE 21: LIKELIHOOD TO RECOMMEND FINCA % Very likely 29.3 Likely 62.8 Neutral 0.3 Unlikely 7.3 Very unlikely 0.5 Source: Evaluation survey 25 often use Pamoja for agriculture but repay with non-agricultural income. They also frequently use proceeds generated from a loan in a non-agricultural business to invest in their agriculture. Lesson Learned: Tanzania lags behind other Sub-Saharan African countries in access to formal accounts at financial institutions. FINCA’s innovations in lending led to FINCA making a large number of loans to people who had no alternative source of financing (75% percent said they would not have otherwise been able to finance their investment. Consequently, those loans had a significant economic impact on the lives of borrowers. Lesson learned: There remain opportunities for Pamoja to improve further to better serve the challenging agricultural sector. For example, FINCA’s most successful Pamoja borrowers are restricted by the risk tolerance of less successful borrowers in their lending group. A successful borrower that could expand his/her business with a large loan at reasonable risk is unlikely to receive the total amount because of the rest of the group’s unwillingness to take responsibility for the large amount should the successful borrower fail. Successful borrowers are often lower risk than other borrowers because of their long track record of paying back loans, and the size and diversification of their businesses. Thus, good business opportunities are underfunded and good entrepreneurs constrained because of more risk averse group members. FINCA’s profitability is also likely held back. During the evaluation, this issue was illustrated by several clients who reported that they were unable to borrow the loan sizes they wished for even though they felt they had a good opportunity for business expansion. Recommendation: FINCA should create a fund within Pamoja that successful entrepreneurs may access to top-off the loans they receive through their lending group. For example, for a loan request by a client with an outstanding track record and a business plan verified by a loan officer, FINCA could absorb the risk on the difference between the loan request and group approval amounts. This additional portion of the loan could be at a slightly higher interest rate to help mitigate some of FINCA’s risk. This hybrid could increase FINCA’s profitability by removing obstacles that FINCA’s best customers face in lending groups. It could be the beginning of a pathway to graduation to FINCAs individual loan program which provides larger loans to individual borrowers. Lesson learned: The majority of Pamoja lending went to off farm agriculture such as food retail ($50 million vs $12 million). FINCA’s crop producer borrowers report receiving only a quarter of the loan size they request, the lowest among all FINCA’s borrowers. Together this indicates that producer level agriculture lending still remains a challenge. Recommendation: One strategy to both mitigate and diversify risk in lending to agricultural producers is to share it across multiple value chain actors. Through value chain financing, FINCA would lend to buyers who would provide inputs and a guaranteed market and price. A partner organization could provide TA to these producers. Together this would increase farmer profitability, reduce risk of crop 26 failure and diversify risk between three parties instead of two, with farmers taking more of the production risk and buyers the market risk. FINCA is already looking into value chain finance. However, it is not clear how many farmers are connected to strong value chains with progressive buyers willing to work with such a scheme. Therefore, this must be considered as just one component of FINCA diversifying its risk rather than a replacement for Pamoja. Recommendation: Another possibility for reaching more agricultural producers is by diversifying by client product. For example, the weather, pest and disease, and market-related risks faced by maize producers and cashew producers may not be highly correlated. Similarly livestock and coffee producers may not have highly correlated risks. USDA or FINCA should commission research to quantify these risks by agricultural product which would help diversify agricultural production-level lending risks. Branchless channels FINCA’s two branchless channels are critical to reaching more agricultural clients. FINCA Express has been particularly successful, with the number of agents expanding by a factor of ten over the past year and agents reporting FINCA Express to be their most profitable line of business. Additionally, over half of clients give FINCA Express the highest satisfaction score, with only three percent report being unsatisfied. In terms of client recommendations for improvements, increasing the number of agents ranks as clients second highest priority behind lower interest rates. The service makes FINCA convenient to more clients by expanding banking service locations and reducing wait times. This convenience is critical to FINCAs success: Over half of FINCA’s clients surveyed said location was the most important reason they chose FINCA over competitors. FINCA Mobile appears to be less successful in expanding than FINCA Express, though still well regarded by those who use it. While use has stagnated at about 10% of transactions over the past year, 51% of those who do use it gave it the highest satisfaction score, with only one percent reporting dissatisfaction. Lesson learned: FINCA Express is a successful model that has grown exponentially and looks likely to continue to rapidly expand FINCA’s client base. It also brings down the cost of financial service delivery. The combination of decentralized financial service delivery and quick service provided through biometric identification offers clients the convenience that many say is a deciding factor in choosing FINCA. Lesson learned: Of the 22% of FINCA clients who do not use one of the two branchless channels, 68% say that it is because they have not heard of the channels. Clients who do not use either FINCA Express or Mobile are further from branches or agents than FINCA Express or Mobile users. Their lack of awareness may indicate that FINCA is not reaching some of its more remote clients with campaigns to raise awareness about its branchless services. 27 Additionally, marketing of these branchless channels to potential new clients could be improved. Recommendation: Add a module to the Pamoja training manual on how to inform clients of the benefits FINCA Express and Mobile and how to use the services. Consider creating a FINCA Mobile lending group officer position – a group member who in return for free FINCA mobile transactions provides assistance to other group members on how to use FINCA Mobile. Additionally, provide a bi-annual update for agents on FINCA products and services, including FINCA Mobile, so they are better at selling FINCA to their other customers. Such training is provided by competitor MMOs whose services FINCA Express agents also offer. Lesson learned: FINCA is interested in other services, such as bill payments, that it can offer through FINCA Mobile to expand its popularity and increase revenue. However offering these services means going head to head with similar services offered by well-established MMOs such as MPesa. Given the ease of transferring money between FINCA Mobile and these MMOs, it’s not obvious what clear advantage FINCA Mobile has over these alternatives, even for FINCA clients. In fact, FINCA allows deposits from some of these other MMOs to client accounts which further undercuts the advantage of FINCA Mobile. The one clear advantage the service does have is that it is the only way for clients to make withdrawals with their telephone. Recommendation: Rather than compete with the variety of services offered by established MMOs, FINCA Mobile should be positioned as a last mile platform for remote or busy clients to access their FINCA accounts. From a business standpoint, FINCA Mobile can be viewed as service that makes FINCA attractive to more customers by offering increased convenience rather than a product providing a new revenue stream. In fact, convenient location is the primary reason clients gave for choosing FINCA over competitors. As such, FINCA should offer reliable service by reducing customer reported , competitive fees, and a user-friendly experience. Training and technical assistance Pamoja clients value FINCAs training program. Evidence of client’s appreciation for training includes theirability to cite specific topics they found useful and the high rating clients give the impact of training. Some areas that FINCA could improve include adjusting follow-up training to limit unnecessarily redundant material as well as including awareness raising and training on branchless channels. Lesson learned: While clients are pleased overall with training, some express a desire to receive agricultural technical assistance to help them maximize the return on their loan. 28 Data from the evaluation survey confirms this intuitive finding. Sixty-six percent of clients who rate themselves as successful or very successful in fulfilling the purpose of the loan report income increases over 20%, compared to 18% for those rating themselves as slightly successful or unsuccessful. FINCA itself is aware of the benefits of such assistance: better agricultural practices by its clients would mean more profits and less risk of loss which would mean reduce risk for FINCA. FINCA made several attempts at agricultural TA partnerships with NGOs but none have been successful due in part to limited overlap of clients and beneficiaries. Recommendation: USDA and other donors can help facilitate partnerships between agricultural financial service providers and NGOs working in agriculture. One way is to require grant applicants to identify a partner and plan for partnership as part of the proposal process. Another way of forming partnerships would be for USDA to host events that bring together financiers and agricultural TA providers to identify possible areas for partnerships. New branches The new USDA bank branches were originally supposed to be located in rural areas to better serve agricultural clients. However after conducting market research FINCA decided that they could reach more clients basing them in urban agricultural trade hubs, which had a higher concentration of prospective clients. However, this decision did not lead to a higher percentage of agricultural clients at new branches than at other branches. The new branches were intended as FINCA’s flagship branches, with more modern and spacious facilities that would help establish FINCA as more professional than competitors. From the Evaluation Team’s comparison of USDA and non-USDA branches, FINCA has achieved this modernization. The evaluation team was unable to document approval of the decision to switch the target for new branches from six to four. Progress towards PMP targets FINCA exceeded the great majority of its USDA FFPr targets, in many cases by a wide margin. For example it exceeded its value of off farm loans target of 1.2 by a multiple of nearly 10 and extended $50 million dollars of off farm loans compared to the target of 15 million. The impact target of increasing incomes by 5% also appears to have been shattered. Lesson learned: Though it was resoundingly successful in exceeding many of its targets, FINCA failed to collect much of the data required for its performance monitoring plan. Additionally, the data it did collect was not organized and reported as per the requirements of the PMP. Therefore performance data reported here has been assembled and organized by the Evaluation Team from various sources including project records, the mid-term evaluation and the survey conducted for this report. In general, FINCA collected data that was useful to running its business. 29 FINCA’s failure to track many of the indicators required through its FFPr agreement with USDA can be viewed within the incentives faced by FINCA Tanzania as well as its core mission. As mentioned, the indicators FINCA did track had value to FINCA in managing success of their core business – extending credit to underserved populations. FINCA did not track the impact of this financing on clients’ businesses or lives, as presumably the cost of doing this was not matched by its value to them in achieving their mission. With the margins on lending to small farmers tight and often negative, diverting staff and resources away from core business activities and towards social impact data collection implies reaching fewer perspective clients at much higher cost. This choice for FINCA could have been avoided had the FFPr grant an M&E budget including salary for a full-time M&E staff. While the contract requires at least three percent of funds to go towards M&E, it’s not apparent whether or not this was achieved and if the target was adequate. Recommendation: To ensure that its implementers collect the data it needs, USDA should include an appropriate budget for M&E and particularly a dedicated M&E staff person. This would avoid placing implementers in the position whereby funds can either by used towards achieving their mission or fulfilling other donor requirements. Annexes 30 Annex I: References External references The Food and Agriculture Organization (FAO). 2016. AQUASTAT – United Republic of Tanzania. Accessed at: http://www.fao.org/nr/water/aquastat/countries_regions/TZA/ Ministry of Agriculture, Food Security and Cooperatives, The United Republic of Tanzania. 2013. National Agriculture Policy. Accessed at: http://www.fao￾ilo.org/fileadmin/user_upload/fao_ilo/pdf/ICA_MLW_and_TZ/NATIONAL_AGRICULTURAL_POLIC Y-2013.pdf Stein, Howard et. al. 2016. The Formal Divide: Customary Rights and the Allocation of Credit to Agriculture in Tanzania. The Journal of Development Studies, 52(9). United States Department of Agriculture Foreign Agricultural Services (USDA FAS). 2017. Food for Progress. Accessed at: http://www.fas.usda.gov/programs/food-progress The World Bank Global. 2014. Financial Inclusion Data Base. Accessed at: http://databank.worldbank.org/data/reports.aspx?source=1228 FINCA documents and data 1. Analysis of total borrower vs Pamoja borrower, 30 April 2016 2. Bi-Annual Project Narratives (5) 3. Client Presentation Flip Chart “Pamoja Loan” 4. Customer Satisfaction Survey 2015 Report 31 5. Delivery Channels Report for the Month of October 2016 6. Final Evaluation Terms of Reference 7. FINCA Customer Satisfaction Survey, April 2014 8. Grant Agreement between the Government of the United States of American and FINCA International FCC-621-2012/044-00 dated 12 September 2012 and amended on 9 August 2015 9. Mid-term evaluation, December 2014 10. Mobile Banking Policy Manual, November 2015 11. Plan of operations UDSA Handoff 12. Pamoja Loan Portfolio, 30 September 2016 13. Pamoja Loan Credit Policy, January 2015 14. USDA Savings Activity April 2015 to October 2015 15. Savings Policy, August 2016 16. Training Guide “Pamoja Loan” 17. USDA Client Database, December 2016 18. USDA Food for Progress: Customer Satisfaction Report, September 2016 Annex II: List of Acronyms and Abbreviations EQ Evaluation Question FFPr Food for Progress FGD Focus Group Discussion ISG International Solutions Group KII Key Informant Interview MMO Mobile Money Operator NGO Non-Governmental Organization SSA Sub-Saharan Africa TA Technical Assistance USDA United States Department of Agriculture 32 Annex III: KIIs, FGDs and Sites Visited KIIS AND SITES VISITED KIIS AND SITES VISITED CONTINUED Type15 Gender Site Date Type Gender Site Date HQ Staff M Dar es Salaam 12.5.2016 Client F Arusha 12.10.2016 HQ Staff M Dar es Salaam 12.5.2016 Client F Arusha 12.10.2016 HQ Staff M Dar es Salaam 12.5.2016 Client F Arusha 12.10.2016 HQ Staff M Dar es Salaam 12.5.2016 Agent F Arusha 12.10.2016 Staff M Morogoro 12.6.2016 Staff M Arusha 12.10.2016 Agent M Morogoro 12.6.2016 Staff M Arusha 12.10.2016 Staff F Morogoro 12.7.2016 Staff M Arusha 12.10.2016 Staff F Morogoro 12.7.2016 Staff F Arusha 12.10.2016 Staff F Morogoro 12.7.2016 Staff F Arusha 12.10.2016 Staff F Morogoro 12.7.2016 Staff F Arusha 12.10.2016 Staff M Morogoro 12.7.2016 Staff F Mara 12.13.2016 Staff M Morogoro 12.7.2016 Staff M Mara 12.13.2016 15 Note that staff includes Branch Managers, Loan Officers, (Branchless) Channels Managers and Pamoja/Pamoja Plus Supervisors. All names and staff positions are withheld to protect respondent anonymity. 33 Staff M Morogoro 12.7.2016 Staff M Mara 12.13.2016 Agent M Morogoro 12.7.2016 Staff M Mara 12.13.2016 Client M Morogoro 12.7.2016 Staff M Bunda 12.13.2016 Client M Morogoro 12.7.2016 Client M Bunda 12.13.2016 Client M Morogoro 12.7.2016 Client M Bunda 12.13.2016 Client F Tanga 12.8.2016 Agent M Tanga 12.8.2016 Staff M Tanga 12.8.2016 Staff F Tanga 12.8.2016 Staff M Tanga 12.8.2016 Staff F Tanga 12.8.2016 CLIENT FOCUS GROUP DISCUSSIONS Female Male Site Date 7 3 Morogoro 12.6.2016 6 6 Tanga 12.8.2016 Annex IV: Composition of Evaluation Team Jason Wares – Project Supervisor Jason Wares, Director of Programs at ISG, is an international development expert with 15 years’ experience specializing in catalyzing development and aid programming through the lens of livelihood development. He holds an MBA from the Thunderbolt School of Global Management. Mr. Wares provided technical oversight throughout the evaluation, led the phone-based survey and managed communications with client and implementer to ensure efficient evaluation delivery. Christopher Root – Team Leader Mr. Root is an agricultural economist with 10 years of experience including in monitoring and evaluation (M&E), and performance and impact evaluation. He holds an MA in International Development from American University and will shortly receive his MS in Agricultural Economics from Michigan State University. Mr. Root led evaluation tool development and report writing. Mr. Chiwango Erasto, FINCA Tanzania’s Research and Social Performance Manager, provided logistical support for the Evaluation Team’s fieldwork in Tanzania. 34 Annex V: Indicator Performance Tracking Tables ACTIVITY RESULTS Activity Objective Output Indicators Target Achieved Develop New Agricultural Lending Product Number of clients participating in pilot of new rural loan product 500 929c Number of clients reached through scale up of new rural loan product 4,500 27,332b Total number of loans provided under new rural lending product 10,000 65,600b Provide Agricultural Loans New loan capital made available to FINCA clients $5,000,000 $5,000,000d Number of loans dispersed to farmers and others as a result of USDA assistance 60,000 152,894b Value of agricultural and rural loans provided with USDA assistance $15,000,000 $62,300,000b Establish New Financial Service Delivery Channels Mobile banking service launched 1 1 b Number of point of sale banking outlets established 87 124d Establish New Bank Branches Number of new branches established 6 4 b Number of new staff 114 100d Promote Savings Accounts and Mobilize Deposits Number of established savings accounts 35,000 124,254b Number of new and prospective clients participating in information sessions on savings 35,000 65,595 b Research Client Satisfaction and Number of clients surveyed for satisfaction and quality of life improvement 1,500 1,991b 35 Social Impact Sources: b. FINCA documentation; c. Mid-term evaluation; d. Conversation with FINCA PERFORMANCE INDICATORS Result Indicator Target Achieved Increased Agricultural Productivity Average annual percent increase in income of all on-farm clients 5 93% over 5% increasea Number of individuals benefiting directly from USDA-funded interventions 65,000 76,210b Number of clients reporting a positive unit change in their land, animal, or selected product 3,000 >5,301c Increased Use of Financial Services Number of farmers and others receiving financial services as a result of USDA assistance 35,000 63,833b Percent of female loan recipients as a result of USDA assistance 50% 49%a Number of agricultural producer (on-farm) clients receiving loans 5,000 >14,709c Total value of on-farm loans $1,260,000 $12,000,000b Number of off-farm clients receiving loans 30,000 38,123b Total value of off-farm loans $15,000,000 $50,000,000b Average per capita value of credit accessed by off-farm clients in the target region $250 $1,282b Number of FINCA savings clients 35,000 65,595b Improved Farm Management (Operations, Financial) Percent of clients with improved financial management of their farm and productive activities 50% 61%a Expanded Trade of Agricultural Products Average annual percent increase in income of all off-farm clients 5 95% over 5% increasea 36 (Domestic, Regional, and International) Number of jobs attributed to USDA assistance 3,000 >12,951c Percentage of women in rural households reporting enhanced business opportunities 50 Not Available Number of clients that report an increase in business sales 15,000 Not Available Increased Access to Markets to Sell Agricultural Products Number of clients that report an increase in number of market sales outlets 7,500 Not Available Improved Market and Trade Infrastructure Number of savings and credit clients utilizing branchless banking delivery channels. 46,500 82,381b Improved Transaction Efficiency Percent of transactions that are conducted electronically 50 46b Percent of clients reporting reduced travel time and/or cost savings when conducting financial transactions 50 Not Available Sources: a. Current survey; b. FINCA documentation; c. Mid-term evaluation Annex VI: Methods Document review The evaluation team undertook an extensive document review that informed all stages of the evaluation. The team used the document review initially to learn about the project and inform data collection tools. Later the review was useful in explaining different features of program design. It was also the source of most of the data the evaluation team used to compare performance results against targets. Qualitative research The evaluation team conducted key informant interviews (KIIs) and focus group discussions (FGDs) to provide qualitative information. The team used qualitative field research to create a narrative that provided context to survey findings and detail to evaluation question answers. Because the evaluation survey featured mostly closed-ended questions to gauge satisfaction (described below), the qualitative field research focused on explorative open-ended questions. The evaluation team asked questions to expand the its understanding of the FFPr project, answer evaluation questions, verify the team’s tentative hypotheses, and confirm information gleaned from other sources. Question guides were used to lead semi-structured interviews. The guides ensured that the evaluation team asked the same key questions across similar respondent types while allowing the freedom for conversational-style interviews. Survey The evaluation also includes a brief client phone survey to collect client perceptions with statistical validity. Simple random sampling was used to complete 400 surveys. The survey team randomly selected respondents from FINCA’s client list of 27,333 active Pamoja loan clients. Enumerators 37 then called clients on the list in order until 400 interviews were complete. This final sample size results in a margin of error of 5% with a confidence level of 95%. Annex VII: Survey tools Question checklist for KIIs – Clients Background □ Are you currently a FINCA client? o If yes, for how long have you been a client o If not, how long ago did you complete your loan? □ How did you hear about FINCA? □ Which FINCA products have you used? [Pamoja, old loan product] □ Which FINCA branch do you use? □ How far are you from the branch/agent you use? Use, impact and alternatives □ What did you use the most recent loan you received from FINCA for? o Roughly how much more money have you made/do you expect to make because of this investment? o Are there any other benefits you expect from this loan? □ How would you have financed this investment without FINCA? □ Did you consider these options before taking FINCA credit? o Why did you choose FINCA? □ Has FINCA changed the way you do business with your suppliers or customers? How? □ Has FINCA changed the way you internally manage your business or farm? How? Application, terms and repayment □ Describe the application and approval process with FINCA. How was it to navigate? o What would you change about the application and approval process? □ What about the terms of the loan (e.g. interest rate, repayment period, eligibility requirements)? Do you think they are reasonable? o What would you change about the terms of the loan? □ Have you managed to make all your repayments on time? o If not, why not? □ How helpful have FINCA staff been throughout the loan process? Explain. Delivery channel □ Do you use branchless banking services for your FINCA loan? If yes, which? [MPesa, Agent, ATM] 38 o If yes, would you have taken a FINCA loan if a branchless option was not available? o How would you rate your overall experience using branchless banking services Savings and loans group □ Are you a member of a savings and loan group? o If yes, did you receive any training from FINCA? o What was most useful about the training? o What could have been improved? Overall □ What has been the biggest surprise about working with FINCA? □ What is one thing that FINCA does really well? □ If you could improve one thing about the way FINCA works with you, what would it be? □ What is the most important thing FINCA could improve? Question checklist for KIIs – Staff and agents Background □ Where do you work? □ What services is your branch/company you providing? □ Do you work with other projects than FFPr? Which ones? □ What is your current job? □ How long have you worked for/with FINCA? How long have you been doing your current job? Clients and impact □ What would be your guess as to how most of your clients would have got financing if not from here? Where did most previously? □ Do you think most use the financing as intended? How do you follow up on that? □ There’s a social component to FINCAs work – how do you track the impact you’re having? □ What do you think FINCA’s clients appreciate the most about the products and services FINCA offers? □ What do you think FINCA’s clients see as their biggest challenge in dealing with FINCA? Operational □ What is the process for clients to apply for and receive loans? □ About what percentage are successful in applying? o What are the main reasons applicants are not approved? o How can the process be improved? □ What about the loan terms? Are they where they should be? □ What’s the difference between Pamoja and the older lending product? o What are the advantages of Pamoja for you and for clients? o Are there any disadvantages or challenges? □ What is the process for providing follow-up advice or assistance to clients on the use of their loans? □ What are the most common issues that clients have with their loans? Training and other interaction with FINCA Tanzania □ Did you receive training from FINCA? What were the topics? o How useful was the training? o How could training have been better? □ What other interactions do you have with FINCA Tanzania? Describe. 39 New Branch staff only □ Is the branch up and running at full capacity? Why or why not? □ What challenges have you seen in getting the branch up and running? □ What would you have done differently during startup in retrospect? Branchless staff only □ Do you think you are reaching clients who otherwise wouldn’t get formal financing? Explain. □ What challenges have you had so far? □ What do you need to do to continue to expand these services to new clients? 40 Focus Group Discussion Guide Introduction for the Focus Group Leader ISG is conducting research on behalf of FINCA International. The objective of the research is to understand the goals, needs, and preferences of microenterprises that have borrowed money under FINCA’s Food for Progress project and to understand how FINCA Tanzania has worked to meet clients’ expectations. ISG’s research will be conducted via a survey, focus group discussions, and key informant interviews. This focus group guide serves as a manual for the focus group leader and note taker conducting focus groups. ISG will conduct two types of focus groups; a client group and a FINCA staff group. The client focus group will cover five topics:  The role and potential of microfinance in the Tanzaniaan food and agriculture sectors;  Client experience and satisfaction in working with FINCA;  The opportunities for, and challenges to, growing FINCA’s portfolio of Tanzaniaan food and agriculture sector loans;  Client experience and opinion of the Tchotcho mobile repayment service; and;  The potential of microfinance to improve the quality of life for people that work in food and agriculture. The staff focus group is intended to capture information about project design, project management, and customer management. The purpose of these focus groups is to present broad issues and to let participants generate ideas and build on each other’s ideas. The focus group leader’s goal is to encourage participation and ask questions that build on participants thoughts. This guide provides follow up questions that you may use. You may ask specific participants to compare their opinion or thoughts to those of other participants through prods such as “Participant B, how does your experience compare to the experience of participant A that we just heard?” or “Participant C, what would you add to participant B’s comments?” The interaction that the focus group leader encourages will provide depth to the information we are gathering through the survey and interviews. Preparation  Make sure there is a sign-in sheet, pen, and paper for each participant.  Make sure the room is set up with flip charts or whatever materials you need.  Set up refreshments.  If available, set up small gifts to thank the focus group participants. Introduction, Ground Rules, Logistics and Sign-in Sheet [FOCUS GROUP LEADER: Introduce yourself and the note taker.] Thank you all for participating in this focus group. My name is _________ and I will lead the group, and this is ____________ who will take notes. Our discussion will last no more than an hour and a half. Before we get started, let me mention that there are drinks and refreshments available (point them out), and the bathroom is (indicate location of bathroom). Please feel free to get up and move around or help yourself to refreshments during the discussion. The purpose of this focus group is to understand the importance of FINCA’s loans to food or agriculture sector businesses. Particularly, we’d like to understand: 41  If the loans increase the productivity of your farm or business?  If the loans increased your income and the income of people around you?  How FINCA could improve its products and services to better serve you and reach other businesses? I wanted to mention a few important points before we start.  The purpose is for us to learn from you, both your positive and negative experiences and opinions.  We’re gathering information, not trying to achieve consensus. Please contribute your experience and opinion. It’s ok for there to be disagreement or different ideas  We’re doing interviews and focus groups in our research. The purpose of focus groups is generate new ideas and to see where there are areas of agreement and disagreement.  Please take a minute to read over the consent form that is in-front of you, and if you agree to participate, sign it and we’ll collect them at the end of the session. We have a few ground rules that we hope you’ll agree with:  Please participate. Your contribution is valuable and will improve FINCA’s products and services in the future.  Please agree to keep the information provided in the focus group confidential. We won’t discuss any sensitive topics. Still, people may share ideas that they’d rather not be discussed privately.  Stay with the group and please don’t have side conversations.  Turn off cell phones if possible. Are there any questions before we get started? FOCUS GROUP LEADER: Go around the table and ask people to introduce themselves by their name and the type of business they manage. Questions 1. Think about the businesses you know that, like yours, are in food and agriculture sector. Why do these businesses most commonly need loans? What are their goals? Follow Up Questions a. What are the opportunities to grow a business in food and agriculture in Tanzania? What are common obstacles to growing business in food and agriculture? b. How important is obtaining a loan from FINCA to achieving business goals? What else is needed in addition to the loan? Is taking a loan ever a bad idea for a business like yours? c. What are the ways in which taking a loan has assisted you in achieving your goals? 2. FINCA provides a few services that are intended to make banking easier. For example, they provide mobile banking, access to ATMs, and point-of sale services. [note: define these services if necessary. Are you aware of all, some, or none of these services? If some, which? (If none, skip to follow up b) Follow Up Questions 42 a. How did you first find out about the service (e.g. Loan officer told me, I saw a flier around town) b. Which of the services do you use most frequently? Why c. What do you see as the benefits of the services? What are the drawbacks? d. If you don’t use the service, what sounds appealing about it? What is unappealing? e. Thinking about the business people you know, why would they not use the services if it were available to them? f. What could FINCA do to increase use of the service among businesses like yours? 3. One purpose of FINCA’s services is to increase agricultural productivity, either on farms or in businesses that make and sell food products Does FINCA’s services increase productivity for people like you? Follow Up Questions a. Which of FINCA’s products or services most increased productivity for you? How? b. How do people know if their business’s productivity has increased? 4. Many people take loans from FINCA to invest in activities that will increase their income and directly improve their lives. Do you think that obtaining a loan from FINCA has directly led to increases in your income? In what ways? Follow Up Questions a. Has your loan led to improvements in other people’s income? For example, did you hire people with the money, spend more with current suppliers or buy from new suppliers? b. What could FINCA do to make sure that loans lead to increases of income for customers? 5. I’d like to talk about your experiences working with FINCA. Have you found the process of obtaining and managing a FINCA loan easy or difficult? Follow Up Questions a. What aspects of working with FINCA do you like the best? b. Which aspects have been most difficult? c. If you could change anything about the loan and service you received from FINCA, what would you change? (e.g. Loan amount, repayment time, application process.) 6. Is there anything else that you would like FINCA to know about their products and services or about the Food for Progress program? 43 Client survey Introduction Hello. Thank you for participating in this interview. My name is _________ and I am working on a project for International Solutions Group, a research organization based in the United States. ISG is conducting a study on behalf of FINCA International. The purpose of this survey is to better understand how FINCA customers in the food and agriculture sectors use and benefit from FINCA products and services and how they might be improved. This survey should take about 20 minutes. Your participation in this interview is voluntary. If you don’t want to answer any question, you don’t have to. Your responses that you provide are confidential: we won’t mention your name in any report we write and we will never connect your name with anything you say. Your responses will be combined with responses from other people. Do you have any questions before we begin? Background Questions 1. Date of interview (day/month) 2. Name of interviewer 3. Gender of respondent a. Male b. Female) 4. Are you currently a FINCA client? (1 = yes, 0 = no) a. Yes [GOTO Q5] b. No [GOTO Q6] 5. For how many months have you been a client? [GOTO Q7] 6. How many months ago were you last a client? 7. Which FINCA Branch do/did you use? a. Eastern Region/Dar Es Salaam i. Magomeni Mwembechai ii. Ilala iii. Tegeta iv. Temeke b. Central Region i. Dodoma ii. Ifakara iii. Kibaya iv. Morogoro v. Mpwapw c. Lake Region i. Bunda ii. Mara iii. Mwanza iv. Tarime d. Northern Region i. Arusha ii. Arusha Meru iii. Moshi iv. Muheza 44 v. Tanga e. Southern Region i. Iringa ii. Mbeya iii. Njombe iv. Songea f. Western Region i. Bukoba ii. Geita iii. Kahama iv. Shinyanga 8. How long does it take you to travel to reach this branch? (in minutes) 9. How do you travel? a. Walk b. Car c. Bus d. Bicycle e. Other 10. Which products do or did you use? (choose all that apply) a. Loans [Yes: Include Q11; No: Goto next indicated question] b. Agency banking [Yes: Include Q 12; No: Goto next indicated question] c. Mobile phone banking [Yes: Answer Q 13; No: Goto next indicated question] d. ATMs [Yes: Answer Q 14; No: Goto next indicated question] e. Savings [NOTE: IF B,C, or D is NOT SELECTED – include Q16,] [NOTE: IF B,C, AND D ARE NOT SELECTED – GOTO Q18] 11. How many loans have you taken from FINCA in the past 12 months? 12. About how often have you typically used agency banking in the past 6 months? a. More than once a day b. Once a day c. At least once a week d. At least once a month e. Less frequently than once a month 13. About how often have you typically used mobile phone banking in the past 6 months? a. More than once a day b. Once a day c. At least once a week d. At least once a month e. Less frequently than once a month 14. About how often have you typically used ATMs in the past 6 months? a. More than once a day b. Once a day c. At least once a week d. At least once a month e. Less frequently than once a month 15. Would you have taken a FINCA loan if branchless banking was not available? a. Yes b. No 45 16. On a scale from 1 to 5, with 1 meaning very well and 5 meaning very poorly, how well does branchless banking serve your needs? a. 1. Very well b. 2. Well c. 3. Neutral d. 4. Poorly e. 5. Very poorly 17. What is the main reason you don’t use an agent, mobile phone banking or ATMS? a. I don’t know anything about them b. They are not available near me c. I don’t trust them d. I have heard bad things about the services e. Different reasons for each f. Other (write). Use, Impact, and Alternatives 18. What type of business is your main source of income? a. Agriculture/Growing Crops b. Animal or Fish Production c. Forestry/Timber Production d. Dairy Production e. Other Food Product Production f. Food processing g. Retail Fish or Meat h. Retail Vegetables/Nuts/Fruits/Grains i. Retail Grains j. Retail live plants k. Other 19. Why did you initially obtain a loan from FINCA? a. To increase farm production/output b. To access new sales outlets c. To improve quality of your product d. To purchase new equipment e. To purchase inputs (Seeds/Fertilizer) f. To hire new labor g. To purchase inventory h. To replace old equipment i. To cover daily operating costs j. To pay back old debt k. For personal/family use l. Other (write below) 20. Did you use the FINCA Loan in the way you initially intended? a. Yes [GOTO Q22] b. No [GOTO Q21] 21. Why couldn’t you use the loan in the way you intended? a. The loan wasn’t a sufficient amount for my original plan b. I had an emergency that changed my financial priorities c. I found another opportunity that would earn more money 46 d. Other (write) 22. How would you have financed these investments if you did not have a FINCA loan? a. A loan from a different bank or MFI [GOTO Q23] b. Income from another business [GOTO Q23] c. A village lender [GOTO Q23] d. Family member or friend [GOTO Q23] e. Sell livestock or equipment or some other asset [GOTO Q24] f. I would not be able to finance the investment [GOTO Q24] g. Other [GOTO Q22] 23. What monthly interest rate would you have paid for this other loan? (%) 24. Why did you chose FINCA over these options? (check all) a. Lower interest rate b. No collateral requirement c. Better location d. More trustworthy e. Other (write) 25. Did the loan you obtained from FINCA increase your income? a. Yes [GOTO Q26] b. No [GOTO Q27] 26. How much did your income increase? a. Less than 5% b. 5% - 10% c. 10% - 20% d. 20% - 30% e. More than 30% 27. Are you a member of a savings and loan group? a. Yes [GOTO Q25] b. No [GOTO Q 28. Did you receive any training from FINCA? (1 = yes; 0 = no) a. Yes [GOTO Q26] b. No [GOTO Q 29. On a scale from 1 to 5, with one meaning very useful and 5 meaning completely useless, how useful was the training? a. 1. very useful b. 2. Useful c. 3. neutral d. 4. Not very useful e. 5. Completely useless 30. Did the training you received improved your financial management of your farm or business? a. Yes b. No 31. Do you intend to use FINCA again if you need credit for your business? a. Yes b. No 32. How likely are you to recommend FINCA to someone else? a. 1. Very unlikely b. 2. Unlikely 47 c. 3. Neutral d. 4. Likely e. 5. Very likely) 29. What is the one improvement or change FINCA could make to increase the chance you use them again or recommend to a friend? Thank you! 48 Annex VIII: Evaluation Scope of Work Project duration: 09/01/2012 – 8/31/2016 (anticipated completion date) Geographical implementation: Morogoro, Dodoma, Ifakara, Iringa, Mbeya, Njombre, Mwanza, Mara, Shinyanga, Tabora, and Bukoba. Other targeted areas will likely include Lindi, Singida, Mtwara, and Tanga. Position Final Evaluation Consultant for FINCA Tanzania USDA Food for Progress Program Contract Type and Duration Service Contract. ASAP starting/travelling approximately on 07/15/2016 Location US, Tanzania Final Evaluation Purpose The purpose of the final evaluation is “to assess whether the project has achieved the expected results as outlined in the results framework. The final evaluation should assess areas of project design, implementation, management, lessons learned and replicability. It should seek to provide lessons learned and recommendations for USDA, program participants and other key stakeholders for future food assistance and capacity building programs. The evaluation will likely use mixed methods approaches as outlined in the agreed upon evaluation plan. In general the final evaluation should assess: relevance, effectiveness, efficiency, impact and sustainability.”16 Similar to the interim evaluation, “USDA requires that the evaluation is managed by the organization’s evaluation unit. If the organization does not have a dedicated evaluation unit the evaluation should be managed by a project staff person or organizational staff person with significant knowledge and expertise concerning evaluation. Ideally, the organization would maintain an evaluation unit that was separated from the staff or line management function of the project being evaluated. Such a structure helps to ensure the independence and impartiality of the evaluation process and report of findings, conclusions and recommendations. In cases where the organization does not have a dedicated evaluation unit or organization staff with significant expertise in evaluation, USDA may decide to manage the project level evaluation through its own monitoring and evaluation unit.”17 The survey work, data analysis and prepared written report will be conducted in compliance with plan outlined in the FINCA USDA Evaluation Plan – TZA 27 Dec 2012 (to be provided). The final evaluation report will be delivered to FINCA International by no later than October, 2015. 16 Food Assistance Division, Office of Capacity Building and Development, Foreign Agricultural Service, U.S. Department of Agriculture; FY 2013 Food Aid Proposal Guidance: Annex IV, page 14. http://apps.fas.usda.gov/excredits/foodaid/2015Solicitation/Annex%20IV_MEPolicy.pdf 17 Food Assistance Division, Office of Capacity Building and Development, Foreign Agricultural Service, U.S. Department of Agriculture; FY 2013 Food Aid Proposal Guidance: Annex IV, page 15 http://apps.fas.usda.gov/excredits/foodaid/2015Solicitation/Annex%20IV_MEPolicy.pdf 49 Key Evaluation Questions These key questions will be addressed by the final evaluator. The evaluator should also asses the areas of management, including staffing and monitoring, along with coordination and collaboration. 9. To what extent have the planned objectives been achieved for the program? Specifically, were the annual monitoring indicators (activities and outputs) successfully reached? 10. Are new rural expansion offices appropriately located and equipped to serve rural and agriculture dependent populations? How can they be improved with respect to operational efficiency and effectiveness? 11. Are the terms of the new Agricultural Loan appropriate and effective? How can the product be strengthened? 12. Is Branchless banking technology improving access to financial services for rural and agriculture dependent communities? How can the mobile delivery of financial services be strengthened? 13. Are the Technical Assistance training materials and delivery appropriate? Which materials and/or delivery techniques need strengthening, if any, and how? 14. Which Food for Progress project activities were the most effective in increasing agricultural production and strengthening trade of agricultural products? Why? Which program components were the least effective? Why? 15. What is the perspective of beneficiaries (both FINCA clients and training participants) with regards to the services provided under the Food for Progress program? 16. What are lessons learned from this project and how can they be ad3dressed prior to the final evaluation? (*The primary document to guiding the evaluation focus and key questions of the project should be the “Annex 2: Results framework”). Methodology The final evaluation will determine the effects and benefits that the FINCA-implemented Food for Progress grant activities and outputs had on clients in Tanzania. The performance evaluation will be both quantitative and qualitative and will be implemented during the final two quarters of the last program year. Through participatory methods a team composed of an independent, third party external lead consultant (the Evaluator) supported by local independent, third party surveyors will examine FINCA’s Food for Progress program results. A visit to activity sites will allow field staff and beneficiaries to provide their inputs to the review process. The Final Evaluation will be conducted in the following stages: Stage I: Review of Existing Documentation The Evaluator will conduct a thorough review of existing data and information from FINCA’s field activities and other lending programs. In particular, the baseline and midterm evaluations and other primary data collected from clients will be reviewed and assessed. The Evaluator may consult additional documentation from the headquarters office, FINCA field offices, or other sources. The Evaluator will outline any data gaps or areas for further investigation as well as preliminary field visit plans. Stage II: Refinement of Review Methods The Evaluator will design and prepare evaluation tools to gather and assess the required information to meet Terms of Reference Requirements. Per the USDA- approved M&E plan, one of the primary 50 functions of the external evaluator will be to review and test the client survey and data collection and cleaning methodologies utilized by FINCA Tanzania, which include the tools used for baseline and midterm data collection. For some performance indicators related to credit expansion, quantitative data is obtained directly through FINCA’s financial records (client/credit reports) which are official, legal documents. The outcome of this review will be an opinion if the measurement and evaluation program in use to manage the USDA program is sufficiently well designed and implemented and that there are enough controls to ensure that the information is accurate and reliable. It will also point to potential gaps in the survey data that should be completed to properly report to USDA. Unless the review indicates material challenges to the reliability, accuracy and completeness of FINCA’s internal management information, the Evaluator will rely upon these FINCA’s data to include in the evaluation. In addition, the Evaluator will develop additional qualitative and quantitative tools based upon the review of existing program documentation and data to gather any required information not already collected and which are specifically outlined in Annex 3: PMP – see indicators designated for collection by independent, third party evaluators. Tools for supplemental data collection will be presented for FINCA and USDA review prior to field evaluation activities. The data collection tools will combine both quantitative and qualitative approaches. Quantitative data will also be drawn from client surveys and interviews, and credit officer interviews. For other indicators, qualitative methods will be a better way to gather more useful information. Qualitative methods will include, but not be limited to, semi-structured interviews, surveys, direct observation, focus groups, secondary data review, key informant interviews, and random spot checks (visits to field offices and target population homes/fields). The Evaluator will advise on choosing a sample of beneficiaries to be interviewed/surveyed. Additional sources of information for the evaluation will include monitoring project data and recommendations made in the mid-term review. Stage III: Field Data Collection The Evaluator will travel to FINCA Tanzania administrative headquarters and to Tanzanian communities and households served with USDA funding in order to collect required data for project analysis. The Evaluator will work with FINCA’s field representative to plan and coordinate all the necessary logistics for the qualitative and quantitative collection of data at the field level. The field review will be conducted in a randomly selected number of offices selected based on locations where USDA-supported agricultural loans were disbursed and where new offices were opened with USDA funds. Stage IV: Write Evaluation Report Upon completion of the field data collection, the team leader will compile, clean and analyze all data collected and then prepare the detailed report outlining the purpose of the review, methodology, primary questions, findings, lessons learned to date, and recommendations. The team leader will present FINCA staff with the findings, lessons learned, and recommendations and then will prepare the final report based on FINCA’s feedback. FINCA will submit the final report to USDA no later than the activity completion date. Responsibilities 1. Provide a Work Plan & detailed outline of the evaluation with key questions, major tasks, and specific schedules prior to initiating the evaluation. Work plan will be approved by USDA and FINCA prior to implementation. Methodology will include:  Participatory approaches to involving key stakeholders including USDA representatives in D.C. and in-country, implementing partners or sub-contractors, project beneficiaries and other relevant program partners. 51  Key questions will relate specifically to the programmatic technical areas of focus, e.g. the development of new agricultural lending product, establishment of new branches, establishment of new financial service delivery channels, promotion of saving accounts and mobilization of deposits, provision of agricultural loans and research on client satisfaction and social impact. 2. Compile all programmatic documentation from the field including operational reports and data and other materials necessary to assess the programmatic goals and objectives 3. Interview key management and operational staff in head office and branches as necessary to report on the program implementation.  USDA Washington and the regional agricultural attaché will be considered as a key informants, and will be interviewed by the evaluation team, preferably prior to the field data collection.  The evaluation team will also interview with other key stakeholders, Government officials involved in the project, branch office staff, and other organizations and individuals the project works closely with, including USG or non-USG donor funding projects. 4. Serve as the team leader to complete the qualitative / quantitative surveys including:  Orienting the local survey team by providing them with a orientation of the program and the purpose of the surveys  Coordinating with FINCA Tanzania and HQ M&E staff on logistics and local surveyor work,  Manage a team of local surveyors to implement the surveys and ensure that all data is correctly entered  Insure that all team members fulfill their obligations, and organizing and directing the team interaction. 5. Compile and clean all data necessary for the quantitative client evaluation including:  Data from the initial and final field work  MIS and other operational data, as necessary  Transform the data to SPSS and complete the cleaning process 6. Review and run basic frequencies of all variables, as necessary. Build on the analysis done by FINCA Tanzania M&E unit, determine in consultation with FINCA Tanzania M&E staff appropriate additional correlations to run, run correlations, and provide cross tabs, 4x4 tables and analysis with confidence intervals. USDA expects the lead external evaluator will have the ability to conduct a full and independent analysis of the data collected through the evaluation field work and surveys. This allows for a comprehensive and objective picture of the project’s achievements and outcomes and enhances the credibility of the evaluation. 7. Prepare a concise report of the findings of the quantitative survey. 8. Identify any weakness in survey design and implementation and make suggestions for improvements for future surveys. 9. Compile both the programmatic evaluation findings and the quantitative survey results into a summary report answering the key questions proposed in the evaluation plan for the final review 10. Present findings to senior leadership and program management staff of FINCA Tanzania, FINCA International and USDA through a PowerPoint presentation, with an emphasis on:  Identifying what the contracted outputs/outcomes are and evaluating what is actually being accomplished.  Identifying strategic opportunities and/or problems and constraints and working with local staff to develop actionable recommendations to deal with them to improve the design and implementation of the program.  Summarizing the findings of the baseline quantitative survey 52 11. Make relevant, useful and realistic recommendations for interventions based on the results of the survey and or the need for further investigation 12. Prepare final evaluation report as specified below. Final evaluation timeline The final evaluation will be scheduled for the final two quarters of Year three of this three-year program. The proposed timeline of evaluation activities is outlined below. Stage Task Deliverables Estimated Person Days Status I Begin 9 months before end date Prepare Terms of Reference and Select Evaluator Terms of Reference completed, External Consultant retained. 15 (15 days x 1 FINCA staff) In process II Duration: 2 Weeks Review Existing Documentation None 8 (4 days x 2 team members) To be completed by independent, third party evaluator III Duration: 3 Weeks Preparation of Tools and Field Team for Data Collection Field visit schedule and itinerary, respondent selection, and data collection tools. 22 (10 days x 1 Evaluator + 6 days x2 team members) To be completed by independent, third party evaluator IV Duration: 4 Weeks Field Data Collection Completed survey tools and data collected 60 (20 days x 3 team members) To be completed by independent, third party evaluator V Duration: 4 Weeks Write Final Evaluation Report Draft to FINCA for comment by 3 weeks before project end date Final report to FINCA 1 week prior to project end date 28 (16 days x 1 team member + 6 days x 2 team members) To be completed by independent, third party evaluator Deliverables 1. Work plan/timeline 2. Final Evaluation TOR 3. Quality Assurance Plan 4. Draft Evaluation Report 5. Interim Evaluation Report 6. Final Evaluation Report 7. Presentation of evaluation The evaluation report will follow the outline/format below:  Cover Page w/ photo  Table of Contents which identifies page numbers for the major content areas of the report.  Executive Summary (2 to 3 pages) should be a clear and concise stand-alone document that gives readers the essential contents of the evaluation report, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the 53 detailed information within the report. In addition, the Executive Summary helps readers determine the key results and recommendations. Thus, the Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of targets and intended outcomes; areas of meaningful under or over achievement; and possibly a few lines describing the action plan developed to follow up on evaluation recommendations and how the evaluation report will be disseminated.  Introduction: including objective of SOW and Brief Description of Project  Methodology: sampling method including strengths and weaknesses of method used, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken, including justification and software package used. The discussion of any random sampling used should include details on how the random respondents were identified and invited to participate. This section should also address constraints and limitations of the evaluation process and rigor. The methodology section should also include a detailed description of data collection techniques and primary questions used throughout the evaluation.  Updated Indicator Performance Tracking Table  Discussion of Performance Results: This is to describe the key findings from the survey as well as providing all materials from the research program including raw and final cleaned data sets, analysis files and output tables. This section should be prepared in line with the outline below. o Brief Description of Interventions o Achievement of Results  Meeting targets (annual and performance indicators)  Other achievements o Discussion of general evaluation questions  Cross-Cutting Issues  Lessons Learned. This is space for the evaluator to think about the data and results, and make concrete recommendations for current or future project improvements/changes or to build on successes, pull out organization lessons learned, and generally comment on data and results.  Recommendations for further institutional strengthening activities: Everything presented in this section must be directly linked back to the information presented in the Results section of the report. Ideally, items discussed here will not be completely new to the reader, but rather will refer to previous discussions. Recommendations that are not directly tied to Results can be included in an Evaluator Comments section for the report.  Annexes: o Evaluation SOW o Composition of the team o Methods o List of sites visited o List of key informants o References o Indicator Performance Tracking Tables (IPTT) o Survey tools o List of acronyms Payment Payment will be made at the end of the contract. Payment will be approved upon receiving the final deliverables. 54 Qualifications  Masters in Economics, international development, sociology or related field.  Minimum of ten years of professional experience.  Program evaluation experience (quantitative research, design and implementation) with USDA, USAID and other international agencies in area of economic development, microfinance, agricultural development, or related fields.  Field experience leading and managing small and medium-scale data collection activities and surveys, using a mix of quantitative and qualitative methods.  Strong analytical, writing and editing skills. Candidate must provide copies of reports primarily produced by the candidate.  Experience with Microsoft Office Suite (Excel, Word, Access and Power Point), Data Spreadsheets and statistical packages like STATA and SPSS.  Proficiency in a foreign language, such as Swahili (preferred).  Experience working in developing countries.  Ability to travel overseas for short-term assignments.  Complies with all third-party independent evaluator requirements outlined in 7 CFR Part 1599.13 and articulated in FY 2013 Food Aid Proposal Guidance: Annex IV pp. 6-7. Reporting FINCA International is the USDA grant holder and will manage the evaluation contract. The FINCA International team managing the consultant evaluation process is comprised of Audrey Burke and Jennifer Schmidt. Audrey Burke is a Senior Customer Research Manager for FINCA International with the responsibility to review and assure the validity of all M&E technical approaches and data analysis. Jennifer Schmidt is the Director for New Business Development for FINCA International and is responsible for managing the USDA grant and that the evaluation proceeds in-line with donor and contract requirements. The ground day-to-day coordination will occur with the FINCA Tanzania management and M&E Team, who will provide access to program records, staff and clients and will support logistics as needed for the evaluation team to complete their work. In advance of the final evaluation, FINCA Tanzania will take steps to notify clients of the following:  There will be an independent evaluation of this project, so clients may be contacted by an evaluator and asked to speak about their experience with the project. The evaluator who contacts clients will identify themselves as an independent evaluator of this project.  FINCA will not be physically present during the independent data collection process, but will work with the evaluator to provide logistics and ensure the evaluator has access to FINCA clients and contact information to ensure a thorough evaluation.  FINCA supports this evaluation. We are eager to learn about participants’ experiences with this project, both positive and negative.  Please participate in data collection if asked and able. By participating and sharing their honest feedback, clients are helping to improve and continue these types of projects in the future.  All client input will be confidential.  Client participation in data collection will not affect participation in this project, or any future project.