WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) MAY 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) Management Systems International Corporate Offices 600 Water Street, SW Washington, DC 20024 Contracted under Contract Number: RAM-00-04-00049-A-FY05-051, Task Order: 051 EVALUATION REPORT: WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) i CONTENTS ACKNOWLEDGMENTS .......................................................................................................III PROJECT SUMMARY........................................................................................................... IV ACRONYMS............................................................................................................................ VI EXECUTIVE SUMMARY ....................................................................................................VII SUMMARY EVALUATION REPORT................................................................................. IX ANNEX 1: WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM).................................................................................................................1 I. INTRODUCTION................................................................................... 2 II. THE DEVELOPMENT PROBLEM..................................................... 3 A. Problem Statement ........................................................................... 3 B. USAID Intervention......................................................................... 3 III. EVALUATION METHODOLOGY...................................................... 7 A. Evaluation Purpose and Questions................................................... 7 B. Evaluation Design............................................................................ 8 C. Data Collection Sources and Methods ............................................. 9 D. Data Limitations............................................................................... 9 IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ON THE OVER ARCHING QUESTIONS......................................................................................... 10 A. Relevance....................................................................................... 10 B. Effectiveness .................................................................................. 13 C. Impact ............................................................................................ 29 D. Efficiency....................................................................................... 36 E. Sustainability.................................................................................. 40 F. Replication ..................................................................................... 43 V. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ON THE CROSS CUTTING QUESTIONS......................................................................................... 43 A. Gender............................................................................................ 43 B. Reporting........................................................................................ 44 C. Communications and Outreach...................................................... 44 D. Coordination................................................................................... 45 VI. LESSONS LEARNED .......................................................................... 46 ANNEX 2: BIBLIOGRAPHY .................................................................................................47 ANNEX 3: LIST OF INDIVIDUALS INTERVIEWED .......................................................48 ANNEX 4: GETTING TO ANSWER MATRIX APPLIED TO EVALUATION QUESTIONS OF THE WHAM PROJECT......................................................49 ANNEX 5: SURVEYS ..............................................................................................................53 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) ii FIGURES Figure 1: WHAM Results Framework.......................................................................................... 4 Figure 2: Distribution of Project Loans [Downscaling – 813 total loans] .................................. 12 Figure 3: Distribution of Project Loans Downscaling and Scaling Up....................................... 12 Figure 4: WHAM Quarterly Total Credit Disbursement and Number of Loans Disbursed.................................................................................................................... 19 Figure 5: Distribution of NBP Loans.......................................................................................... 20 Figure 6: Regional Share of Total Loans Disbursed (2006 and 2007) ....................................... 20 Figure 7: ASASAH: WHAM Quarterly Total Credit Disbursement and Number of Loans Disbursed......................................................................................................... 25 Figure 8: Use of Loans................................................................................................................ 32 Figure 9: Projected Project Impact ............................................................................................. 35 Figure 10: Disposition of Expenditures ...................................................................................... 39 Figure 11. Selected Expenditure Ratios......................................................................................39 TABLES Table 1: Getting to the Answers for Overall Evaluation Questions.............................................. 9 Table 2: WHAM Loan Activity: Indicators and End-of-Project Targets ...................................14 Table 3: WHAM Direct Loan Activity – NBP and Asasah........................................................ 15 Table 4: Industry Strengthening – Indicators and....................................................................... 15 Table 5. Primary Monetary And Non-Monetary Benefits Of Microfinance Lending ................ 29 Table 6. Reported Change in Profits........................................................................................... 31 Table 7. Impact of Loan on Business Practices .......................................................................... 33 Table 8. Loan History ................................................................................................................. 33 Table 9. Estimated Current Impacts............................................................................................ 34 Table 10. Sensitivity of Project Impacts to Projection Assumptions.......................................... 35 Table 11. Costs and Benefits of Upscaling Component by Year................................................ 37 Table 12. Summary of Efficiency Measures for Scaling Up Component at Project Completion................................................................................................................. 38 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) iii ACKNOWLEDGMENTS The evaluation team wishes to express our special gratitude to Mr. Gregory Chen and his staff of the Widening Harmonized Access to Microfinance Project, for their overall orientation and generous support to execute the evaluation tasks. They made themselves available with no reservations. Their willingness to offer frank and open views on activities and problems of the project was refreshing. This greatly facilitated the ability of the team to conduct this evaluation. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) iv PROJECT SUMMARY Project Name and Strategic Objective (SO) Widening Harmonized Access to Microfinance SO6: Increased Opportunities for the Poor IR6.3: Improve Private Sector Growth IR6.4: Improve Financial Sector’s Contribution to Economic Growth Project Dates March 2005 to March 2008 Project Location Nationwide Implementing Partner Shorebank International Ltd. Cooperative Agreement Request for Assistance (RFA) extended to Shorebank International Ltd., Activity Number 391-05-02 Budget $4,999,514 Summary of Efficiency Measures for scaling up activities at Project Completion1 Measure Value Present value of project costs (scaling up only) $1.8 million2 Present value of benefits (scaling up only) $1.8 million Net present value $0 Internal rate of return 2% Benefit cost ratio 1.0 Annual cost per beneficiary $84 Number of direct beneficiaries 19,882 SMEs/households Average annual benefit/beneficiary $83 Note: The summary measures in the table refer only to the approximately one-third of the project that focused on scaling up. It includes benefits only to the estimated 56% of scaling up loan recipients who ran retail businesses. 1 The Impact Section of Annex I projects benefits for five years beyond project completion. The net present value of benefits through 2012 is $61.8 million with no additional project costs. 2 The present value of project costs exceeds the project budget because the present value calculation inflates costs incurred prior to 2007. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) v WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) vi ACRONYMS AKAM Aga Khan Agency for Microfinance CEO Corporate Executive Officer CGAP Consultative Group to Assist the Poor DFID Department for International Development FATA Federal Administrated Tribal Areas FMFB First Microfinance Bank FSSP Financial Sector Strengthening Program GDP Gross Domestic Product GOP Government of Pakistan MFBs Microfinance Banks MSI Management Systems International NBP National Bank of Pakistan NGOs Non-Government Organizations NRSP National Rural Support Programme PPAF Pakistan Poverty Alleviation Fund PMN Pakistan Microfinance Network RFA Request for Assistance RSPs Rural Support Programmes SAS Shorebank Advisory Services SBF Small Business Finance SBP State Bank of Pakistan SDC Swiss Development and Cooperation SMEs Small and Medium Size Enterprises USAID United Agency for International Development WHAM Widening Harmonized Access for Microfinance WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) vii EXECUTIVE SUMMARY For the past sixty years, Pakistan experienced an uneven pattern of economic growth with widespread poverty and unequal distribution of income. In addition, many people do not engage in dynamic market transactions. Large numbers of households and businesses have no access to formal financial services. In fact, although Pakistan has a population of 165 million, the total number of borrowers in the formal financial system barely surpassed 4 million households and businesses.3 This financial outreach is significantly below potential. When USAID resumed operations in 2002 here, Pakistan did not have dynamically developing micro and SME finance sectors. In response to this problem, USAID launched three finance projects: Developing Non-Bankable Territories for Financial Services, to support micro loans below $500 to low income households in Baluchistan, Sindh and the Federal Administered Tribal Areas; the Enterprise Development Facility, to wholesale loans to credit-granting non-government organizations that extend credit between $500 and $1,666; and Widening Harmonized Access to Microfinance (WHAM). WHAM set out to improve financial access to customers in a financial “missing middle.” This refers to those who seek loans between $500 and $25,000. Credit-granting NGOs did not address these loan demands, nor did smaller specialized banks or the larger commercial banks. To achieve this objective, USAID designed a two-pronged strategy. The first part aimed to develop support for a microfinance industry. This consisted of many activities, such as raising awareness among stakeholders, improving knowledge of the market, and the identifying growth-constraining factors in the industry. At the project’s outset, stakeholders seemed uninformed or possessed only partial knowledge on issues like the real cost of delivery of financial services, effective interest rates, and size of the market. Clearly, Pakistan lacked effective education on the known best practices of microfinance, such as cost-recovery loan rates, character-based lending, and financial sustainability. In addition, stakeholders did not fully understand how microfinance contributed to poverty alleviation. The second part of the strategy involved training and building capacities of financial institutions interested in venturing into the aforementioned “missing middle.” To work most effectively in this area, the project used both downscaling and scaling up approaches. Conceptually, downscaling concentrates on commercial banks “going down market,” which means these banks penetrate lower income niches that were traditionally alien to them. Scaling up, on the other hand, focuses on increasing the loan capacity of financial entities with a long history of extending credit to low income households and micro enterprises. These are primarily credit-granting NGOs, but also formally supervised microfinance banks that have been purposefully created to serve such clienteles. In late 2007, USAID requested a final evaluation of WHAM, which is set to end in March 2008. Conclusions The evaluation focused on 10 primary themes. The following table summarizes the main conclusions for each theme. 3 Source: Shorebank International Ltd. “The SME Market in Pakistan. Initial Analysis of Markets and Trends.” (Power Point Presentation to Crescent Commercial Bank, April 2007., with data collected from the State Bank of Pakistan in June 2006). WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) viii Evaluation Theme Key Conclusions Relevance ƒ WHAM succeeded in addressing real needs of beneficiaries by facilitating strategic decisions to target distinct financial entities -- commercial banks, microfinance banks and credit-granting NGOs. The project also designed the contents of technical assistance services that corresponded to the unique characteristics of each type of entity and the particular loan demand profiles of their clienteles. Effectiveness ƒ WHAM contributed directly to the extension of 1,967 loans with a total value of $28 million. The numbers surpassed initial targets by 257% and 180% respectively. ƒ WHAM enlisted the participation of three commercial banks. Only one of these led directly to increased loan activity. ƒ WHAM was effective in enlisting six microfinance partner organizations and in providing each of them useful and relevant technical assistance. ƒ WHAM contributed to raising the skills of 724 finance sector practitioners, 114 of them women. ƒ WHAM improved knowledge and enhanced the visibility of the microfinance industry. Impact ƒ A survey of borrowers from one of the scaling up partner organizations (Kashf) produced limited evidence that WHAM-supported lending had a positive impact on business profits. ƒ On average, survey respondents reported diverting about one-third of additional profits to their households, so WHAM-supported lending increased household incomes. Efficiency ƒ The project was likely reasonably efficient, although the measures of benefits are not refined enough to make a firm determination. Sustainability ƒ The sustainability of downscaling activities and results cannot be conclusively ascertained. As regards scaling up activities, given the application by the Contractor of microfinance best practices, it is possible to conclude that they enhanced sustainability. By lending support to the State Bank of Pakistan and the Pakistan Microfinance Network, WHAM contributed to strengthening the microfinance industry and its prospects for long-term sustainability. Replication ƒ Project replication would create uneven results. Industry strengthening and scaling up are likely to yield positive results, but downscaling is likely to be largely ineffectual, at least in a three to five year timeframe. Scaling up activities would replicate well in all regions except areas of insecurity, very low population density, and abject poverty. Gender ƒ WHAM effectively contributed to increasing outreach to women borrowers, albeit modestly. Asasah made slightly more than 1,000 loans to women. Reporting ƒ WHAM fulfilled reporting requirements and in a useful manner. Communication and Outreach ƒ WHAM was very effective in getting its story out. Coordination ƒ WHAM effectively coordinated its efforts with key stakeholders of the microfinance industry. Recommendations The WHAM project ended in March, 2008 so this evaluation is a final evaluation. The following recommendations can be considered if USAID maintains a focus on finance projects. 1. USAID should define with as much precision as possible the contours of market segments upon which activities will be applied and should not support in the short run projects that consider downscaling activities. 2. USAID should consider a timeframe of no less than five years for projects that attempt to build institutional capacities of micro and small enterprise finance providers. 3. For entities that benefit from technical assistance and training, USAID should explicitly define indicators and targets that measure performance towards expanded knowledge and skills, and increased use of good/best practices. 4. USAID should commission studies to shed light on (a) the opportunities and constraints that underline potential interventions that target poor women; and (b) demand for financial services other than micro loans. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) ix SUMMARY EVALUATION REPORT4 I. INTRODUCTION In Pakistan, the financial sector’s contribution to economic development remains substantially unrealized. Large numbers of households and businesses have no access to formal financial services. As of June 2006, the total number of borrowers in the formal financial system barely surpassed 4 million households and businesses.5 In a country with a population of 165 million people, this financial outreach is significantly below potential. Economic duality underlines the low financial outreach. Large commercial banks focus on corporate clients and on extending consumer loans to customers located in Karachi, Lahore and other urban areas. They also serve a small number of small and medium enterprises (SMEs), and largely stay away from rural areas with a concentration of low-income and poor populations.6 In all, the total of SME firms that are currently served by commercial banks reach approximately 160,000, which is equivalent to no more than 8% of potential customers.7 Regarding micro enterprises, microfinance providers currently serve approximately one million customers, just 6% of potential micro loan demand.8 The vast majority of households, individuals and business units with potential to demand micro loans obtain credit from informal sources, such as moneylenders, suppliers, traders, shopkeepers, relatives and friends. In March 2005, as part of a comprehensive effort,9 USAID launched the $4.99 million project, Widening Harmonized Access to Microfinance (WHAM). WHAM aimed to improve financial access for customers not served by credit-granting NGOs or smaller specialized banks, or by the larger commercial banks. WHAM focused on enterprises straddling a financial “missing middle,” which means a credit demand between $500 and $25,000. With credit-granting NGOs, the project planned loan operations typically concentrated in credit-worthy micro enterprises engaged in income generating activities. With commercial banks, the project targeted loans for small firms with demonstrated capacity to generate higher revenues and, eventually, more jobs. The project did not plan to concentrate in a particular province or region of Pakistan, and stated no geographical emphasis or limits. To achieve this objective, USAID designed a two-pronged strategy. The first part aimed to develop support for a microfinance industry. This consisted of several activities, such as raising awareness among stakeholders, improving knowledge of the market, and the identifying growth-constraining factors in the industry. The second part of the strategy involved training and building capacities of financial institutions interested in setting foot in the aforementioned “missing middle.” The project 4 The full evaluation report is included as Annex 1. 5 Source: Shorebank International Ltd. “The SME Market in Pakistan. Initial Analysis of Markets and Trends.” (Power Point Presentation to Crescent Commercial Bank, April 2007., with data collected from the State Bank of Pakistan). 6 In Pakistan there is no single definition and classification of enterprises. Various government agencies use their own definitions. For reasons of simplicity, this evaluation report will consider the definition by employment that is commonly used by most international development organizations, including the United Nations Development Program and the World Bank. Accordingly, micro enterprises are those businesses that employ up to 10 workers, small enterprises are those which employ between 11 and 50 workers, and medium-sized firms are those employing between 51 and 250 workers. 7 Source: Crescent Commercial Bank, from data collected in June 2007 from the State Bank of Pakistan and the Lahore University of Management Science 8 In May 2006, Shorebank International estimated that the market size of micro credit comprises 7 million households and no less than 10 million individuals. This minimum estimate was based on an analysis of the Household Income and Expenditure of Survey of 2001. Within poverty bands, the households and individuals who make up the market for micro credit are classified as transitory vulnerable and transitory poor. 9 The effort included two other finance projects: Developing Non-Bankable Territories for Financial Services, otherwise known as the Khushhali Bank (KB Project), and the Enterprise Development Facility (EDF). WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) x executed this by pursuing both downscaling and scaling up approaches to trigger increased financial outreach. Conceptually, downscaling pins hope on commercial banks “going down the market,” that is, their penetration into low-income niches that were traditionally alien to them. Scaling up, on the other hand, focuses on building the loan size capacity of financial entities with a long history of extending credit to low income households and micro enterprises. II. EVALUATION METHODOLOGY The evaluation team consisted of MSI staff member Jorge L. Daly who served as the team leader. Aroona Kamal and Memoona Khan participated as independent consultants who conducted interviews with stakeholders, collected additional data and provided further analysis. Douglas Krieger led the tasks centered on efficiency and impact analyses. Maliha Hussein, a microfinance specialist, reviewed the draft for content, quality, and contextual accuracy. The team designed this evaluation as the final assessment of the WHAM project, and intended it to be independent, objective, and findings-based. The team evaluated the answers based on a general approach that progressed from findings to conclusions to recommendations. The evaluation methodology utilized the Getting to Answers Matrix (GTA). This tool creates a format in which evaluation questions drive the identification of relevant data sources and data collection methods. See the complete GTA matrix in Annex 4. III. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ON OVERARCHING ISSUES A. Relevance of WHAM Evaluation question: How well was the project focused on the needs of the beneficiaries? Findings The evaluation team interviewed 18 key informants. These informants all agreed that the project goals were relevant and appropriate. However, several of them raised questions about the project design and approach. Five of the 18 key informants noted that market conditions during the life of the project were not ideal for launching downscaling activities. Also, these five pointed to financial spreads in the commercial banking sector that were abnormally high. In fact, this is still true today (according to the World Bank, “abnormally high” is no less than 4.5%).10 Additionally, the Chief of Party of WHAM noted that the banking sector is consolidating. Most banks focus on seeing how the market evolves before they decide to venture into the more risky Small and Medium Enterprise (SME) segment. The evaluation team reviewed project documents and met with the contractor’s Chief of Party. The team was not able to obtain evidence of analysis before the project began. To provide an in-depth understanding of the target market, this analysis would have included defining the target loan range and developing the most applicable approaches and instruments to address the target market (loans between $500 and $25,000). The project design emphasized increasing loan volumes to customers straddling the missing middle, even though the project created the possibility of other financial instruments being introduced. This was emphasized through the use of a Cost Share requirement which measured only loan disbursements rather than a wider array of financial instrument objectives. As a result, WHAM developed and launched only loan products, even though anecdotal evidence shows that targeted groups, especially women, do have the capacity to save and thus demand savings type instruments.11 Project activities created loans above the upper limit of the definitional range of the missing middle. In downscaling, the National Bank of Pakistan (NBP) provided loans above the upper limit of 10 However, experts in Pakistan, including managers of commercial banks and other financial entities, believe that actual spreads are actually much higher – no less than 7%. By way of comparison, normal spreads in countries that exhibit more competitive banking systems hover around 2%. 11 It is important to point out, however, that WHAM produced a research paper on the potential of micro savings in Pakistan. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xi $25,000, which is approximately 40% of the 813 loans issued (see Figure 1 below). By contrast, scaling up produced 1,154 small business finance (SBF) loans in the low end of the definitional range of the missing middle (Asasah provided all these loans, and is the only scaling up financial institution that received intensive support from WHAM to create the SBF loan product). By considering the two sets of loans together, the picture is substantially more in line with project intent. Approximately 84% of the loan recipients come from the missing middle (see Figure 2 below). Figure 1: Distribution of Project Loans [NBP – 813 total loans] Figure 2: Distribution of Project Loans (NBP and Asasah) Finally, increasing access to finance for marginalized women was never an explicit part of the project, as this objective was not included in project design. Nevertheless, from the beginning, WHAM did target entities that exclusively provide loans to women. Conclusions The project succeeded in addressing real needs of beneficiaries by facilitating strategic decisions to target distinct financial entities, such as commercial banks and microfinance banks and credit-granting NGOs. The project also designed the contents of technical assistance services that corresponded to the unique characteristics of each type of entity and the particular loan demand profiles of their clienteles. Regarding downscaling, however, the project design did not sufficiently factor in market conditions relevant to commercial banks. Unfortunately, commercial banks did not have sufficient incentive to move out of their typical loan markets. The project focused on increasing loan activity, and therefore did not develop other financial products, such as savings. The project design did not establish the range of the missing middle with use of prior data or analysis that supported demand for these financial services. A typical analysis would include types of products, volumes, frequency of demand and intended usage from customers in that segment. Without this analysis, WHAM could not determine the precise location of the true market failures. Not surprisingly, 19% of all loans corresponding to the downscaling activity were equal or above $50,000, which is twice the upper limit of the missing middle range. The definitional range of the missing middle ended up lumping under one single diagnosis the needs of beneficiaries that emerge from different realities, i.e., from the extreme ends of the missing middle. This resulted in negative consequences for project relevance. To address this problem, WHAM focused on clearly separating downscaling of commercial banks from scaling up microfinance institutions, and sought out a variety of products and loan sizes that fitted the targets of selected implementing partners. Finally, the project experience has shown that targeting the missing middle does not necessarily allow strong outreach to women. Very few women occupy the market segment of owning and managing their own small enterprise in Pakistan and this fact is reflected in the overall outreach to women. Recognizing this, WHAM calibrated the importance of this gap and actively proceeded to give priority to the enlisting of microfinance providers that cater their services exclusively to women. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xii Recommendations To the extent that it stays engaged in finance projects, USAID should: (a) Propose interventions that target more specific market segments, by collecting data on size and customer-profile characteristics inhabiting such segments, and on the existing financial entities that can potentially serve them; and (b) commission studies to shed light on (i) the opportunities and constraints that underline potential interventions that target poor women; and (ii) the demand for financial services other than micro loans. B. Effectiveness of WHAM Evaluation question: Has the project accomplished its objectives? WHAM provided technical assistance to nine partner organizations. More than other partner organizations, the activities of NBP and Asasah led to increased volumes of loan activity focused on the missing middle. By December 2007, WHAM surpassed the June 2006 targets for the number and value of loans, 257% and 180% respectively. Also in December 2007, the number of trained loan officers of partner organizations increased to 555, or ten times as high as the original target. 1. Downscaling WHAM intended to support downscaling activity with technical assistance and training, and assumed the banks themselves funded, designed and extended credit products to small enterprises straddling the upper limit of the “missing middle.” Findings WHAM developed and applied a rigorous methodology to select and enlist commercial banks. After three years of operation, WHAM established relationships with only three commercial banks: Crescent Commercial Bank (Cresbank), Standard Chartered Bank (SCB), and National Bank of Pakistan (NBP). Only the relationship with NBP resulted in the disbursement of loans to the target population. In October 2005, the project initiated a pilot program with NBP for technical assistance to 18 branches in four regions: Lahore Central, Lahore East, Gujaranwala and Sialkot. During the pilot program, WHAM used score cards to measure assimilation of knowledge and skills of the trained loan officers. According to WHAM Chief of Party, however, these score cards were discontinued in late 2005 because NBP management did not receive them well. The score cards were replaced by a monthly loan activity reporting tool that tracks the number, value and breakdown according to size of disbursed loans, the portfolio at risk and the number of loan officers trained. Unfortunately, this system does not directly track the development of skills and knowledge of trained staff. After discontinuing the score cards, WHAM did not develop other tools to track changes in knowledge and skills and increased use of improved SME lending techniques. One year after the NBP project began, the number of loans issued surpassed initial targets by nearly 100%. At the request of senior management of NBP, WHAM extended the program to 50 additional branches located in 7 new regions. After sluggish performance in the first quarter of 2007, loan activity grew quickly, reaching a total of 813 loans for a value of $27.3 million by December 2007 (see Figure 3 below). Further, portfolio at risk was low: as of November 30, 2007, out of 505 active loans, the number of delinquent loans was 8 and constituted 2% of the outstanding loan portfolio. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xiii Figure 3: NBP: WHAM Quarterly Total Credit Disbursement and Number of Loans Disbursed NBP: WHAM Quarterly Total Credit Disbursement and Number of Loans Disburesed - 1 2 3 4 5 6 7 8 9 Jan-Mar 06 Apr-Jun 06 Jul-Sep 06 Oct-Dec 06 Jan-Mar 07 Apr-Jun 07 Jul-Sep 07 Oct-Dec 07 Millions Quarter Value (US$) - 50 100 150 200 250 300 Number Value of Loans Disbursed Number of Loans Disbursed Number of loans disbursed in 2006 was heavily concentrated in the branches of Lahore Central (46%) and Gujaranwala (33%) for a total of 79%. In 2007, loan concentration in Lahore Central and Gujaranwala declined to 47% and the branches of Faisalabad contributed to 21% of total loans. Together, these accounted for a total relative contribution of 68% of all loans among branches of these three regions. Conclusions Although WHAM applied a sound selection strategy for technical assistance and training for loan activity, only three commercial banks participated in the project. Furthermore, only the state-owned National Bank of Pakistan’s (NBP) participation directly led to increased loan activity. Apparently this result occurred because of negative factors largely outside WHAM’s control; more specifically, a large interest rate spread removed any incentive for commercial banks to consider entering new markets. During the project, commercial banks have been very busy and focused in reaping huge profits from non-credit financial operations and loans targeted to market segments other than the SME sector. High-quality loan activity grew considerably even though total SME lending across the country declined by 5.8% during the period December 2006 – September 2007. At the same time, program loans were heavily concentrated in three regions of the Punjab province. In recent years, these regions experienced rapid growth of bank loan activity of all kinds and sizes, even more than in other regions of Punjab and certainly more than the rest of the country. Consequently, external forces might also be playing an important role in driving the loan activity. Thus, it is difficult to precisely attribute such growth to WHAM given such a dynamic banking market. Technical assistance and training activities focused on helping NBP develop a line of business that could trigger the production of high quality loans. WHAM tracked progress more on the basis of the lending performance than on the intermediate goal of improved staff capacity. Although this emphasis aligns with the project objective of producing loans, the evaluation team finds it possibly shortsighted in retrospect. WHAM trainers engaged with loan officers on a very intensive level, and mentored them in virtually all aspects of the loan cycle. Although the trainee undoubtedly benefits, the project cannot measure if trainees can apply the learning on their own without objective indicators for improved knowledge, skills and application of improved practices. Therefore, skills might not effectively transfer to the trainees, meaning that when the trainers leave, the NBP SME program will lose those skills. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xiv 2. Scaling Up The objective of this activity was to persuade credit-granting NGOs, rural support programs and microfinance banks to design and extend credit products to small enterprises straddling the lower limit of the “missing middle.” Before the project started, these entities generally extended group￾collateralized loans in the range of $160 to $250. Scaling up to loans above $500 requires improvement of institutional capacities to process individual loans. Clientele profiles are not the same in each case, and methodologies to assess and mitigate risks are consequently different. WHAM signed technical assistance agreements with four entities during April 2005 – November 2006: (a) Kashf Foundation; (b) First Microfinance Bank (FMFB); (c) Tameer Microfinance Bank; and (d) Asasah. The agreement with Kashf Foundation involved assistance to improve the design of an individual loan product the organization was developing, for an average size of $666. The agreement also included the design of a plan to pilot test a mobile phone cash payment system, and the development of credit-risk management strategies. For FMFB, WHAM committed to determine the bank’s needs as regards its readiness to use technological innovations leading to increased outreach and improved overall organizational efficiency. As part of the agreement, WHAM provided recommendations on the adoption of electronic banking systems applicable to urban and rural contexts and has supported the development of an internal training curriculum to equip FMFB loan officers with the skill to provide scale up loans and has helped disseminate this curriculum across FMFB branches. Tameer Microfinance Bank enlisted the assistance of WHAM to (a) review the quality of the loan portfolio; (b) revamp entirely the loan processing system and procedures; and (c) improve human resource development policies and procedures. In addition, WHAM is footing the salary bill of two trainers and the start up of a training center in Lahore. For Asasah, WHAM designed a small business finance (SBF) loan product with a targeted minimum loan value of $500. Disbursements of this product increased steadily from 20 (July - September 2005) to 250 two years later (July - September 2007), which reached an accumulated total of 1,154 in December 2007. Of these loans, SBF issued 96% to women. However, in the last quarter of 2007, the number of disbursed loans declined to 55, because Asasah was not able to access funds for loans. Value of loans correspondingly rose and declined. Repayment rates hovered around 95%. Asasah’s management attributes low arrears to the quality of technical assistance delivered by WHAM. Figure 4: Asasah: WHAM Quarterly Total Credit Disbursement and Number of Loans Disbursed ASASAH: WHAM Quarterly Total Credit Disbursement and Number of Loans Disbursed - 20 40 60 80 100 120 140 160 180 Jun-Sep 05 Oct-Dec 05 Jan-Mar 06 Apr-Jun 06 Jul-Sep 06 Oct-Dec 06 Jan-Mar 07 Apr-Jun 07 Jul-Sep 07 Oct-Dec 07 Thousands Quarter Value (US$) - 50 100 150 200 250 300 Number Value of Loans Disbursed Number of Loans Disbursed In the last quarter of 2007, WHAM signed agreements with the National Rural Support Program (NRSP) and Khushhali Bank. These 2007 agreements commit WHAM to assist NRSP in the WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xv preparation of a business plan for its eventual transformation into a microfinance bank, and to deliver to Khushhali Bank trainings on sales force management and financial analysis. Conclusions WHAM was effective in enlisting partner organizations that have an important presence in the microfinance market. NRSP, enlisted in late 2007, and Kashf Foundation are second and third respectively as regards client outreach among providers. Tameer Microfinance Bank is also quickly establishing itself as a major player in the market. FMFB established branch presence in all four provinces of the country and Asasah is focused on delivering financial services to women. WHAM support to Asasah contributed directly and substantially to increased access to loans for borrowers that straddle the lower end of the missing middle. Previously, these borrowers mainly received microloans, but now obtained substantially larger value loans. With the SBF loan product, WHAM effectively increased outreach to women borrowers in the missing middle (women make up 1,006 of the Asasah SBF loan recipients). Though WHAM monitored progress more on the basis of the lending performance than on the intermediate goal of improved staff capacity, all the senior managers from partner organizations stated in interviews that the technical assistance and training activities provided by the project were effective, useful and relevant. 3. Industry Strengthening Industry strengthening activities focused on the following objectives: (a) Taking stock of the level of awareness among stakeholders of basic facts and held opinions regarding the microfinance industry; (b) Updating and disseminating knowledge of market size, industry data and trends; (c) Enhancing knowledge of factors that restrain long-term growth; and (d) Delivering training tailored to downscaling and scaling up practitioners. Findings WHAM delivered 42 training events on topics relevant to micro and SME lending for 724 participants, which largely surpassed the end-of-project target of 500. 114 of the participants were women. Each event lasted one to three days. The project also organized and sponsored five seminars and events to increase the visibility of the microfinance industry. In addition, the project conducted seven research papers, such as the potential for microsavings or the level of indebtedness in the Lahore market. In 2006, WHAM did not succeed in its attempt to enlist local and international institutions to run a training center for the entire microfinance sector of Pakistan. However, in mid 2007, WHAM designed and launched a Human Resource Development Initiative that targeted microfinance middle management. This initiative, which is to be carried forward by the branch office of ShoreBank International in Pakistan after the project end, has already resulted in the selection of trainers, the development of course materials, the delivery of a training of trainer workshop, marketing among prospective participants, and planning for future sustainability. WHAM collaborated with the Pakistan Microfinance Network (PMN) to strengthen its institutional presence among key stakeholders. During project implementation, the number of microfinance providers that submitted financial reports to the PMN increased from 18 to 25. Conclusions WHAM improved knowledge and enhanced visibility of the microfinance industry through the delivery of seminars and conferences and also by disseminating updated industry information. On average, WHAM conducted fourteen training events per year on the principles and practices of micro enterprise and SME individual lending, which is more than one event per month. During the training, 16% of trainees were women. The individual lending techniques, when and if applied by the attendees, is likely to steer their organizations along a path of faster growth. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xvi As a result of WHAM’s assistance, PMN now creates benchmarks based on the performance of microfinance providers. These providers now report more information than before WHAM’s assistance. After dedicating efforts since mid 2006 to enlist local and international venues to establish a training center, WHAM launched in the second half of 2007 the Human Resource Development Initiative. Given the very recent establishment, this evaluation did not examine the effectiveness of the initiative. Recommendations To the extent that USAID continues its engagement in finance projects, USAID should: (a) Dismiss in the short run projects that consider downscaling activities, as the banking industry is not likely to penetrate the SME market within the next three to five years; (b) Require that contractors use tools that measure and monitor acquisition of new skills, improved knowledge, and the eventual use of improved practices by trainees in projects that emphasize capacity building of financial institutions; and (c) Require that activities directed at laying the groundwork and developing local self-sustaining institutional capacities to address lack of local skilled human capital be undertaken early in the project life. C. Impact of WHAM Evaluation question: To what extent has the project benefited the people of Pakistan? The evaluation team was able to estimate monetary measures of project impacts for only the one-third of the project that focused on scaling up. Furthermore, it obtained statistically reliable estimates of impacts only for the estimated 56% of scaling up loan recipients who ran retail businesses. The findings in this section and in the Efficiency section thus refer only to the costs and benefits associated with the scaling up component of the project and only for retail businesses. Findings Based on the evaluation team’s survey of borrowers from one of the project’s scaling up partners (Kashf), project-supported lending appears to generate increases in business profits and household income. Business profit increased by an average of Rs. 15,000 annually for retail businesses that received loans relative to similar business that did not receive loans. On average, businesses retained two-thirds of the increased profits in the business and diverted one-third to household expenses. Because the survey lacked a good baseline, the results are not particularly reliable. They do, however, support similar conclusions from other impact assessments of microfinance in Pakistan and throughout the world. The present value of benefits to retail businesses who received loans under the scaling up component was $1.8 million at the end of the project and an estimated $61.8 million by 2012. In addition to monetary impacts, microfinance lending through WHAM generated non-monetary benefits. Asasah gives women an outlet for their savings, which almost 70% of Asasah’s 27 thousand total beneficiaries did. This totaled $80,000 worth of savings as of June 2007. Also, Asasah contributed to increased empowerment of women borrowers. In group interviews conducted with women borrowers of Asasah, almost all interviewees noted that the loans gave them the chance to make decisions for their children and family. Asasah also tailored products to fit the particular needs of women. For example, Asasah introduced an educational loan facility that enables women and/or their children to attend schools and colleges at an affordable price. The project’s downscaling and industry strengthening components (two-thirds of project activities) also had impacts although it was not possible to estimate the monetary measures of those impacts. The evaluation team was unable to obtain data from the handful of downscaling clients in time to complete the evaluation. In addition, industry strengthening activities did not generate easily measured monetary impacts. Conclusions The project almost certainly benefited the SMEs that received loans and the households of the SME owners although quantifying the impact is difficult. A survey of borrowers from one of the scaling up WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xvii partner organizations (Kashf) produced limited evidence that WHAM-supported lending had a positive impact on business profits for retail businesses. Focus groups with borrowers revealed impacts associated with empowerment of women and improved education of family members. These conclusions are generally consistent with other impact assessments of microfinance in Pakistan. On average, survey respondents reported diverting about one-third of additional profits to their households so WHAM-supported lending increased household incomes. Recommendations Plan for future impact assessment. Assessing the monetary impact of microfinance lending is notoriously difficult. The absence of good baseline data makes it particularly difficult. If USAID desires reliable measures of monetary impact, and related measures of efficiency, it should require implementers to plan for, and implement, rigorous impact assessments of future projects. D. Efficiency of WHAM Evaluation question: How efficient has the project been in utilizing its resources to achieve results? Findings A full accounting of social costs of microfinance lending was beyond the scope of the evaluation. The evaluation does not, therefore, produce estimates of the net present value, benefit cost ratio, or internal rate of return that are comparable to those of the other evaluations. Limited estimates of the efficiency of the scaling up portion of the project suggest that it returned monetary benefits about equal to costs during the project’s period of performance. Projected for five years beyond the end of the project, scaling up activities generated benefits well in excess of costs. Conclusions The project was likely reasonably efficient, although the measures of costs and benefits are not refined enough to make a firm determination. Recommendations Many measures of efficiency depend on reliable measures of monetary impacts. If USAID wishes to assess project efficiency, then it must emphasize monetary impact assessment of future projects. E. Sustainability of WHAM Evaluation question: Are the activities and results likely to be sustained after the project is completed? Findings Regarding downscaling, the NBP experienced an increase from 40% to 60% in the operational sustainability of SME lending during the period of technical assistance.12 At the request of bank management, WHAM prepared a plan to extend the assistance to 600 additional branches. If implemented, this plan might greatly enhance the availability and sustainability of SME lending in Pakistan. During the project life, NBP did not institutionalize policies and procedures that are necessary to sustain the autonomous development of an SME Unit. WHAM carefully chose scaling up partners based on their established orientation towards sustainability. For example, NRSP is seriously contemplating legal transformation into a microfinance bank. Since its inception in 2002, FMFB operated at break-even. WHAM knew that Tameer Microfinance Bank and Kashf Foundation were designed to price their services on cost-recovery and 12 Operational sustainability is defined as the generation of enough revenues to cover operating expenses and financing costs,. Financial sustainability indicates whether or not enough revenue has been earned to cover operating expenses, financing costs, provision for loan losses and the cost of funds due to inflation. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xviii profit-making bases. In Asasah, its CEO brought previous experience from a tenure at the Kashf Foundation. Officials of Tameer, Kashf and Asasah underscored that WHAM offered technical assistance and training inputs that emphasized principles of self-sufficiency and long-term sustainability WHAM assisted the State Bank of Pakistan with a microfinance paper presented to the Prime Minister. This paper recommends policy actions to encourage long-term industry sustainability. The project also strengthened the institutional presence of the PMN, which now receives financial performance from 95% of microfinance providers. Conclusions By lending support to the State Bank of Pakistan and the Pakistan Microfinance Network, WHAM contributed to strengthening the microfinance industry and its prospects for long-term sustainability. Regarding downscaling, the team cannot unequivocally ascertain whether the activities and results will be sustainable in the long run. If NBP implements the plan to extend the program to 600 branches, the long-term sustainability of SME lending is undoubtedly enhanced. However, the bank has not institutionalized systems for SME lending, which suggests lack of long-term commitment. Regarding scaling up organizations, the implemented activities enhanced sustainability, given WHAM’s emphasis on microfinance best practices in the delivery of technical assistance and training. Recommendations In case it decides to stay engaged in finance interventions, USAID should: (a) Consider a timeframe (no less than five years) that simultaneously builds institutional capacities of micro and small enterprise finance providers, and executes activities directed at strengthening the industry; a three￾year execution timeline is just too short for projects that pursue these very ambitious objectives; and (b) Explicitly define indicators and targets that underscore measurable performance towards operational and financial sustainability of microfinance providers. F. Replication Evaluation question: To what extent can the activities and results of the project be replicated? The evaluation team does not have sufficient data to provide comments on findings, conclusions, and recommendations for replication. In all likelihood, project replication would create uneven results. Industry strengthening and scaling up are likely to yield positive results, but downscaling is likely to be largely ineffectual, at least in a three to five year timeframe. Scaling up activities would replicate well in all regions except areas of very low population density and abject poverty. Scaling up projects will also fail in areas suffering from entrenched insecurity, such as parts of Baluchistan and most especially, FATA. In these regions, the current violence and general insecurity will not permit effective financial outreach. IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ON CROSS CUTTING ISSUES A. Gender Evaluation question: To what extent has the project benefited women? Findings WHAM provided assistance to two partner organizations (Kashf and Asasah) focused on extending financial services to low-income women. In all, 2,179 women received loans as a result of WHAM technical assistance and training activities to partner organizations. Of this total, only six corresponded to the downscaling component. Downscaling operates at much larger loan sizes for businesses owned by men. The remaining 2,173 women were extended loans by scaling up microfinance providers. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) xix Conclusions WHAM modestly contributed to increasing loans extended to women. As stated above, there are very few women who manage or own businesses in the missing middle. Recognizing this, WHAM calibrated the importance of this gap and actively proceeded to give priority to the enlisting of microfinance providers that cater their services exclusively to women. The project intensively engaged in building the institutional capacity of Asasah, an organization managed and staffed by women. Recommendations Provided it continues to focus on finance projects, USAID should: a) Commission a study on the opportunities and constraints that underline potential interventions targeting low-income women; b) Organize a high-level seminar with the participation of key stakeholders to highlight findings, conclusions and recommendations of the study; and c) Facilitate access to lessons learned from other countries that have enjoyed high levels of success in pushing financial outreach to women. These efforts could take many forms, including, teaching seminars led by visiting experts, study tours, etc. B. Reporting, Communication and Outreach and Coordination Evaluation question: Has the prime contractor reported on time and in a useful manner? WHAM adequately fulfilled reporting requirements. Evaluation question: How effective has the project been in getting its story out? The project effectively raised awareness among stakeholders concerning the project’s objectives, content, and benefits. Channeling key information through the PMN was relevant, as it directly contributed to elevating the profile of such an important local institution. In addition, WHAM strengthened the PMN, which will lead to further benefits for the microfinance industry after the project ends. This includes the PMN providing information, producing research pieces, and interacting with government officials. Evaluation question: How effectively has the project coordinated with other parties? WHAM effectively coordinated efforts with key stakeholders of the microfinance industry, including the State Bank of Pakistan, the Pakistan Microfinance Network, and representatives of the donor community. V. LESSONS LEARNED After evaluating WHAM, the team found the following lessons: First, USAID must carefully execute project design. Without a proper assessment of the prospects of downscaling, the project defined a loan range that was too wide for the “missing middle.” The project mistakenly assumed both commercial banks and microfinance providers were equally interested. Second, WHAM needed a longer period, such as five years, for more effective delivery of technical assistance. This allows for the necessary time to foster trust and establish partnerships beforehand. Third, projects that offer technical assistance and training untied to the supply of loan for capital or credit guarantees can be effective. Fourth, activities that support industry strengthening require small investments but have the potential to yield substantial and lasting returns. Fifth, USAID must plan for impact assessment of future activities. Quantitative impact assessment of microfinance is particularly difficult. Planning for impact assessment from the beginning of an activity will greatly enhance prospects for credible impact assessment of future activities. In particular, activities should establish baselines soon after inception, plan to follow up on the baselines, and collect the quantitative and qualitative data necessary for a comparison of activity outcomes with outcomes if the activity had not taken place. The lack of baseline information and reliable implementation data severely hampered the present evaluation effort. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 1 ANNEX 1: WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 2 I. INTRODUCTION In August 2007, the United States Agency for International Development (USAID) commissioned Management Systems International (MSI) to conduct the evaluation of eight projects within the purview of its Pakistan Economic Growth Activities. Three of these projects correspond to activities that seek to increase access to financial services to the benefit of households and micro and small businesses in rural and urban areas across Pakistan. This report addresses the final evaluation of one of these finance projects – Widening Harmonized Access to Micro Finance (WHAM) – which was awarded to ShoreBank Advisory Services (SAS), under conditions stipulated in the Request for Assistance (RFA) 391-05-02. WHAM is a $4.99 million project that started operations in April 2005 and is due to terminate in March 2008.13 In 2003 USAID, following several years of suspended activities, committed itself to support efforts aimed at achieving a more balanced economic growth, in turn enhancing economic opportunities and increasing incomes of the vast poor. In the decades that followed decolonization, overall economic development performance was less than satisfactory, in part because, and not unlike the majority of developing countries, Pakistan had applied economic policies that privileged state activism over private sector entrepreneurship. Restrictions to international trade, sheltered local industries, state ownership of large companies, price controls and over regulation of business activities were part and parcel of the political-economic milieu until the mid to late 1990s. In the financial realm, this overall development strategy would find expression in what has been typically characterized as features of financial repression: (a) government-control and ceilings on subsidized loan rates; (b) directed credit targets mandated by the State Bank of Pakistan (SBP), with strong preference for resource allocation to government programs that underlined political goals; (c) low remunerated, high reserve requirements applied to commercial banks; (d) easy rediscount credit from the SBP; and (e) massive presence of state-owned banks. Under these circumstances, a highly uneven pattern of economic development evolved, one in which high rates of growth were arrested by recurring balance of payments crises. In the 1960s the annual average rate of GDP growth was 6.8%, in the 1970s it declined to 4.8%, in the 1980s it surged again to 6.5% and in the 1990s it dove back to 4.6%.14 In these decades, high inflation sapped incomes of consumers and sliced profits of micro and small entrepreneurs, job creation lagged very substantially behind the supply of new entrants into the workforce, and informal activities increasingly dotted the business landscape. Not surprisingly, by the end of the last decade, alarming signs of unacceptable socioeconomic exclusion were for all to see. The 1990s, in particular, is known by Pakistani and international experts as a “lost decade.” Drawing from surveys conducted in 1998 and 1999, a substantial increase in poverty, especially in rural areas, was reported during the decade. By the close of the 90s a third of the population lived below the national poverty line, while 13% and 66% earned incomes below $1 a day and $2 a day respectively. And with respect to the distribution of income, the share of the lowest 20% of income earners on total income was 8.8%, while that of the highest 20% reached 42.3%.15 Early in the present decade, the government made successful attempts to control inflation and to institute more business-friendly, market-oriented reforms in a bid to catalyze more vigorous private investment. As a result, the rate of gross domestic product (GDP) rose steadily, from 2.2% in 2001 to 8.6% in 2005 and 6.6% in 2006, this latest indicator notwithstanding the spike in oil prices and the devastating earthquake of October 2005. The country also succeeded in enacting financial reforms that largely eliminated the features of financial repression.16 The reforms have led to more vigorous commercial bank activity. Unshackled from stringent government controls, commercial banks can now pursue market signals that point to profitable opportunities. In doing so, there is little doubt that they are making an important contribution to economic growth. 13 ShoreBank Advisory Services changed its name to ShoreBank International Ltd in 2006. 14 Source: Web site of the Association of Pakistani Professionals (AOPP), with data obtained from the State Bank of Pakistan, Federal Bureau of Statistics and the Central Board of Revenue. 15 The World Bank [2005] 16 Highlights of the financial reform are extracted from The World Bank [2006]. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 3 Still, notwithstanding the vigorous expansion of commercial bank activity, finance’s contribution to economic development remains substantially unrealized. This is expressed in the large numbers of households and businesses which have no access to formal financial services. As of June 2006, the total number of borrowers in the formal financial system barely surpassed 4 million households and businesses.17 In a country with a population of 165 million people, this financial outreach is significantly below potential. II. THE DEVELOPMENT PROBLEM A. Problem Statement As noted above, a large portion of the Pakistani population – and Pakistani businesses – do not have the opportunity to access financial services. Large commercial banks focus on corporate clients and on extending consumer loans to customers located in Karachi, Lahore and other urban areas. They also serve a small number of small and medium enterprises (SMEs), and largely stay away from rural areas, where low-income and poor populations are mainly concentrated.18 In all, the total of SME firms that are currently served by commercial banks approximate 160 thousand, equivalent to no more than 8% of potential customers.19 As regards micro enterprises, it is estimated that microfinance providers currently serve approximately just 5% of potential micro loan demand.20 The vast majority of households, individuals and business units with potential to demand micro loans obtain credit from informal sources – money lenders, suppliers, traders, shopkeepers, relatives and friends. Recent market-friendly financial reforms have resulted in some penetration of formal financial services to marginalized households and businesses in Pakistan. However, these reforms do not appear to be sufficient to incorporate, within a reasonable timeframe, substantial portions of these under-served households and businesses. In short, prior to the start of the project, financial reform appeared to be a necessary but not sufficient condition to substantially increase access to financial services for the vast majority of households and SMEs. B. USAID Intervention By 2002, when USAID resumed operations in the country, micro and SME finance were not developing dynamically. The response of USAID to this problem was to launch three finance projects – Developing Non-Bankable Territories for Financial Services, the Enterprise Development Facility, and WHAM. The first of these was activated in March 2003, the second, Enterprise Development Facility, in September 2004 and WHAM, as noted in the previous section, in March 2005. The objective of WHAM was to improve financial access to customers that are served neither by credit￾granting NGOs or smaller specialized banks, nor by commercial banks. The RFA stated that these customers, positioned in a financial “missing middle,” would demand loans within a range between $500 in the lower limit and $25,000 in the upper limit. With respect to credit-granting NGOs, loan operations would be typically concentrated in credit-worthy micro enterprises engaged in value-added income generating activities. As regards commercial banks, loans would be targeted to small firms 17 Source: Shorebank International Ltd. “The SME Market in Pakistan. Initial Analysis of Markets and Trends.” (Power Point Presentation to Crescent Commercial Bank, April 2007., with data collected from the State Bank of Pakistan) 18 In Pakistan there is no single definition and classification of enterprises. Various government agencies use their own definitions. For reasons of simplicity, this evaluation report will consider the definition by employment that is commonly used by most international development organizations, including the United Nations Development Program and the World Bank. Accordingly, micro enterprises are those businesses that employ up to 10 workers, small enterprises are those which employ between 11 and 50 workers, and medium-sized firms are those employing between 51 and 250 workers. 19 Source: Crescent Commercial Bank, from data collected in June 2007 from the State Bank of Pakistan and the Lahore University of Management Science 20 In May 2006, Shorebank International estimated that the market size of micro credit comprises 7 million households and no less than 10 million individuals. This minimum estimate was based on an analysis of the Household Income and Expenditure of Survey of 2001. Within poverty bands, the households and individuals who make up the market for micro credit are classified as transitory vulnerable and transitory poor. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 4 with demonstrated capacity to generate higher revenues and, crucially, jobs. The project was not to be concentrated in a particular province or region of Pakistan. In fact, no geographical limits were stated. It is important to note that the timing of WHAM was advantageous. WHAM was designed and implemented four years after the Government of Pakistan (GOP) made its first attempt at instituting a more enabling regulatory environment for the provision of micro loans. Indeed, the enactment in 2001 of the Microfinance Ordinance was a watershed event. Though not a perfect legal instrument, as attested, for example, by the sanction of a too-low cap on the maximum size of a micro loan, the Ordinance contains norms and regulations that unquestionably paved the way for a more dynamic growth of the industry. For example, it authorized the establishment of specialized microfinance banks (MFBs) with a minimum capital requirement of $8 million for a national license, a fifth of what is required from commercial banks. Even lower requirements apply for provincial and district level licenses. Lower minimum capital requirements clearly constitute an incentive for the entry of new providers which can more easily tailor their services to the loan demand profiles of poorer, non￾traditional clienteles. The fact that in 2007 there were six MFBs (three with full national licenses) in operation and another two due for licensing in 2008, suggests that this capital requirement is not unduly high, i.e., that it does not constitute a significant barrier to entry into the market by new providers. It is nonetheless balanced by a preference to facilitate entry of strongly capitalized financial entities, a fact that is likely to benefit the industry in the long-run. Two other important measures of the 2001 Ordinance were the opening of a microfinance unit within the SBP. And roughly one year earlier this Pakistan Poverty Alleviation Fund (PPAF) initiated wholesale lending to credit-granting NGOs. Against this backdrop of financial sector reform, WHAM pursued its objective of improved financial access for marginalized customers through a two-pronged strategy. FIGURE 1: WHAM RESULTS FRAMEWORK The first element of the strategy consisted of activities aimed at providing support for the development of a micro finance industry, including raising awareness among stakeholders, improving knowledge of the market, and identifying growth-constraining factors that the industry faces. Prior to project implementation, best practices of microfinance, such as cost-recovery loan rates, character￾based lending, and financial sustainability, were not in wide use. Through seminars and publication of research pieces that informed on the status and prospects of the microfinance industry, WHAM aimed to disseminate knowledge and highlight the presence of structural flaws that blunt more dynamic industry growth. One of the most important constraints recognized during project design was the lack of skilled practitioners, among them loan officers and middle managers. To this end, WHAM was built to focus on the design and delivery of training workshops to improve knowledge and skills in the industry at large. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 5 The second element of the strategy involved training and building capacities of financial institutions interested in setting foot in the aforementioned “missing middle.” The execution of this element of the strategy called for pursuing both, downscaling and scaling up approaches to trigger increased financial outreach. Conceptually, downscaling aims to persuade commercial banks to “go down the market,” that is, penetrate into low income niches that were traditionally alien to them. Scaling up, on the other hand, focuses on financial entities which have a long history of extending credit to low income households and micro enterprises. They primarily include credit-granting NGOs, but also formally supervised microfinance banks that have been purposefully created to serve such clienteles. Scaling up can be understood indistinctively as processes leading to significant increases in the customer base, loan volumes, or average loan size. The concept can be also referred to as the transformation of a credit-granting NGO into a commercial, formally regulated and supervised small bank or non-bank financial entity. The microfinance market segment in Pakistan is littered with an array of entities of differing institutional arrangements. These include: • The recently established six MFBs – Tameer Microfinance Bank, First Microfinance Bank, Rozgar Microfinance Bank, Pak-Oman Microfinance Bank, Network Microfinance Bank, and the state-directed Khushhali Bank. • Credit-granting NGOs, which include those entirely focused on financial operations and those that feature multipurpose goals. The most important among the former includes Asasah, Kashf Foundation and Thardeep Rural Development Program. • Rural Support Programmes (RSPs), which can be considered as the forerunners of micro lending in Pakistan. Established in the 1980s with government and donor assistance, RSPs are NGO-type multifunctional entities that provide financial services to support integrated rural development programs. The most important is the National Rural Support Programme (NRSP) which, by September 2007, was the largest micro credit lender in Pakistan with over 370,000 active borrowers. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 6 21 Sources: CGAP [2007], Pakistan Microfinance Review [2006], and interviews with industry stakeholders conducted in the field by the evaluation team. 22 Financial Sustainability “indicates whether or not enough revenue has been earned to cover both direct costs, including financing costs, provisions for loan losses, and operating expenses, and indirect costs, including the adjusted cost of capital.” [Ledgerwood (1999)] 23 Pakistan Microfinance Review [2006] 24 Ibid. 25 A further push into a larger loan size market was given by the EDF project. According to terms of the Cooperative Agreement between USAID and PPAF, the latter would wholesale funds for loans to micro entrepreneurs in the range of $500 - $1,500. Basic Features and Challenges of the Microfinance Sector in Pakistan While the microfinance sector in Pakistan has great development potential, it faces stiff challenges. Some of the most important are the following:21 • The institutional foundations of most micro finance providers are very weak. According to CGAP, as of 2007, only two providers were sustainable. By the end of 2006, the combined indicators of financial sustainability22 for all providers in Pakistan was 62%, significantly below those reached by South East Asian countries (109%), and even regions of lesser economic and financial development like Sub-Saharan Africa (101%). • Growth in recent years is led by Khushhali Bank, an entity that is partially owned by the government. This bank accounts for approximately one third of total micro lending outreach. Its operational efficiency however, by the end of 2006, stood at 75%.23 • As regards women outreach, the number of active women borrowers constitutes 52% of the total number of male borrowers.24 • The commercial wholesale market is for all practical purposes undeveloped. Linkages between commercial banks and the micro finance providers are a new event, with the first transaction accessing commercial capital closing during 2007. • There is one government-sponsored APEX organization, the Pakistan Poverty Alleviation Fund (PPAF). The Chief Strategic Planning Officer of this entity revealed to the evaluation team that wholesale funds to micro finance providers were delivered at below-market rates. PPAF also limits loan sizes to under $500 from its main credit window – which was one major factor in the original design of WHAM to push into the larger loan size market.25 • A few reforms have yet to be introduced in policy and regulatory framework. Some of these include, for example, removing tax disincentives to microfinance banks, authorization for their mobilizing of institutional deposits, and the enactment of borrower protection laws. • Security in certain sectors of the country – the Federal Administrated Tribal Areas (FATA) and some areas of Baluchistan and the North West Frontier Province – greatly limits the supply of microfinance services to poor households and micro enterprises. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 7 III. EVALUATION METHODOLOGY The evaluation team consisted of MSI staff member Jorge L. Daly, Team Leader, with Aroona Kamal and Memoona Khan, independent consultants who participated in interviews with stakeholders and additional data collection and analysis. Douglas Krieger led the tasks centered in efficiency and impact analyses, and Maliha Hussein, a microfinance specialist, reviewed the draft for content, quality, and contextual accuracy. The evaluation tasks began with the arrival of the Team Leader in Islamabad on October 12th 2007. Because his arrival coincided with the Eid Holidays, effective kick-off of activities started on October 17th. Ms. Kamal was immediately available, but Ms. Khan was not incorporated into the team until November 1st. The Team Leader returned to Washington DC on November 20th after conducting a partial data collection in Islamabad, Karachi and Lahore, and assisting in the survey design for impact analysis. Efficiency and impact analyses were conducted during the period of February 18th – March 15th 2008. On November 19th, right before the return of the Team Leader to Washington DC, a presentation on preliminary findings was made to USAID. A. Evaluation Purpose and Questions The purpose of this evaluation is to conduct the final assessment of the WHAM project. In doing so, the evaluation will determine and inform USAID of the impact and sustainability of the project’s activities and results. The evaluation provides answers to both general and specific questions. The general questions are the following: • Relevance: how well was WHAM focused on the needs of the beneficiaries? • Effectiveness: has WHAM accomplished its objectives? • Efficiency: how efficient has WHAM been in utilizing resources to achieve intended results? • Impact: to what extent has WHAM benefited the people of Pakistan? • Sustainability: are activities and results likely to be sustained after WHAM is completed? • Replication: to what extent can the activities and results of WHAM be replicated? • Gender: to what extent has WHAM benefited women? • Reporting: has WHAM reported in a timely and useful manner? • Communications and Outreach: how effective has WHAM been in getting its story out? • Coordination: how effectively has WHAM been in coordinating with other parties? Specific questions of this evaluation included the following: • Have project activities directed to partner organizations increased their capacity? • Has the increased capacity generated new loans? • How many loans have been generated as a result of project activities? • How many loans have been disbursed in each quarter since WHAM started operations? (by region, lender, loan size, and gender) • What are projections for loan disbursements in the future? (by region, lender, loan size, and gender) • How many loans would have been generated without the project? WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 8 • How many loans would have been disbursed without WHAM intervention over the life of the project and into the future? (by region, loan size, and gender) • What would business income have been without the loans? (by region, business type, gender) • What were the monetary and non monetary impacts of the loans on recipient businesses? • What impact have loans had on the income of recipient businesses? (by region, lender, loan size, and gender) • How were the loans used and what impact did they have on the business? • What are the household level impacts of the loans? • To recipients? • How, if at all, have the benefits of the loan to the business affected the income of the owner and the welfare of the owner’s household? • Have the loans generated any secondary impacts as a result of increased business activity? (by region, loan size, and gender) • Have the trainings and media campaigns been effective? • What is the partner bank and NGO’s perception of the value of ShoreBank’s assistance? • Has the project been effective in changing the lending practices of the banks and NGOs? • What constraints, if any, do you face in offering new loans in the target range? • Under what conditions would you be willing to make more loans in the target loan range? B. Evaluation Design The WHAM evaluation is designed to be independent, objective, and findings-based. The evaluation questions are answered based on a general approach that progresses from findings to conclusions to recommendations. The process starts with findings which are observed and collected facts. These are “produced” during the data collection phase of the evaluation. Using analysis, interpretation and judgment, conclusions are drawn from the findings. In turn recommendations are based on the conclusions. Recommendations aim to identify practical actions for the consideration of project managers in the event USAID decides to continue to focus on finance projects. This approach gives report reviewers a clear view of the evidence that supports each of the study’s conclusions and recommendations. The evaluation methodology can be best illustrated and summarized by the “Getting to Answers Matrix” (GTA). This key evaluation design tool provides a format that allows the key evaluation questions to drive the identification of relevant data sources and data collection methods. This process – the completion of the GTA matrix – creates the point of departure for the collection of data necessary to the key evaluation questions. Table 1 below provides an illustration of the GTA as applied to the design of the WHAM evaluation. The full GTA matrix for the WHAM project evaluation is provided in Annex 4. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 9 TABLE 1: GETTING TO THE ANSWERS FOR OVERALL EVALUATION QUESTIONS Questions - Design Considerations - Specific Methods/Sources - Data Analysis - Answers Evaluation Questions Type of Answer or Evidence Needed Method of Data Collection Data Source Selection Criteria Data Analysis Methods 1. Relevance: How well was the project focused on the needs of the beneficiaries? Comparative of targets related to beneficiaries with results, Comparative of what was targeted versus felt needs, Strategic or cause and effect ƒ Review of project documents ƒ Interviews ƒ Surveys, ƒ Project data and reports ƒ USAID and project personnel, government officials, finance industry officials ƒ Borrowers Knowledgeabl e Persons, Randomly selected beneficiaries, stratified as appropriate for the project in PEGED surveys Comparisons Strategic analysis 2. Effectiveness: Has the project accomplished its objectives? Quantitative comparison of targets to baseline data, if available Strategic or cause and effect ƒ Review of project documents ƒ Individual interviews ƒ Group interviews ƒ Surveys ƒ Project data and reports ƒ USAID & project staff ƒ Lender staff & borrowers ƒ Borrowers Knowledgeabl e persons, random selection of beneficiaries, stratified as appropriate Comparison, quantified number of beneficiaries (disaggregated) and quantified benefits as possible C. Data Collection Sources and Methods The WHAM evaluation relied on the following data collection methods. • Review of relevant documents including (a) the USAID Request for Assistance (RFA); (b) the contractor’s technical proposal, performance monitoring plans, quarterly reports, and technical assistance agreements with partner organizations; (c) documents supporting the efforts to enlist commercial banks and microfinance organizations; (d) institutional and financial assessment of partner organizations; (e) market assessments of potential demand for SME and micro loans; and (f) several research papers covering topics that include: the impact of project-generated loans on borrowers, the potential of savings products, and the dynamics of micro loan expansion in Lahore. The review was conducted with the arrival of the Team Leader in Islamabad on October 12, 2007. • Interviews with key informants. These included (a) the Chief of Party and technical staff of WHAM; (b) three representatives of the donor community; (c) representatives of two commercial banks, including managers and loan officers of selected partner organizations; (d) two representatives of microfinance banks; (e) representatives and technical staff of two credit-granting NGOs; (f) officials of the public sector, including representatives of the Pakistan Poverty Alleviation Fund and the State Bank of Pakistan; and (g) two representatives of a civil society organization – the Pakistan Microfinance Network. The interviews were conducted in Islamabad, Lahore and Karachi, during the period October 17 – November 19 2007. A list of these key informants is included in Annex 3. • Group interviews with selected borrowers of two partner organizations, conducted in Lahore on November 6 – 8 2007. • Survey of borrowers. In order to determine project impact of loan activity, information was collected from a sample of 200 borrowers. The survey instrument is included in Annex 5. The survey was conducted from March 3 to March 7. D. Data Limitations In the process of collecting data, the evaluation encountered the following limitations: WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 10 • The task coincided with a period of high political tension in the country at large. The data collection stage was dominated by important events such as: a) the attempt on the life of opposition leader Benazir Bhutto on October 19th and her subsequent assassination on December 27th 2007; b) the imposition of a state of emergency by President Musharraf on November 1st 2007; and c) the postponement of national elections. These events created unintended obstacles, including sudden cancellation of flights and appointments with key informants. Given the above, some interviews with key informants were rescheduled and eventually conducted in less allotted time. Others could not be conducted at all, notwithstanding the efforts of the team. The most important key appointments that were missed were with the Ministry of Finance, the World Bank, and First Microfinance Bank. • The analysis of project impact was constrained by the lack of baseline data collected prior to or shortly after the start of the project. It was not possible, therefore, to identify reliable before-project data to be used as a point of comparison. Nor was it possible to identify a suitable loan recipient comparison group, furthering limiting the comparative analysis included in this report. IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ON THE OVER ARCHING QUESTIONS A. Relevance The purpose of this section is to determine if the design and implementation of WHAM was focused on the needs of the beneficiaries. Section II revealed that a minority of micro and small businesses has direct access to loans from formal financial institutions. WHAM was premised on the understanding that improvements in access to formal loans benefiting larger numbers of marginalized customers were going to be byproducts of technical assistance and training activities delivered to commercial banks, credit-granting NGOs and microfinance banks, and also to agencies directly engaged in the strengthening of the industry. Critically, design did not include using donor funds for loan capital. What this means is that none of the financial entities WHAM supported had to be enticed through cheap donor credit or guarantees to enter the scaling up and downscaling markets. 1. Findings • Among stakeholders, the evaluation team conducted interviews with representatives of four donors, two commercial banks, two microfinance banks, two credit-granting NGOs, and officials of the Pakistan Microfinance Network, the Pakistan Poverty Alleviation Fund, and the State Bank of Pakistan for a total of 13 organizations and 18 key informants. During interviews, each of these key informants stated that the project goals were relevant and appropriate, i.e., that they addressed important and unmet needs. • The existence of a large market segment that is not well served by the financial sector is also evidenced by market research and analysis supported by WHAM and conducted by SBP, Crescent Bank and the Lahore University of Management Science (LUMS). This research indicates that there exists between 500 thousand and 1 million business units with an average credit demand of $33,000 thousand which are considered “bankable new markets” but which have yet to be served by financial entities. • Looking at the “other end” of the missing middle, before the project started, very few microfinance providers had strategic interests in loans above $500. The average outstanding loan balance comprising the 19 microfinance providers that reported financial performance to the Pakistan Microfinance Review was $158. Not a single one of these entities reported average loan balances above $500.26 • Though the 18 key informants referenced above agreed on the relevance of the project goals and the importance of those goals to target beneficiaries, several raised questions about the project design/approach. Five of the 18 key informants interviewed noted that market conditions during the life of the project were not ideal for launching downscaling activities. In explaining their respective positions, each of the five pointed to financial spreads in the 26 Source: Pakistan Microfinance Review [2006]. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 11 commercial banking sector that were – and that remain – abnormally high (according to the World Bank no less than 4.5%)27. Additionally, the Chief of Party of WHAM noted that banking sector is consolidating and most banks are focused on seeing how the market evolves before they decide to venture into the SME segment. Finally, one of these stakeholders underscored remnants of a commercial bank culture that still view SME lending not as an attractive business opportunity, but as an endeavor fundamentally permeated by the need to be perceived, by the government or the public at large, as being socially responsible. • Also related to the relevance of project design, a review of project documents and an interview with the contractor’s Chief of Party revealed no evidence of pre-project analysis aimed at creating an in-depth understanding of the target market, at defining the target loan range, or at developing the most applicable approaches and instruments to address the target market (loans between $500 and $25,000). • When the project started, the Microfinance Ordinance of 2001 capped the maximum size of a micro loan at $1,666. This ceiling was not an initial constraint, as there was ample room for microfinance banks to offer loans between the $500 and $1,666 range. Subsequently, the Pakistan Microfinance Network, during the life of WHAM, successfully cajoled the State Bank of Pakistan to raise the micro loan limit to $2,500. • One specific aspect of the initial project design seems to have been ignored during project implementation, thereby missing an opportunity to address the needs of the target population. That is, while the original design emphasized increasing the volume of loans to the “missing middle” via the delivery of technical assistance and training to financial intermediaries, it also requested prospective implementers to propose the introduction of other financial instruments, including long-term and contractual savings, micro insurance and micro venture capital. However, the project proposal and implementation only developed and launched loan products. Though anecdotal, during group interviews conducted with 16 low-income women that are served by Asasah, six of the women indicated they had the capacity to save and expressed a need/demand for safe outlets for their savings. • Related directly to the point above, it must also be noted that two factors inhibited such outreach by the implementing partner. First is that NGOs are not licensed to mobilize savings. Second is that there was a contractual requirement for a cost share in the Cooperative Agreement of $10 million measured in terms of loans disbursed; hence the emphasis on lending from the outset. 27 However, experts in Pakistan, including managers of commercial banks and other financial entities, believe that actual spreads are actually much higher – no less than 7%. By way of comparison, normal spreads in countries that exhibit more competitive banking systems hover around 2%. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 12 • Looking again at actual implementation when considering the question of relevance and the needs of target clients, it is useful to note that project activities resulted in the provision of a substantial number of loans that were well above the upper limit of the definitional range of the missing middle. In the case of downscaling, nearly 40% of the 813 loans issued, all extended by the National Bank of Pakistan (NBP), were above the upper limit of $25,000. When considering WHAM’s scaling up work with Asasah – the only scaling up financial institution that received comprehensive support from WHAM – the picture becomes substantially more in line with project intent. As Figure 3 shows, the portion of loan recipients from the missing middle increases to approximately 84%. • A final point related to the relevance of WHAM concerns the question of gender. Increasing access to finance for marginalized women was never an explicit part of the project, as this objective was not included in project design. Nevertheless, WHAM, since project kick off, targeted entities that exclusively provide loans to women. (This issue is more fully discussed under both the effectiveness and gender evaluation questions that are presented later in this report). 2. Conclusions First, the project succeeded in addressing needs of beneficiaries. Its approach facilitated strategic decisions to target distinct financial entities – commercial banks and microfinance banks and credit￾granting NGOs – and design the contents of technical assistance services corresponding to the unique characteristics of each type of entity and the particular loan demand profiles of their clienteles. Barring use of donor funds for loan capital was also appropriate. Offering this incentive to potential “takers” would have accentuated donor-dependence behavior, with the corresponding negative consequence for long-term operational and financial sustainability. Second, design did not sufficiently factor in market conditions relevant to commercial banks when considering the downscaling aspect of the project. The lack of incentives for commercial banks to move out of their typical loan markets due to high loan margins undermined overall project effectiveness. Third, project relevance was undermined by too much focus on activities aimed at increasing the volume of loan activity, to the detriment of other financial products, especially savings.28 This was at least partially if not largely due to the contractual cost share requirement measured by loan volume. In many countries across the world, and Pakistan is no exception, the poor have capacity to save but they are offered neither the instruments nor the facilities to entrust their savings to financial institutions. As demonstrated by numerous studies, the economic welfare of poor customers can be enhanced more by the supply of savings instruments and facilities than by loans. 28 It is important to point out, however, that WHAM produced a research paper on the potential of micro savings in Pakistan. Figure 2: Distribution of Project Loans [Downscaling – 813 total loans] Figure 3: Distribution of Project Loans Downscaling and Scaling Up 16 % >500 <$25,000 61% 39% >$500 <$25,000 >$25,001 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 13 Fourth, the range of the missing middle was determined without adequate analysis. The range was established with no use of prior data and analysis supporting knowledge of the demand for financial services – types of products, volumes, frequency of demand, intended usage – from customers that straddle the segment. This was a critical constraint that made it difficult for WHAM to determine the precise location of the true market failures. Not surprisingly, 19% of all loans corresponding to the downscaling activity were equal or above $50,000 which is twice the upper limit of the missing middle range although 84% of loans fell within the range set out at project outset. Fifth, the definitional range ended up lumping under one single diagnosis needs of beneficiaries that emerge from different realities – for example the needs of a microentrepreneur who might require a $1000 loan for inventory versus a small businessperson looking for a $20,000 loan for capital improvements - with negative consequences for project relevance. To address this problem, WHAM focused on clearly separating downscaling of commercial banks from scaling up microfinance institutions, and sought out a variety of products and loan sizes that fitted the targets of selected implementing partners. Sixth, project design did not anticipate the benefits from targeting women customers. However, right after the kick off of implementation activities, WHAM rapidly calibrated the importance of this gap and actively proceeded to give priority to enlisting microfinance providers that cater their services exclusively to women. This section does not discuss the Industry Strengthening element (one-third) of the project. 3. Recommendations First, to the extent that USAID proposes interventions that target specific market segments, it is necessary prior to project kick off to obtain data and/or conduct comprehensive research regarding the size and customer-profile characteristics inhabiting such segments, and on the existing financial entities that can potentially serve them. Market failures within targeted segments would be more easily identified, and activities to address them more properly designed. In this way, project relevance gains considerably. Second, if USAID continues to focus on finance projects, USAID should commission studies to shed light on (a) the opportunities and constraints that underline potential interventions that target poor women; and (b) the demand for financial services other than micro loans. B. Effectiveness The purpose of this section is to determine if WHAM accomplished its objectives. The central objective of the project was to expand financial outreach, and to increase the capacity of Pakistani commercial banks, microfinance banks and credit-granting NGOs to support access of micro and small enterprises to credit services. Specific objectives included: (a) helping microfinance providers scale up loans above $500; (b) helping commercial banks downscale to provide small business up to $25,000; and (c) strengthening the microfinance industry of Pakistan. The discussion that follows seeks to determine effectiveness of the project in delivering activities, the outputs derived from the activities, and the eventual outcomes and results to which those outputs are intended to contribute. For ease of exposition, the discussion is organized along the following sections: (a) project indicators and targets; (b) findings and conclusions on the downscaling activity; (c) findings and conclusions on the scaling up activity; (d) findings and conclusions related to industry strengthening activities; and (e) overall recommendations. 1. Project Targets and Indicators The RFA provided a set of illustrative results and indicators from activities that contribute to Strategic Objective 6 – Increased Economic Opportunities for the Poor. At the moment of project inception, no end-of-project (EOP) targets had been predetermined for each of the three components of WHAM. With the nod of USAID, defining the EOP targets was left to SAS, which had to first pursue field analysis in order to propose realistic targets. This analysis consisted of developing quarter by quarter projections for each prospective partner organization. The figures were consolidated into project totals that were subsequently tracked as project execution proceeded. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 14 WHAM submitted to USAID a detail of indicators and proposed end-of-project targets by the end of 2005, that is, no less than six months after project kick off. These were finally approved by USAID in mid 2006, culminating in a combined delay of approximately 14 months. Table 2 depicts the indicators and targets proposed in June 2006 for both downscaling and scaling up loan activities, actual performance as measured in June and December 2007, and revised end-of-project targets projected for March 2008, which were approved by USAID in July 2007 following consultation with the contractor. TABLE 2: WHAM LOAN ACTIVITY: INDICATORS AND END-OF-PROJECT TARGETS Indicator End of Project Targets (stated as of June 2006) Actual Results (Downscaling, to June 2007) (1) Actual Results (Scaling Up, to June 2007) (2) Actual Results (Accumulated to June 2007) (1) + (2) Actual Results (December 2007) End of Project Targets (March 2008) # of Loans Disbursed/ Borrowers 550 564 13,905 14,469 37,036 39,333 Value of Loans Disbursed (US$) 10 million 15.7 million 7.2 million 22.9 million 47.9 million 46.2 million Loan Portfolio at Risk (%) 10 -0- 2.5 2.5 2.0 5 # of Female Borrowers 200 4 1,076 1,080 2,179 5,174 # of Repeat Borrowers 250 -0- 2,844 2,844 9,831 1,228 Annualized Revenue Generated by Loans (US$) 1.08 million 1.8 million 0.66 million 2.46 million 5.04 million 3.42 million # of Loan Officers Trained 50 173 154 327 555 722 Source: Quarterly Reports April – June 2006 and 2007, and October – December 2007 Table 2 shows that, in the twelve months following June 2006, the number of loans disbursed increased manifold and their value more than doubled. Original targets therefore were largely surpassed, and subsequently adjusted to capture a significant pick up of engagements with partner organizations that were engineered since late 2006. Table 2 also highlights that in the six months spanning June to December 2007 the dynamism of loan activity was maintained -- the number of loans increased by 156% during this timeframe and the total loan value jumped by 109%. As noted in Table 2, the number of loans disbursed by December of 2007 is 6% short of meeting the EOP target, while the total loan value has already surpassed the EOP target by 3.6%. With regard to female borrowers, as of December 2007, loans disbursed to women had achieved only 42% of end-of￾project target. Contrasting this lagging performance is the number of repeat borrowers, which by December of 2007 had surpassed the March 2008 target by a factor of eight. It is important to also look beyond loan activity at the project’s overall success in delivering training and related WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 15 skills/knowledge building services. As noted in Table 2, the number of lenders trained under WHAM, while ten times higher than the original June 2006 target, is still 23% short of meeting the end-of￾project target. When considering project effectiveness, however, it is critically important to recognize that the numbers presented in Table 2 represent project support to financial institutions that is both direct and indirect. Direct technical assistance and training can be defined as comprehensive, entailing a range of interventions that are clearly and closely associated with the provision of new or expanded financial services by the target institution. By contrast, indirect technical assistance is narrower, delivering very specific technical support. In the case of indirect support, the delivery training and technical assistance activities, at best, contributed only marginally to increased loan activity during the life of the project. For this reason, a more accurate assessment of in-progress project performance starts by recognizing that agreements with only two partner organizations – NBP in downscaling and Asasah in scaling up – can be closely attributed to USAID-supported loan activity. Table 3 presents results in these two entities as of December 2007 against the targets stated in June 2006. TABLE 3: WHAM DIRECT LOAN ACTIVITY – NBP AND ASASAH Indicator End of Project Targets (stated as of June 2006) Actual Results Downscaling (NBP, to December 2007) Actual Results Scaling Up (Asasah, to December 2007) Total Loans Difference # of Loans Disbursed/Borrowers 550 813 1,154 1,967 (+) 1,417 Value of Loans Disbursed (US$) 10 million 27.3 million 723 thousand 28 million (+) 18 million Source: WHAM Actual to Targets From Table 3 it can be seen that eighteen months after the original targets for the entire project were set, the total number and value of loans disbursed by NBP and Asasah surpassed June 2006 targets by 257% and 180% respectively. Table 4 presents the indicators, actual performance and end-of-project targets for the industry strengthening component. This training and seminars reflected in Table 4 are aimed at broadly at the financial sector and are not a part of the institution-specific training and technical assistance captured by Table 2. As Table 4 indicates, the project has generally exceeded expectations with regard to the delivery of activities aimed at overall industry strengthening and awareness raising. TABLE 4: INDUSTRY STRENGTHENING – INDICATORS AND Indicator Actual Performance (Accumulated to June 2007) End of Project Targets (March 2008) Trained Professionals 724 500 Trained Professionals – Female 114 26 Trainings Delivered 42 30 Industry Seminars and Events 5 6 New Publications Produced and Disseminated 7 5 Source: WHAM Actual and Targets 2. Findings and Conclusions on Downscaling Activities As noted, the objective of the downscaling activity was to persuade commercial banks to design and extend credit products to small enterprises straddling the upper limit of the “missing middle.” Such WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 16 lending would be funded with commercial banks’ own resources. WHAM would support the effort with the timely delivery of technical assistance and training to participating banks. Background on the downscaling approach and related project activities are detailed below: • WHAM proceeded to pursue a two-step strategy to enlist the participation of potentially interested commercial banks. First, it developed and followed a set of criteria for selecting commercial banks which included (a) the existing credit rating of the bank; (b) the existence, or lack of existence, of a SME policy and plan within the business and corporate strategy of the bank; (c) the allocation of loan-able funds and capital to small businesses during the project period; (d) the number of total branches devoted or suited to SME lending; (e) the availability of already skilled loan officers requiring only refreshing orientations or non￾intensive training on SME lending, as opposed to banks that needed to devote resources to the hiring of new specialists; and (f) the existence, or lack of existence, of adequate management information systems. Second, it proactively sought to enlist the participation of as many commercial banks as possible. To this end, WHAM sent letters of intent to twenty-eight commercial banks. Of these, a short list of eight potentially participating banks was constructed. Key decision makers in this short list were actively lobbied on the objective and content of a package of capacity building-technical assistance services required to successfully serve the SME sector. • During the first quarter of the project, particular emphasis was given to nailing down agreements with prospective partners that had signed Letters of Commitments to the contractor’s proposal. For the downscaling component, these were two – National Bank of Pakistan and United Bank Limited. Comprehensive institutional assessments to determine the capacity of these two entities to lend to small enterprises below the $25,000 loan size were conducted. A technical assistance agreement was signed with the NBP in October 2005, but further work with the United Bank was discontinued because the bank’s management was slow to respond. • After three years of operation, WHAM managed to establish relationships with only three commercial banks – Crescent Commercial Bank (herein Cresbank), Standard Chartered Bank (SCB), and National Bank of Pakistan (NBP). Only one of these relationships – NBP - resulted in the disbursement of loans to the target population. • The technical assistance agreement with Cresbank was signed in May 2007. The engagement included support for the assessment of market opportunities in SME lending and joint visits to areas in the country where WHAM is engaged in promoting SME finance development. It obligated WHAM to design a business strategy and plan for the launching of SME loan products. The final strategy, including financial projections, was presented to the bank’s board in August 2007. The Head of the SME Unit of Cresbank was interviewed and expressed the view that the services provided by WHAM were of high quality and very valuable for the objectives of the bank. By January 2008, the bank had not launched the SME loan products. At the moment of conducting this evaluation, technical assistance delivered to Cresbank had not resulted in an increase in the availability of loans to the target group. • The technical assistance agreement with SCB was signed in February 2007. Under the terms of the agreement, WHAM analyzed the root causes of the bank’s delinquencies in its Kissan Credit Card for small scale farmers, which can be used to purchase inputs from suppliers. It operates as a line of credit within a range of $333 to $2,083. In March 2006, the number of active cards was 9,227 and the average outstanding balance per loan was $417, which is below the lower limit ($500) of the definitional range of the “missing middle.” However, and because the credit limit is higher, the outstanding balance does not necessarily reflect the loan size itself. The project maintains that, while the Kissan card straddled the lower end of the “missing middle”, the product’s design was focused within the range of target groups. WHAM assisted SCB in the diagnosis of the repayment problem, which had increased from 5% in March 2005 to 27% a year later, in March 2006. Delinquencies were attributed to the fact that the credit card was not offered as a typical micro loan and its terms and conditions WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 17 did not follow microfinance best practices, including the benefits of having the loan officer handle initial assessment, monitoring, and collection of approved loan limits. WHAM proposed recommendations to fix the problems that led to arrears. By January 2008, bank management had yet to decide whether to implement them or not. Technical assistance delivered to SCB did not result in an increase in the availability of loans to the target group. Findings Related to Technical Assistance to NBP • As noted, the relationship that WHAM established with NBP is to be considered as the only one that directly led to loan activity under the downscaling component of the project. NBP is a state-owned commercial bank that “has the largest branch network in the country: over 1,200 branches employing over 14,000 people with total assets of $10.5 billion.29 Of late, as mentioned in Section I, the bank has been going through a process of modernization that is leading to increased profitability and higher prestige in the market. • Early in 2004 the bank started penetrating the SME market. The assessment conducted by WHAM in mid 2005 revealed the following problems: (a) scarcity of qualified credit officers devoted to SME lending; (b) very little knowledge on part of credit officers of cash flow analysis; (c) weak MIS system; (d) burdensome loan documentation requirements; and (e) high arrears.30 • The technical assistance agreement that WHAM signed with the bank obligated the project to initiate, via a pilot test in 18 selected branches, the application of cash-flow based lending methodologies. Eleven of these branches were located in Lahore Central and Lahore East, four in Gujaranwala, and three in Sialkot. The target was to roll out a total of 100 SME loans in these 18 branches. Under the pilot project WHAM committed itself to (a) assisting senior bank managers in the identification of staff that were to undergo initial training; (b) providing in-class training for loan officers; (c) delivering proprietary and confidential training curriculum to the bank; (d) reviewing at least 50% of the project loans and attending no less than 50% of the regional management committees (the venue where the project loans are reviewed); (e) assisting loan officers in loan memo development and financial analysis; (f) reviewing problem asset reports; and (g) conducting monitoring site visits. Beyond the pilot test, WHAM offered technical assistance for streamlining and modifying loan processes suitable to SME credit, propelling long-term development of staff, assessing successes and lessons learned, and determining roll out potential of SME lending strategy across the bank.31 • The evaluation team conducted meetings with the Head of the SME Department of NBP at the Regional Office in Lahore, two branch managers (also located in Lahore), four loan officers (two in each branch), eight borrowers (four in each branch), and the Chief of Party and two WHAM trainers. Findings from these meetings reveal the following activities and outputs: • During the first year of the pilot, until October 2006, 25 loan officers had been trained – 11 active in branches of Lahore Central, 11 in branches of Lahore East, two in branches of Gujaranwala and one in Sialkot. • A total of 173 loan officers were trained through November 2007. Training was conducted both in classroom and on the job. During the pilot program, WHAM used score cards to measure assimilation of knowledge and skills of the trainees but, according to WHAM Chief of Party, they were discontinued because their use was not well received by NBP management. The score cards were discontinued in early 2006 and replaced by a monthly loan activity reporting tool that tracks the number, value and breakdown according to size of disbursed loans, the portfolio at risk and the number of loan officers 29 Qureshi [2007] 30 Overview/Analysis of SME Lending Activities and Needs Assessment for the National Bank of Pakistan [June 2005] 31 Technical Assistance Plan 1.0 between SAS and NBP – October 25, 2005 to September 30, 2006. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 18 trained (but does not attempt to directly track the development of skills and knowledge of trained staff). • After the elimination of the score cards, WHAM did not develop other tools to track changes in knowledge and skills and increased use of improved SME lending techniques. Instead, the project assumed that the effectiveness of both classroom and on-the-job training activities would be evident in the volume and quality of loans the trainees processed. As stated by the Chief of Party, “if staff were well trained they could make good loans and if they were not, they were unable to make good loans.” In addition and because WHAM “provided supporting letters or recommendation to good loans and did not for bad loans, “it was clear to us which trainees were doing well and which were not.” • Each of the four loan officers that were interviewed praised both classroom and on-the￾job training but placed more value on the latter. On-the-job training is based on a mentoring approach. It involves WHAM trainers accompanying loan officers in all aspects of the loan cycle – identification and selection in situ of potential borrowers, collection and verification of relevant data, financial analysis of repayment capacities, risk assessments, approval or rejection criteria, and determination of loan size and payment terms. • The training activity to NBP started by focusing on providing training to non-bank trainers who in turn would provide training to bank staff. This approach was rapidly discontinued. Instead, WHAM relied on its own staff to provide training the staff of the Bank Training College and directly to loan officers. • The Head of the SME Department and the two branch managers expressed satisfaction with the content and quality of the technical assistance provided by WHAM. • Two loan officers of the same NBP branch indicated that the training imparted by WHAM had made possible the transfer of skills to the borrowers on the principles and practices of building more transparent and accurate financial statements. This finding was corroborated by the four borrowers of that same branch who were interviewed. • All eight borrowers who were interviewed stated that loan officers and managers do have a good understanding of their businesses and the constraints they face. They also praised the fact that loan documentation requirements are simple, and that processing approvals is relatively quick. • All eight borrowers expressed satisfaction with the terms of NBP loans – tenor of one year, 13% interest rate and collateralized by fixed assets. • Six of the eight borrowers, however, noted that NBP is not providing other services offered by competitors that have a direct impact on the efficiency and ease of bank transactions, such as electronic banking. • Two of the eight borrowers stated that they had obtained loans before from other commercial banks. Switching to NBP was largely determined by better terms, most concretely, a cheaper loan rate. In their view, this factor outweighs the alleged disadvantages of not having access to a more complete set of services, including electronic banking. • Looking beyond activities and outputs, a review and analysis of project documents, as well as interview and survey data evidences the following outcomes and results: • The 100 SME loan target established for the NBP pilot phase was surpassed, reaching 196 loans disbursed in the first twelve months of the pilot project (i.e., through the end of October 2006). • A follow-on to the original agreement with NBP was signed in February 2007 under which an additional 34 branches were incorporated into the pilot project for a total of 52 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 19 branches. The program also expanded into new regions, including Multan, Faisalabad, Jhang, Sahiwal, and Bahawalpur. By September 2007, the program was operating in 68 branches, including new additions in Rawalpindi and Gujrat. • The number of loan disbursements grew rapidly. In the six months corresponding to the period January – June 2007, 266 loans were disbursed, thus surpassing by 36% the total number of loans extended during the first twelve months of the technical assistance program. In the period July – September, 2007 an additional 127 were disbursed, and the total accumulated loans disbursed thru December 2007, that is, twenty six months after the program was launched, reached 813, with a total value of $27.3 million. By the end of 2007, the average SME loan size disbursed by NBP was $33,580. [See Figure 4] FIGURE 4: WHAM QUARTERLY TOTAL CREDIT DISBURSEMENT AND NUMBER OF LOANS DISBURSED NBP: WHAM Quarterly Total Credit Disbursement and Number of Loans Disburesed - 1 2 3 4 5 6 7 8 9 Jan-Mar 06 Apr-Jun 06 Jul-Sep 06 Oct-Dec 06 Jan-Mar 07 Apr-Jun 07 Jul-Sep 07 Oct-Dec 07 Millions Quarter Value (US$) - 50 100 150 200 250 300 Number Value of Loans Disbursed Number of Loans Disbursed • As clearly highlighted in Figure above, in the first quarter of 2007, loan activity sustained a steep decline, as only 6 SME loans were disbursed by NBP. The WHAM Chief of Party, during interviews with the evaluation team, attributed the decline to fatigue in NBP from the heavy volume of loans extended in the previous quarter and to time spent in negotiations to renew the technical assistance agreement which was eventually signed in February 2007. Another, less pronounced decline occurred in the third quarter of 2007 but loan growth picked up steam again in the following quarter. The June and December spikes in lending reflect the timing of the twice-yearly lending targets and incentive plans for branches and loan officers. • The distribution of borrowers according to loan size is illustrated in Figure 5 Approximately 60% of loans fall within the range of the “missing middle.” Borrowers in this range are devoted to varied activities that include grocery stores, beauty parlors, small manufacturing, retail of garments, auto workshops, medical stores, electro domestic stores, etc. Those above the upper end of the range include larger manufacturing concerns, large-scale traders, and even exporters. • The number of loans disbursed in 2006 was heavily concentrated in the branches of Lahore Central (46%) and Gujaranwala (33%) for a total of 79%. Lahore East, in spite of contributing the largest number of branches in the pilot project (six of eighteen pilot branches), only accounted for 15% of loans. In 2007 loan concentration in Lahore Central and Gujaranwala declined to 47% while the branches of Faisalabad contributed to 21% of total loan for a total relative contribution of 68% of all loans among branches of these three regions. The contribution of Lahore East declined dramatically. [Figure 6] WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 20 FIGURE 5: DISTRIBUTION OF NBP LOANS FIGURE 6: REGIONAL SHARE OF TOTAL LOANS DISBURSED (2006 AND 2007) • The four regions selected in the pilot, with the exception of Lahore East, sustained increased loan volumes in 2007. Lahore Central and Gujaranwalla saw their relative contribution decline in 2007. The performance of branches in Lahore East was particularly disappointing in 2007, as loan disbursements were 97% below target. • In all, 22 of 68 branches that were targeted for technical assistance were “dormant”, that is, showing no loan disbursements during 2007. • As of the end of 2007, the number of active SME loans was 813 for a portfolio value of $27.3 million. Approximately 2% of the outstanding loans were classified as delinquent. Conclusions on Downscaling Activities WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 21 First, the strategy and procedures that WHAM followed to identify and select partner banks was sound. In a first approximation, WHAM correctly attempted to rule in all 28 commercial banks. Expressions of interest however were quickly followed by respondents’ self assessments and quick appraisals of bank conditions that were or were not conducive to rolling out SME loans in a quick fashion. The selection criteria used to screen and select commercial banks were correct. The approach used by WHAM bore fruit because at the beginning of the second quarter of the project WHAM already had a short list of eight potentially suitable partners with whom it could craft technical assistance agreements. Second, notwithstanding the application of such a sound strategy, as regards the delivery of technical assistance and training for loan activity, WHAM could enlist the participation of only one commercial bank -- the state-owned National Bank of Pakistan. It appears that this result cannot be attributed to lack of effort of the project. The more plausible reasons lie in the negative impact of factors that were largely outside the control of WHAM. During the life of the project, commercial banks have been very busy and focused on reaping huge profits from loans targeted to market segments other than the SME sector, and also from non-credit financial operations. Under these circumstances, there are no pressures to venture into new market segments. WHAM did not fully appreciate the limited interest of commercial banks in the SME sector. To effectively enter the sector, investments are necessary to hire staff who are experienced and competent in dealing with SME firms, to set up management information systems, to design and launch a tailored marketing campaign, and the like. Due to consideration of both actual and opportunity costs related to such requirements, banks will be cautious when considering entry into the SME sector. It is indeed very telling that the only enlisted entity was one with whom Shorebank International had previously developed a successful relationship. To its credit, WHAM was very opportunistic by taking advantage of this relationship to launch the downscaling activity. NBP was more receptive and more easily swayed than private commercial banks to the benefits of extending financial services to a clientele whose creditworthiness has been traditionally suspect. Furthermore, NBP is not subject to the consolidation game occurring in the rest of the industry, and therefore offered a more stable partner who could take more risks and a longer term view to downscaling potential. Third, WHAM was not entirely focused on supporting loan activity within the range of the pre￾determined “missing middle.” The average loan size of the program with NBP exceeds the upper limit of the missing middle range by 34%. This issue is of little relevance. For one, as noted in the previous section, the range was determined arbitrarily. It is the perception of the providers of financial services who are in daily contact with the market, more than estimates derived from desk-studies, that really matters. From experience, they know the qualitative differences between corporate and small enterprise borrowers, and build an approximate range of loan sizes that the latter demand. Such an approximation, it is worth underlining, is seldom perfect, but is not devoid of practical utility to bankers: when extending loans, whatever loan-size deviations from the range approximation is not important as long as they know that the bank has enlisted (a) new client(s) whose profile conforms to the characteristics of the targeted clientele. The distribution of loans across size intervals is more relevant to determine what groups of borrowers benefited most WHAM intervention, and which of them were left out. Data from Figure 5 suggests that the 60% of borrowers located within the missing middle were all served in different loan-size intervals, with larger clusters in lower half of the range and in the $15,001 - $20,000 range. Fourth, as 40% of the SME loans issued by NBP were above the upper limit of the missing middle, it is plausible that the program with NBP may be targeting borrowers who can easily obtain loans from other commercial banks. This is almost certainly the case for those 19% of SME loan recipients who secured loans in excess of $50,000. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 22 Fifth, loan activity in 2007 was concentrated in three regions of Punjab – Central Lahore, Gujaranwala and Faisalabad. In clear contrast with late nationwide trends of SME loans,32 these are regions which have witnessed in recent years rapid growth of bank loan activity, more so than in other regions of Punjab and of the rest of the provinces in the country. It is plausible, therefore, that external forces and a very dynamic market were driving some of the increase in loan activity. Thus, it is difficult to precisely attribute the level of WHAM impact in this context. Sixth, loan disbursements have indeed grown dynamically and considerably. By the end of the first year of the pilot program, the number of disbursed loans had surpassed the target by almost double. And in the 14 months that followed, until December 2007, disbursements had increased by 317%. However, the performance across branches of the program has been uneven. The issue goes beyond the previously noted concentration of loan activity in the three regions of Punjab. For one, there is reason to believe that the branch selection for the pilot launched in October 2005 was far from optimal, as the branches in Lahore East underperformed badly (as regards meeting loan targets) during the first year of the pilot and in 2007. Second, as regards support extended to the new branches that were incorporated since early 2007, each of these branches, with the exception of those in Faisalabad, had a slow roll out and have failed to meet targets. Two corollaries from this discussion follow. The first is that there does not seem to be correlation between number of loan officers trained and loan activity performance. As noted, 11 of the 25 loan officers that were trained during the pilot phase hailed from Lahore East, whereas the branches of Gujaranwala, which have performed far more dynamically, had only two active trained staff. Second, the presumed effectiveness of technical assistance and training activities delivered to the branches incorporated in 2007, with the exception of those in Faisalabad, does not match that obtained in the pilot branches of Lahore Central and Gujaranwala. Seventh, technical assistance and training activities were focused on helping NBP develop a line of business that could trigger the production of high quality loans. WHAM tracked progress more on the basis of the lending performance than on the intermediate goal of improved staff capacity. While this emphasis is aligned with the project objective of producing loans, in retrospect it may have been shortsighted. For it must be underscored that the engagement of WHAM trainers with loan officers was very intensive, involving them as mentors in virtually all aspects of the loan cycle. In circumstances like this, the trainee undoubtedly benefits, but absent objective indicators that attest to improved knowledge, skills and application of improved practices, there is no way to ascertain when and if they will “do it all on their own.” The risk then is that skills are not effectively transferred and with the departure of the trainers, those skills will be lost to the NBP SME program. 3. Findings and Conclusions on Scaling Up Activities The objective of this activity was to persuade credit-granting NGOs, RSPs and microfinance banks to design and extend credit products to small enterprises straddling the lower limit of the “missing middle.” Before the project started, these entities generally extended group-collateralized loans in the range of $160 to $250. Scaling up to loans above $500 requires improvement of institutional capacities to process individual loans. Clientele profiles are not the same in each case, and methodologies to assess and mitigate risks are consequently different. Background on the scaling up approach and related project activities are detailed below: • During the first quarter of project execution, WHAM focused on enlisting partner organizations that had signed letters of commitment in the proposal, as well as on several institutions that did not. These organizations included the Khushhali Bank, the Pakistan Poverty Alleviation Fund, First Microfinance Bank (FMFB), National Rural Support Program, Damen, Asasah, Kashf Foundation and the Urban Poverty Alleviation Program (UPAP). A rationale for targeting most of these organizations was prepared in the second quarter. 32 Quoting data from the State Bank of Pakistan, WHAM revealed that total value of SME loans in the country declined from 408 billion rupees in December 2006 to 384 billion rupees in September 2007, a decline of 5.8%. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 23 • Though delays occurred in negotiating agreements with several potential partner organizations, agreements were eventually signed with six partner groups. During the period April 2005 – November 2006 WHAM signed technical assistance agreements with: (a) Kashf Foundation; (b) FMFB; (c) Tameer Microfinance Bank; and (d) Asasah. In the last quarter of 2007, WHAM signed additional agreements with the National Rural Support Program and Khushhali Bank. These 2007 agreements commit WHAM to assist NRSP in the preparation of a business plan for its eventual transformation into a microfinance bank, and to deliver to Khushhali Bank trainings on sales force management and financial analysis. The cost of this training is split in half between the project and the bank. • Kushshahali Bank, NRSP and Kashf Foundation, as regards client outreach, are the most important microfinance providers in Pakistan. The total number of active clients of these three entities represents 67% of all micro borrowers in Pakistan.33 Kashf Foundation and Asasah are focused on delivering loan services almost exclusively to women.34 • The agreement with Kashf Foundation was signed in March 2006. It involved assistance to improve the design of an individual loan product the organization was developing, for an average size of $666. The agreement also included the design of a plan to pilot test a mobile phone cash payment system, and the development of credit-risk management strategies. • The evaluation team conducted an interview with the general manager of Kashf Foundation. He expressed satisfaction with both the content and quality of technical assistance provided by WHAM. Before taking up this position, the general manager was employed by WHAM, from October 2005 to March 2007. One of his responsibilities was to set up the project’s operations and hire local trainers for the Lahore office, from where technical assistance and training activities are conducted and overseen. • With FMFB, WHAM committed to determine the bank’s needs as regards its readiness to use technological innovations leading to increased outreach and improved overall organizational efficiency. As part of the agreement, WHAM provided recommendations on the adoption of electronic banking systems applicable to urban and rural contexts and has supported the development of an internal training curriculum to equip FMFB loan officers with the skill to provide scale up loans and has helped disseminate this curriculum across FMFB branches. Notwithstanding repeated attempts to establish contact with the officials of this entity, the evaluation team was not able to conduct an interview. • The agreement with Tameer Microfinance Bank was signed in October 2006. Since its inception in 2005, this bank sustained phenomenal high rates in the growth of its individual loans which, unfortunately, led to a significant deterioration in the quality of its portfolio. The bank enlisted the assistance of WHAM to (a) review the quality of the loan portfolio; (b) revamp entirely the loan processing system and procedures; and (c) improve human resource development policies and procedures. In addition, WHAM is footing the salary bill of two trainers and the start up of a training center in Lahore. The evaluation team conducted an interview with the Tameer Microfinance Bank. He considers that the technical assistance provided by WHAM has been very valuable to his entity. • Of the six technical agreements signed with partner organizations, only the agreement with Asasah resulted in the delivery by WHAM of substantially comprehensive assistance that is clearly attributable to the provision of loans to the defined target group. Findings Related to Technical Assistance to Asasah Asasah is a Lahore-based credit-granting NGO, established in 2003, that provides both financial services and non-financial services, including micro-enterprise training, to low-income households. 33 Pakistan Microfinance Review [2006] 34 According to the Pakistan Microfinance Review [2006], 98.4% of Kahsf’s borrowers were women. In the case of Asasah, the percentage of women borrowers was 100%. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 24 The principal objective of Asasah is to contribute to the eradication of poverty. The average annual income per capita of its clients is $320, which is less than half of the national average. All of its financial and non-financial services are targeted exclusively to women. Outreach has steadily increased from 4,500 borrowers in June 2004 to 27,711 in June 2007. Loan portfolio growth has been largely driven by one loan product, labeled “productive loan,” whose average size is $220. Because it extends group-collateralized loans, the default rate is very low. Still, the entity is remains highly dependent on government and donor largesse: PPAF contributes over 90% of its loan funds with a credit facility that is offered at below market rates (8%). Asasah also benefits from a $250,000 grant from Save the Children and another for $100,000 from the Swiss Agency for Development and Cooperation (SDC). WHAM’s assistance to Asasah has been intensive. Three technical assistance agreements were signed to support the processing of the small business finance loan (SBF) which is offered to customers that, in average, earn $1,500. This figure represents almost five times the annual income of borrowers who access Asasah’s typical “productive loan.” The agreements committed WHAM to (a) train branch level staff in financial analysis; (b) support Asasah trainers to train new SBF loan officers, developing training modules for SBF modifications; (c) monitor the productivity, efficiency and quality of the SBF line of business; (d) modify the monitoring system of the SBF product and train the staff to conduct monitoring; (e) establish procedures for branch managers to approve loans up to $666; and (f) provide advice to the CEO on growth and institutional development strategy.35 The evaluation team conducted interviews with a management consultant of Asasah, three managers (Training, Human Resources and Marketing), the Corporate Executive Officer, two branch managers, and one WHAM trainer. Group interviews were conducted with a total of 16 women borrowers and existing project documents and data sets were reviewed. Findings from these data collection efforts are detailed below: • WHAM training was focused on improving the knowledge and skills of managers and loan officers. As regards loan activity in particular, WHAM mentored the three managers and also the Head of Operations who, in turn, trained 81 loan officers. WHAM remains engaged with on-site support post-training. Results on loan activity are reported to the Corporate Executive Officer on a weekly basis. • The three managers and the Corporate Executive Officer who received training directly from WHAM were satisfied with both the content of quality of training. They were exposed to a three-day workshop on the basics of the SBF loan product and principles of loan processing and risk mitigation. The Manager of Training, who supervises all overall training to loan officers, expressed the view that increase of loan disbursements and the good repayment performance they exhibit are tied to the technical assistance delivered by WHAM. The two branch managers also subscribed to this statement during interviews. • The Manager of Marketing promotes the SBF loan product. He stated that there are two major obstacles related to the product that must be overcome so as to enlist new clients: small loan size and bi-monthly repayment schedules. WHAM is aware of these constraints but, for the time being, does not recommend any changes on grounds that such changes will increase risk significantly. • Disbursements of the SBF loan product increased steadily from 20 in the period from July - September 2005, to 250 during the same time period two years later. However, in the last quarter of 2007 the number of disbursed loans declined to 55 – a decline attributed to the inability of Asasah to access funds for loans. By the close of 2007 an accumulated total of 1,154 SBF loans had been disbursed by Asasah. [Figure 7]. 35 Technical Assistance Agreement 3.0 January 1 – December 31 2007 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 25 FIGURE 7: ASASAH: WHAM QUARTERLY TOTAL CREDIT DISBURSEMENT AND NUMBER OF LOANS DISBURSED • 96% of the individuals receiving SBF loans, a total of 1006 borrowers, were women. • All disbursed SBF loans are within the range of $500 to $1,666, with the average loan size being approximately $650. Loans are collateralized by groups of eight to ten borrowers and repayment rates hover around 95%. The managers who were interviewed attribute low arrears to both the requirement for group collateral and to the quality of technical assistance delivered by WHAM. Conclusions on Scaling Up Activities First, WHAM was effective in enlisting partner organizations that have an important presence in the microfinance market. NRSP, which was enlisted in late 2007, and Kashf Foundation are, respectively, the second and third largest providers of micro loans in Pakistan. Tameer Microfinance Bank is quickly establishing itself as a major player in the market and FMFB has established branch presence in all four provinces of the country. Finally, Asasah, whose CEO worked previously with Kashf Foundation, is focused on delivering financial services to women. Second, WHAM support to Asasah contributed directly and substantially to increased access to loans for borrowers that straddle the lower end of the missing middle. Borrowers who traditionally were only able to receive micro loans, benefited by obtaining substantially larger value loans. Third, by improving the design of Kashf’s loan product, and by providing intensive technical assistance and training for the delivery of Asasah’s SBF loan product, WHAM effectively contributed to increased outreach and financial access for women borrowers, albeit modestly. Fourth, as in the case of NBP, technical assistance and training activities were focused on helping Asasah develop a line of business that could trigger the production of high quality loans. Fifth, WHAM tracked performance of the scaling up component of the project primarily in terms of the volume, value and quality of loans made. Indicators related to the quality and effectiveness of training and technical assistance, i.e., changes in knowledge, skills and practices, were not tracked by the project, thus limiting the ability of WHAM to make useful adjustments to its training and technical assistance activities. Sixth, though WHAM monitored progress more on the basis of the lending performance than on the intermediate goal of improved staff capacity, all of the senior managers from partner organizations who were interviewed stated that the technical assistance and training activities provided by the project were effective, useful and relevant. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 26 Seventh, from a conceptual point of view, assisting NRSP to help its transformation into a microfinance bank is an activity that, if it is well implemented, is bound to have high impact. Given that the process of legal transformation seldom bears fruit within a short time span, and because technical assistance was launched with only three months remaining in the life of the project, it is not possible to determine the effectiveness, or likely effectiveness, of this activity. 4. Findings and Conclusions on Industry Strengthening Activities This project component was not initiated until late 2005. Activities that were implemented were focused on the following objectives: (a) taking stock of the level of awareness among stakeholders of basic facts and held opinions regarding the microfinance industry; (b) updating and disseminating knowledge of market size, industry data and trends; (c) enhancing knowledge of factors that restrain long-term growth; and (d) delivering training tailored to downscaling and scaling up practitioners. Review of project documents and interviews held with stakeholders reveal the following findings: • As of December 2007, a total of 42 one to three-day training events were delivered. Total attendees number 724, of which 114 were women. The training events addressed topics relevant to both micro and SME lending. The former included basic concepts on microfinance and key principles and procedures of individual lending such as loan tracking, credit risk management, institutional risk management, credit committees, and information management. In some of the training events tests were handed out to grade degree of acquired knowledge. Attendees included loan officers and branch managers of credit-granting NGOs and microfinance banks. Some of the trainees were selected to become trainers of loan officers of their respective organizations. Training events focused on SME lending included presentation of techniques of cash-flow methodology, imparted mainly to loan officers and branch managers of commercial banks, including members of the NBP Staff College in Lahore. Higher-level training on SME markets and lending policies were delivered to vice presidents and managers of commercial banks. • WHAM organized and sponsored five seminars and conferences that increased the visibility of the microfinance industry. An example of one such event is the conference conducted in Islamabad in June 2006 to facilitate exchanges of experiences among microfinance providers and commercial banks. The two-day event drew the participation of 32 representatives from various microfinance providers, and four commercial banks. In October 2006, in collaboration with the Pakistan Microfinance Network, WHAM organized a conference on the commercialization of microfinance, a roundtable on approaches multi purpose organizations can use to more effectively extend financial services to new borrowers and markets. The event drew the participation of eight microfinance providers. The four key informants interviewed from the donor community and private microfinance providers attended one or more of these events and noted that they were of high quality and valuable for the industry. • WHAM conducted seven “knowledge-generation initiatives,” including a research paper on the dynamics of microfinance expansion in Lahore that was a joint undertaking with the Financial Sector Strengthening Program (FSSP) of the Swiss Agency for Development and Cooperation (SDC), the Department for International Development (DFID) of the United Kingdom, and the Pakistan Microfinance Network.36 Other initiatives have included a research paper on the potential market for micro savings, a report on the microfinance industry during the period 1999- 2005, and the launching of Micro Watch, a quarterly bulletin that provides up-to-date information on microfinance outreach. The four key informants interviewed from the donor community and private microfinance providers praised the quality and importance of these deliverables. • In close collaboration with the PMN, WHAM is currently working on a capital market research initiative that has already resulted in a concept paper on factors that impede long￾term funding to the microfinance industry, and principles for the establishment of a credit 36 Bano and Mehr Shah [2007] WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 27 enhancement facility. The managing director of the PMN expressed satisfaction on the assistance extended by WHAM. • WHAM’s collaboration with the Pakistan Microfinance Network was aimed at strengthening its institutional presence among key stakeholders. During the period of project implementation, the number of microfinance providers that submit financial reports to the PMN augmented from 18 to 25. • In the middle of 2006 WHAM launched efforts to build a training center for the entire microfinance sector of Pakistan which would outlast the project. The initiative took first the form of assessments of the training market. Thereafter, WHAM attempted to enlist local and international institutions to house the center. These efforts bore no fruit. In mid 2007, WHAM designed and launched a Human Resource Development Initiative (HRDI) that targets microfinance middle management. The initiative resulted in the selection of trainers, the development of course materials, the delivery of a training of trainer workshop, marketing among prospective participants, and planning for future sustainability. Finally, in late in 2007, WHAM signed a contract with Khushhalibank by which HRDI will be carried forward beyond the end of the project. • WHAM conducted two attitude and awareness surveys of key microfinance stakeholders, including officials from microfinance providers, government, commercial banks and donors. The first survey, conducted in September 2006, was distributed to 225 individuals, a third of which responded. The main findings were: (a) approximately half of the respondents were unaware of the costs of delivering microfinance services; (b) roughly a third of respondents considered loan rates above 20% as unfair to clients; (c) over half the respondents stated that financial sustainability of microfinance providers is essential (which typically requires interest rates in the range of 30%); (d) more than half of the respondents also noted their belief that microfinance contributes to poverty alleviation; and (e) there is general support for the development of inclusive, gender-balanced microfinance sector. A year later a second survey was conducted. Complete results of the survey were unavailable at the time this evaluation was conducted, but preliminary findings indicate that while respondents still see financial sustainability of microfinance institutions as important, over a third of (36%) stated that effective interest rates above 20% were unfair to clients. Views on microfinance contribution to poverty alleviation, costs and gender appear to be positive and consistent with the 2006 survey. • Finally, WHAM delivered several formal presentations to key stakeholders in the private and public sector on the size, characteristics, and trends of the SME and microfinance markets. Conclusions on Industry Strengthening Activities First, in less than three years of operation, WHAM conducted on average fourteen training events per year on the principles and practices of micro enterprise and SME lending, which is more than one event per month. This is a high output. Sixteen percent of trainees were women. The average number of attendees per event was 17. The training events related to microfinance imparted techniques applied to individual lending to organizations that have hitherto relied in group lending methodologies. The individual lending techniques, when and if applied by the attendees, is likely to steer their organizations along a path of faster growth. Training events that addressed SME lending were conducted in a period in which downscaling activity on part of commercial banks is not particularly dynamic. If commercial banks enter into the SME segment sooner rather than later, the benefits of the events will not be lost. Second, WHAM has contributed to improving knowledge of, and within, the microfinance industry. The research piece on the Lahore microfinance market sheds light on threats of borrower indebtedness which, if not addressed soon, can trigger a systemic crisis as evidenced in experiences elsewhere. The research piece on micro savings highlights the opportunities and constraints of developing savings instruments. When and if delivered by microfinance providers, savings products can enhance the economic welfare of poor customers, and strengthen the growth prospects of deposit-taking institutions. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 28 Third, WHAM has contributed to enhancing visibility of the microfinance industry through the delivery of seminars and conferences and via the dissemination of updated industry information. In this regard, assistance extended to the Pakistan Microfinance Network, including the efforts that this entity makes to use benchmarks and make information on the performance of microfinance providers more transparent, has been particularly effective. Still, as derived from results of the awareness surveys, a significant proportion of microfinance providers and stakeholders still cling to the view that loan rates should be determined by perceptions of what is “fair” to the borrower and not by criteria that allow for – or constrain - institutional sustainability. As this example illustrates, much work remains as regards the education of stakeholders on microfinance best practices. Fourth, much time and effort was spent early on in trying to identify local and international institutional venues to establish a training center for microfinance practices. Filling this need is critical because improving the skills and knowledge of middle-level managers is a key condition for the long￾term growth of microfinance providers. WHAM finally launched in the second half of 2007 the Human Resource Development Initiative, which will be housed and carried forward by the branch office of ShoreBank International after the project’s completion. Given its very recent establishment, it was not possible for this evaluation to examine the effectiveness of the Initiative. 5. Recommendations First, if it continues engaged in finance interventions, USAID should define with as much precision as possible the contours of market segments upon which activities will be applied. Too broad a definitional range of intervention does not help, as sub-segments require different financial instruments. Projects should clearly identify targeted sector characteristics so as to address more effectively the needs of beneficiaries. In the case of WHAM, for example, the characteristics and requirements of borrowers at the low end of the missing middle were vastly different than those at the high end of the missing middle. Second, in an eventual further involvement in finance projects of Pakistan, USAID should not support in the short run projects that consider downscaling activities, as the banking industry is not likely to penetrate the SME market within the next three to five years. More generally, to the extent that it keeps its focus on finance projects, USAID should pay special heed to existing market conditions. In projects that critically depend on partnering with financial entities, and where no money for loan capital is provided, this includes appraising their interest and assessing prospects for their intervention in particular market segments (e.g., downscaling). Such an assessment also includes pinpointing possible segments upon which economic and financial activity is likely to “take off” independently, that is, without the catalyst effect of the project intervention. Third, if and when sponsoring future finance projects, USAID should monitor closely patterns of distribution of loan activity to determine if loans are unduly concentrated in particular regions and if other important segments are consequently underserved. Early attention to these problems gives USAID and the contractor an opportunity to introduce necessary modifications to project focus and activities. Fourth, in eventual finance projects that emphasize capacity building of financial institutions, USAID should require that contractors use tools that measure and monitor acquisition of new skills, improved knowledge, and the eventual use of improved practices by trainees. In addition, USAID should require that the contractor use benchmarks to track progress on institutional change. Anchoring project results just on loan activity will not do, for there is no guarantee that authentic capacity building of partner organizations will have occurred once the project terminates operations. Fifth, in case it contemplates further interventions in finance, USAID should establish well-defined indicators and targets as close to the beginning of project implementation as possible. If indicators and targets are not defined at the onset, there is a risk of diverting precious time and resources for more productive activities. Certainly, as a project gathers new information, indicators and targets can be modified accordingly. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 29 Sixth, to the extent that it stays actively engaged in the Pakistan context, where the regulatory frameworks allow the legal transformation of a credit-granting NGO into a commercial bank, USAID should nudge the contractor to exhaust efforts aimed at (a) identifying, early in the project life, those entities that are willing and ready to do it; and (b) crafting technical assistance agreements that will contribute to the transformation. Pursuing such a transformation strengthens the institutional capacity of donor-dependent microfinance providers and consolidates their path towards long-term sustainability. Seventh, if it chooses to stay focused in microfinance in Pakistan, where the availability of skilled human capital is a constraint for long term growth of the SME and microfinance industry, USAID should require that activities directed at laying the groundwork and developing local self-sustaining institutional capacities to address this constraint, be undertaken early in the project life. C. Impact The monetary impacts of microfinance lending are notoriously difficult to quantify even with well designed impact assessments (Hussein & Hussain, 2003). The absence of baselines in the three microfinance projects of USAID’s Economic Growth portfolio make estimating reliable measures of monetary impact particularly difficult. This section reviews the available evidence and conclusions regarding monetary and non-monetary impacts associated with the WHAM project. Table 5 summarizes the types of monetary and non-monetary benefits typically associated in the literature with microfinance lending. Evidence of these benefits, particularly the monetary benefits, is largely anecdotal. Evidence from more rigorous approaches to identifying these impacts is mixed: changes are often small, marginally significant, and inconsistent across studies. TABLE 5. PRIMARY MONETARY AND NON-MONETARY BENEFITS OF MICROFINANCE LENDING Monetary benefits Non-monetary benefits • Increased business income/profit • Increased household income • Increased asset value (business and household) • Increased business activity (sales) • Increased household expenditure and consumption • Increased spending on education • Increased employment • Women’s economic empowerment (for loans to women) and status in household • Improved family health and hygiene • Increased school enrollment • Improved family nutrition • Improved food security This chapter first reviews the measurable monetary impacts of WHAM lending followed by a review of non-monetary benefits. 1. Findings – Monetary Impacts The evaluation team was able to estimate monetary measures of project impacts only for the one-third of the project that focused on scaling up. Furthermore, it obtained statistically reliable estimates of impacts only for the estimated 56% of scaling up loan recipients who ran retail businesses. The findings in this section and in the Efficiency section thus refer only to the costs and benefits associated with the scaling up component of the project and only for retail businesses. The evaluation team conducted a survey of borrowers from Kashf, one of the WHAM scaling up partner organizations, to estimate the quantifiable impacts of WHAM scaling up lending.37 The survey focused on Kashf because of the size of its portfolio and the availability of data for sampling. In addition to estimates of monetary impacts of loans, the survey collected information on borrower characteristics and use of loans to produce a quantitative profile of borrowers. Annex 4 provides detail 37 The original sampling plan called for a sample from the National Bank of Pakistan (NBP) as well to represent the downscaling component. The bank, however, was not willing to share client details with an outside survey firm and the evaluation team ran out of time waiting for the necessary clearances. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 30 on the survey including experimental design, questionnaire development, sampling strategy, and copies of the questionnaires in English and in Urdu. The survey included both treatment (loan recipients) and control groups (businesses that had been approved for a loan but had not yet received the money.) Comparison of the two groups, both with measures of change in business profits between pre and post-loan periods, provided the data necessary to estimate impacts of the loans on business profits.38 The final sample consisted of 129 observations on the treatment group and 102 observations on the control group. The following two sections review, respectively, survey findings relative to current and projected (future) monetary impacts. Current Monetary Impacts Because of the difficulty in obtaining reliable estimates of change in business profits (see Annex 4), the survey asked about change in profits from several perspectives. All questions asked respondents to compare their typical monthly profit before receiving the loan to their typical monthly profit after receiving the loan. The first question asked generally whether profit had changed as a result of the loan. The subsequent three questions attempted to quantify the magnitude of the change beginning with relatively exact measures and progressing to less exact measures depending on a respondent’s willingness or ability to answer. The first question asked directly for the approximate change in profit as a result of the loan. If the respondent would not answer (either because he/she would not or could not), the interviewer attempted to obtain a less exact response by asking if the change in profit fell between broad thresholds. If the respondent still would not answer, the interviewer asked for the likely percentage change in profit. To provide at least one consistent measure over the entire sample, interviewers asked the percentage format of all respondents. Table 6 summarizes responses to the four questions about change in business profit. The table shows responses for the treatment group (those who received loans at least one year ago) and the control group (those approved for loans who had not yet received the loan). The difference between the treatment and the control group is the impact of the loan. For each measure, the table shows the number of observations (N) for both the treatment and control groups and the value of the profit measure. The “significance” column shows the statistical significance of any observed differences between the treatment and control groups. 38 The approach, which asks for retrospective data on pre-loan profits, is not ideal but was the best approach available given the absence of a pre-project baseline. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 31 TABLE 6. REPORTED CHANGE IN PROFITS Treatment Control Measures of profit change N Value N Value Significancea Change in profits (% of respondents) % % Increased 114 88% 63 62% *** No change 4 3% 1 1% Decreased 3 2% 0 0% * Don't know 8 6% 38 37% *** Average increase in profit (Rs./month) Rs. Rs. All business types 95 12,782 39 10,153 Manufacturing 26 16,192 16 12,312 Retail trading 45 9,367 20 5,575 ** Services 3 3,000 0 0 Restaurant or food 1 2,000 1 150,000 Other 10 6,900 1 10,000 Average approximate increase in profit (Rs.) 78 9,212 49 7,298 * Average percent increase in profits (%) 64 22% 52 20% a. * = α of at least 0.10, ** = α of at least 0.05, *** = α of at least 0.01 The “change in profits” section of the table suggests that those who had received loans were more likely to have experienced an increase in business profits, slightly more likely to have experienced a decrease, and much more likely to know if they had experienced either. A majority of loan recipients were willing to answer the question asking for the actual change in business profits. Because much of the control group did not know whether profits had changed or not, they provided fewer responses to the question. The second section of the table compares stated change in business profits for the entire sample and for particular categories of businesses. Over all business types, loan recipients reported a larger average change in profits (Rs. 12,782) than the control group (Rs. 10,153). However, the difference was not statistically significant. The only statistically significant difference in change in profits was for retail businesses, the largest category of business types among Kashf clients. On average, retail business who received a loan reported a Rs. 3,792 larger increase in monthly profits than did retail businesses that had not received a loan. More respondents were willing to give interviewers an estimate of change in profit if they could report it in a range rather than an exact figure. The third section of the table reports average monthly increase in profit based on categorical responses. The average is calculated from the midpoint of each range. This produces a less reliable measure of change but has the advantage of more observations. The average change for the treatment group is larger than that for the control group by Rs. 1,914 per month. The difference is marginally significant. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 32 FIGURE 8: USE OF LOANS Overall, the survey provides limited evidence that the scaling up component of WHAM, which represents about one-third of project expenditure, increased business profits and household incomes relative to businesses that did not received a WHAM loan, especially for retail businesses. The evidence is limited for a number of reasons including: • The experimental design of the impact assessment lacked a baseline. Reconstructing a baseline from retrospective questions is not ideal. • The quality of data on reported changes in profit is not high. The design relied on respondents’ recall for pre-loan profits. Accurate recall is difficult, especially when most businesses do not keep financial records. Respondents have an incentive to report increased profits so it appears they are doing better, rather than worse, over time. On the other hand, they may underreport profits for fear of taxation. Nevertheless, the survey results do support the findings of several other impact assessments of microfinance lending in Pakistan. In particular, a recent study of impacts associated with another WHAM scaling up partner (Asasah) found weak evidence of positive impacts on business profit (Qureshi & Zaka, 2007). The survey reported an average monthly increase in profit of Rs. 7,620 although the author had little confidence in the reliability of the result. Two studies commissioned by PPAF and administered by Gallup found that microfinance lending increased personal and household incomes although the studies did not quantify the change. The survey also collected data to determine how recipients used loans and how the loan affected their business practices. Figure 12 illustrates that 78% of recipients used the loans to increase inventory of inputs or finished goods. This is consistent with the majority of recipients being engaged in trading or light manufacturing. Less than 10% went to household use or for adding machinery or space. Very little went to hiring additional workers. Table 7 provides further data on how the loan affected business practices. This evidence further supports the quantitative finding that the loans had an impact on business profits. Lower costs for inputs or inventory translate straight to a firm’s bottom line. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 33 TABLE 7. IMPACT OF LOAN ON BUSINESS PRACTICES Use of loan Frequency Percent Increased inventory of inputs or finished goods Discounts on bulk purchase 75 75% Discounts for cash purchase 22 22% Reduced production interruptions 11 11% Increased number of goods for sale 8 8% Other 1 1% Added machinery or space Increased amount produced 8 67% Increased worker productivity 3 25% Other 1 8% Increased business profits also increased household incomes. On average, respondents who experienced an increase in business profits reported using one-third for household expenses and two￾thirds for the business. Based on the estimated Rs. 3,792 per month average increase in business profits and an average transfer to the household of 33%, Kashf borrowers’ household incomes increased by an estimated Rs. 1,251 per month or Rs. 15,016 annually. This increase represents about 13% of the average annual household income in Pakistan. These results are impacts of WHAM-supported scaling up lending only if the borrowers would not have had access to the loans without WHAM. Given the competitive microfinance market in Lahore, it seems likely that borrowers would have had access to other sources of credit. The survey reviewed respondents’ loan histories to determine whether the treatment group had better access to loans than the control group. Table 8 summarizes findings. TABLE 8. LOAN HISTORY Year 2004 2005 2006 2007 Number with loans 3 9 45 22 Treatment group 1.55 5.43 28.68 10.85 Control group 0.98 1.96 7.84 7.84 Average loan amount (Rs.) 11,667 31,222 39,733 48,318 Lender Kashf 0% 89% 100% 96% Bank 0% 11% 0% 0% Family 33% 0% 0% 0% Other 67% 0% 0% 4% WHAM does seem to have increased access to loans, particularly in the “missing middle” above Rs. 30,000. Access to credit increased for both treatment (borrowers with a history with Kashf) and control groups (new Kashf clients) but at a much faster rate for the treatment group. Furthermore, the average loan size increased quickly to above the Rs. 30,000 lower threshold of the “missing middle.” WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 34 Table 9 summarizes the potential impact of WHAM-supported scaling up. Estimates are based on an average annual increase in business profits to retail businesses of Rs. 15,016, an estimated 56% of clients in retail business, a 33% diversion of profits to the household, and the pace of growth in WHAM scaling up loans from WHAM annual reports. TABLE 9. ESTIMATED CURRENT IMPACTS 2004 2005 2006 2007 Total Number of upscaling clients 0 73 2,580 35,504 35,504 Number of clients in retail 0 41 1,445 19,882 19,882 Number attributable to WHAM 0 41 1,445 19,882 19,882 Annual impacts Business profits ($ 1,000) $0 $7 $242 $3,334 $3,583 Household income ($ 1,000) $0 $3 $119 $1,642 $1,765 Based on these findings, the project may have generated $1.8 million in increased household income and $3.6 million in increased business profits retained in the business. The estimate is limited in many respects including: • It is based on an experimental design that did not include a baseline. • It includes only benefits to retail businesses that are clients of scaling up microfinance institutions and does not include other types of businesses, the downscaling portion of WHAM, or industry strengthening activities. The downscaling portion represents many fewer borrowers but, since loan sizes are larger, the potential impacts per loan are larger. • It does not include non-monetary or non-quantifiable benefits associated with loans. • Attribution to WHAM is difficult to determine. Kashf, for instance, was starting to lend above the Rs. 30,000 threshold before WHAM began. It is likely that WHAM clients would have obtained access to loans without WHAM but it would likely have taken longer. Projected (Future) Monetary Impacts Projecting benefits into the future requires assumptions about the trajectory of loan activity with and without the project. Given the competitive microfinance market in Pakistan and the fact that one partner, Kashf, was already beginning lending in the missing middle range, it is certain that lenders would have begun serving the missing middle even without the project. However, the project’s work with the partner organizations probably got them involved in the market segment more quickly and also affected the rate at which they disbursed loans in the missing middle range.39 The projected future impact of the project is the difference (annually) between benefits associated with the trajectory of lending activity with the project and the trajectory without the project. The analysis uses the partner organizations’ projections of growth in missing middle loans as the “with project” scenario. Gregory Chen, the Country Representative for Shorebank International in Pakistan, believed the partner organizations’ projections were realistic. He also believed that the WHAM project probably shifted the growth in lending up by about two years and increased the rate of growth of loans by about 50%. Figure 13 illustrates “with project” and “without project” benefits based on the following assumptions: • Partner organizations’ projections of loan activity through 2012, 39 Discussion with Gregory Chen, Country Representative, Shorebank International. More dynamic loan activity is also tied to the gradual entry of microfinance banks into the “missing middle”, which is spurring keener competition in this market segment WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 35 • A two-year rightward shift of lending projections without the project, • A 50% reduction in the growth rate of loans without the project, and • Estimated annual impacts of the loan on household income of Rs. 5,000 applied to the 56% of loan recipients engaged in retail trade. The figure does not show the estimated Rs. 10,000 average annual increase in retained business profits as a project benefit The vertical axis of Figure 13 represents aggregate benefits in each year. The impact of the project is the present value of the area between the two lines. The present value calculation accounts for different values of money over time by using a discount rate (10%) to adjust costs and benefits that occur at different points in time to a common year. In this illustration, which represents the best guess scenario, the present value of the impact of the scaling up portion of the project is $61.8 million in 2012. FIGURE 9: PROJECTED PROJECT IMPACT $0 $20,000 $40,000 $60,000 $80,000 2004 2005 2006 2007 2008 2009 2010 2011 2012 ($ 1,000) With project Without project PV = $61.8 million The estimated $61.8 million impact depends on the assumptions that lending activity would be lagged by two-years without the project and its growth would be 50% slower. These assumptions represent best informed guesses but are far from certain. Table 10 illustrates the impact of alternative assumptions on the estimated present value of impacts. The “year of lag” is the number of years the “without project” trajectory lags the “with project” trajectory. The reduction in rate of growth is the “without project” rate of growth in loans as a percent of the “with project” rate of growth. TABLE 10. SENSITIVITY OF PROJECT IMPACTS TO PROJECTION ASSUMPTIONS Reduction in rate of growth (%) Years of lag 25% 50% 75% 1 $63.20 $51.50 $39.90 2 $68.30 $61.80 $55.30 3 $71.70 $68.60 $65.40 Even with the most pessimistic assumptions (i.e., that the project only speeds up the process by one year and that “without project” rates of growth are 75% of “with project” rates), benefits of the project are still 64% of the best guess scenario. The most optimistic assumptions yield benefits of 116% of the best guess scenario. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 36 2. Findings – Non-monetary Impacts WHAM collected no beneficiary level baseline data and tracked almost no indicators related to outcomes and impact, other than loan volumes, values and quality, which are thoroughly reviewed earlier in this section and in the discussion on project effectiveness. As a result, it is difficult to present much in the way of evidence or analysis related to project impact beyond loan activity. Nonetheless, a couple of points are important to note. First, assistance to Asasah yielded important benefits to women clients beyond the provision of loans, as the entity offers an outlet for women to place their savings. As of June 2007, almost 70% of Asasah’s beneficiaries, that is approximately 27,000 productive and SBF loan clients, had entrusted Asasah with their savings, for an accumulated total of $80 thousand. Admittedly, the average savings per client is very small, but it is nonetheless important, as poor women slowly learn the advantages inherent in placing deposits in a financial entity. Second, support to Asasah also contributed to increased empowerment of women borrowers. In group interviews conducted with women borrowers of Asasah, virtually all interviewees noted that the loans they received had enabled them make expenditure decisions on their own to the benefit of their children and the family at large. In addition, Asasah has made efforts to tailor products that better fit the particular needs of women. For example, it introduced an educational loan facility that enables women and/or their children to attend schools and colleges at an affordable price. 3. Conclusions Given the limitations of data and approach, it was not possible to estimate quantitatively the impact of the bank downscaling or the industry strengthening portions of the WHAM project. These portions each represented roughly one-third of project activities and expenditures. There was sufficient data on the scaling up third of the project to make informed estimates of the impact. The project almost certainly benefited the SMEs that received loans and the households of the SME owners although quantifying the impact is difficult. A survey of borrowers from one of the scaling up partner organizations (Kashf) produced some evidence that WHAM-supported lending had a positive impact on business profits. Focus groups with borrowers revealed impacts associated with empowerment of women and improved education of family members. These conclusions are generally consistent with other impact assessments of microfinance in Pakistan. On average, survey respondents reported diverting about one-third of additional profits to their households, implying WHAM-supported lending increased household incomes. D. Efficiency The purpose of this section is to determine the efficiency of WHAM in delivering benefits. There are a number of possible measures of project efficiency. They have in common a comparison of costs to some measure of output. This section presents measures of economic efficiency (i.e., net present value, benefit cost ratio, and internal rate of return) and financial efficiency (i.e., expenditure ratios). 1. Findings Table 11 documents current and projected (future) project costs and benefits and presents three common measures of economic efficiency: net present value (NPV), the benefit cost ratio, and the internal rate of return (IRR) associated with the project. The cost and benefit measures presented in the table consider only the measurable monetary benefits of the scaling up component to retail businesses. The reported efficiency measures thus apply only to that portion of the project. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 37 TABLE 11. COSTS AND BENEFITS OF UPSCALING COMPONENT BY YEAR Costs ($1,000) Benefits ($1,000) Year Actual Discounted Actual Discounted 2004 $0 $0 $0 $0 2005 $531 $642 $3 $4 2006 $612 $673 $120 $132 2007 $524 $524 $1,655 $1,655 2008 $0 $0 $5,566 $5,060 2009 $0 $0 $11,061 $9,141 2010 $0 $0 $16,879 $12,681 2011 $0 $0 $22,908 $15,647 2012 $0 $0 $28,124 $17,463 Totals (2004-2007) $1,839 $1,792 Totals (2004-2012) $1,839 $61,784 As of end of project Projected for 5 years beyond project end Net Present Value -$48 $59,944 Benefit Cost Ratio 1.0 33.6 Internal Rate of Return 2% 57% Note: Efficiency measures for projected benefits consider only costs associated with USAID support of WHAM. Consideration of the full social costs of microfinance lending were beyond the scope of the evaluation. The present values in Table 11 are based on a 10% discount rate that adjusts for the different value of money over time. Net Present Value (NPV) – The project’s net present value is the difference between the present value of benefits and the present value of costs. A positive value indicates that a project has returned benefits in excess of costs. Since the measure is independent of the size of a project’s budget, however, it provides no evidence of the return on the project investment. For example, the returns implied by a NPV of $1 million are very different for a $5 million project than for a $20 million project. Benefit Cost Ratio – The benefit cost ratio is the present value of benefits divided by the present value of costs. A benefit cost ratio of one implies that a project returned one dollar for each dollar expended. A benefit cost ratio provides a rough measure of economic returns because it represents the proportional return on investment. For example, a benefit cost ratio of 1.5 implies that project benefits are 150% of costs. It is not, however, a rigorous measure of returns as it does not account for the timing of costs and benefits. Internal Rate of Return (IRR) – The IRR is the rate of return on a stream of costs that equates the present value of costs with the present value of benefits. It thus represents the financial return on the investment. The IRR, however, provides no information on the distribution of benefits. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 38 The efficiency measures presented in Table 11 reflect a very narrow view of efficiency. They consider only the costs incurred by USAID to support WHAM and exclude the broader social and other costs associated with supporting the partner organizations. These include operational costs, subsidies, and support from other donors. Consideration of a broader range of costs exceeded the scope of the evaluation. Because the measures of efficiency are not complete, they are not entirely comparable with estimates from the other evaluations which consider a more complete range of social costs. Based on a very partial accounting of project costs and benefits that (a) does not include the many non-quantifiable benefits; (b) includes only a limited accounting of monetary benefits; and (c) excludes many relevant costs, the scaling up portion of the WHAM project generated household-level benefits about equal to costs during the life of the project. Through 2012, project benefits are projected to exceed costs by a substantial margin. Table 12 summarizes selected measures of project performance that will facilitate comparison across projects on a number of dimensions. TABLE 12. SUMMARY OF EFFICIENCY MEASURES FOR SCALING UP COMPONENT AT PROJECT COMPLETION Measure Value Present value of costs $1.8 million Present value of benefits $1.8 million Net present value $0 Internal rate of return 2% Benefit cost ratio 1.0 Annual cost per beneficiary $84 Number of direct beneficiaries 19,882 retail SMEs/households Average annual benefit/beneficiary $83 Benefit as % of average annual income 4% Note: The table refers only to costs and benefits of the scaling up component of WHAM and the benefits to the 56% of loan recipients that are retail businesses. Another way to assess efficiency is how costs are distributed across different cost categories. Figure 10 summarizes project expenditures by category. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 39 FIGURE 10: DISPOSITION OF EXPENDITURES The project spent 44% of its budget on administration but this is not be unusual for a project that focuses on capacity building and training. The project also spent a lot on labor but it was, by its nature, a TA dominated project. Figure 11 presents some basic expenditure ratios. FIGURE 11. SELECTED EXPENDITURE RATIOS 2. Conclusions The project appears to have been reasonably efficient although the measures of benefits are not refined enough to make a firm determination. $0.2 $0.5 $2.1 $2.2 $5.0 $0.0 $1.0 $2.0 $3.0 $4.0 $5.0 $6.0 Outputs and activities ODCs Labor Administrative Total $, millions 43% 54% 4% 0% 10% 20% 30% 40% 50% 60% Labor as % of total Administrative & ODCs as % of total Outputs and activities as % of total Percent of expenses WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 40 E. Sustainability The purpose of this section is to determine whether project activities and results are likely to be sustained after the project is completed. WHAM is a comprehensive project that is wide in scope, with objectives that are addressed in a three￾year timeframe. Its contractor is reputed to have an excellent grasp on micro and SME finance best practices, and on the benefits of merging the financial bottom line objective, i.e., profit-making by financial entities, with social development. 1. Findings Principal findings as regards sustainability of activities and results for the three components are detailed below: • In mid 2007, at the request of Senior Management of NBP, “WHAM drafted a proposal to roll out SME lending to 600 branches of NBP across Pakistan [with the] goal of making SME lending a profitable line of business by achieving economies of scale.”40 The plan was presented and discussed with NBP senior management in the last quarter of 2007. If implemented this plan has the potential to greatly enhance the availability and sustainability of SME lending in Pakistan • NBP has not institutionalized policies and procedures that are necessary to sustain the autonomous development of an SME Unit. They include, inter alia, the design of a long-term staff development strategy, systematized training, the introduction of incentive-based pay, and the establishment of management information systems. These policies and procedures address important constraints to sustainability – e.g., WHAM trainers acknowledged that the NBP loan officers they train are reluctant to embrace change and are very poorly motivated. • The Head of the SME Development Unit of NBP revealed that prior to technical assistance of WHAM, operational sustainability of SME lending was 40% i.e., revenues covered only 40% of operation costs. Subsequently, operational sustainability has risen to 60%. • WHAM chose scaling up partners based on their basic orientation towards sustainability. As noted in the last section, NRSP is seriously contemplating legal transformation into a microfinance bank. Since its inception in 2002, FMFB has operated at break-even. Tameer Microfinance Bank, Kashf Foundation were designed to price their services on cost-recovery and profit-making bases. The CEO of Asasah was schooled via prior experience in Kashf Foundation. Prior to signing technical agreements with WHAM, these three entities charged effective loan rates that were 75% to 100% above the average among other microfinance providers. Officials of Tameer, Kashf and Asasah underscored that the technical assistance and training inputs provided by WHAM were underpinned by the principles of self￾sufficiency and long-term sustainability. • The Asasah SBF loan product that has been developed with assistance of WHAM represented less than 5% of total portfolio by June 2007. Despite improvements in the operational sustainability of SBF (see above), the overall operational sustainability of Asasah declined slightly from 54% in September 2006 to 51% in June 2007. In the last two quarters of 2007, WHAM scaled back its assistance because this entity “encountered substantial slowdowns due a lack of funding.”41 To address this constraint, WHAM assisted Asasah in the preparation and presentation of a proposal to access lines of credit from commercial banks. At the moment of drafting this report (early 2008), accessing commercial credit had not materialized. • WHAM assisted the State Bank of Pakistan in the preparation and presentation to the Prime Minister of a paper on the microfinance industry. The paper identifies constraints on operations of microfinance providers, on accessing commercial lines of credit, and on 40 Quarterly Report July – September 2007 41 Quarterly Report October – December 2007. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 41 building a human resource base. It provides recommendations for policy actions in these areas that are conducive to long-term industry sustainability. • As noted in the previous section, WHAM has strengthened the institutional presence of the Pakistan Microfinance Network among industry stakeholders, as 95% of microfinance providers submit indicators on financial performance to this entity. The PMN subsequently makes the data available to the public at large. The PMN, however, has not reached financial sustainability, as acknowledged by its top official. At present, and notwithstanding organizational improvements, the PMN still relies on non-USAID donor funding. • The Head Unit of the SME Unit of Cresbank revealed that a pilot testing of its SME loan products will be launched from the bank’s consumer finance platform in 2008. The pilot will be conducted on basis of the business strategy developed by WHAM in early to mid 2007. According to this bank official and the Chief of Party of WHAM, the delay is attributed to changes in ownership and top management of the bank. The Chief of Party of WHAM stated that no further involvement from the project is warranted. • NBP charges on loans one percentage point less than private banks. The cheaper loan rate is partly explained by the fact that NBP is recipient of large government deposits, which gives way to the benefits of zero or low cost of funds. 2. Conclusions First, with respect to the downscaling component, it is not possible to unequivocally ascertain whether or not the outcomes and results pursued by WHAM will be sustainable in the long run. For one, with respect to Cresbank, WHAM did everything except launch the SME loan pilot program. This decision is the responsibility of bank management. The Head of the SME Unit is a capable manager who definitively champions the introduction of the product probably, in his view, in 2008. If and when this happens cannot be precisely ascertained because of factors that he cannot control. With respect to NBP, it is fair to conclude that management request to study a plan by which the pilot program can be extended from 64 to 600 branches denotes, at least, increasing interest in the benefits of establishing a SME lending platform. There is no doubt that the chance for long-term sustainability of SME lending at NBP will be significantly enhanced if this plan comes to pass. It is a concern, however, that the bank has failed to yet institutionalize systems for SME lending; this suggests that long term management commitment to SME lending that has yet to fully materialize. Lastly, because it enjoys zero or low cost of funds, NBP can offer lower loan rates than potential competitors. As noted before, private commercial banks have not shown much enthusiasm for downscaling under present conditions in the financial market. Therefore, the pricing policy of NBP is not leading to the “crowding out” of private operators. These conditions however could change in the future if NBP loan rates are kept artificially low. If this scenario comes to pass, the benefits of the technical assistance to NBP that can be derived to other banks through the impact of [positive] “demonstration effect,” will be substantially limited. Second, as regards technical assistance delivered to scaling up organizations, it is fair to conclude that, given the content of the technical assistance and training that were delivered, as well as the principles and practices that underline the historical trajectory of the contractor in the field of micro and SME finance, no activities were implemented that led to increased donor dependence. If and when NRSP transforms legally into a microfinance bank, it will be possible to assert that WHAM contributed to the long-term sustainability of this large institution -- it will not be possible, however, to be at all specific in characterizing the extent of WHAM’s contribution. Kashf Foundation and FMFB were already operationally sustainable prior to the signing of their respective technical agreements with WHAM. It is reasonable to conclude that project assistance bolstered further their sustainability. Both Tameer Microfinance Bank and Asasah were not operationally sustainable prior to WHAM intervention and, as of early 2008, still were not. The scope and content of the technical assistance delivered to the former (overhaul of the credit process) and the approval it drew from top management suggests that WHAM inputs support improvements in long-term sustainability. The picture is different with respect to Asasah, the only scaling up organization with which WHAM partnered to launch a loan activity. The entity, unfortunately, is under stress, due to lack of external WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 42 funding, and this situation has forced WHAM to scale back loan activity support. It must be noted that, prior to the signing of the first technical assistance agreement in October 2005, WHAM had already identified lack of access to funding as an important risk and key constraint. The project nevertheless took the risk and, for over two years, devoted time and labor intensive resources to support this entity. WHAM has correctly and sensibly tried to assist Asasah to access commercial lines of credit. In the world of credit-granting NGOs however, success is never guaranteed in the short run. A new board of directors has been created and Asasah is currently undergoing an internal process of management restructuring. WHAM involvement will slow down even more in the months ahead. Thus, there is a risk that the investment in Asasah will come to naught. Third, the effectiveness of WHAM and the sustainability of its impact depend to a very large degree on the relationships it is able to build with both the financial organizations it aims to assist, and with other stakeholders in the financial sector. Building such relationships – to the point sufficient mutual trust and confidence exist – requires substantial levels of resources and energy, and even more importantly perhaps, a long period of time. The timeframe of WHAM was just three years. This is likely an insufficient period of time to establish and expand the depth of relationships necessary to achieve broad sustainable project results. Fourth, by lending support to the State Bank of Pakistan and the Pakistan Microfinance Network, WHAM has contributed to strengthening the microfinance industry and its prospects for long-term sustainability. 3. Recommendations First, to the extent that it decides to stay actively engaged in projects that simultaneously attempt to build institutional capacities of micro and small enterprise finance providers, and execute activities directed at strengthening the industry, USAID should implement them within a timeframe of no less than five years. A three-year execution timeline is just too short for projects which pursue these very ambitious objectives to achieve sustainable outcomes and results. Second, if it continues to focus on finance projects that seek to build institutional capacities of micro and small enterprise finance providers, USAID should explicitly define indicators and targets that underscore measurable performance towards operational and financial sustainability. Third, in case it decides to continue spending resources for capacity building of credit-granting NGOs, USAID should monitor closely their selection and exercise veto on those crippled by weak internal management structures and inability to access external funds to sustain high rates of growth. Fourth, and also with respect to eventual support to credit-granting NGOs, USAID should require the contractor to devote time, energy and resources to promote financial linkages between the NGOs and commercial banks. In this way, NGOs will be less dependent on donor funding and become used to accessing loan capital at commercial rates. Fifth, contingent to further involvement with projects that support credit-granting NGOs, USAID should emphasize assistance to microfinance providers that have the potential to transform into deposit-taking institutions, as they usually exhibit good and sustainable growth prospects. Sixth, in case it decides to consider more projects in finance, USAID should impose strict criteria and limited resources for the extension of financial assistance to commercial banks, as these entities have financial wherewithal. When commercial banks are enlisted, technical assistance and training commitments should not be open ended. USAID should make resources available contingent to the bank’s meeting in-progress targets not of loan activity, but of institutional transformation and operational sustainability. USAID should relax technical assistance criteria and resources in the case of microfinance banks because they have a far smaller capital base. Seventh, in the eventual case of further involvement, USAID should promote further donor assistance to the Pakistan Microfinance Network until it reaches financial sustainability. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 43 F. Replication 1. Findings The conclusions presented below are drawn from information and evidence presented in earlier portions of the report, most significantly from the sustainability and effectiveness narratives. 2. Conclusions First, results from eventual project replication would be very uneven: more activities in industry strengthening and scaling up are likely to yield positive results. Based on the lessons learned from the current experience, additional activities focused on downscaling are likely to be largely ineffectual, at least within the next three to five years.42 Second, the microfinance industry of Pakistan is still young and with limited outreach. Scaling up loan activity can likely be replicated in most areas of the country. Exceptions would include areas of very low population density and abject poverty. Scaling up projects will also fail in areas suffering from entrenched insecurity, such as parts of Baluchistan and, most especially, FATA. In these regions, the current violence and general insecurity will not permit effective financial outreach. V. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ON THE CROSS CUTTING QUESTIONS A. Gender This section explores the extent of which WHAM benefited women. 1. Findings • Increasing access to finance for women was never an explicit part of the project. It was not part of the request for assistance from USAID, nor was it contemplated in project design. A review of project documents revealed no analysis pointing to the obstacles that prevent more extensive financial outreach to women. • Notwithstanding the preceding finding, WHAM provided assistance to two partner organizations that are focused on extending financial services to low-income women – Kashf and Asasah. Technical assistance and training delivered to Asasah resulted in the direct generation of loans to 1,006 women. Asasah borrowers are mainly engaged in trade and commerce and, to a lesser extent, in manufacturing and processing. • As of December 2007, a total of 2,179 women had received loans as a result of WHAM technical assistance and training activities to partner organizations. Of this total, only 6 corresponded to the downscaling component. The rest, 2,173, were extended loans by scaling up microfinance providers. • WHAM undertook a study on micro savings. As demonstrated by the assistance delivered to Asasah, low-income women do have the capacity to save. Asasah offers its women clients an outlet for their savings. As of June 2007, almost 70% of its beneficiaries had entrusted Asasah with their savings, for an accumulated total of $80 thousand. • WHAM provided training on best practices of individual lending methodologies to 114 women finance professionals. 2. Conclusions First, WHAM contributed to expanding, albeit modestly, the number of loans extended to women. Of all the loans extended women by providers supported by WHAM, 99.7% were provided by scaling up providers. 42 The Chief of Party of WHAM is guardedly optimistic. He believes that the market for downscaling could ripen in three years. The large majority of stakeholders that were inquired on this matter believe that it will take longer than that. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 44 Second, WHAM was intensively engaged in building the institutional capacity of Asasah, an entity that is managed and staffed by women, and that focuses on the delivery of financial services to women. Third, given the relatively low outreach indicators with respect to the female borrowers that currently prevail in Pakistan, it could be argued that much more could have been accomplished by the project. Raising such an expectation, however, would ignore one of the key lessons learned by WHAM, which is that very few women occupy the targeted loan audience of WHAM (the missing middle). Asasah would appear to contradict this, but closer examination even of these loans would show that the female borrower aspect is largely for show. The businesses financed by Asasah’s SBF product are predominantly owned and managed by men. Fourth, the project missed an opportunity by not undertaking a research project focused on identifying and investigating the factors that impede a higher volume of financial services addressed to women. 3. Recommendations First, if USAID continues to focus on finance projects, it should commission a study on the opportunities and constraints that underline potential interventions that target low-income women. Second, contingent on its further involvement in finance projects, and following completion of the above mentioned study, USAID should organize a high-level seminar with the participation of key stakeholders to highlight findings, conclusions and recommendations of the study. Third, in the case it decides to continue its support to finance interventions, USAID should facilitate access to lessons learned from other countries that have enjoyed high levels of success in pushing financial outreach to women. These efforts could take many forms, including, teaching seminars led by visiting experts, study tours, etc. B. Reporting Review of project documents reveals the following: 1. Findings • WHAM fulfilled reporting requirements • Branding guidelines were followed • Reported results were accurate and verified with information provided by partner organizations. 2. Conclusions First, the WHAM implementing partner, Shorebank, was effective in meeting the reporting requirements of the project. Project reports included information useful to USAID managers and were prepared and delivered in a timely fashion. C. Communications and Outreach This section addresses the effectiveness of WHAM in getting its story out. 1. Findings Review of project documents reveals the following findings: • The project’s work plans did contain communications and outreach activities. • The project implemented activities that focused on microfinance awareness building. For example, it designed a poster depicting ten years of microfinance of Pakistan that has been widely and effectively disseminated across the country. More high-impact products, such as the publication “Microfinance in Pakistan 1999-2005: Growth, but a Structural Flaw Persists”, and the most recent Micro Watch Bulletin, have also reached large audiences among stakeholders in the private and public sector. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 45 • Key information generated by the project was channeled through the Pakistan Microfinance Network. • Success stories of the impact of project-generated loans on low-income borrowers were drafted and highlighted in friendly-reading material, and disseminated across the country. • Branding guidelines were appropriately followed. 2. Conclusions First, WHAM was very effective in “getting its story out”, and raised, in this way, awareness among stakeholders of the objectives, content, and benefits brought about by the project. Second, channeling key information through the PMN was relevant, as it directly contributes to elevating the profile of such an important local institution. Indirectly, such a decision critically underlined the execution of an adequate exit strategy that is largely beneficial for the microfinance industry.43 D. Coordination This section addresses the effectiveness of WHAM in coordinating its activities with other parties. 1. Findings • WHAM coordinated efforts with the State Bank of Pakistan, the most important stakeholder in the public sector. These efforts were praised by the Director of Microfinance of the State Bank of Pakistan. • Outside the government, WHAM coordinated efforts with the Pakistan Microfinance Network. Its top official praised the assistance delivered by WHAM. • WHAM reached out to other important non-government stakeholders, such as the Lahore University Management Science to develop high-impact research projects. • WHAM enlisted the contribution of SDC to boost the technical assistance package offered to Asasah. It also collaborated jointly with SDC and DFID to sponsor and publish a research paper on Lahore’s dynamics of microfinance expansion. • The Chief of Party of WHAM regularly attends coordination meetings. For the last year, WHAM acted as the Secretariat for the Informal Donor Group to facilitate USAID’s role as chair. • While on occasions WHAM directly interfaced with PPAF and Khushhali Bank, there is no evidence that WHAM coordinated efforts with the USAID-funded microfinance projects that these entities implement. 2. Conclusions WHAM has been very effective in coordinating its efforts with stakeholders of the microfinance industry. 43 WHAM will close operations in March 2008, but the PMN will stay. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 46 VI. LESSONS LEARNED The major lessons that evolve from the WHAM project are the following: First, project design should be carefully executed. Prospects of downscaling should have been assessed properly. The project defined a too wide loan range in the “missing middle” and assumed that interest from commercial banks and microfinance providers alike was there. Second, the type of technical assistance provided under WHAM would have been more effectively delivered in a five year project, as it requires fostering trust and establishing partnerships before the assistance is actually delivered. Third, projects that offer technical assistance and training untied to the supply of loan for capital or credit guarantees can be effective. Fourth, activities that support industry strengthening require small investments but have the potential to yield substantial and lasting returns. Fifth, plan for impact assessment of future activities. Quantitative impact assessment of microfinance is particularly difficult. Planning for impact assessment from the beginning of an activity will greatly enhance prospects for credible impact assessment of future activities. In particular, activities should establish baselines soon after inception, plan to follow up on the baselines, and collect the quantitative and qualitative data necessary for a comparison of activity outcomes with outcomes if the activity had not taken place. The lack of baseline information and reliable implementation data severely hampered the present evaluation effort. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 47 ANNEX 2: BIBLIOGRAPHY Asasah. Quarterly Progress Report April – June 2007. Bano Burki Hussan and Mehr Shah. 2007. The Dynamics of Microfinance Expansion in Lahore. Pakistan Microfinance Network and Shorebank International Ltd. Islamabad, Pakistan. BWTP. Asia Resource Center for Microfinance. Undated. Pakistan Country Profile. Off the Internet. CGAP. 2007. Country-Level Effectiveness and Accountability Review with a Policy Diagnostic. Pakistan. CGAP. 2006. Microfinance Consensus Guidelines. Hussein, M., & Hussain, S. 2003. The Impact of Micro Finance on Poverty and Gender Equity: Approaches and Evidence from Pakistan. Pakistan Micro Finance Network. The Independent Evaluation Group. 2006. Pakistan: An Evaluation of the World Bank’s Assistance. The World Bank, Washington DC Ledgerwood, Joanna. 1999. Sustainable Banking for the Poor. Microfinance Handbook. An Institutional and Financial Perspective. The World Bank, Washington DC. Micro Watch. Issues 3 (January – March 2007) and 4 (April – June 2007). A Quarterly Update on Microfinance Outreach in Pakistan. Pakistan Microfinance Network and Shorebank International Ltd. Islamabad, Pakistan. Montgomery, H. 2005. Meeting the double bottom line: the impact of Khushhali Bank's microfinance program in Pakistan. Tokyo, Japan: Asian Development Bank Institute Publishing. Pakistan Microfinance Review. Shades of Growth. 2006. Pakistan Microfinance Network. Islamabad, Pakistan Qureshi, U., & Zaka, S. 2007. Survival or take-off: a study of MSEs in the Lahore small business finance markets. Shorebank Tameer Microfinance Bank Limited. 2006. Annual Report. USAID. 2006. Microfinance Performance in Pakistan. Islamabad, Pakistan. Valenzuela, Liza. 2001. Getting the Recipe Right: The Experiences and Challenges of Commercial Bank Downscalers. Microfinance Best Practices. Development Alternatives Inc. Bethesda, Md. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 48 ANNEX 3: LIST OF INDIVIDUALS INTERVIEWED Key Informants Aban Haq, Research Analyst. Pakistan Microfinance Network. Ahmad Jamal, Chief Strategic Planning Officer. Pakistan Poverty Alleviation Fund. Ghazanfar Azzam, CEO Designate for the Microfinance Bank Gregory C. Chen, Country Representative, Pakistan. Shorebank International Haroon Sharif. Senior Advisor Private Sector Development. Department of International Development Iftikhar Qureshi, Consultant Management. Asasah. Kamran Amin, Head of SME Division, National Bank of Pakistan Khalid Nawaz. Project Director. Financial Sector Strengthening Programme. Mohammed Azim Hashimi, Investment Officer. Pakistan Resident Mission. Asian Development Bank. Mueen Batlay, Executive Vice President. Crescent Commercial Bank Ltd. Nadeem Hussain, President & CEO. Tameer Microfinance Bank Limited. Qasim Nawaz, Director, Microfinance Department. State Bank of Pakistan Rashid Bajwa, CEO. National Rural Support Programme. Syed Mohsin Ahmed, General Manager, Pakistan Microfinance Network Tabinda Alkanz Jaffery, Chief Executive Officer. Asasah. Tariq Khan Baluch, General Manager. Credit & Enterprise Development. Pakistan Poverty Alleviation Fund. Uzma Qureshi, Consultant, Shorebank International Zach Orend, United States Agency for International Development. Other. Dr. Abid Hameed, Economist. State Bank of Pakistan. Asghar Memon. Manager. Credit & Enterprise Development. Pakistan Poverty Alleviation Fund. Aqeel Ahmed Zeeshan Malik, Senior Analyst. Shorebank International Bilal Ahmad, Credit Analyst. Shorebank International Hussan Bano Burki, Consultant. Shorebank International Jasim Sheik, Credit Analyst. Shorebank International Kirsten Weiss, Senior Consultant. Shorebank International M. Sualeh Faruqi, Head of HRD & Adm. Network Leasing Corporation Limited. Muhammad Farooq, Senior Finance Executive. Network Leasing Corporation Limited. Syed Ali Shabbar Naqvi. Manager, Shahalam Market Branch, Lahore. National Bank of Pakistan. Umeed Ayad Method, Vice President. National Bank of Pakistan. Zulqarnain Rafiq, Sr. Management Executive, Credit & Enterprise Development. Pakistan Poverty Alleviation Fund. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 49 ANNEX 4: GETTING TO ANSWER MATRIX APPLIED TO EVALUATION QUESTIONS OF THE WHAM PROJECT Evaluation Questions Type of Answer or Evidence Needed Method of Data Collection Data Source Selection Criteria Data Analysis Methods Have project activities directed to partner organizations increased their capacity? Work with project experts to determine appropriate questions to assess capacity building. How many loans have been disbursed in each quarter since WHAM started operations? (by region, lender, loan size, and gender) Comparison with number of loans without WHAM Document review Project documents review Describe trajectory of loan disbursements since WHAM began operations. Break down by province/district, lender, loan size, and gender. What are projections for loan disbursements in the future? (by region, lender, loan size, and gender) Interviews Key informants interview Partner organization interview Project staff interview Project number of loans by province/district, lender, loan size, and gender. How many loans would have been disbursed without WHAM intervention over the life of the project and into the future? (by region, lender, loan size, and gender) Comparison with number of loans disbursed with WHAM Interviews Document review Surveys Key informants interview Review of historic records of partner organizations Potential survey of financial institutions not partnered with WHAM Project trend in lending from pre-WHAM period. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 50 Evaluation Questions Type of Answer or Evidence Needed Method of Data Collection Data Source Selection Criteria Data Analysis Methods What would business income have been without the loans? (by region, business type, gender) Serves as control for estimating loan impacts Document review Surveys Review of business statistics if timely and Applicable Control group businesses survey Loan recipient surveys If control group surveyed, then stratify by region and business type. If from loan recipient survey, then stratify survey of loan participants by loan size and by region (and by lender?) Given the number of loans and nationwide scope, a clustered sample may be necessary. Need detail on location of loans for sampling. Estimate business income without WHAM support. Determine appropriate control based on available data and feasibility of control group survey. What impact have loans had on the income of recipient businesses? (by region and gender) Comparison of business income with and without loans. Document review Surveys Project documents (including recent survey in Lahore) Loan recipient surveys Stratify survey of loan participants by loan size and by region (and by lender?) Given the number of loans and nationwide scope, a clustered sample may be necessary. Need detail on location of loans for sampling. Stream of with and without impacts over time. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 51 Evaluation Questions Type of Answer or Evidence Needed Method of Data Collection Data Source Selection Criteria Data Analysis Methods How were the loans used and what impact did they have on the business? Descriptive Surveys Loan recipient survey Stratify survey of loan participants by loan size and by region (and by lender?) Given the number of loans and nationwide scope, a clustered sample may be necessary. Need detail on location of loans for sampling. Narrative descriptions of impacts of loans on the business How, if at all, have the benefits of the loan to the business affected the income of the owner and the welfare of the owner’s household? (by region, loan size, and gender) Descriptive Quantitative Surveys Loan recipient survey Stratify survey of loan participants by loan size and by region (and by lender?) Given the number of loans and nationwide scope, a clustered sample may be necessary. Need detail on location of loans for sampling. Narrative descriptions of impacts of loans on the business Estimates of current and lasting impacts on household income. Have the loans generated any secondary impacts as a result of increased business activity? (by region, loan amount, and gender) Surveys Existing model or multipliers Loan recipient survey Stratify survey of loan participants by loan size and by region (and by lender?) Given the number of loans and nationwide scope, a clustered sample may be necessary. Need detail on location of loans for Existing multipliers applied to estimates of additional employment or income for recipient businesses. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 52 Evaluation Questions Type of Answer or Evidence Needed Method of Data Collection Data Source Selection Criteria Data Analysis Methods sampling. Have the trainings and media campaigns been effective? Comparison Document review Market demand studies and research Change in demand for new loan products What is the partner bank and NGO’s perception of the value of ShoreBank’s assistance? Descriptive Interviews Interviews with partner organizations Narrative with content analysis of common perceptions of value Has the project been effective in changing the lending practices of the banks and NGOs? Comparison Document review Records of partner organizations Trend in number of loans in target range. Has there been a change with the project? What constraints, if any, do you face in offering new loans in the target range? Narrative Interviews Interviews with partner organizations Narrative with content analysis of common constraints Under what conditions would you be willing to make more loans in the target loan range? Narrative Interviews Interviews with partner organizations Narrative with content analysis of common Conditions WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 53 ANNEX 5: SURVEYS Survey Design Experimental Design Ideally, a study of the impact of microfinance would establish a baseline for both a treatment group (businesses that would receive loans) and a control group (businesses similar to the treatment group except for not receiving loans) and then update the baseline at some point after the treatment group had received the loans. This approach would permit comparison of how loans had affected businesses that received them to, inferring from the control group, how those same businesses would have fared had they not received the loan. Implementing such an approach is fraught with difficulties. One of the most important is establishing a suitable control group. If the control group differs from the treatment group, it is possible that the differences, rather than receipt of the loan, caused observed differences between the groups. To reliably attribute impacts to receipt of the loan, the two groups must be similar in every respect except receipt of the loan. For this survey, the control group consisted of businesses that had been approved for a loan but had not yet received it. Another difficulty in this particular study is the absence of a baseline. Comparison of the treatment with the control group at one point in time provides an ex post measure of the difference between the groups but provides no information about where the two groups started. For example, if the objective is to measure change in income, the design would identify differences in income between the two groups but would not identify change attributable to the loan because it covers only one point in time and provides no evidence that the two groups had similar incomes prior to the survey. In the absence of a baseline, the experimental design had to employ the second-best approach of asking respondents to recall their profits prior to receipt of the loan. In an environment where few business keep financial records and they have valid reasons for wariness about revealing profit information, this approach is not likely to yield very precise measures of change in profits. It was, however, the best that could be accomplished under the circumstances. Questionnaire Development The questionnaire design process began with the evaluation team developing draft questionnaires, one for the treatment and one for the control group, in English. The drafts drew from questionnaires developed for two previous studies of the impacts of microfinance in Pakistan: a survey of WHAM borrowers conducted in 2006 (Qureshi & Zaka, 2007) and a survey of Khushhali Bank borrowers conducted in 2005 (Montgomery, 2005). Borrowing from existing questionnaires takes advantage of questions that have already been field tested and may produce data that is comparable across different surveys. The survey firm hired for the field work translated the English questionnaires into Urdu, the language in which it would be administered. The evaluation team and the survey firm then compared the Urdu and English questionnaires to identify and resolve any discrepancies in meaning. The survey firm then conducted 12 pretest interviews in the field with a sample of potential respondents. The evaluation team debriefed the interviewer who conducted the pretest interviews and revised the questionnaires to resolve issues identified in the pretest. Changes included simplifying a question that asked about employees, correcting skip instructions, inserting a question to ask about how the business used profits, revising the question about change in profits to accommodate some respondents’ unwillingness to provide a monetary amount, and simplifying a series of questions on household expenditures. The survey firm translated the modifications from English to Urdu and incorporated them into the questionnaires. The evaluation team reviewed the revised questionnaires again to confirm that they were accurate. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 54 Sample Selection The survey targeted clients of Kashf, one of the microfinance institutions partnering with WHAM.44 The experimental design called for a total sample of 200 consisting of 100 borrowers who had obtained WHAM loans at least one year ago and 100 businesses who had applied for, but not yet received, a loan. Kashf was able to provide a sample meeting these criteria. The final sample included 231 interviews, 129 with the treatment group (i.e., current borrowers) and 102 with the control group. The evaluation team had to rely on Kashf to select the samples using sampling instructions provided by the evaluators. Since the evaluation team could not observe the selection process, however, it is not possible to know for sure how representative the sample is. Field Procedures A team of interviewers and field supervisors conducted interviews with clients of five Kashf branches in Lahore and Gunrajwala between March 10 and March 16, 2007. A Kashf loan officer accompanied the teams to show them the location of clients but did not attend the actual interviews. Questionnaires The following sections reproduce the survey questionnaires, first the English and then the Urdu versions. The questionnaires were administered in Urdu. The English versions therefore do not reflect the formatting used in the field. 44 Originally, the survey also included clients of the National Bank of Pakistan (NBP) to represent the downscaling portion of the WHAM activity. However, in the competitive finance market in Lahore, NBP was reluctant to provide access to their clients to an outside survey research firm. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 55 SR NO USAID-FUNDED Pakistan Economic Growth Evaluation and Design Project (PEGED) TREATMENT GROUP Household Survey 2008 Management Systems International (MSI) Country Office: International Office: MSI/PEGED Office, Apt 48, Management Systems International (MSI) Mayfair Block Royal Apartments, 600 Water Street, SW Diplomatic Enclave, Islamabad Washington, D.C. 2004 Pakistan. Ph: 051-8314731, 4301365 USA www.msiworldwide.com MSI CRCP WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 56 Section A: IDENTIFICATION Fill in from information received from the bank or MFI branch A1 Lending institution Kashf NBP A3 Date of loan disbursal A4 Amount of loan (Rs.) A5 Name and Code of Interviewer: A6 Date of Interview: A7 Interviewer’s Arrival Time: a Hours [Code00 – 24] b Minutes [Code00 – 59] A8 Interviewer’s Departure Time: a Hours [code00 – 24] b Minutes [code00 – 59] Visit Interviewer’s Name Date Result Code A9 1 A10 If code is not 01, Give reasons? ……………………………………………………………………………………………………….. ……………………………………………………………………………………………………….. A11 2 A12 If code is not 01, Give reasons? ……………………………………………………………………………………………………….. ……………………………………………………………………………………………………….. Result codes: 01 Complete 02 Incomplete 03 Refused 04 Non-Availability of respondent 05 House Locked 06 House Replaced Others [Specify] ____________________ Name and code of Field Supervisor Name of Editor/Coder in Office Name and Code of Data Entry Operator A13 Name and code of editing staff [Different staff will use different color pencils for editing] WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 57 Section B: HOUSEHOLD INFORMATION a. Total no of persons [including children and adults] b. Total no of males [including male children and adults] c. Total no of females [including female children and adults B1 How many people are there in your household? By your household I mean all those who live in the same house for at least 5 days per week and who eat from the same table (share a common kitchen). d. Total number of persons before receiving loan. [including children and adults] Instruction: Check that the total number of males and females equals the total number of persons and make corrections if necessary. B2 How many male children between a. Number of male children the ages of 5 and 15 years are in your household and how many are enrolled in school? b. Number of male children enrolled in school B3 How many female children between a. Number of female children the ages of 5 and 15 years are in your household and how many are enrolled in school? b. Number of female children enrolled in school Section C: BUSINESS INFORMATION The following questions ask about the business for which you obtained the loan. on _______ [fill in date from front page]. Please answer only for that business. For what type of business did you obtain the loan? (mark one answer) C1 1. Manufacturing (making finished goods from other materials, e.g., making clothing, parts, handicrafts, jewelry, etc.) 2. Retail or trading (shops, buying and selling finished goods or products) 3. Services (renting equipment, operating schools, laundry, transportation/driver, beauty salon) 4. Restaurant or food (preparing and selling ready-to-eat food items) 5. Agriculture (growing crops) 6. Livestock (buying livestock, raising livestock) 7. Other (specify) _____________ C2 In what year did you start the business? (enter year) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 58 What is the ownership structure of the business? (mark one answer) C3 1. Sole proprietorship 2. Family partnership 3. Non-family partnership 4. Private limited company 5. Public limited company 6. Other (specify) ____________ From where do you operate the business? (mark one answer) C4 1. Out of your home 2. In a building that you own other than your home 3. In a building that you rent other than your home 4. Other (specify) _____________________ Can you tell me about the people who work in your business now? (fill in numbers for permanent and for temporary employees) Total number How many are men How many are family members How many are paid How many before you received loan Permanent employees Temporary employees C5 Instructions to interviewers Same as current questionnaire Instructions to Interviewers: A permanent employee is a paid employee that is contracted for one or more years, receives a monthly salary, and will continue working unless the employer makes a specific decision to end their employment. Temporary workers are hired for a specific period, e.g. a day, a week or even a month but it is assumed that their employment will not be continued unless the employer specifically decides to hire them for a longer period. Prompt to ensure that respondent has included all family members who may regularly work in the business whether they are paid or not. How many permanent and temporary workers have you laid off and not replaced since before you received the loan? a. Number of permanent workers C6 b. Number of temporary workers Section D: LOAN HISTORY D1 Had you ever borrowed money for this business before you received the loan? 1. Yes 2. No (Go to Æ Q no. D4 ) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 59 D2. If yes, can you please tell me about the other loans you received? Interviewer: ask questions to fill in loan history since 2004 in table below) Year Source of loan Amount borrowed (Rs.) 1 Family or friends 2 MFI 3 Kashf 4 Bank 5 NBP 6 Money lender 7 Supplier’s credit 8 Other (specify) ___________________ 2004 2005 2006 2007 If respondent received any other loans, ask: If you have borrowed money from other sources, why did you choose to borrow from Kashf this time? (mark all that apply) D3 1. Easier process, not as much paperwork 2. Able to get amount needed 3. Lower interest rate 4. Loan approved and disbursed more quickly 5. Proximity of lender 6. Friendly staff 7. Know loan officer If you had not received the current loan from Kashf, what would you have done? (mark one answer) D4 1. Used personal funds (skip to section E) 2. Borrowed the money from some other source 3. Not made the expenditures (skip to section E) 4. Other (specify) ________________ (skip to section E) Which source would you have borrowed from? (mark one answer, then skip to next section) D5 1. Family or friends 2. MFI 3. Bank 4. Money lender 5. Supplier’s credit 6. Other (specify) ____________________ Section E: USE OF THE LOAN The following questions ask about the way you used the loan you received from Kashf on ________ [fill in date from front page]. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 60 E1 What did you use the loan for? (mark all that apply) 1. Increased inventory of inputs or finished goods 2. Bought, built, or rented new machinery. 3. Repaired existing machinery 4. Bought, built, or rented more work space 5. Hired more workers 6. Increased the salaries of existing workers 7. Paid consultant services to improve my business 8. Paid off debt 9. Family education expenses 10. Family health expenses 11. Emergency expenses 12. Household expenses (food, clothes, trips, etc.) 13. Other (specify) _________________ Interviewer instruction: If respondent marked “1” in question E1, then ask: E2 If you used the loan to increase inventory of inputs or finished goods, how did it affect the way you operate your business? (write in response. Probe to learn how the loan affected the way they were able to run their business. Examples below) Code responses into these categories, DO NOT READ 1. Able to buy larger quantity and get discounts 2. Able to get lower price by paying cash 3. No longer had to go through as many middlemen 4. Larger inventory of inputs led to fewer interruptions in production 5. Larger inventory of goods for sale gave customers more choices 6. Other (specify)______________________ Interviewer instruction: If respondent market “2”, “3”, or “4” in question E1, then ask: If you used the loan to buy or rent more work space or add machinery, how did it affect your business? (write in response. Probe to learn how the loan affected the way they were able to run their business. Examples below) E3 Code responses into these categories, DO NOT READ 1. Increased the amount you could produce in the peak season 2. Reduced your cost of production 3. Increased the amount each worker can produce 4. Other (specify) _____________________ WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 61 Section F: USE OF THE LOAN Ask for all respondents How does the total profit from your business in 2007 compare to your profit in the year before you received the loan from Kashf? Is it... (read answers) F1 1. Definitely increased profit 2. Probably increased profit 3. No affect on profit (skip to question F3) 4. Probably decreased profit 5. Definitely decreased profit 6. Don’t know (skip to question F3) About how much larger/smaller was your profit from the business in 2007 compared to your profit for the year before you received the loan from Kashf? (record answer in Rupees per month) F2 a. Fill in rupees per month _____________________ If respondent can’t or won’t answer, then prompt with the following: b. Would you say it was.... 1. Less than 500 rupees more/less? 2. Between 500 and 1,000 rupees more/less? 3. Between 1,001 and 5,000 rupees more/less? 4. Between 5,001 and 10,000 rupees more/less 5. Between 10,001 and 15,000 rupees more/less 6. Between 15,001 and 20,000 rupees more/less 7. Between 20,001 and 25,000 rupees more/less 8. Between 25,001 and 30,000 rupees more/less 9. Between 30,001 and 35,000 rupees more/less 10. Between 35,001 and 40,000 rupees more/less 11. Between 40,001 and 45,000 rupees more/less 12. Between 45,001 and 50,000 rupees more/less 13. More than 50,000 rupees more/less Ask of all respondents c. What was the approximate percentage increase/decrease in profit (fill in percent) Ask only if profit increased in question F2 About what percent of your increased profits do you use for...? (enter percent for each) 1. Household expenses 2. Reinvesting in the business 3. Other (specify) ___________________ F3 Interviewer instructions: Make sure respondent is answering only for the increase in profit. Make sure percentages add up to 100. WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 62 Have you taken on any additional workers since you received the loan that you would not have taken on if you had not received the loan? a. Number of permanent workers F4 b. Number of temporary workers Pretest note: Make sure that interviewers and respondents understand that we are looking for additions to employment as a result of receiving the loan Section G: HOUSEHOLD INCOME, CONSUMPTION, & EXPENDITURE How does your household’s normal monthly expenditure on __________ now compare to your normal monthly expenditure on ________ before you received the loan? (ask for a. through c. below and mark one choice for each) G1. Food G2. Health care G3. Education G4. Other 1. Large increase 2. Moderate increase 3. Small increase 4. No change 5. Small decrease 6. Moderate decrease 7. Large decrease 8. Don’t know Section H: EMPOWERMENT (ask only of women) To what extent would you say your preferences/opinions are taken into consideration in making the following decisions within your household? (ask for a. through h. below and mark one choice for each) a. Decisions regarding children’s schooling b. Decisions regarding children’s marriage c. Type of medical assistance to seek if a child is ill d. Whether to have another child e. Repair or construction of home f. Borrowing money g. Your participation in community or political activities H1 h. Whether or not you should work for an income 1. Always 2. Most of the time 3. Some of the time 4. Rarely 5. Never 6. Not applicable WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 63 Section I: RESPONDENT INFORMATION I1 Gender 1. Male 2. Female I2 Are you married, unmarried, or a widow or widower? (mark one response) 1. Married 2. Unmarried 3. Widow/widower 4. Separated I3 In what year were you born? ___________ (fill in year) I4 1. No formal education 2. Did not complete primary school 3. Primary school 4. Middle school 5. Matric 6. Intermediate school 7. B.A./B.Sc. 8. M.A./M.Sc. 9. MPhil/Ph.d WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 64 SR NO USAID-FUNDED Pakistan Economic Growth Evaluation and Design Project (PEGED) CONTROL GROUP Household Survey 2008 Management Systems International (MSI) Country Office: International Office: MSI/PEGED Office, Apt 48, Management Systems International (MSI) Mayfair Block Royal Apartments, 600 Water Street, SW Diplomatic Enclave, Islamabad Washington, D.C. 2004 Pakistan. Ph: 051-8314731, 4301365 USA www.msiworldwide.com MSI CRCP WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 65 Section A: IDENTIFICATION Fill in from information received from the bank or MFI branch A1 Lending institution Kashf NBP A3 Date of loan disbursal A4 Amount of loan (Rs.) A5 Name and Code of Interviewer: A6 Date of Interview: A7 Interviewer’s Arrival Time: a Hours [Code00 – 24] b Minutes [Code00 – 59] A8 Interviewer’s Departure Time: a Hours [code00 – 24] b Minutes [code00 – 59] Visit Interviewer’s Name Date Result Code A9 1 A10 If code is not 01, Give reasons? ……………………………………………………………………………………………………….. ……………………………………………………………………………………………………….. A11 2 A12 If code is not 01, Give reasons? ……………………………………………………………………………………………………….. ……………………………………………………………………………………………………….. Result codes: 01 Complete 02 Incomplete 03 Refused 04 Non-Availability of respondent 05 House Locked 06 House Replaced Others [Specify] ____________________ Name and code of Field Supervisor Name of Editor/Coder in Office Name and Code of Data Entry Operator A13 Name and code of editing staff [Different staff will use different color pencils for editing] WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 66 Section B: HOUSEHOLD INFORMATION a. Total no of persons [including children and adults] b. Total no of males [including male children and adults] c. Total no of females [including female children and adults B1 How many people are there in your household? By your household I mean all those who live in the same house for at least 5 days per week and who eat from the same table (share a common kitchen). d. Total number of persons before applying for the loan. [including children and adults] Instruction: Check that the total number of males and females equals the total number of persons and make corrections if necessary. B2 How many male children between a. Number of male children the ages of 5 and 15 years are in your household and how many are enrolled in school? b. Number of male children enrolled in school B3 How many female children between a. Number of female children the ages of 5 and 15 years are in your household and how many are enrolled in school? b. Number of female children enrolled in school Section C: BUSINESS INFORMATION The following questions ask about the business for which you applied for the loan on _______ [fill in date from front page]. Please answer only for that business. For what type of business did you apply for the loan? (mark one answer) C1 8. Manufacturing (making finished goods from other materials, e.g., making clothing, parts, handicrafts, jewelry, etc.) 9. Retail or trading (shops, buying and selling finished goods or products) 10. Services (renting equipment, operating schools, laundry, transportation/driver, beauty salon) 11. Restaurant or food (preparing and selling ready-to-eat food items) 12. Agriculture (growing crops) 13. Livestock (buying livestock, raising livestock) 14. Other (specify) _____________ C2 In what year did you start the business? (enter year) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 67 What is the ownership structure of the business? (mark one answer) C3 7. Sole proprietorship 8. Family partnership 9. Non-family partnership 10. Private limited company 11. Public limited company 12. Other (specify) ____________ From where do you operate the business? (mark one answer) C4 5. Out of your home 6. In a building that you own other than your home 7. In a building that you rent other than your home 8. Other (specify) _____________________ Can you tell me about the people who work in your business now? (fill in numbers for permanent and for temporary employees) Total number How many are men How many are family members How many are paid How many before you applied for loan Permanent employees Temporary employees C5 Instructions to interviewers Same as current questionnaire Instructions to Interviewers: A permanent employee is a paid employee that is contracted for one or more years, receives a monthly salary, and will continue working unless the employer makes a specific decision to end their employment. Temporary workers are hired for a specific period, e.g. a day, a week or even a month but it is assumed that their employment will not be continued unless the employer specifically decides to hire them for a longer period. Prompt to ensure that respondent has included all family members who may regularly work in the business whether they are paid or not. How many permanent and temporary workers have you laid off and not replaced since before you applied for the loan? a. Number of permanent workers C6 b. Number of temporary workers Section D: LOAN HISTORY Had you ever borrowed money for this business before you were approved for the loan? D1 3. Yes 4. No (Go to Æ Q no. D4 ) D2. If yes, can you please tell me about the other loans you received? Interviewer: ask questions to fill in loan history since 2004 in table below) Year Source of loan Amount borrowed (Rs.) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 68 1 Family or friends 2 MFI 3 Kashf 4 Bank 5 NBP 6 Money lender 7 Supplier’s credit 8 Other (specify) ___________________ 2004 2005 2006 2007 If respondent received any other loans, ask: If you have borrowed money from other sources, why did you choose to borrow from Kashf this time? (mark all that apply) D3 8. Easier process, not as much paperwork 9. Able to get amount needed 10. Lower interest rate 11. Loan approved and disbursed more quickly 12. Proximity of lender 13. Friendly staff 14. Know loan officer If you had not been approved for the current loan from Kashf, what would you have done? (mark one answer) D4 5. Used personal funds (skip to section E) 6. Borrowed the money from some other source 7. Not made the expenditures (skip to section E) 8. Other (specify) ________________ (skip to section E) Which source would you have borrowed from? (mark one answer, then skip to next section) D5 7. Family or friends 8. MFI 9. Bank 10. Money lender 11. Supplier’s credit 12. Other (specify) ____________________ Section E: USE OF THE LOAN The following questions ask about the way you plan to use the loan you received from Kashf on ________ [fill in date from front page]. E1 What do you plan to use the loan for? (mark all that apply) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 69 14. Increased inventory of inputs or finished goods 15. Bought, built, or rented new machinery. 16. Repaired existing machinery 17. Bought, built, or rented more work space 18. Hired more workers 19. Increased the salaries of existing workers 20. Paid consultant services to improve my business 21. Paid off debt 22. Family education expenses 23. Family health expenses 24. Emergency expenses 25. Household expenses (food, clothes, trips, etc.) 26. Other (specify) _________________ Interviewer instruction: If respondent marked “1” in question E1, then ask: E2 If you plan to use the loan to increase inventory of inputs or finished goods, how do you expect it to affect the way you operate your business? (write in response. Probe to learn how the loan affected the way they were able to run their business. Examples below) Code responses into these categories, DO NOT READ 7. Able to buy larger quantity and get discounts 8. Able to get lower price by paying cash 9. No longer had to go through as many middlemen 10. Larger inventory of inputs led to fewer interruptions in production 11. Larger inventory of goods for sale gave customers more choices 12. Other (specify)______________________ Interviewer instruction: If respondent market “2”, “3”, or “4” in question E1, then ask: If you plan to use the loan to buy or rent more work space or add machinery, how do you expect it to affect your business? (write in response. Probe to learn how the loan affected the way they were able to run their business. Examples below) E3 Code responses into these categories, DO NOT READ 5. Increased the amount you could produce in the peak season 6. Reduced your cost of production 7. Increased the amount each worker can produce 8. Other (specify) _____________________ Section F: USE OF THE LOAN Ask for all respondents WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 70 How does the total profit from your business in 2007 compare to your profit in the year before you applied for the loan from Kashf? Is it... (read answers) F1 7. Definitely increased profit 8. Probably increased profit 9. No affect on profit (skip to question F3) 10. Probably decreased profit 11. Definitely decreased profit 12. Don’t know (skip to question F3) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 71 About how much larger/smaller was your profit from the business in 2007 compared to your profit for the year before you applied for the loan from Kashf? (record answer in Rupees per month) F2 a. Fill in rupees per month _____________________ If respondent can’t or won’t answer, then prompt with the following: b. Would you say it was.... 14. Less than 500 rupees more/less? 15. Between 500 and 1,000 rupees more/less? 16. Between 1,001 and 5,000 rupees more/less? 17. Between 5,001 and 10,000 rupees more/less 18. Between 10,001 and 15,000 rupees more/less 19. Between 15,001 and 20,000 rupees more/less 20. Between 20,001 and 25,000 rupees more/less 21. Between 25,001 and 30,000 rupees more/less 22. Between 30,001 and 35,000 rupees more/less 23. Between 35,001 and 40,000 rupees more/less 24. Between 40,001 and 45,000 rupees more/less 25. Between 45,001 and 50,000 rupees more/less 26. More than 50,000 rupees more/less Ask of all respondents c. What was the approximate percentage increase/decrease in profit (fill in percent) Ask only if profit increased in question F2 About what percent of your increased profits do you use for...? (enter percent for each) 1. Household expenses 2. Reinvesting in the business 3. Other (specify) ___________________ F3 Interviewer instructions: Make sure respondent is answering only for the increase in profit. Make sure percentages add up to 100. Have you taken on any additional workers since you applied for the loan that you would not have taken on if you had not applied for the loan? a. Number of permanent workers F4 b. Number of temporary workers Pretest note: Make sure that interviewers and respondents understand that we are looking for additions to employment as a result of approval for the loan Section G: HOUSEHOLD INCOME, CONSUMPTION, & EXPENDITURE How does your household’s normal monthly expenditure on __________ now compare to your normal monthly expenditure on ________ before you applied for the loan? (ask for a. through c. below and mark one choice for each) WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 72 G1. Food G2. Health care G3. Education G4. Other 1. Large increase 2. Moderate increase 3. Small increase 4. No change 5. Small decrease 6. Moderate decrease 7. Large decrease 8. Don’t know Section H: EMPOWERMENT (ask only of women) To what extent would you say your preferences/opinions are taken into consideration in making the following decisions within your household? (ask for a. through h. below and mark one choice for each) a. Decisions regarding children’s schooling b. Decisions regarding children’s marriage c. Type of medical assistance to seek if a child is ill d. Whether to have another child e. Repair or construction of home f. Borrowing money g. Your participation in community or political activities H1 h. Whether or not you should work for an income 1. Always 2. Most of the time 3. Some of the time 4. Rarely 5. Never 6. Not applicable Section I: RESPONDENT INFORMATION I1 Gender 3. Male 4. Female I2 Are you married, unmarried, or a widow or widower? (mark one response) 5. Married 6. Unmarried 7. Widow/widower 8. Separated I3 In what year were you born? ___________ (fill in year) I4 10. No formal education 11. Did not complete primary school 12. Primary school 13. Middle school 14. Matric 15. Intermediate school 16. B.A./B.Sc. 17. M.A./M.Sc. 18. MPhil/Ph.d WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 73 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 74 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 75 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 76 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 77 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 78 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 79 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 80 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 81 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 82 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 83 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 84 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 85 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 86 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 87 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 88 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 89 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 90 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 91 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 92 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 93 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 94 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 95 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 96 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 97 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 98 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 99 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 100 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 101 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 102 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 103 WIDENING HARMONIZED ACCESS TO MICROFINANCE (WHAM) 104