Final Evaluation For the Pro Mujer Matching Grant December 2003 Submitted by: P.O. Box 53296 Washington, DC 20009 Tel: (202) 483-7781 Mail@alternative-credit.com 2 TABLE OF CONTENTS LIST OF ACRONYMS..............................................................................................................43 EVALUATION QUESTIONS....................................................................................................54 1.1 EVALUATION PROFILE SHEET........................................................................................54 1.2 SUMMARY OF CONCLUSIONS AND RECOMMENDATIONS...............................................65 1.3 PROGRAM BACKGROUND .............................................................................................76 1.4 PROGRAM EFFECTIVENESS ...........................................................................................87 1.4.1 Program Model or Approach for PM.........................................................................87 1. PRO MUJER PERU................................................................................................................98 1.4.2.1 PMP Achievement of Objectives.............................................................................98 1.4.3.1 PMP Lessons Learned and Observations...........................................................1312 1.4.4.1 PMP Cross-Cutting Issues..................................................................................1514 1.5.1.1 PMP Management Approach..........................................................................252423 1.5.2.1 PMP Quality and Status of Detailed Implementation Plan (DIP)..............262524 1.5.3.1 PMP Financial Management.........................................................................272625 1.5.4.1 PMP Monitoring and Evaluation (M&E) System .........................................302928 1.5.5.1 PMP Information Systems..............................................................................323130 1.5.6.1 PMP Staffing and Supervision.......................................................................333231 1.5.7.1 PMP Program Recommendations..................................................................343332 2. PRO MUJER BOLIVIA....................................................................................................383635 1.4.2.2 PMB Achievement of Objectives....................................................................383635 1.4.3.2 PMB Cross-Cutting Issues..............................................................................424039 1.4.4.2 PMB Program Effectiveness Lessons Learned and Observations................424039 1.5.7.2 PMB MIS Recommendations..........................................................................464342 3. PRO MUJER NICARAGUA..............................................................................................484544 1.4.2.3 PMN Achievement of Objectives:...................................................................484544 1.4.3.3 PMN Cross-Cutting Issues.............................................................................504746 1.4.4.3 PMN Lessons Learned and Observations......................................................514847 1.5.5.3 PMN Information Systems..............................................................................514847 4. PRO MUJER INTERNATIONAL.......................................................................................524948 1.4.2.4 PMI Achievement of Objectives......................................................................524948 1.4.3.4 PMI Cross-cutting Issues................................................................................605756 1.4.4.4 PMI Program Effectiveness Lessons Learned and Program Recommendations ......................................................................................................................................656261 1.5.1.4 PMI Management Approach...........................................................................666362 1.5.2.4 PMI Quality and Status of Detailed Implementation Plan (DIP)...............676463 1.5.3.4 PMI Financial Management..........................................................................686564 1.5.4.4 PMI Monitoring and Evaluation (M&E) System ..........................................696665 1.5.5.4 PMI Information Systems...............................................................................716766 1.5.6.4 PMI Staffing and Supervision........................................................................716867 3 1.5.7 PVC Program Management...............................................................................737069 1.5.8.4 PMI Program Management Recommendations.............................................747069 1.6 EVALUATION METHODOLOGY..............................................................................757271 4 List Of Acronyms AC Asociación Comunal – a village banking group APCI Peruvian International Cooperation Agency BOD Board of Directors CA Cooperative Agreement CF Central Focal – a branch COPEME Consorcio de Organizaciones Privadas de Promoción al Desarrollo de la Micro y Pequena Empresa- a network organization of MFIs in Peru DIP Detailed Implementation Plan EDPYME Empresa de Desarrollo de la Pequena y Micro Empresa- a commercial microfinance institution EMDAP Emerging Markets Development Advisor Program EOP End of Project GL General Ledger GOS General Operating Support LOP Life of Project LP Loan Portfolio Management System M&E Monitoring and Evaluation MEDA Mennonite Economic Development Associates MG Matching Grant MIS Management Information System NGO Recipient Country Non-Governmental Organization PLP Practitioner Learning Program—The SEEP Network PM Pro Mujer Network PMB Pro Mujer Bolivia PMI Pro Mujer International PMM Pro Mujer Mexico PMN Pro Mujer Nicaragua PMP Pro Mujer Peru PS Peruvian Soles PVC Office of Private and Voluntary Cooperation PVO U.S. Based/Affiliated Private Voluntary Organization SG Solidarity Group USAID United States Agency for International Development 5 Evaluation Questions 1.1 Evaluation Profile Sheet · PVO name and Cooperative Agreement Number Pro Mujer (Programs for Women), Inc. FAO-A-00-99-00056-00 · Country Program Sites and Names of Principal Partners Peru – Pro Mujer Peru (PMP) Bolivia – Pro Mujer Bolivia (PMB) Nicaragua – Pro Mujer Nicaragua (PMN) · Duration of Grant (Month/Year) September 1999 to September 2002 On August 2, 2002, a one-year no-cost extension was requested to extend the project to September 2003 to complete the implementation of the MIS system, to complete the process of establishing a local NGO in Peru, and to begin the process for establishing a local NGO in Nicaragua. On June 19, 2003, a second extension was requested for three months, to December 2003 to complete the final evaluation of the Matching Grant (MG) and full debriefing of the findings with USAID/PVC and PMI · Beneficiary Populations (includes borrowers and savers): As of June 30, 2003: PMB: 34,973 PMN: 11,096 PMP: 15,946 · PVC-PM match totals ($) and PVC-PM match funds disbursed to date ($): Budget Actual Expenditures (as of 12/31/02) 6 USAID 1,500,000 1,337,760 PM 1,500,000 1,486,465 · Date DIP was first approved by PVC and changes made to DIP: Approved: April 2000 Changes: none · Evaluation Start Date (begin implementation of SOW) and End Date (submission of final report to PVC) Start: July 30, 2003 End: No later than December 31, 2003, including acceptance by USAID/PVC and debriefing 1.2 Summary of Conclusions and Recommendations Provide a summary of the evaluation’s main conclusions and recommendations. Pro Mujer (PM) has had considerable success in implementing its MG program, particularly in Peru. It has surpassed most of its targets, evolving from a two-country program at the beginning of the grant period into a four program network reaching over 75,000 clients. PM maintains talented and committed staff and Board of Director (BOD) membership at both the international and country level. This fact, along with strong marketing and fund raising capabilities, has been fundamental to the success the network has had to date. Currently, PM has significant momentum and organizational experience; however, there are several steps that PM must take in order to harness this momentum and experience into an efficient network capable of scaling-up and replicating its success. Necessary steps include: 1. Pro Mujer International (PMI)1 must address and clarify the network/affiliate relationship. They should evaluate various existing models, such as Women’s World Banking (local autonomous organizations) or FINCA (tightly controlled affiliates). PM’s current organizational form and management practices apply various elements of each of these structures in different ways across the country programs. This has led to confusion regarding leadership, management and reporting responsibilities within the network. Though there is an affiliate agreement in place, clearer definitions are needed, in particular, of the roles of country directors, Executive Directors, and local and PMI BODs. For example, the affiliate agreement does not clarify the reporting relationships nor who is responsible for evaluating the country directors. Clarifying the reporting and oversight responsibilities should include approval and implementation of the internal audit system. 1 It is important to clarify the structure of PMI to establish the correct context for the following comments. PMI is legally responsible for all of the country programs in the network. PMN and PMB are branches that report directly to the PMI BOD. PMP and PMM have local BODs, but they are each controlled by a two-person assembly, which has the right to remove or replace the local BOD. The assembly is made up of one member from the PMI BOD, and one person appointed by the PMI BOD. Thus, the ultimate control lays with the assembly, hence the PMI BOD, for the PMM and PMP programs as well. 7 2. PMI has an active and talented BOD that should examine ways to strengthen the leadership and skills for moving the network forward in its growth and new stage of development. The current Executive Directors are both so deeply involved in daily management responsibilities that there is little time for strategic planning and vision for the growth of the network. 3. PM is shifting its culture from that of a ‘family owned business’ to that of an institution with organizational needs for formal supervision and backstopping. The MG came at a critical time to begin this “institutionalization” of PM. PM should develop a management structure that can survive without relying too heavily on key founders or stakeholders. 4. A participatory planning process must be designed that enables the entire network to envision long￾term goals, and to develop an actionable strategic plan. This plan should guide programmatic and operational decision-making and field-program annual plans and budgets. 5. PM should develop a long-term strategy for non-credit services in its programs. While it is likely that these services will attract private partners and/or foundation sponsors, the network should develop a long-term strategy for these services and consider whether such services are best provided through different legal entities. 6. PM must address the lack of documentation of network policies and procedures. A systematic effort to define and document the PM program is critical for implementing new programs and facilitating PMI oversight and support of field programs. Finally, PM must increase investment in two key resources: 7. The MIS needs of the network must be assessed and fulfilled. 8. PMI should systematize and expand its support and training for staff at the field and PMI office levels to invest in and retain the talented staff which has been so valuable to the success of the network. 1.3 Program Background PM is an international women’s development organization, founded in 1990 to empower women to improve their social and economic status. PM accomplishes this by establishing sustainable microfinance institutions that provide the credit and training in business development that women need to improve their small businesses and increase their incomes. PM also provides health education and links women and their families to health services. In 1999 when the MG was awarded, PM had operations in Bolivia and Nicaragua and was serving a total of 26,100 clients with a loan portfolio of $2.5 million. 8 PM received a MG of $1.5 million from USAID in order to: 1. strengthen PM’s financial, managerial, technical and analytical capacity to develop and manage field operations in Bolivia, Peru and Nicaragua; 2. establish a sustainable microfinance organization in Peru; 3. increase women’s access to credit in three countries; 4. increase the self-sufficiency of the credit programs in three countries; 5. develop a new management information system. PM’s match to USAID’s funding was $1.5 million. 1.4 Program Effectiveness 1.4.1 Program Model or Approach for PM Briefly describe the program’s general approach/model(s) and hypotheses; determine if the hypotheses and assumptions under-pinning the program model were sound. ß PM has been developed in Latin America FOR Latin America – This is a strength of PM in that the programs are tailored to local conditions and realities. ß PM is focused on serving women: Women are statistically poorer and more vulnerable throughout Central and South America. PM has been successful in serving this clientele not only with microcredit, but also in terms of the non-financial integrated services that are available to the PM clients – namely health education. ß PM does not have a standardized operating system. This is a weakness in the sense that programs during the start up phase have to duplicate efforts of other programs when developing standard operating procedures. It would be worthwhile to have a manual with PM’s general principles, policies, procedures, and basic products/systems that could be adapted by each new country program to the local conditions or modified based on lessons learned in other programs. ß PM has a flexible program structure and methodology; each country program has had a different beginning, mission, products, and governance structure. This adaptability is a strength when operating in different cultural and economic environments. It would be more effective if PMI documented the difference in models to enhance lateral learning and network development. ß PM has a ‘support and coordination office’ (PMI office) instead of a ‘headquarters’ to work with and provide support to country programs. This is a strength of PM in that overhead costs are kept low. The PMI office has been remarkably successful at servicing the country programs with a small and extremely dedicated staff and Executive Director. The office and salaries are modest as well. There is a true commitment to keeping back office expenses to a minimum. 9 1.4.2 Achievement of Objectives NOTE: During the preparatory meeting on July 30, 2003 with PVC, all parties agreed that Sections 1.4.2 to 1.5.6 will be reported on a country-by-country basis, as this facilitates report writing and presentation. The order of reporting will be: 1. Pro Mujer Peru 2. Pro Mujer Bolivia 3. Pro Mujer Nicaragua 4. Pro Mujer International It was agreed that PMP and PMI would have in-depth reporting, whereas PMB and PMN would primarily report on indicators and the MIS system. 1. Pro Mujer Peru 1.4.2.1 PMP Achievement of Objectives This section should identify the program objectives and assess progress made to date toward achieving each major objective defined in the DIP, for PMI and each country program. Program Objectives Strengths Weaknesses 1. Establish a local MFI · Locally incorporated association with talented local BOD · Leading MFI in Puno market · Recognized nationally as a leading NGO in microfinance · Motivated staff · High operational self-sufficiency · Untrained local BOD · Unclear roles of PMI and PMP BOD · Uncertain access to future funding · Unclear strategy for long￾term expansion 2. Implement credit program · Village Banking product widely accepted · New products popular · No delinquency at PMP level2 · Effective management of rapid growth · Integrated approach to client services popular with clients · Effective linkages with other · Inadequate MIS · Dependence on subsidies for non-credit services 2 While some delinquency exists within Asociaciones Comunales (ACs), the ACs are all current in their payments to PMP. 10 service providers 1.4.2.1.1 Changes made since the DIP On February 3, 2003, Lori Pommerenke sent an approval to Lynne Patterson of ProMujer to confirm that their request to extend the implementation period for the USAID/PVC Matching Grant (#FAO-A-00-99-00056-00) had been approved by the USAID procurement division. The new grant ending date was 9/26/03. A budget reallocation request, dated 23 Sept. 2002, was also approved by USAID. This request reallocated $26,380 from salaries to other direct costs for the Peru portion of the program and to reallocate $16,563 from consultants to travel for the headquarters portion of the program. 1.4.2.1.2 Results Assess progress toward each major objective. Provide evidence, criteria for judgment and cite data sources. Objective 1: Establish a local MFI PMP has successfully completed this objective. The indicators measure steps towards the establishment of legal status for PMP and the creation of a local governing body, which have both been completed. In fact, PMP’s institutional development has exceeded these initial goals: It is firmly established in its markets as a major microfinance institution and is recognized nationally as a leading microfinance NGO. At the same time, PMI’s strategy for using its legal status and local BOD to govern PMP is unclear. It is not yet clear to PMP’s management or the BOD what the roles and responsibilities of PMI and the local BOD are. Indicators · Board of Directors. PMP was established in April 2002 and registered in the public registry of Lima in July 2002 and recognized by the Peruvian International Cooperation Agency (APCI) in October of the same year. The BOD conducted its first meeting on the first day. A second meeting was conducted in January 2003 and the third meeting was conducted in September 2003. · Written By-laws. By-laws were completed in April 2002 (Constitución de Asociación Civil Sin Fines de Lucro). The by-laws provide for the creation of a non-profit association, called Pro Mujer Peru, and specify its activities and governing structure. It should be noted that according to the by-laws, PMP’s governing structure consists of three bodies: an assembly, a BOD and an executive committee. Final authority is held by the assembly which currently consists of two associates, PMI and the former Chairperson of PMI, Rosemary Werrett. 11 · Legal status as local NGO. PMP is registered as an Asociación Civil, a non-profit organization in the public registry of Lima in July 2002 and recognized by APCI in October 2002. · Strategic and financial plans. PMP developed a strategic plan following best practices. It also has an annual operating plan and budget for 2003. · Form a National Advisory Committee with 7 members by 31 December 2002. PMI utilized three persons as an advisory committee prior to forming a BOD and a different three persons who assisted in the decision-making to form a local association.3 While these persons did not function as a unified committee, collectively they provided guidance to PMP on key issues. Activities · Identify and train BOD members. PMI identified BOD members who are named in the by-laws. There has been no training for BOD members. · Form local BOD. The BOD has been formed, consisting of eight persons, six of whom are Peruvian.4 · Write by-laws. See the above indicator. · Apply for legal status as local NGO. See the above indicator. · Carry out strategic and financial planning. With the assistance of a consultant, PMP conducted a strategic planning workshop and completed a four-year strategic plan using best practice techniques. PMP also has an annual operating plan and an annual operating budget. Objective 2: Implement Credit Program PMP has been very successful in establishing its credit program, surpassing all indicators proposed in the DIP. Indicators 3 According to the National Director, some issues addressed by the National Advisory Committee were market information in Tacna, drafting by-laws, legal advice on an act of fraud, and facilitating some technical assistance with PMB. The committee did not act a unified body, rather as expert consultants. Those consultants were Victor Miguel Moises Marroquin Merinon, Jacinta Hamann de Vivero, Marcela Maria del Carmen Velasco de Torrez, Guillermo Casteñeda Echegaray, and Lynne Patterson. 4 Lynne Patterson, Susan Maria de la Puente Wiese, Victor Miguel Moises Marroquin Merinon, Jacinta Hamann de Vivero, Marcela Maria del Carmen Velasco de Torrez, Guillermo Casteñeda Echegaray, Eduardo Enrique Vasquez Huaman, Gerardo Pejerrey Piedra 12 Results are shown as of December 2002 and June 2003, respectively. Target As of Dec. 2002 As of June 2003 7,100 clients 12,483 active clients 15,946 active clients 284 banks5 521 active ACs 654 active ACs $715,000 portfolio $1.27mm gross loan portfolio $1.64mm gross loan portfolio $240,000 income $475,000 financial income $341,000 financial income for first six months 87% Operational Self￾sufficiency 134.7% 158.6% Clients will save the equivalent of 20% of their current loan. Client savings reached 49% of the gross loan portfolio Client savings reached 44% of gross loan portfolio Adapt program methodology to needs of Peruvian clients: training manuals for clients revised and adjusted for Peruvian program. PMP incorporated several changes to the methodology. The policies are incorporated into the PMP operations manual and the standard AC training. For a description of changes, see pages 16-19. Note: While the term “client” was not clearly defined in Table 1 of the DIP, for the purpose of this report the term includes all program participants with a loan outstanding, excluding write-offs. Participants who maintain savings in their community banks but who do not have outstanding loans were not counted as active clients. Activities · Recruit and organize 8 groups of 25-30 women each month. Since it began lending, PMP has recruited and organized an average of 17 ACs per month as of December 2002. This has slowed slightly to 16 ACs as of June 2003. · Provide pre-credit training to all groups. As part of its methodology, PMP provides 8 training sessions of two hours each to new ACs. PMP uses the guide Destrezas básicas en negocios 1 and Asociaciones Comunales, developed for PMB. Training includes modules on the form and function of ACs, the importance of the solidarity guarantee, and exercises for determining the appropriate loan size and the capacity to repay (using cash flow). New members joining existing groups have five training sessions and are required to attend the final two AC meetings of a loan cycle. · Supervise loan repayment and saving weekly or bi monthly. ACs are required to meet weekly (cycles 1-3) or bi-weekly (an option beginning in cycle 4). All meetings are conducted by the AC committee and are supervised by a Promoter. Central Focal (CF) Managers attend 5 A “bank” refers to a village bank, or AC 13 or assist in AC meetings as needed, including important meetings (disbursements and final payments). During the meetings, the role of the Promoter is to ensure that AC policies and procedures are being followed and to encourage ACs to find solutions to any problems with members. · Ongoing staff training of existing and newly hired staff. If possible, staff are hired and trained in groups. New recruits are given an overview of the institution, including its mission and objectives, as well as three sessions on policies and procedures. Potential new Promoters spend two weeks, without pay, working with a current Promoter. After the two-week period, the applicant is reviewed by the CF Manager and Supervisor and is then offered a position (with 3 month probation period). Continuing education for staff is ad hoc, including general workshops to promote staff unity or on specific topics related to new products or product changes. 1.4.3.1 PMP Lessons Learned and Observations Identify major successes, challenges and constraints in achieving each objective. A summary of the program’s DIP results is in Annex B. Assess if the main strategies and activities employed to achieve results were effective PMP has been successful in achieving its objectives to develop a locally incorporated, high quality MFI that is a market leader in Puno. While the indicators for the first objective (establishing a local MFI) focus on legal and governance issues, the evaluation team recognizes that the development of a strong institution also requires staffing, infrastructure, technology, and market presence. In each of these areas, PMP has exceeded the expectations of the MG. PMP has qualified and experienced management, a highly motivated and disciplined staff, a network of CFs with back office administrative support, a widely accepted product, and a dominant position in its chosen market. It has also exceeded expectations in terms of achieving the second objective of implementing a credit program. PMP’s implementation of the village banking technology has been successful in achieving rapid growth among very low-income women while maintaining a very high quality portfolio. As the indicators show, PMP has added over 500 clients per month, maintained 0% delinquency by ACs6 , and had a fairly high client retention rate of approximately 80%7 . PMI’s strategy for implementing a credit program in Peru has worked well. First, the geographic proximity and cultural similarity of PMP to PMB allowed PMP to establish itself quickly using a tested methodology. Second, the decision to permit product modifications (as discussed in section, 1.4.4.1.2 6 Pro Mujer lends to the ACs and does not track delinquency between the clients. 7 Based on the formula End of Period Clients/ (Beginning of Period Clients + New Clients added during period). The six month period of January to June, 2003 showed a retention rate of 86%. It was impossible to calculate the retention rate for 2002 due to incorrect client information calculated by the MIS. This does not take into account clients who withdraw and then return, which might increase this retention rate. 14 New Tools, Guidance or Standards, page 16) provided more efficient service provision and a more attractive product to clients. The use of the CFs has been efficient and has allowed for more control over credit activities. Finally, anecdotal evidence suggest that the decision to provide integrated services, bundling health and business training with credit, has enhanced client loyalty. PMP is in the process of finishing an impact evaluation and implementing a set of tools to measure client satisfaction, both of which will better determine the value of the integrated services to clients. It is also clear that PM’s strategy to create local capacity by bringing training and technology from the network (primarily PMB) has served PMP well. PMP management has experienced some frustration with the frequency and the reliability of this assistance, but it is fair to say that PMP’s early development owes much to the expertise of the PM network. The success of the strategy to create a locally incorporated MFI with a local BOD is not as clear. It is likely that local status will (and has) helped PMP secure funding from both local and international sources. It is well known that donors prefer to lend and grant money to local institutions. In terms of governance, PMP’s local BOD members are qualified and have significant expertise in banking, finance and legal issues. However, since its nomination, the BOD has not performed most of the duties assigned to it in the by-laws. It has not performed the traditional duties of a BOD, such as evaluating the National Director, approving annual operating plans and budgets, selecting and overseeing the work of the external auditor, or performing or overseeing internal audit functions. The BOD members have not received training, and an interview with the President confirmed that members are waiting for clearer guidance on which activities they should undertake. Most of the members of the BOD do not have a significant understanding of the operations of PMP. Even with training, it is doubtful that their busy schedules will permit them to perform the required performance monitoring duties. This is more likely because the BOD knows that PMI is already acting as a governing body, requiring reports and providing feedback.8 In short, it is not clear what is the true purpose (or advantage) of creating a local BOD. Discuss the impact of the program on target population PMP was one of the first poverty lenders in the state of Puno. The majority of the clients interviewed stated that they had not had access to credit before joining PMP. The majority stated that they joined PMP because it did not require the same amount of paperwork or collateral as the commercial banks and local savings and loan institutions (Caja Municipales). At present, PMP serves over 15,000 clients with an average disbursed loan size of approximately US$100. While the industry recognizes that loan size is not an accurate measure of poverty, it is fair to say that the type of client willing to participate in weekly meetings and obligatory savings is most likely a client that does not have sufficient assets to receive commercial credit. PMP conducted a baseline study of its clients and a group of non-clients in 2001. It is currently updating this study to determine what impact, if any, its programs have had on the income, assets, and 8 This impression was confirmed by discussions with the President, National Director, and staff members of PMI. 15 other qualify of life indicators of its clients compared to non-clients. Early results indicate that the although clients and non-clients alike have experienced a fall in annual income over the past two years, the decline for PMP client families has been approximately 6% as compared to 21% for the non-client control group. Similarly, PMP clients had fewer of their children working to support the family than non-clients. Another preliminary finding is that clients have increased their expenditures for business and for household consumption over the past two years and their level of savings is higher than that of non￾clients. Finally, there is early evidence to suggest that these improvements appear to be greatest among single mothers participating in the program.9 Interviews with clients confirmed that the majority felt that they were better off than before their participation with PMP. While few could quantify this in terms of income or assets, many expressed having a stronger sense of security as a result of having access to credit, savings, and the support of their AC. 1.4.4.1 PMP Cross-Cutting Issues 1.4.4.1.1 Partnerships: Assess the status and outcomes of PMI’s partnerships with local organizations, NGOs, networks, businesses or intermediate support organizations: PM Network Assess the effectiveness of PM’s approach to building in-country partnerships. Identify the key elements that contributed to success as well as key challenges. PM’s approach to partnership in Peru was to create a local organization rather than partner with an existing NGO. PMP began as a registered branch of PMI. PMI used its network, primarily its materials and personnel from PMB, to train new Peruvian staff in lending methodology, accounting, financial management, and personnel management. Although the PM network did not provide a complete operating system of manuals, forms, and MIS, it provided enough materials and systems from PMB to allow PMP to achieve early success. PMI also placed a full time expatriate, with training from PMN, as National Director for two years: this was a key element to PMP’s ability to start operations quickly. Other key aspects of the partnership included personnel exchanges, on-site training, and specialized international technical assistance from the PM network. Finally, PMP has received significant financial support from PMI. This includes not only the MG funds, but also a loan of $625,000 that has been necessary to fund growth. In general, PMP has been very satisfied with its relationship with PMI. If anything, it has come to depend too much on PMI as its primary source of technical assistance and support. But this 9 Based on an interview with Demecia Mamani Benique, Coordinator of Research. 16 dependence has not been without its problems. PMP expressed concern with the significant change in the personnel of PMI, especially up until the end of 2001. It is not always transparent to whom at PMI they should direct which requests. Up until recently, PMP lacked guidelines on the rights and responsibilities of the members of the PM network. The National Director recently received a copy of an affiliation agreement (in English) which highlights the obligations of PMI and PMP, but it has not been signed. Only in the past few months has PMI done a survey of its affiliates to determine what their greatest needs and concerns are. There is also concern among some affiliates that the network is not a network of equals, namely that PMB has special rights10. PMP is required to pay a fee for any technical assistance it receives (or any visit to) PMB and that such assistance is also made available to PMP’s competitors for a fee.11 Exchanges between other affiliates are free (that is to say, at the expense of the visitor).12 Initially, this made more sense when PMB was the primary source of operating systems. However, now PMP is training PMB on seasonal loans and other adaptations. PMP also expressed concern that it lacks performance information on other PM affiliates. PMI also directed the local incorporation of PMP as an association and selected a BOD. The establishment of a local NGO with a local BOD has not significantly changed the relationship between PMI and PMP thus far. PMP has expressed some frustration over the inactivity of the BOD and anxiety over the future role the BOD may play. The nature of the partnership is currently in transition and it is not clear how PMI and PMP’s relationship shall change. Identify local partners and cite changes in local partners’ institutional capacities that occurred as a result of the program partnerships. PMP is the only local partner of PMI. As noted above, PMP was created by PMI which developed it using the resources for the PM network, primarily PMB. The PMI office and the PM network are the preeminent influences on PMP. With the assistance of a full time expatriate National Director, PMP transplanted the lending methodology and credit operational systems from PMB. These were adapted (and continue to be adapted) by local staff to address the Peruvian environment. PMB provided the credit manuals and methodology, the MIS system, and training on accounting. As noted in the annual reports, targeted technical assistance has helped to develop PMP’s product line and institutional capacity. PMI’s Financial Advisor, Nancy Natilson, provided guidance on setting interest rates, performance monitoring, and developing the seasonal loan product. Outside consultants funded by Ford Foundation and Summit provided assistance in conducting the impact study and a consultant funded by COPEME assisted in reviewing internal controls. 10 Based on conversations with the PMP National Director and the Operations Manager of PMM. 11 According to the PMP National Director, ADRA Peru paid PMB to receive training in village banking before PMP began operations. Now ADRA competes against PMP in Puno and Juliaca using the same methodology. 12 At the time of the Peru visit, the current system for technical assistance was not yet in place. At the Country Directors conference in September 2003, the directors and the PMI staff developed an equitable system for technical assistance where all countries are responsible for paying for assistance received. Agreements are made between the countries and coordinated by the Operations Officer 17 Identify outcomes of program links with any coalitions, networks or associations; and discuss if local partnerships are mutually satisfactory and beneficial. Identify and discuss major constraints to developing durable and productive partnerships. (See Partnership Table in Annex C). One of PMP’s strategic objectives is to take advantage of other service providers to increase services to PMP clients, primarily in the areas of health, legal and business support. PMP has negotiated numerous formal and informal partnerships with governmental and non-governmental agencies, as well as with private companies and individuals to provide quality services to PMP clients. PMP’s strategy is to assist clients in accessing services that are already available and, in some cases, negotiate a lower cost on their behalf. In addition, PMP is involved in working with the providers to improve the quality and appropriateness of the services provided, particularly health services. Some examples are the following: Plan Fami. One of the strongest partnerships is with Plan Fami, a local NGO that provides health services to women on a fee basis. PMP has negotiated an agreement with Plan Fami for them to provide medical and dental services to their clients at a reduced rate. Plan Fami holds a 1-2 day health fair in each CF once per month. Clients may pay for services such as a PAP smear, tooth extraction, blood test, and eye exams for a fee of S5-10 ($1.75-3.50). PMP’s Health Promoters explain the importance of the tests, exams, and services offered during regular health trainings. Interviews with both Plan Fami and PMP indicate that both are very satisfied with the arrangement and both intend to renew the agreement when it expires at the end of this year. For Plan Fami, it provides access to a significant number of clients. This volume allows them to offer services at a lower cost. For PMP, it helps fulfill their objective of providing integrated services to clients with a focus on health. Universidad Andina Nestor. Seven graduates of the law school provide one training per week on legal issues related to family violence, leaving the home, and relations with family members. Based on the requests of the Health Promoters, they provide consultations with clients (256 to date). Universidad Nacional de Altiplano. Students work as interns with the business development services area to provide training to clients in the use of credit, business skills, and marketing. This complements the hands-on technical training and consulting provided by the PMP business promoters. Day Care. PMP partners with individual day care providers. These trained teachers provide full-time and part-time day care in the CFs. PMP provides the space and promotes the service to clients and the teacher provides their time and materials. Day care providers are paid directly by clients for this service. In general, the quality of the partners’ work is fairly good, but almost all partners have required some training and encouragement by PMP to improve their performance. PMP’s experience is that partners do not maintain the same high standards of discipline and reliability. Partners don’t fully recognize that 18 PMP’s success depends on fulfilling its promises to clients to be on time, prepared, and respectful of the client’s time. PMP’s challenge going forward is to structure and support partnerships so that they will reflect PMP’s high institutional quality standards. 1.4.4.1.2 New Tools, Guidance Or Standards: Identify if new tools/guidance, approaches, or program standards were developed under this program. Assess if these tools, approaches etc. were effective and merit broader distribution or application. PMP has introduced a number of changes to the village banking methodology practiced by other PM country programs (see a description of these changes below). In addition, they have created new products (seasonal loans, school loans) which can be replicated by other PM country programs. PMP has already hosted staff from other PM programs to observe these new approaches and products, and Pro Mujer Mexico (PMM) has adopted a number of the PMP practices. PMP is committed to providing integrated services. Although the primary product is credit, PMP is committed to a holistic approach to women’s development. This approach is reflected in PMP’s organizational structure that includes a department for credit, health, and business development. This strategy goes against the current trend in the microfinance industry to separate credit and non-credit services. In addition, PMP does not charge clients directly for health and business development services, although the partner organizations may (and usually do) require a fee for services.13 To minimize the cost of this integrated approach and to keep non-credit services financially separate from credit operations, PMP seeks distinct donor support for non-credit services. It also seeks out partners to provide these services rather than building internal capacity. In the local environment, this strategy has worked well to attract and retain clients, and may provide an example to other microfinance NGOs. However, this success is contingent on PMP’s location in urban areas that have a number of high quality service providers focusing on the same population. PMP is also working with a clientele that traditionally supports highly responsible and active women. The lack of similar conditions may limit the replicability of PMP’s model elsewhere. An important operational approach is the use of CFs as a “one-stop shop” for services. PMP’s CFs have become more than just loan production offices. They also act as community centers for PMP’s members. Members can access training, health services, day care, and other specialized services in the centers, all of which are either provided or arranged by PMP. In return, PMP has asked members to pay the rent of the CFs. Each member pays 50¢ per month which is used to cover the costs of renting and maintaining the CF. This approach is most effective in urban areas, but might be considered on a lesser scale in peri-urban centers. 13 To clarify, PMP employs a health coordinator for nearly every CF. These coordinators are not paid by clients for the counseling they provide or for providing the linkages to other services. This cost is born by PMP and paid out of grants. 19 What changes to the methodology were made at PMP that were successful/unsuccessful which could be applied in other PM programs or other village banking programs? From the beginning, PMP used a modified village banking methodology that eliminated internal account lending. Traditionally, ACs are required to save and have the option to use the accumulated savings to make internal loans to AC members. This internal account lending is managed by the membership and in theory, eventually leads to a self-financing savings and loan group. Experience in many village programs has shown that the management of the internal account is often a cause of significant problems that lead to the erosion of group solidarity. Modifications to the internal account. At PMP, the internal account consists of the obligatory and voluntary savings of members, interest earned on those savings, and any other fees or duties collected by ACs. PMP eliminated the option of using the internal account for lending and instead requires ACs to place these funds on deposit with a local bank. These bank accounts require two signatures, one of which must be a PMP Promoter. Members are only allowed to access the internal account at the end of a loan cycle.14 In PMP’s opinion, the elimination of internal account loans has greatly reduced the management strain on ACs and the corresponding problems with group solidarity. It also reduces the amount of time required for promoters to facilitate payments, as ACs do not have to manage loan payments between the AC and its members. This allows promoters to manage more groups. The internal account serves the dual purpose of providing members with secure savings and being a guarantee against default. In many village banking programs, members come to rely on the internal account to cover the payment of delinquent members or to make final payments. In the case of PMP, the internal account may not be used to cover the delinquency of members or for final payments. This enforces discipline on ACs to collect from all members, regardless of payment amount. It also eliminates the problem of using the savings of members who repay in full to subsidize defaulting borrowers. PMP has also modified the savings requirement so that after the fifth cycle, members are allowed to withdraw much of their obligatory savings. From that point on, they are required to maintain savings equivalent to 20% of the loan amount. PMP’s baseline study revealed client dissatisfaction with the previous cumulative savings requirement. As ACs mature, the ratio of loans to accumulated savings declines, and members see a greater advantage in withdrawing their savings and leaving the AC. To address this, PMP identified the cycle in which the loan/savings ratio became undesirable and reduced the savings requirement to a 20% compensating balance. Linkages with local service providers. PMP has effectively linked with local health, education, and business service providers to provide access to additional services to clients at lower cost to both the clients and PMP. As noted above, PMP primarily operates in urban areas where these providers are 14 Members are allowed to withdraw voluntary savings prior to the end of the cycle. 20 located. Assess the new products that were developed by PMP (seasonal loans and educational loans) to evaluate whether these new products are meeting client needs; how have these products been received by clients? Has the transfer of knowledge about the seasonal loan product to PMB been successful and has PMB introduced these new products? Seasonal Loans. At the beginning of the fourth cycle (approximately one year), members in good standing may request a second short-term loan for seasonal needs. Within an AC, members wishing to receive an additional loan form special solidarity groups to guarantee the loans. The AC must then approve the new group’s loan. Loans are for one or two months with monthly payments of principal and interest. The seasonal loans are popular, accounting for 10-20% of the loan portfolio. In the holiday season, disbursements of seasonal loans exceed those of regular loans. Thus far, there is no delinquency on seasonal loans nor have they caused any apparent disruption of AC solidarity. While this type of loan is not unique to PMP, it has proven to be a successful innovation that could be replicated elsewhere. It also helps to offset the reduction of liquidity for the clients caused by the elimination of internal account loans (see above). Educational Loans. In March and April 2003, PMP conducted a pilot test of educational loans. The purpose of the loan is to provide a small sum (PS50-200) to existing clients to purchase books and uniforms and to pay registration fees for their children’s school. Requirements are very similar to seasonal loans but they are only offered during the beginning of the semester. Clients may not have a seasonal loan and an educational loan at the same time. While the pilot was a success, some challenges included the low loan ceiling and the limit on the number of children that can be supported (maximum 2). Members also expressed interest in borrowing on behalf of other family members, such as nieces and nephews. Also, PMP is still determining a way to verify that funds are used for scholastic purposes. What are the key criteria of PM’s methodology, as applied in PMP, which contribute to success (as measured by 0% delinquency, scale of operations, self-sufficiency and efficiency)? A focus group conducted with Promoters revealed a number of key success factors. First, the methodology is systematic and requires significant discipline. This has allowed for rapid replication of ACs without delinquency. This includes the use of the solidarity guarantee. The local culture, especially among women, is also very disciplined and the product has been widely accepted. This is not surprising considering that the ethnic groups served in Peru are the same as those served for many years in Bolivia. Promoters also pointed to the efficiency and popularity of the CF in strategic locations that allow groups to be served rapidly, securely, and in a comfortable location. Other factors include the provision of non-credit services and the personal interaction between CF staff and clients. The evaluators agree with all of the above observations. Other factors include: · PMP had virtually no competition when it entered its markets. · PMP had a product that had been tested significantly in a very similar environment in Bolivia. · PMP was fully funded for its initial growth, allowing it to build infrastructure quickly. 21 What makes PMP so efficient compared to other PM programs and other village bank programs? PMP’s efficiency will be analyzed statistically (compared to peer group data) and qualitatively (through observation of operations) but no other village bank programs will be visited as part of the evaluation to make efficiency comparisons. PMP is very efficient in comparison to similar organizations. The following tables show PMP in relation to peer group data reported to the MicroBanking Bulletin, Issue 9, July 2003. The peer group is identified as Latin American microfinance institutions that are small (loan portfolio less than USD 1.5 million) and serve low-end clientele (average loan size less than USD 150). 22 Table 1: Peer Group Characteristics INSTITUTIONAL CHARACTERISTICS MFIs BY PEER GROUP Age Total Assets Offices Personn el (years) (US $) (number) (number) Latin America Small, Low End MFI Number of MFIs in Sample 8 10 8 10 Average 6 1,894,792 19 43 PMP As of December 31, 2002 3 1,752,690 9 (focal centers in Puno & Juliaca as of Dec. 2002) 30 While PMP is similar in asset size to its peer group, its higher relative efficiencies result from cost savings on overhead and staff expense. PMP maintains fewer offices (CFs) because it is operating in two fairly dense urban areas: Puno and Juliaca. It also has fewer personnel because it has fewer CFs. Because all of its CFs are close to headquarters, PMP has centralized accounting and administrative functions rather than maintaining accounting and administrative staff in each CF. PMP disburses and collects loans through commercial banks and does not employ tellers in the CFs. Table 2: Peer Group Outreach MFIs BY PEER GROUP Number of Active Borrowers Percent of Women Borrowers Gross Loan Portfolio Average Loan Balance per Borrower Voluntary Savings (number) (%) (US $) (US $) (US $) Latin America Small, Low End MFI Number of MFIs in Sample 10 10 10 10 8 Average 4,282 73.3 1,052,708 267 54,522 PMP As of December 31, 2002 12,483 100 1,274,309 102 105,553 As noted in the table above, PMP is reaching significantly more clients than its peers because of its use of the village banking methodology. While PMP had over 12,000 clients at the end of 2002, it had only slightly more than 500 active loans to ACs. 23 Table 3: Efficiency/ Productivity MFIs BY PEER GROUP EFFICIENCY/PRODUCTIVITY Operati onal Self￾Sufficien cy Yield on Gross Portfo lio (real) Operat ing Expen se/ Loan Portfol io Perso nnel Expen se/ Loan Portfol io Adjust ed Cost per Borro wer Borrower s per Staff Member Borrower s per Loan Officer Personne l Allocatio n Ratio (%) (%) (%) (%) (%) (number) (number) (%) Latin America Small, Low End MFI* Number of MFIs in Sample 10 10 10 8 8 10 10 10 Average 79 55.6 98.2 42.3 199 141 298 44.4 PMP As of December 31, 2002 135 49 47 30 35 416 594 70 The table above clearly outlines PMP’s high level of efficiency. In ACT’s opinion, the most important factor is the effective use of CFs in urban areas. The CFs allow each PMP Promoter to serve up to 4 ACs per day. If all CFs met biweekly, some promoters could serve up to 40 ACs. The current average figure is 21, since many new ACs meet weekly. This efficiency will increase with time as PMP moves more ACs to biweekly payment. In addition, the cost of the CFs is passed on to the members through a monthly fee, thereby reducing the cost to PMP of maintaining the CF infrastructure. Interviews with Promoters confirmed CF staff is nearing full capacity. Additional responsibilities, such as conducting exit interviews and satisfaction surveys, are making it difficult for Promoters to provide personal attention to clients outside of AC meetings. The centralization of accounting, administrative and MIS responsibilities with headquarters also increases efficiency. Each of these departments acts on behalf of all of the CFs in Puno and Juliaca, doing everything from producing loan contracts and payment schedules to purchasing materials. However, this is not the case for the new CFs in Tacna, which requires a regional office to perform these functions. PMP’s overall efficiency may be reduced when these operations are fully integrated into operational evaluations. In terms of financial performance, PMP’s low delinquency results in low loan loss provision expenses and higher operational self￾sufficiency. How effectively is PMP leveraging resources within the community to achieve the greatest impact to its programs? In addition to partnerships providing integrated health, legal and business services described above, PMP has also leveraged local financial resources. PMP has a signed agreement with 24 FONDOEMPLEO and FONCODES. FONDOEMPLEO is providing a combination interest-free loan and grant for approximately $340,000. FONCODES has provided a loan of approximately $80,000 to capitalize the loan portfolio, requiring that 69% of the interest earned may be used for operating expenses and training, 20% must be used to capitalize the loan portfolio, and 11% is used for training activities managed by FONCODES. In addition, PMP has been effective in mobilizing member savings. However, since PMP is not allowed to intermediate these savings they remain in local banks. It is important to note that PMP client savings amount to nearly $750,000, which is a significant source of capital for local banks and savings and loans. 1.4.4.1.3 Advocacy: Determine if the program has engaged in policy or program advocacy. Describe the advocacy efforts and assess if they contributed to strengthening the policy environment or promoted the program in any way. PMP joined COPEME, a network organization for Peruvian MFIs, in 2000 as a member. PMP’s advocacy efforts to strengthen or change the policy environment have been limited to its participation in COPEME. Some issues addressed by COPEME include lobbying congress to lower taxes on microcredit NGOs (no result) and helping NGOs gain access to the national credit bureau, INFOCORP, at reduced rates (achieved). PMP also provides quarterly financial reports to COPEME which are used to maintain statistics on the microfinance industry. PMP also initiated the creation of a local credit bureau for itself and other microcredit NGOs. The management of the credit bureau is beyond PMP’s capacity and there has been a lack of participation by others. On PMP’s behalf, COPEME is discussing having INFOCORP take over the service. How does PMP encourage women to be advocates for better health services (advocacy through action)? PMP is very active at the local level as an advocate for women’s health. They have participated in at least three public campaigns (prevent violence against women, children’s health, and the crusade against poverty), acting as a partner with the local government and other NGOs to offer services and information on specific issues. In addition, it uses the CFs as promotion centers for women’s health. PMP also sponsors a daily radio program that deals with health, business and women’s issues. The evaluators heard a PMP radio discussion related to violence against women while in Juliaca and watched a video developed to educate women about menopause. 1.4.4.1.4 Conflict Mitigation/Prevention: This is not an issue that was addressed by the PMP as it does not operate in conflict situations. 1.4.4.1.5 Sustainability and Scale-Up: 25 Assess program sustainability and scale-up plans of operations PMP has grown from 0 to 15,000 clients in three years, all of whom are women. As a replication, PMP has proven that the PM network has the ability to achieve size and self-sufficiency quickly using slightly modified operating systems. By the end of 2002, PMP was achieving financial self-sufficiency on a monthly basis. The challenge for PMP is to maintain its growth and profitability going forward. The experience of other microfinance NGOs, particularly those which are methodology driven, is that growth overwhelms administrative systems and that management lacks a growth strategy after initial markets are saturated. PMP’s systems are already under strain. The MIS system is not capable of managing the volume of AC loans or the addition of new products. The accounting and administrative systems, which are centralized in the headquarters, were not set up to manage a decentralized network of CFs like those being created in Tacna. In Puno, the market is approaching saturation. The market is also getting crowded in Juliaca. PMP is already dependent on the new market, Tacna, for growth. PMP is very clear that it intends on staying with its target clients and is concerned that adding individual loan products would attract less poor clients. Therefore, it is not clear what PMP will do in Juliaca and Puno once its current markets are saturated with the group lending product. The strategic plan does focus on geographic expansion, selecting Cuzco and Arequipa as the next target markets. But it is less clear on how additional or improved products (e.g. individual loans) or increased services could be used to further penetrate existing markets. PMP estimates that it has sufficient funding for an additional year of growth. It is not clear how PMP will fund portfolio expansion beyond the current year. Fortunately, PMP already has access to funds through PMI ($625,000) and the Inter-American Development Bank ($500,000). Already, delays in the disbursement of a negotiated IDB loan are forcing PMP to choose between funding its Tacna expansion and funding additional growth in Juliaca. Progress program is making toward diversifying resources, for example building alliances with in-country business organizations. See Section 1.5.3.1 Financial Management 1.5 Program Management 1.5.1 Management Approach 1.5.1.1 PMP Management Approach In the context of the program model and the changing country situation, is the overall approach to program management flexible, appropriate and adequate? PMP’s management of its credit operations is quite successful. PMI placed an expatriate as National Director for a two year period to help establish and grow PMP. The National Director trained her 26 replacement, the Peruvian Finance Director, who was well versed in all aspects of the business model by the time of the transition. According to the by-laws, the current National Director reports to the BOD. Practically, she reports to the Executive Director of PMI. The organizational structure of PMP contains many levels. There are technically five levels between the Credit Assistants and the National Director. Three of these levels exist in the CF (Manager, Promoter, and Assistant). However, these three persons function more as a team than as a management structure. Each of them has ACs as clients. Technically, the CF Managers report to regional Supervisors who in turn report to the Finance Director, a position that was vacant for over a year. This violates the appropriate segregation of duties since full responsibility for approving, disbursing and collecting loans falls within one department. This line of reporting was established by the previous National Director who relied on the local Finance Director to directly manage local staff. Administration and the MIS Department also report to the Finance Director. In the case of Tacna, all staff (accounting and credit) report to the regional Supervisor. Technically, staff involved in non-credit services report both to the Manager of the respective CF as well as the Coordinator of those services. PMP will need to reconsider its current organizational structure as it grows, segregating the credit/operations department from the Finance Department and determining if Regional Supervisors are to have executive authority (e.g. report directly to the National Director) or shall be considered credit/operations staff. The level of authority of the Regional Supervisors should reflect their primary functions, which at present are related to credit. Is the program cost effective and timely in converting inputs into outputs and outcomes? As discussed throughout this report, PMP has been very cost effective and timely in executing its credit program. PMP’s lending costs are only 7¢ per dollar lent, while its earnings are nearly 36¢ per dollar lent.15 1.5.2.1 PMP Quality and Status of Detailed Implementation Plan (DIP) Discuss the quality of the DIP, i.e. the clarity and adequacy of the objectives, indicators, baseline studies and activities. Although the DIP contains some key information, such as growth, outreach and self-sufficiency indicators, it is a brief document which was not sufficient to guide the development of PMP. The section for PMP is a single page and the objectives are very general. For Objective 1, the DIP indicators and activities were very similar (e.g. an established BOD was an indicator and establishing a BOD was an activity), resulting in much redundancy. The indicators related to establishing a local MFI primarily focused on the official legal and governance status, rather than on program requirements of a 15 Calculated from the audited financial statements for 2002 and Datos Preliminarios del Pais for 2002, Administrative & Personnel Costs/Value of Loans Disbursed. 27 fully operational MFI. As an example, PMP has fulfilled its objectives and activities under Objective 1, Establish a local MFI, even though the BOD is not fully functional and PMI does not have a clear policy on local governance. Objective 2, Implement a Credit Program, contains measurable quantitative objectives, but the described activities don’t include anything other than establishing a basic village banking model. Comment on the utility of the DIP as a management tool for PM, their partners. The DIP was instrumental in starting the planning process in Peru for PMI. However, it was quickly superceded by the momentum generated by the rapid growth of lending activities. By the end of the first year, it was clear that it would surpass the MG targets making the DIP less relevant. Other than the budget, PMP no longer uses the DIP. The current National Director commented that she does not have a copy of it. With the assistance of a consultant, PMP conducted a strategic planning workshop which resulted in a four-year strategic plan. The plan includes background, vision, mission, as well as an analysis of both the internal and external conditions (SWOT). It contains three strategic objectives, including: (1) satisfying the needs of female entrepreneurs through microfinance services in the regions of Puno, Tacna, and Cuzco, (2) consolidating the provision of integrated development services and ensuring permanence through improving the methodology, coverage, and impact on members, and (3) seeking a comparative advantage through partnerships with local and national associations. Each objective has a number of operational objectives. The plan contains both qualitative and quantitative targets and represents best practices. However, it does not include financial projections or identify the resources required to achieve the stated objectives. PMP has an operational plan that consists of planned activities for each area and shares the targets of the strategic plan. It is structured by department, specifies some targets, and assigns a responsible person. It contains goals for activities and some financial and outreach targets. There is no narrative, but it appears that the management monitors the implementation of this plan. The plan is completed at the end of the year, several months after the completion of the annual budget. This reflects PMP’s need to complete a budget for donor funds according to the budgeting schedule of PMI. Neither plan has been approved by the BOD, primarily because the BOD is still determining its appropriate roles. The plan is supervised by the National Director, and is generally known and respected by the staff. 1.5.3.1 PMP Financial Management Analyze accounting and internal control systems PMP’s financial management is dominated by two factors, namely (1) an outdated MIS system from PMB, and (2) donor requirements. This has led to a system that requires significant manual intervention and over a dozen bank accounts to keep funds segregated. 28 Focal Center At the CF level, accounting is limited to managing the payments and savings of the ACs. Other than petty cash, expense reports, and purchase requests, CFs do not perform any accounting duties or financial reporting. The primary financial operations of the CF, namely the disbursement and collection of loans, are facilitated by the headquarters accounting department and managed by the ACs themselves. The accounting department issues a check in the name of the AC Treasurer. The Treasurer cashes the check at the local bank and, with the oversight of a Promoter and the AC committee, distributes the cash to members. Similarly, payments and savings are collected by the AC, and deposited in the bank by the AC Treasurer. Each AC maintains its own bank account for the collective savings of its members. With very few exceptions, CF staff does not handle cash transactions.16 To control the actions of the AC committee, the CF manager is a co-signer on the AC bank account and the AC Treasurer is required to provide a deposit receipt to the CF on the same day. AC operations are monitored by Promoters and CF Managers. Each meeting is observed by the Promoter and forms are used to verify payments and savings. CF managers accompany AC Treasurers to the bank when checks are being cashed. CF managers also review AC records at the beginning and end of each cycle. As an additional control, PMP has recently created the position of Substitute Promoter. This Substitute Promoter takes the place of an absent Promoter and reports directly to the Credit Supervisor. This permits the discovery of any inappropriate actions by a Promoter. In addition, Promoters are rotated on a regular basis (6 months – 18 months) to prevent any collusion within a single CF. MIS Department The MIS Department is responsible for all data entry, reporting and management of the loan portfolio management system. Operators input client and AC information, set up new loans, produce loan agreements and payment schedules, input payment information, and produce amortization reports. In most cases, data is input from original documents such as loan applications, payment reports, and bank deposit receipts. MIS reports to the Finance Director. Credit PMP credit approval process is driven by the credit policies. ACs may approve credits for clients within the limits of these policies. AC loans must also be approved by the Promoter and the CF Manager and are occasionally modified if the AC is not following the policies. The loans are also reviewed and approved by the Regional Supervisor and Finance Manager prior to disbursement. Credit files, including original contracts, are maintained by the CF and are accessible to all staff. Finance and Accounting 16 The major exception is in the case of no-shows. If a client does not appear to receive her loan, her loan amount is given to the CF Manager who deposits it in the bank. The CF Manager is required to provide the receipt for this deposit to Accounting on the same day. 29 Financial and accounting operations are the responsibility of the Finance Director, a position which was vacant until recently. PMP has an accounting manual which outlines the major procedures for bank accounts, funds, checks, purchases, petty cash, and safekeeping of documents. In addition, the National Director indicated that PMP has instituted policies to minimize foreign exchange risk and asset liability management.17 Cash needs are forecast monthly based on forecasts provided by the CFs, and cash is moved between accounts on a weekly basis. Accounting records are maintained in the Accounting and Administration office and are accessible to all accounting staff. Accounting is responsible for recording all non-credit transactions in the general ledger, preparing payments, managing bank accounts, managing donor funds, calculating and paying taxes, and producing financial reports. For the most part, Accounting inputs transactions based on original documents and completes reconciliations. Issuing checks and payment orders is done by the Office Administrator.18 At present, the Office Administrator maintains the checks and the only authorized signatory is the National Director. PMP maintains insurance for vehicles and fixed assets. It also has liability insurance for death and disability for employees while traveling. It maintains vehicle insurance ($9,700) and insurance against loss of cash ($1,500). It maintains other insurances such as: fire insurance for US$5,000 and insurance against electronic equipment damage for US$33,461. It has established policies for provisions, setting aside 3% of the gross loan portfolio as an allowance for potential losses. As PMP has no delinquent loans, the 3% is very conservative. PMP also has a reserve policy for losses due to foreign currency fluctuations as some of its funding liabilities are denominated in US$. The reserve is a non-cash reserve that is created on the basis of the foreign exchange gap (the difference between US$ assets and US$ liabilities). What provisions were made in response to the analysis that was done/recommendations made by Mennonite Economic Development Associates (MEDA)? In 2000, MEDA Consulting conducted an internal control evaluation. PMP has adapted many of the recommendations outlined in MEDA’s report. Per the scope of work, the evaluation team noted that the following important issues have not been addressed. · The BOD does not yet effectively serve as a governing body of PMP · There is no manual or guide for BOD members outlining expectations and functions. · Financial indicators (CCREP reports) are still prepared quarterly, rather than monthly. · The performance appraisal system still does not include work plans with measurable objectives to allow for objective evaluation. · Salary ranges have not been established for positions, rather PMP still relies on the MG budget. 17 While not documented, PMI requires that PMP maintain both US$ and PS accounts, converting funds to Soles only when they are needed. In addition, PMP provides for foreign currency mismatches (since it has US$ liabilities) by taking a monthly charge equal to the spread between US$ and PS rates and the amount of the exposure. 18 This as well as occasional lack of segregation of duties was noted in the internal control report issued by MEDA. 30 · The selection and installation of the new MIS has not commenced. As a result accounting procedures have not been revised. · There is still no 2nd signature requirement for checks and transfers, nor have authorization levels been established within the organization. With the opening of the CFs in the Tacna region, PMP now faces a new challenge of maintaining control over a distant region. Some of the procedures used to monitor Tacna operations are: · Monthly visit to Tacna CFs by either the Finance Director or National Director. · Transfers are limited to those needed for loan disbursements. · Bank statements for the Tacna accounts are sent to the headquarters · Headquarters has on-line access to view Tacna accounts. Is PMP leveraging additional resources (beyond the match)? In addition to program income which has exceeded $3.6mm since the inception of the program, PMP has sought other funds from local and international sources. According to the 2001 and 2002 audited financial statements, PMP received the following: 2000 · Private Individuals $20,081 2001 · Gems of Hope $43,690 · Summit Foundation $24,500 · PMI Loan $205,000 · FONCODES Loan $73,530 2002 · Gems of Hope $43,295 · Summit Foundation $30,686 · FONDOEMPLEO Grant $10,004 · FONDOEMPLEO Loan $27,065 · PMI Loan $625,000 · FONCODES Loan $79,821 1.5.4.1 PMP Monitoring and Evaluation (M&E) System Does the M&E system supply accurate, reliable and timely performance data? 31 PMP produces monthly financial statements, usually within 15 days of the close of the month. The system allows them to produce these statements in soles and dollars. Management uses this data to produce monthly variance reports, comparing actual performance (loans disbursed, loans outstanding, active clients, active ACs, etc.) to the budget and operating plan. Accounting also produces a monthly variance report. On a quarterly basis, this data is used to complete a CCREP19 report, which is submitted to PMI. PMI analyzes the data and sends analysis and comments back to PMP. The MEDA Internal Controls Report recommended the monthly production of the CCREP, however management does not feel that one month is a sufficient time period for the indicators to reveal a trend and prefer the quarterly reports. The data is mostly accurate. Limitations of the MIS system (see below) require data to be exported or extracted from several databases and then combined using Excel spreadsheets. Whenever significant manual intervention is involved, there is a greater risk of error. Assess if the partner organizations have increased their capacity to monitor and evaluate their work, document program achievements, and use data for decision-making and program advocacy. The reporting systems are based on those used elsewhere in the PM Network. PMP had several technical assistance visits related to performance monitoring and producing financial reports with PMI’s Financial Advisor. While PMP staff does not have an in-depth knowledge of financial and trend analysis, they clearly understand the implications of changes in indicators. PMP has created several reports locally and utilizes the PM approved CCREP report to monitor financial performance. Has the program undertaken to date, any special studies to assess program operations or impact? Comment on the quality and utility of these studies, in particular the impact evaluation funded through Summit and Ford. PMP has undertaken an extensive impact study. A baseline study of clients and non-clients was completed in 2001. PMP is currently in the process of updating the study using the same sample. Of the 128 families initially interviewed, PMP has located and is interviewing 111 of them two years later. The survey consists of three primary instruments: a client questionnaire, a spouse questionnaire, and a non-client questionnaire. PMP is working with a local University to conduct these surveys outside of the CFs. Each interview lasts 45 minutes to 1 ½ hours and consists of questions relating to business, income, assets, living conditions, quality of life and other issues. A consultant, Claudia de la Quinta, assisted with the design of the study, which costs PMP approximately $15,000 per year. While there are no written results, preliminary findings are shared during the weekly meeting of promoters. Some of 19 CCREP stands for: Calidad de Cartera (portfolio quality); Crecimiento (growth); Rentabilidad (profitability); Estructura Financiera (financial strucuture); and Productividad (productivity). 32 the findings were discussed above in Lessons Learned and Observations. In addition to the impact study, PMP and PMB are part of the Practitioner’s Learning Program (PLP) on Client Assessment. This program is funded by USAID Microenterprise Development Office and managed by the SEEP Network. It began with a workshop in October 2002 and will continue through 2004. PMP received a grant of $11,475. With assistance from this program, PMP has developed several tools to incorporate client feedback in its regular operations and to use that feedback to modify its products and operations. To date, PMP has developed an exit questionnaire that is conducted by Promoters for all clients leaving PMP. It also conducts focus groups of both clients and spouses to evaluate the current product offerings and clients’ satisfaction levels. These are being implemented with the assistance of a volunteer, Lisa Sherk (from the US). Finally, active clients are also asked to complete a general survey. Using a sample of 800 every six months, the general information is used to help PMP maintain an up-to-date socioeconomic profile of its clients and understand their opinions and level of satisfaction with PMP services. Assess PMP and partner use of data to make management decisions. The evaluation team asked for specific examples on how the data learned from the impact evaluation and the early client surveys had led to changes in policies or products. Some specific examples included: · Reduction of the obligatory savings rate after the fourth cycle. · Less frequent rotation of promoters between CFs · Determination of the modules used in the business development program. The M&E effort is managed by a single coordinator. PMP admits that the process needs to be more integrated to make it useful for Promoters, showing the linkages between what the clients are demanding and their current socioeconomic situation. PMP also needs to transform the data into useful ideas for product modification and development, which is one of the goals of the PLP. At the same time, the results thus far have narrowed down the areas of investigation for PMP. 1.5.5.1 PMP Information Systems Has the program increased in-country partners’ access to information technologies? PMP started using two software packages developed for PMB by Roberto Cornejo. The MIS consists of general ledger software (GL) and a separate loan portfolio management software (LP) specifically designed for ACs. PMP received the software and initial training in 2000 and 2001. PMB continues to use this software but has been trying to implement a new MIS, SAF2000, since 2001. PMP does not have a contract or service agreement with PMB or Mr. Cornejo and does not receive regular service or consultations. Both systems operate in DOS, which limits their processing capability. Both systems operate in both local currency and dollars with the manual entry of the daily exchange 33 rate. The GL and LP are not integrated and cannot be merged. The GL does not effect dual entries or batch processing. It does not manage payroll, fixed assets, or checks. It is a very basic system, which requires manual entry of both sides of the transaction. It can produce income statements and balance sheets and it is designed for fund accounting, allowing the operator to track and report on transactions by funding sources. The LP is not multi-user and cannot be maintained on a network. It also does not support multiple products. As a result, PMP has been required to create separate databases for each region and each product (i.e. two databases for each region). Crashes are common and the MIS staff backs up frequently to avoid data loss. It does support savings accounts, produce legal documents and loan schedules, and maintain the names and identification numbers of all clients in ACs (segregated by solidarity group and AC). However printing such documents requires significant processing and locks out the operator until printing is complete. It does generate lists of delinquent loans but cannot categorize by age. It does produce portfolio reports and income reports, but testing has shown errors. Statistical reports are available, but when compared with manually calculated reports they reveal inconsistencies. Clients may not be eliminated from the system, leading to difficulties in calculating basic indicators such as number of clients and number of active borrowers. The MIS Department has developed and presently come to rely on a second tool, Socia ARV, which extracts data from the LP by register. This allows the MIS Department greater ability to access and sort data to produce reports. The MIS Department knows the system well and has taken a patchwork approach to maintaining the data. They are currently in the process of cleaning all past data and then saving the databases in anticipation of a new system. Everyone admits that the system is no longer adequate and is taking “defensive measures” to protect against significant loss. Neither system is secure. Computers and databases can be accessed by anyone that gains access to the MIS office. It does not have an audit function. 1.5.6.1 PMP Staffing and Supervision Do PM and partner organizations have an adequate number of staff with relevant expertise for supervising/backstopping the program? Staffing of PMP has been sufficient. PMP has recently added a human resource person to its administrative staff who is responsible for the oversight of hiring, orientation, and employee monitoring. It has developed an employee manual and has an established code of conduct (Reglamento de Trabajo). Recruiting and training procedures appear to be adequate. Employees are hired on a contract basis, for 6 months to one year. Each period, they undergo an evaluation before their contract is renewed. This allows PMP the flexibility to let employees go without having to fire them. There are no annual or period work plans, but employees know that they will be evaluated according to a list of 34 set criteria. Evaluations are not linked to salaries. There are no salary levels at PMP, rather salaries are determined by the MG budget with a guaranteed annual increase of 9% per year. There is no incentive system. Both staff and management were adamant that PMP did not wish to implement a system that gave incentives to individuals. PMP relies heavily on teamwork in the CFs and there is concern that an individual incentive system would reduce employees’ willingness to work as a team. PMP has a manual of all job descriptions, highlighting responsibilities and duties. Hiring of related persons is not allowed and PMP maintains insurances ($9,700) against theft. Employee morale appears to be high. Lack of training and the frequent rotation of staff between CFs were the only major concerns. PMI has provided supervision and backstopping throughout the MG. As discussed above, this consisted of an on-site expatriate, technical assistance visits to and other members of the PM network. Currently it consists of visits by PMI staff. The number of visits has been documented in the MG reports. 1.5.7.1 PMP Program Recommendations Client Level · Find rewards and incentives for the oldest, best clients. Client interviews revealed a quiet dissatisfaction with the lack or rewards or incentives for good clients. This is particularly important with the village banking methodology, as high performing experienced clients often shoulder the responsibility to train and monitor less experienced, less reliable new clients. Good clients are eligible for seasonal loans, which are highly valued by clients. At the same time, clients are aware that the seasonal loans are expensive.20 The MIS makes tracking individual credit history or providing individual incentives difficult. Still, PMP should determine which incentives it can offer its best clients (less frequent payments, reduced interest, gifts) that will show appreciation of their loyalty. · Train clients on competitive advantages of PMP. There is much confusion and disinformation over the terms, conditions and services of the competition. PMP may wish to develop a training session on Consumer Protection to help clients understand the differences (and advantages) of PMP relative to the competition. Materials from the SEEP Consumer Protection Task Force may be of assistance in this effort. Focal Center Level 20 Clients pay 5.35% declining balance per month. 35 · Secure original documents. Original loan documents and applications are kept at the CFs and are accessible to all staff. Best practices dictates that all originals of key documents (applications, approval, contracts, disbursement) should be maintained securely away from the credit staff. · Increase and systematize controls on Tacna CFs. Decentralization is one of the greatest challenges for any MFI, especially when distances are great. PMP should establish a set of regular daily, weekly, and monthly controls on the Tacna CFs that will discourage any fraudulent or inappropriate acts by staff. Controls should include limiting daily bank balances, daily monitoring of bank balances, limiting check writing authorities, limiting approval authorities, spot checks of CFs, and verification of sample ACs during meetings. · Add field audit to responsibility of floating Promoters. Floating promoters are key controls, as they fill in for absent promoters. PMP should consider using these floating promoters to do systematic audits of ACs during their meetings. When not acting as substitutes, these Promoters could attend AC meetings unannounced and review a set list of items to ensure that the ACs are being properly managed. This is particularly important for Tacna. These Promoters would provide written reports of findings. National Office Level · Review the lines of reporting and create controls through distinct reporting lines. As discussed above, it is inappropriate for all credit, MIS and accounting staff to report to a Finance Director. PM must decide if it wishes the Regional Supervisors to be the most senior credit managers or if they will have a central position of operations manager. In the case of Tacna, finance and MIS staff should not report to the local regional supervisor, rather they should report to PMP headquarters finance department. · Determine the role of the BOD and train them in their duties. PMI should determine the amount of power and responsibility it wishes to devolve to the PMP BOD and then train them appropriately. The clarification of roles and responsibilities should go beyond what is already established in the by-laws and the affiliate agreement (e.g. country director evaluations). In addition, the by-laws and affiliate agreement should be revised to ensure that they are consistent with the appropriate roles and responsibilities. Any decision should take into consideration the time requirements and ability of the chosen BOD members to perform those duties. · Make a decision on MIS. PMP feels unable to make a decision related to MIS because it believes that PMI prefers a network-wide package. At the same time, both PMB and PMM are testing different systems. PMP needs to act quickly to identify, test, and install a new MIS system. As this could take a year or more, it is important for PMI to provide the assistance necessary to choose new software before the end of the year. 36 · Plan and budget at the same time. PMP’s planning process is driven by the MG budget and PMI budget, both of which are done in advance of the PMP annual planning process. With the MG ending, PMP should first conduct its planning process and then develop the budget to support its objectives. It should also conduct planning prior to the PMI, rather than after, so its plan contributes to the PMI plan rather than follows it. As in all plans, it is likely that objectives will need to be revisited once budget realities are addresses. However, PMP should use its plan as a marketing tool it needs to secure the funding required to realize it. · Plan now for 2005 funding and conduct an analysis of becoming regulated. PMP has funds to support another year of growth. If it does not wish to lose momentum, it needs to identify a source of funding for expansion in 2005 as soon as possible. Given its proven ability to mobilize savings, PMP should consider conducting a cost-benefit analysis of becoming a cooperative or EDPYME that might enable it to intermediate savings for growth. · Utilize client savings as leverage in negotiating funding. As PMP has mobilized over $750,000 in savings, most of which will remain on deposit with local institutions, PMP should negotiate with the same local banks and cajas for funding. In exchange for continuing to utilize certain banks for AC savings, the bank may be willing to provide a lower interest rate. Alternatively, clients could be asked if they would be willing to allow their savings balance to guarantee PMP borrowing. Clients should be compensated by PMP for this and the risks clearly explained to them. The lending agreement should be either an overdraft facility or revolving credit line. Any lending agreement should include strict covenants that limit PMP’s use of borrowed funds for expanding the loan portfolio and require PMP to take immediate action to remedy any late payment on the bank credit line. · Look for more local options for technical assistance. The PM network is not large enough to provide for all of the needs of PMP. Indeed, there is already some concern that PMP is not receiving enough attention. PMP needs to actively seek other technical assistance to providers that can meet PMP’s needs on a timelier basis. In addition, PMI needs to clarify what role, if any, it wishes to have in the prioritization and selection of technical assistance. · Develop a long-term strategy for non-credit services. In the next two years, funding for PMP’s non-credit services will run out. While it is likely that these services will attract other sponsors, PMP should develop a long-term strategy for these services. This should be considered when investigating the option to become regulated and whether such services are best provided through a different legal entity. · Develop compensation policy and incentive program. PMP should develop salary levels commensurate with experience and responsibility for all positions. In addition, annual increases should consider not only the cost of living but also merit. While the evaluators respect the desire not to introduce individual incentives, it is recommended that PMP establish an incentive system linked to PMP’s objectives (self-sufficiency, growth, low delinquency) that can be used to 37 reward all employees. · Consider reducing the loan loss allowance required on current loans. 3% is fairly conservative for current loans. PMP may wish to consider 1-2% as a loan allowance for current loans. 38 2. Pro Mujer Bolivia 1.4.2.2 PMB Achievement of Objectives Program Objectives Strengths Weaknesses 1. Modify and test the new MIS, SAF2000 · SAF2000 software has the capability to manage internal accounts of PMB’s community banks (detailed savings and internal loans) · The software is well integrated between the loan tracking, savings and accounting modules · The software has a good set of parameters which gives users the capability to set up loan products · SAF2000 does not have the capability to consolidate information of decentralized, off￾line transactions, a necessary attribute · The technical service provided by SYSDE to PMB lacks organization and control · The software given to PMB has not proven to be reliable 2. Establish a local NGO · PMB is actively connected to the PMI BOD: the Executive Director is a member on the BOD while another member makes regular monthly visits to the program to support in audit and review of financial statements · Process of forming a local BOD for PMB has been halted due to recent incidences of Bolivian NGO BODs replacing institutional management with personal contacts. · PMB therefore needs more attention than that given by the visit from one Int’l BOD member; especially since PMB acts very independently from the rest of PM 3. Expand training and credit program · PMB has achieved its targets for community banks, with 2114 as of June 2003 · PMB has achieved its outstanding portfolio targets, reporting $4,521,205 as of June 2003 · PMB reported having reached financial self-sufficiency as of March 2003 (102.8%) · PMB’s increase in number of active borrowers was slower than anticipated, reporting 31,535 in December 2002 (target 35,000). However, as of June 2003, it has 34,973. · PMB has earned less income than projected 39 1.4.2.2.1 Changes made since the DIP No changes were made to the indicators. Note: While the term “client” was not clearly defined in Table 1 of the DIP, for the purpose of this report the term includes all program participants with a loan outstanding, excluding write-offs. Participants who maintain savings in their community banks but who do not have outstanding loans were not counted as active clients. 1.4.2.2.2 Results Objective 1: Modify and test new MIS, SAF2000 · Information needs assessed. MIS plan finalized. PMB utilized the document called “General and Specific Requirements for the implementation of the integrated Management Information System,” completed in November 30, 1998 as a foundation for its MIS search. This document was created jointly with the other PM offices existing at the time: The PMI office and PMN. The document registers important requirements for the system at the management level; however, it lacks a concrete description of the operational capacity required of the MIS by the institution. Based on this document, PMB created the implementation plan for its MIS, which was later to be replicated in other Latin American affiliates. · Existing software assessed After creating the above-mentioned requirements document, PMB contacted different software companies catering to MFIs. None of the companies contacted offered software designed to track internal account activity for community banks, a requirement of PM. The MFI evaluated three alternatives in 1999: SIEM of Guatemala, CALYX of Seattle, USA and SAF2000 of SYSDE in Costa Rica. · Decision to purchase or develop in-house system PMB selected SAF2000 in December 1999. SAF2000 was selected after consulting five Costa Rican companies on the quality of services received from the software provider, SYSDE Costa Rica. · MIS developed Between 2000 and 2003, PMB and SYSDE tried to successfully modify the SAF2000 software to meet PMB’s specific MIS needs, namely, the capability to manage the internal account activity at the community bank level. This process was scheduled for completion in June 2003; however, as of September 2003, PMB’s opinion is that the software still contains deficiencies which should be resolved by the software provider. As a result, PMB has decided 40 to abandon the use of this software. · MIS installed PMB installed the software in their regional office of El Alto, as of June 2003. The software was scheduled to be installed in the remaining offices after successful completion of parallel testing. · MIS tested The tests and customizations/adjustments to the software SAF2000 were conducted in three sessions: November 2001, August 2002 and August 2003. · Parallel systems run At the time of the MIS analysis, PMB was conducting parallel testing in the regional office of El Alto with loan information from two focal centers and the regional office accounting information. · MIS reliability confirmed The results of the parallel tests were evaluated at the close of operations, September 2003. At that time the system still proved to be unreliable, resulting in the abandonment of the software. · All data transferred to new MIS At the time this report was written, only the information of the regional office of El Alto had been transferred. This activity was pending until the successful conclusion of parallel tests. · Staff trained to use MIS Only the personnel of the regional office of El Alto were trained in the use of the new software. This activity was pending until the successful conclusion of parallel tests. · Only new MIS in use Abandoned · MIS installed in other countries Due to delays in the development of PMB’s MIS project, PMM began an independent process of developing its own MIS. At present, this software is in the trial stage. · Other country staff trained Abandoned · Adjustments made and MIS maintained Abandoned Activities The activities under this objective are to complete the above indicators. As PMB had not completed 41 Phase 2 of the plan, Phase 3 could not be implemented. Objective 2: Establish a local NGO Indicators · Local BOD established. The Country Director of PMB has decided against forming a local BOD in the immediate future. This decision was based in part on concerns that a local BOD might choose to replace current institutional management with personal contacts of BOD members. This fear is not unfounded, considering allegations that this has recently occurred in several Bolivian institutions with local BODs, including CRECER. Another reason for delaying formation of a local BOD is the current political instability in Bolivia: it is believed that having an international BOD may shield the institution from political pressures better than having a local one. For example: Reportedly, there has been some discussion in the government of placing local NGO money under government control. While PMB has not fully ruled out forming a local BOD, it has decided to postpone this activity until it may be done more securely. Foundation Established. This cannot be accomplished until a local BOD is formed. Activities: The activities under this objective were to achieve the target indicators. Objective 3: Expand training and credit program Indicators: Results are reported as of December 2002 and June 2003. · Number of Banks: 1,346. PMB reported 1,778 and 2,114 respectively. · Number of Clients: 35,000. PMB reported 31,535 and 34,973 respectively.21 · Gross Loan Portfolio: $3,400,00022 . PMB reported $4,387,825 and $4,547,992 respectively23 . · Income earned24: $1,600,000. PMB reported earned income of $1,329,612 and $848,389 respectively25 . · Operational Self-sufficiency26: 160%. PMB reported operational self-sufficiency of 112.8% 21 Additionally, Pro Mujer Bolivia maintained 10,074 and 11,006 active savers with no outstanding loan balance, respectively. 22 Gross loan portfolio refers to the outstanding principal balance of all outstanding loans, including current, and delinquent loans, but not loans that have been written off. 23 Using historical exchange rates of 7.734 and 7.876 Bolivianos per US$, respectively. 24 Earned Income refers to the interest earned on investments as well as interest collected on loan portfolio and all commissions and fees. 25 Using historical exchange rates of 7.734 and 7.876 Bolivianos per US$, respectively. 26 Pro Mujer Bolivia uses the following formula for calculating Operational self-sufficiency: Financial income/ (financial costs + loan loss expense + administrative costs) 42 and 124.2% respectively. Activities The activities under this objective were to achieve the target indicators. 1.4.3.2 PMB Cross-Cutting Issues 1.4.3.2.1 Partnerships: Assess the status and outcomes of the partnership of PMB with Pro Woman. Identify the key elements that contributed to success. The Director of PMB stated that it is clear that, as a part of the PM network, PMB has more possibilities than it would have on its own. The network provides PMB greater visibility and exposure while adding to the positive image of the institution. Additionally, this positive image and visibility acts as motivation for the programs within the network to succeed, as they realize that poor results will damage more than their individual program. The PMI BOD sends a member to PMB every month to go through the auditing process and to review the financial statements, a form of support which is very useful to the program. PMB provides technical assistance to other programs within the network for a fee. Identify local partners and cite changes in PMB’s institutional capacities that occurred as a result of the PM Network. Identify what activities were most instrumental in strengthening PMB. One PMB staff went to PMP to observe their seasonal loan product. This product has not yet been implemented at PMB; however, based on the results, PMB has hired a consultant to assist them in designing and implementing a similar product. 1.4.4.2 PMB Program Effectiveness Lessons Learned and Observations Identify major successes, challenges and constraints in achieving each objective. Include a chart that summarizes the program’s successes and weaknesses. See summary of the program’s DIP results status Annex B Successes · PMB was able to incorporate management of the community banks’ internal accounts into the MIS’ capabilities, including detailed tracking of savings and internal account loan transactions at the community bank level. This capability permits PMB to support each group in management of their 43 internal accounts; however the reliability of the functionality has not been adequate. 44 Challenges · Selecting a provider with no local physical presence in Bolivia created complications in interaction for PMB, which lead to severe delays in customizing and implementing the system. · The communications infrastructure in the geographic areas where PMB works is deficient. This aspect was not properly analyzed and considered at the beginning of the project by either the MFI or SYSDE. As a result, the SAF2000 system was implemented with serious limitations with respect to integration and consolidation of information in the regional offices and head office. Constraints · Even though the added capability to manage community bank internal accounts was an important achievement in the software modification, this function contains some errors which the provider was unable to correct. This caused PMB to lose confidence in the technical support provision capabilities of SYSDE. · PMB lacked sufficient technical and data processing personnel at the beginning of the Project and, as a result, was obligated to invest greater financial resources to contract external technical assistance in order to become better counterparts for the software provider. · PMB contracted SYSDE after conducting a client satisfaction survey of SYSDE clients and after verifying that the company would have local representation and technical support. A short time after the initiation of the project, SYSDE withdrew its local representation and never provided local technical service. Due to the absence of local technical service, PMB had to send their personnel and technical consultant to Costa Rica for more than 15 days in order to help SYSDE in the development of the product. · There was a lack of opportune training of technological personnel in the tools required to implement and operate the SAF2000 software. · PMB delayed making critical decisions, trusting that SYSDE would respond, despite repeated failure to attend to PMB requirements. · Selection of centrally-based software with on-line users was a constraint, as PMB needs a system that operates on and off-line simultaneously. Centralized, online software is not possible for all PMB’s offices (agencies and regional offices) given the lack of infrastructure in some cases and the elevated cost of connection in all cases. SYSDE, based on the flexibility of its product, claimed it had adequate software—in the end, it did not. · Excessive cost of licensing and tools for operating the SAF2000 software--it requires licensing for each region or agency operating in off-line mode. 45 Lessons Learned · If the requirements of the institution are too ambitious, the institution may get confused and find it difficult to communicate such requirements to a software provider. In the end it may be impossible to integrate the requirements into the software. · If the rights and responsibilities are not clearly established in writing between the MFI and the software provider, the goals may not be achieved. · To work remotely during the process of customizing the software is a complicated venture, one which, if it does not fail completely, will at least take too much time. · In the final two days of this evaluation, PMB made the decision to abandon the plans to work with SYSDE Costa Rica and not to utilize SAF2000 for its MIS needs. PMB cites the following reasons for its decision: o During the three years of development, modification, implementation and adjustment of the software to suit PMB, SYSDE has not given the necessary support required for an undertaking of this magnitude. o The remote software support that SYSDE provided in this project was too slow and too complicated to be successful, especially considering the type of changes required by PMB. The time required to make changes and the poor communication generated by using email have caused incalculable delays, making good communication between SYSDE and PMB impossible. o The lack of local technical support has caused the project to become too delayed. Furthermore, the current problems experience cannot be resolved in a timely manner, such that the software can function properly. o In making required changes, SYSDE did not rely on sufficient quality control, which resulted in further delays and increase in the demand for system error resolution. In many cases, changes realized by SYSDE generated errors in other processes. · Finally, PMB has learned that in order to achieve success, it is essential to: o Define from the beginning with utmost clarity as possible the functional requirements for the MIS, simplifying as much as possible the operating processes in order to cause minimal complication the processes of software, achieving positive results. o To define a formal plan of implementation with rights and responsibilities clearly established for all parties o To manage the project in agreement with the plan o To count on a local strategic technical partnership to provide local technical support. Discuss any unintended impacts to date - positive or negative. · (+) PMB personnel have achieved a deep knowledge of the organization’s financial services methodology. · (+) PMB is better prepared to face computerization projects in the future. · (+) Improvements in the MIS department have been achieved. The IT department has more personnel now and they are taking training in the new technologies they are using. · (+) Despite not being able to implement SAF2000, PMB utilizes an alternate system (preceding SAF2000,) which provides them information sufficient for making operational and management 46 decisions · (-) PMB attempted to carry out the implementation of SAF2000 without the input of the other PM programs. · (-) Lack of sufficiently budgeted resources in the initial definition of the project · (-) PMB and its Director set very high expectations which were not reached. · (-)PMB did not have sufficient documentation of the system requirements of PM programs in other countries (both in terms of the detail and completeness) when the software provider was contracted · (-) The PMB offices had insufficient technical support to successfully work remotely with the provider · (-) The attempt to successfully implement SAF2000 was not carried out adequately, principally by the software provider. This fact has generated an enormous erosion of confidence in the system within PMB and has jeopardized the Pro Mujer Network idea of consolidating MIS systems. This is demonstrated in the fact that PMM has been implementing an alternative software system because it did not have an alternative MIS to fall back on. Assess if the main strategies and activities employed to achieve results were effective. Identify any changes made during implementation. Have the strategies and activities been effective in achieving project targets? The strategies and activities defined at the beginning of the project were correct; however, in the administration of the project PMB waited too long to confront SYSDE for their long delays in meeting requirements and achieving goals. During the implementation of the project, PMB and SYSDE agreed to share the burden of resources required to customize the software. However, the details of the promised responsibilities and activities were never clearly defined in writing. As a result, neither party completed their obligations, while both parties complained of the other’s failure to complete their own obligations. The final result is that SAF2000 was not implemented at PMB to the satisfaction of the MFI and in-turn, the MFI chose to abandon the implementation and start over with a new MIS. 1.5.7.2 PMB MIS Recommendations · Reviewing the process of customizing and implementing the SAF2000 realized to date, especially considering the quality of technical service received until now; it is recommended that PMB look for an alternative MIS. (Already done.) · Once making this decision, PMB should begin by redefining or revising the functional requirements of the system. This revision should emphasize the simplification of requirements with respect to the management of the internal accounts of the community banks. It should also take into account the need to harmonize the MIS with those of other affiliates. The MIS need not be integrated between the affiliates; but it should allow for standard reporting and consolidation. · Establish and/or define a new plan of implementation of the MIS. Priority should be given to implementation of software that can be supported locally. 47 Discuss key success factors in PM’s achieved results and identify institutional systems that it needs to put in place to continue to expand (i.e., identify threats to continued growth); · The growth of the program might not be supported without consistent and efficient information: this could threaten the stability of the institution as it expands. 48 3. Pro Mujer Nicaragua 1.4.2.3 PMN Achievement of Objectives: Program Objectives Strengths Weaknesses 1. Establish a local MFI · PMN is in the process of forming a local BOD and has the support and experience of its PMI BOD to guide the process · According to the DIP, PMN was supposed to have fully formed the BOD and have registered as a local NGO by the end of the MG. · PMN is anticipating a difficult, bureaucratic process as it registers locally 2. Expand training and credit program · PMN has reached operational self sufficiency and is close to full financial self￾sufficiency · PMN operates in four departments in Nicaragua · PMN has a loan portfolio of over US $1,000,000 as of June 30,2003 · PMN has reached fewer clients than anticipated (11,096 vs. 12,400) · PMN has lagged behind targets in both operational self￾sufficiency (89.7% vs. 100%) and interest income earned ($414,908 vs. $470,000) 1.4.2.3.1 Changes made since the DIP No changes were made to the DIP. However, it is important to note that PMN made the decision to postpone establishment of a local MFI (Objective 1) until after the institution had attained operational self-sufficiency. This was communicated in the DIP… “PMN will develop a local advisory board that will become the BOD sometime during the next three years, once the program’s self-sufficiency has been ensured.” Note: While the term “client” was not clearly defined in Table 1 of the DIP, for the purpose of this report the term includes all program participants with a loan outstanding, excluding write-offs. Participants who maintain savings in their community banks but who do not have outstanding loans were not counted as active clients. 1.4.2.3.2 Achievement of Objectives Objective 1: Establish a local MFI Indicators To have a BOD, to have written by-laws, to gain legal status as a local NGO. 49 As mentioned above, these activities were postponed until PMN had achieved operational self￾sufficiency. The MFI reportedly achieved 116% operational self-sufficiency in March 2003 and has maintained levels greater than 100% since. As a result, it is now pursuing establishment of a local MFI. To have in place strategic and financial plans PMN has a five-year strategic plan covering 2002-2007. Creation of the plan was facilitated by an external consultant who met with all levels of the organization through workshops and conducted an evaluation of the institution in the process. The staff then used Microfin to create the plan. The plan includes outreach projections and projected financial statements for the five years covered. The plan was approved by the PM Executive Director and, while it has required some adjustments, serves as a useful management tool. In addition to the strategic plan, PMN utilizes annual operating plans derived from the strategic plan. In creating operational plans for subsequent years, the MFI conducts an annual review. Activities Identify and train BOD members PMN has begun contacting persons considered for potential membership on the local BOD. The institution is currently looking for representatives from the commercial banking sector, a member of the BID, people with high profiles, experts in MF and people without financial background. They are stressing that potential members not have political ties. PM’s Executive Director is advising the institution on potential BOD members. It is believed that there will be a minimum of five members on the BOD. In their contact with potential members, PMN has communicated the needs of the NGO, its need for a local BOD, as well as the responsibilities and authority of BOD members. While it is clear that some form of training will be given to the BOD members in their duties, there is no fixed plan for doing so yet. Form local BOD PMN management has been meeting with potential BOD members since September 2003. One activity conducted in these meetings is discussion of possibilities for the final legal status of the NGO. Write by-laws While the PMI BOD has bylaws, none have been created at this stage of development of the local BOD. However, it is likely that the PMP and PMM bylaws will serve as a starting point. Apply for legal status as a local NGO PMN is currently registered as an international, non profit NGO with a “perpetual number.” The application for local registration has not yet been conducted, as it is still unclear what will be the best legal structure as a local NGO. Carry out strategic and financial planning 50 See the indicator above. Objective 2: Expand training and credit program Indicators Results as of December 2002 and June 2003. · Number of Clients: 12,400. PMN reported 10,733 and 11,096 respectively. · Gross Loan Portfolio: $900,00027 . PMN reported $974,919 and $1,010,998 respectively.28 · Income earned29: $470,000. PMN reported $414,908 of earned income as of December 31, 2002 and $257,937 as of June 30, 2003.30 · Operational Self-sufficiency31: 100%. PMN reported 89.7% and 102.6% respectively. Activities · Recruit and organize 8 groups of 25-30 women each month. The average group size is 26, however, PMN has not met the target of 8 groups per month. · Provide pre-credit training to all groups. All groups must receive pre-credit training in order to receive a loan. PMN employs “group formers” whose responsibility is to train groups before loan disbursement. Their work is verified by the department supervisor, who is responsible for verifying that each group has been properly trained. Verification includes both oral and written exams. · Supervise repayment and savings on a weekly or bi-monthly basis. All groups are supervised by loan officers at bi-weekly payment meetings. 1.4.3.3 PMN Cross-Cutting Issues 1.4.3.3.1 Partnerships: PMN has received technical assistance from PMB. The PMI office also provides the MFI with TA and program supervision, particularly in the area of financial management and administration, which PMN considers to be of good quality. PMN has provided technical assistance to PMP in the start-up phase. The director of PMP visited PMN for a time to observe operations. The Director of PMN reported that she sees benefits to being a part of the greater network, including: interchange of experience between programs, increased visibility and access to donor funds, cost 27 Gross loan portfolio refers to the outstanding principal balance of all outstanding loans, including current, and delinquent loans, but not loans that have been written off. 28 Using historical exchange rates of 7.734 and 7.876 Cordobas per USD, respectively. 29 Income earned refers to Interest on investments, interest on loan portfolio, commissions and fees. 30 Using historical exc hange rates of 7.734 and 7.876 Cordobas per USD, respectively. 31 PMN uses the following formula for calculating Operational self-sufficiency: Financial income/ (financial costs + loan loss expense + administrative costs) 51 sharing for technical assistance and better program supervision. 1.4.4.3 PMN Lessons Learned and Observations Identify major successes, challenges and constraints in achieving each objective. Include a chart that summarizes the program’s successes and weaknesses. See summary of the program’s DIP results status Annex B Lessons Learned In creating the strategic plan, it is important to involve all levels of the institution in order to have buy-in and commitment to the plan. Constraints and Challenges In identifying and selecting local BOD members, PMN has observed that there is only a small pool of candidates with both sufficient experience and time to dedicate to the BOD. While they are only beginning the process of registering the institution as a local NGO, PMN foresees that this will be a difficult and bureaucratic process. 1.5.5.3 PMN Information Systems PMN utilizes the old PMB MIS. While the system provides the institution with sufficient information to track the portfolio and make management decisions, it has lead to problems of efficiency. The MFI must work off line and outside the system in some tasks. As the network hopes to consolidate the MIS systems of all countries, currently PMN is evaluating PMM’s system. To this end the institution has contracted an external consultant to evaluate the MIS. If they can determine that the PMM system will serve their needs, they will use it. If it will not, they will look for another system in coordination with the rest of PM. 52 4. Pro Mujer International 1.4.2.4 PMI Achievement of Objectives Program Objectives Strengths Weaknesses 1. Enhance PMI’s managerial and technical capacity to establish and support local autonomous affiliates and coordinate activities of the network. · Strong and active PMI BOD · Low-Cost ‘Support and Coordination Office’ · Motivated, talented, highly productive staff were recruited during the MG · Strong capacity to fundraise for the network · Strong grants management · Strong coordination of network activities and development of PM shared policies and procedures is underway as of June 2003 with the hire of new program officer (Operations Officer) · Lack of clear and uniform roles, responsibilities, and reporting relationships between PMI and the country programs · Unclear long-term vision for the PM Network · Did not meet goal of creating all local autonomous programs · Little local BOD training · No standard Network policies, procedures, or manuals other than financial policies · Little documentation of lessons learned and program innovations 1.4.2.4.1 Changes made to the DIP A budget reallocation request, dated 23 Sept. 2002, was approved by Lori Pommerenke of USAID. This request reallocated $16,563 from consultants to travel for the headquarters portion of the program. 1.4.2.4.2 Results Objective 1. Enhance PMI’s managerial and technical capacity to establish and support local autonomous affiliates and coordinate activities of the network. PMI accomplishments under this objective were delayed by the lack of necessary PMI staff during the majority of the MG period. Important steps were taken during the second half of the MG period that will significantly improve PMI’s technical and managerial capacity to serve the network – mainly through the hiring of a full-time Financial Officer and full-time Operations Officer. The current PMI office staff and BOD are talented and present a capable and motivated team for going forward. In addition, PMI recognizes the areas where there are shortcomings and has implemented appropriate projects to address these needs. In particular, they are in the process of recruiting technical advisors to service the network needs in MIS and internal audit functions. 53 An important area PMI has yet to address is the lack of documentation of policies and procedures for the network other than financial policies. A systematic effort to define and document the PM guidelines and policies for operations (e.g credit, accounting, human resources) is critical for implementing new programs and facilitating PMI oversight and support of field programs. The accreditation system will be a way to develop common standards to which PM programs will be accountable to and this is an effort underway. a. Supervise and Support Country Programs: Indicators · Five year strategic and financial plans. PMI has a five-year strategic and financial plan that was written in October 2002 by the PMI office without the opportunity for input or participation by the country programs. It serves primarily as a marketing document to describe program goals and objectives, but does not contain the strategies and implementation steps that comprise a true strategic plan. It does not provide internal programmatic, planning or decision-making guidance at the network and country levels. · Annual Plans and Annual Budgets. PM does not have a policy that requires annual plans, although most of the country programs create them as part of their annual budgeting process. Country programs submit annual plans and budgets to either the PMI office, the PMI BOD, and/or to their local BOD. However, the country programs have not been trained in producing valid and coordinated plans and budgets, as evidenced by submittals of annual budgets prior to completion of annual operating plans. PM lacks a coordinated process for developing annual plans and budgets that are consistent across the network and which reflect the PMI strategic plan. · Quarterly Program Reports. Every quarter, each country director is responsible for submitting a detailed program report to the PMI office and PMI BOD. The standardized reports are comprehensive and require the following information: o analysis of the political and economic environment; o results reporting on financial and non-financial services; o description of activities; o analysis of challenges and institutional opportunities; o goals for the year. The reports are well done and contain much of the information that should be documented for lateral learning within the network. · Quarterly Financial Reports. Every quarter, country programs submit their financial reports to the PMI financial officer in the form of a CCREP report that conforms to best practices, SEEP financial ratios, and generally accepted accounting principles. The financial officer reviews the reports on a quarterly basis to monitor performance and to highlight any red flags. Also on a quarterly basis, the 54 financial officer consolidates these reports and produces a summary report that is reviewed by the finance committee of the PMI BOD. The PMI BOD works with the PMI financial officer to monitor the financial information closely. In addition, the country programs submit their financial statements and variance reports on a monthly basis. · Annual Program Evaluations. Annual program evaluations are carried out by the PMI Executive Director and financial officer. This is not a formalized or documented process. It consists primarily of an annual visit to each program by the Executive Director and by the financial officer (previously performed by the financial advisor). Each programs’ operations are evaluated during the visit. Reports of the Executive Director’s findings and observations are written and submitted to the PMI BOD and discussed with the country staff and board. · Loan Resources Identified. The PMI office helps country programs to gather resources to meet the needs of their programs. PMI far exceeded its MG match requirement for the loan fund of $300,000, raising a total of $1.2 million in donations and loans during the MG period. $625,000 has been lent to the PMP to fund micro lending. In addition, Unitus has issued a $1.5 million line of credit for PMM, which has not been accessed yet. · Annual Impact Data Collected. All of the country programs are encouraged to collect impact data on their clients and programs. However, there is no systematized collection or reporting process. Several formal one-off impact evaluations have been carried out. Available impact information is included in the quarterly reports to the PMI BOD. PM has several current research initiatives (with Imp-Act/Ford and SEEP/PLP32) to assess client satisfaction and monitor client information. PM is making significant progress and contributions to these efforts according to sources at Imp-Act and SEEP. Activities · Increase time of Financial Advisor. For the first year of the grant, PMI operations were run by the Executive Director and a part-time financial advisor. The two were responsible for all program start-up and monitoring. The part-time financial advisor was responsible for setting up the financial reporting system that is used as a monitoring system and training tool for country programs. She monitored the financial progress of the organizations. She also acted as the program advisor. After the successful establishment of PMP program, it became clear that a full-time financial officer would be needed. The financial advisor trained the current financial officer of PMI to take over the job on a full time basis in June 2001. Service to the country programs has dramatically increased as a result of having a full time person dedicated to the task. The financial officer redesigned the accounting and grants management 32 The SEEP Network manages a Practitioners’ Learning Program (PLP) grant program, which provides grants to a group of MFIs and/or MFI network organizations to work on similar projects. The grantees meet annually to compare results and share lessons learned. See www.seepnetwork.org for more information. 55 process – budgeting, reporting, budget management and compliance with donor’s rules, regulations, and requirements. She monitors the programs’ financial performance, reviews monthly and quarterly financial reports, and works with the Finance Committee of the PMI BOD to ensure program and financial oversight. In addition, she manages the PM loan fund. She closely interacts with the financial staff in the country programs in regards to: audit, financial reporting and analysis; strategic planning; financial projections and budgeting; coordinates the external audit process of PMI and country programs combined audit; and assists the Executive Director in the development of financial proposals for new grants and/or investments. Surveys from the September 2003 country directors’ conference noted a significant increase in the level of support and input they now receive from PMI. The financial advisor remains available to the network on an as-needed basis and is a member of the finance committee on the PMI BOD. · Hire Program Officer. A program officer was hired during the first year of the MG. It became clear during her year of employment that the position required stronger interpersonal skills and that PM really needed a strong full-time financial officer to take her place. She left the position in June 2001, and it remained unfilled until June 2003. However, the financial advisor fulfilled the role the program officer in the meantime as she transitioned her responsibilities to the financial officer. The new full time program officer has already made significant progress towards coordination of network activities including technical assistance and lateral learning. This program officer is a strong addition to the PMI office, and brings important experience to the network. She will also be more involved in the management of the network as reflected in the title from program officer to Operations Officer. · Provide program support and supervision. The PMI office in New York is a support and coordination office which fulfills five important functions: oversight of programs; coordination of technical assistance (identifying resources within the network for inter-program technical assistance); documenting and sharing program innovations; start-up of new programs; and fundraising. PMI’s primary emphasis has been providing up-front support to new programs and monitoring the activities of other country programs to provide support as needed. It is the intention of the new Operations Officer to make ongoing support a higher priority. · Provide financial management and training. The financial officer provides initial training to all new programs on the standard reporting formats and correct calculation of the required financial indicators. For established country programs, the financial officer will monitor performance indicators and reporting. If problems arise, the financial officer provides on-site technical assistance and close monitoring. For example, when PMN experienced financial difficulty during 2003, the financial officer traveled on-site three times to provide audit and recovery management assistance. · Develop DIP for each country. The PMI office worked with each of the country programs to develop a DIP for purposes of reporting to USAID. The DIPs were a helpful management tool for the country programs, particularly by providing measurable reporting targets. 56 · Midterm evaluation of grant program. PMI relied on annual external audits to evaluate performance of the country programs. A formal midterm evaluation of the MG was not carried out because the country programs were performing beyond expectations. In addition, USAID approved the MEDA internal control report of PMP to serve as the midterm evaluation. PMI provided written annual updates on progress to USAID. · Provide resources identified by affiliates. PMI’s loan fund resource was established in 2000 and is managed by the PMI office. The loan fund has two products: direct loans to country programs and loan guarantees to leverage locally available low-interest rate loans. PMI raised $1.2 million in donations and loans for the loan fund during the course of the MG period to meet the growing demand for loan capital by the country programs as well as a $1.5 million line of credit from Unitus to fund PMM. PMP drew $625,000 in the form of loans from the loan fund to finance its faster than anticipated growth. · Provide technical assistance identified by affiliates The PM network’s technical assistance model is for country programs to provide technical assistance to each other. The PMI office is not charged with providing technical assistance, but does provide support and coordination. The PMI office has developed a technical assistance listserve for the country programs, and helps to identify resources within the network to address the expressed needs. b. Grants Management Indicators · Loan Resources Identified. See above · Number of private and corporate partnerships From 2000-2003 PMI received grants from 30 organizations. 24 of the grants came from private foundations or corporate partnerships. (Note that this does not include additional grants from individuals.) · Number of proposals submitted and grants received. From 2000-2003, PMI submitted proposals to organizations to support non-financial and financial service activities of country programs, and general operating support (GOS). Of the 66 proposals written, 31 grants were received or pledged. Fourteen proposals for GOS are still outstanding and 38 letters of inquiry have been sent in 2003. The total amount raised/pledged for the MG period and future years at the time of this report was $4.1 million. In addition, as stated above, $2.7 million was raised for the loan fund including a $1.5 million line of credit from Unitus for PMM. c. Coordinate Network Activities Indicators 57 · Annual meeting of BOD and program directors. The PMI BOD meets quarterly. One of the four meetings is conducted in a program country in order to provide the PMI BOD members with direct program contact. Additionally, one annual meeting is held with all local BOD Presidents and the PMI BOD. Prior to 2003, the PMI BOD met twice a year and the country directors had their annual meeting for two to three days before the BOD meeting. The most successful country directors meeting took place in September 2003, which jumpstarted many of the current network coordination initiatives. The proceedings of this meeting are extremely well documented and were distributed to all the country programs for follow up purposes. · Annual country to country exchanges. Exchanges have occurred on an ad-hoc basis primarily related to technical assistance. The country programs do not currently have adequate information about each other. The PMI BOD and PMI office are well informed, but the information has not been communicated to the country level. The PMI office is currently developing an intranet system that will post an internal newsletter highlighting program developments, host the technical assistance listserve, and track ongoing discussions and developments based on the annual program directors meeting. PMI will also be starting an internal accreditation process for the country programs in 2004 to promote network standards and evaluate each institution based on these standards and on industry best practices. The accreditation team will include a staff member from another PM country program. This structure will help promote staff learning across the network. · Website and public relations materials developed. The website was updated in Summer 2003. · Number of publications- 0 · Newsletters – Two external newsletters per year have been produced and an internal newsletter will be developed; · Annual reports – annual reports are produced every two years; · Brochures – Two informational brochures were created - one on the loan fund and the other on PM in general; · Articles – Two articles were written – one for the National Council of Jewish Women Journal on Building Self-Sufficiency by Lynne Patterson (ED), and the other was a case study of PMB for the Micro Banking Bulletin by Nancy Natilson (Financial Advisor); · Studies- An EMDAP (Emerging Markets Development Advisor Program) advisor who spent a year in PMB wrote a case study titled “Controlling Delinquency in Microlending”; the study was published in Portraits of Business Practices in Emerging Markets, volume 3, editor Richard G. Linowes, 2002. · Annual Research and Evaluation Activities. PM is actively involved in two Practitioner Learning 58 Programs through the SEEP Network. The first incorporates PMP and PMB in research on Client Assessment. The second program involves all 4 country programs in a study of Efficiency and Human Resources Management starting in 2004. This study in particular will be critical for the PM network. These activities also promote country to country information exchanges, cross-network learning, and network organizational development. · Staff Development activities. Human capital in the field programs and the PMI office has been the key to PM’s success. Evaluations by MEDA (mid-term) and ACT (final) both point to highly committed and well-motivated staff as contributors to PM’s success. However, more can be done to invest in supporting and developing human resources. As noted in the MEDA evaluation, the single most important factor in the prevention of fraud is a well-motivated staff. Staff development at PM has only taken place on an ad-hoc basis, primarily in the form of participation in the annual SEEP and Microcredit Summit conferences. In addition, some staff have been trained in MICROFIN. · Dissemination of Results (conferences, workshops, papers). Internally: PM has not had an organized or effective process for sharing information. The new Operations Officer has initiated efforts to improve the networks’ dissemination capabilities. PMI BOD reports are being translated for the country programs. An extensive report from the Country Director’s meeting was distributed for program review and feedback. Also, a report summarizing the lessons learned from the recent PLP workshop on client assessment attended by PMP and PMB is being written and will be distributed. Externally: PM has shared results of its PLP research at SEEP meetings and has contributed sector learning about PMB in the Micro Banking Bulletin. PM has also reported externally to the MIX website and the Micro Banking Bulletin, and has participated in Microcredit Summit and SEEP panels. Activities · Coordinate network activities. See above · Disseminate program results. See above · Promote studies and research needed to improve programs. PMP and PMB participate in the PLP on Client Assessment. All four PM country programs will participate in the upcoming PLP on improving human resource management to increase the efficiency of programs. · Carry out strategic planning by BOD and staff. The current five-year PMI strategic plan was written in October 2002. However, it was developed by the PMI office without the opportunity for input or participation by the country programs. At the country level, PMP and PMN have developed strategic plans that are submitted to the PMI office for comments before being approved by the local and/or PMI BOD. PMB submits its plan directly to the PMI BOD. PMM has not submitted a strategic plan to the PMI office because it has not been required by their local BOD. To 59 improve strategic planning, the September 2003 country director’s conference included a discussion of strategic planning not only for their respective programs but for the network as a whole. The country directors are now developing a “bottom up” strategic planning process to propose to the PMI BOD. · Hold annual meetings for country directors. See above. The September 2003 meeting provided strategic analysis for network relations. · Sponsor program staff at conferences. All country directors attended the Microcredit Summit in 2001 and 2002 except for the PMB director. The Executive Director and financial officer attended the Microcredit Summit in 2002 as well. The financial officer, Executive Director and PMP director attended the SEEP Conference in 2002. Five staff members from the network have attended the MICROFIN training over the last three years. There is not, however, a policy for dissemination of information and learning from these events to the entire network · Develop guidelines for network relationships with affiliates. Although PMI has developed and distributed affiliation agreements, these agreements did not adequately clarify the roles at each level of the network. In addition, it is clear that in practice the relationship between PMI and each of the country programs varies greatly. This is most apparent in PMB, where the country director is also an Executive Director of PMI. This unique relationship is cause for concern among the other country programs. In particular, there is no clear consensus across the network about the roles and responsibilities of the PMI office, PMI BOD, local BODs, and Executive Directors. For example, since the Executive Directors also serve as PMI and local BOD members, there is confusion among country directors about whether they report to their local BODs or to the Executive Directors. Assess if the main strategies and activities employed to achieve results were effective. Identify any changes made during implementation. Have the strategies and activities been effective in achieving end of project targets? PMI chose to focus resources initially on the start-up and management of the country programs, and on financial and grants management. This strategy has resulted in excellent operating results at the country program level. The PMP start-up is performing far above the expectations outlined in the DIP. PMN, which had serious financial management difficulties at the start of the MG period, has shown significant improvement. The hiring of the full time financial officer has resulted in effective network-wide financial and grants management procedures. In addition to effective grants management, PMI met the MG requirement of $1.5 million. For the first two years of the grant period, PMI’s resources remained focused on the more immediate goals of the country programs, as well as financial, and grant management An attempt was made early in the MG period to fill the program officer position for supervision and support to country programs as 60 well as the coordination of network activities. As noted above, the first person hired for this task lacked the necessary skills. The new Program Officer has already set several processes in place (strategic planning, annual country director meetings, etc.) that are highly participatory and which will build a sense of cohesion within the network. However, the delay in hiring the correct person for this position has caused delays in the complete fulfillment of DIP goals and objectives. Did the PVO conduct a mid-term evaluation or data review? Was program implementation on￾track? Were mid-course corrections made and what are the effects of these changes? PMI relied on annual external audits to make sure they were on track in terms of financial performance. The results of these audits were a driver in maintaining emphasis on performance of the country programs and financial management. As described above, appropriate and successful actions were taken as a result of these audits. However, there was no formal midterm evaluation. In 2001 MEDA carried out an internal control external evaluation for PMP. USAID accepted this evaluation in lieu of a midterm evaluation. Also as described above, a lack of staff contributed to PMI’s difficulties in achieving the DIP indicators for network coordination. This problem was not addressed until after the mid-point of the MG period. With respect to MIS, PMI communicated to USAID that the progress was behind schedule and requested an extension to carry out the activities. A closer evaluation at the midterm of the project of the difficulties that PMB was facing with the software provider in repeated failures to complete deadlines and promises might have resulted in a different course of action and earlier retreat from the SAF2000 software. See the PMB section for details. 1.4.3.4 PMI Cross-cutting Issues Partnerships: It is important to clarify the structure of PMI to establish the correct context for the following comments. PMI is legally responsible for all of the country programs in the network. PMN and PMB are branches that report directly to the PMI BOD. PMP and PMM have local BODs, but they are each controlled by a two-person assembly, which has the right to remove or replace the local BOD. The assembly is made up of one member from the PMI BOD, and one person appointed by the PMI BOD. Thus, the ultimate control lays with the assembly, hence the PMI BOD, for the PMM and PMP programs as well. This relationship is not well understood by the country programs or by some BOD members. The PMM and PMP programs are not clear about the roles and responsibilities of the local BOD and reporting relationships overall. Assess the status and outcomes of PM’s partnerships with local organizations, NGOs, networks, businesses or intermediate support organizations: Assess the effectiveness of PM’s approach to building in-country partnerships. Identify the key elements that contributed to success. 61 PMI excels at building in-country partnerships and monitoring the quality and financial health of its partners. During the period of the MG, PMI established a new program in Peru and in Mexico and significantly improved the operating results of its other country programs, particularly PMN. PMI’s commitment to flexibility and innovation has created local programs uniquely suited to local conditions and which take advantage of local strengths. Country programs do not rely on PMI for direct technical assistance and/or management but forge their own way. For example, PMM was established with a very strong local BOD, while PMP has grown under the leadership of a strong country staff. PMP was also allowed the flexibility to drop the internal account and it started with an untested seasonal loan product which contributed to its success early on. A second key element for the success of the partnerships is PMI’s ability to raise funds for the loan capital needs of its country programs. PMI’s strong fundraising skills have allowed it to “keep its side of the bargain” as well as providing the flexibility for country programs to grow at the required rates suited to their local conditions. Identify local partners and cite changes in local partners’ institutional capacities that occurred as a result of the program partnerships. Identify what activities were most instrumental in strengthening local partners; PM’s model has been to start programs rather than to identify existing local partners. See section on PMP for description of the strong and effective in-country partnerships PM has formed for non-financial services. Cite changes in PM’s institutional capacities, as a result of learning from in-country partners; Although there has been a great amount of learning over the period of the MG, the challenge for PMI remains to document what they have learned through the establishment and growth of each country program. This will ensure that the learning is maintained within the organization and is available for the future and for sharing outside of the organization. It will also allow for better analysis of the strengths and weaknesses of the variety of structures and products within the network, which will deepen PM’s capacity to succeed going forward. Discuss if these partnerships are mutually satisfactory and beneficial. Identify and discuss major constraints to developing durable and productive partnerships; Each of the country program’s operations is running satisfactorily, with support and supervision from PMI. The network as a whole has been able to raise adequate loan capital and program support funds, partly as a result of strong performance at the country level. However, there have been two main constraints to creating even more beneficial and productive partnerships. The first major constraint is the lack of clear network/affiliate roles and responsibilities (see discussion under Objective 1, part c) Coordinate Network Activities). This is particularly challenging in the division of management and reporting responsibilities between country programs and PMI. PMP has a 62 talented but untrained local BOD, and looks to the PMI office for guidance and supervision. PMM has a very strong BOD, which has been supporting PMM staff from the beginning and wants more control to govern the country program. PMB and PMN do not have local BODs and are directly responsible to the PMI BOD. Further, PMB has a unique status within the network. PMB is considered the flagship program, and has the potential to provide leadership in both technical and organizational development for the network. Unfortunately, this potential has not been developed and PMB’s unique status has, in fact, begun to create tensions within the growing PM network. As a highly developed program, PMB does not perceive that it receives significant benefits from participation in the overall network. Therefore, the program is managed with a high degree of independence from the rest of the network. In addition, the dual role of the director of PMB as also the co-Executive Director for PMI has contributed to confusion throughout the network about appropriate management and reporting responsibilities. These differences in capacity and relationship to the PMI office and PMI BOD have limited the benefits and productivity of the network as a whole. The second major constraint is the lack of intra-network technical assistance and information sharing. This has limited the connections among country programs and the realization of potential benefits (learning, innovation, capacity building). This is described as an important part of the PM model, but resource limitations and the confusion over appropriate roles and responsibilities (described above) have delayed the implementation of these activities. PMB has had limited incentive to share materials, manuals, policies and procedures, or innovations with other network members. This has reduced the network benefits received by PMI start-up programs in other countries, which have had to develop these tools individually or from outdated materials. It has also limited capacity building throughout the network. Recent initiatives by the Operations Officer to coordinate and develop a formal and equitable system of direct country to country technical assistance, and the start-up of the intranet and listserve, will begin to address this constraint. Identify outcomes of program links with any coalitions, networks or associations; See PMP section below and discussion of PM’s active participation in the SEEP network PLP programs above. See the Partnerships Table in Annex C. New Tools, Guidance Or Standards: What changes to the methodology were made in PMP that were successful/unsuccessful which could be applied in other PM programs or other village banking programs? See PMP section for the majority of the analysis. Comments on the other programs are included below. 63 PMM works with a more geographically dispersed population, so many of the elements that worked well in PMP will not apply to PMM. The lessons learned in Mexico will help the network understand how to operate in areas that do not have the same population density as the other PM programs. How effective has the PM loan fund, managed out of headquarters, been in leveraging existing resources and funding the growth of PMP? The loan fund, established in 2001, has been highly effective for the PM network in providing needed resources to country programs. This was particularly the case for PMP to fuel its quick growth. There is $1.2 million in the loan fund and the fund has made direct loans to PMP totaling $625,000. The loan fund offers both direct loans and loan guarantees to attract commercial financial institutions as investors in each country. Going forward the loan fund will be used primarily to pursue guarantees with local banks. In addition, Unitus has committed a $1.5 million line of credit to PMM which has not been accessed yet. Direct loans to PMP were made for a period of 3-8 years, with a concessional interest rate (8% for PMP) set depending on the cost of funds and with a grace period for principal repayment. PM sets aside between 10% and 20% for reserves depending on the risk of the particular programs (10% for programs with an above-average track record, and 20% for start-up programs). The loan fund can also be used to cover short-term liquidity needs of the country programs. The loan fund is technically a line item on the PMI balance sheet. At present, this form is adequate for its stage of development and for the services it provides. The loan fund is managed by a ‘Loan Fund Committee’ on the PMI BOD which continues to investigate options for the ultimate form that the fund will take. The committee reviews all the loan fund proposals, analyzes the financial condition of the country programs applying, and monitors the activities of the fund. The loans are supplemented with technical assistance and training for staff in country programs as appropriate. If the loan fund continues to grow (PMI is hoping to raise $5 million for the fund by the end of 2004) the fund should be managed by a full time officer with investment and liquidity management experience, credit analysis experience, and foreign exchange experience to manage the excess liquidity available within the network across countries MIS Since the SAF2000 has not demonstrated the necessary reliability and capability, it will not be replicated in other countries. Advocacy: This is not an issue that was addressed by the PMI office. See PMP section. 64 Conflict Mitigation/Prevention: This is not an issue that was addressed by the PM MG program. The country programs do not operate in conflict situations. Sustainability and Scale-Up: The PM self-sufficiency model emphasizes a low-overhead support and coordination function at the PMI office and a reliance on resources within the network to provide technical assistance across the country programs. Payment for direct technical assistance from one country program to another is agreed to by the country programs and coordinated by the PMI office. Only if the required resources are not available within the network, external consultants will be hired to work on high priority areas (e.g. MIS). These consultants will be paid on a cost-share system by the country programs. The PMI office has successfully kept costs very low. They share office space with two other organizations and pay modest salaries. Currently country programs do not contribute to the costs of the PMI office. PMI is exploring an affiliate fee structure but has not come to a conclusion. Fundraising has provided a stable source of funds for the PMI office. Progress and potential for scale-up or replication; PMM was started in November 2001 with a strong local BOD that identified the local resources to begin operations. The program has the financial support of the Mexican federal and state governments as well as corporate sponsors such as Wal-Mart, Citibank and JP Morgan Chase. Unitus, a social investment company in the US, is also supporting the program. PMI is planning to expand in the next year to another country and is seeking the resources to do so. PMI projected that for the period, 2003 to 2006, they would need to raise the following sums: $16 million for the Loan Portfolio; $2 million for financial services; $3 million to cover the costs of non￾financial services; and $3 million to cover the costs of PMI. There is significant progress to be made in identifying loan portfolio resources. In addition to using the loan fund, the loan fund committee of the PMI BOD is exploring options with existing loan funds, such as Blue Orchard, to access resources for the network. Progress made by the program toward achieving sustainability and if PM is planning to measure post-grant sustainability; PMB and PMP are financially sustainable institutions. PMN is operationally self sufficient and close to full financial self-sufficiency. PMM is moving more slowly to financial self-sufficiency because of the disperse population density in the areas where it is operating, but has adequate funding to support its operations in the meantime. The PMI office is currently covering the costs of its operations through 65 fundraising. It is also investigating the possibility of creating an affiliate fee structure to partially cover the costs of the support office going forward. Progress program is making toward diversifying resources, for example building alliances with in-country business organizations. PMI is diversifying away from USAID funding, i.e., the start-up of PMM occurred with local funding. Further, PMI is trying to build the loan fund to service the loan capital requests of the country programs and is focusing on building relationships with local banks. 1.4.4.4 PMI Program Effectiveness Lessons Learned and Program Recommendations Lessons Learned · A coordination and support office can provide important services without having to be a large back office. The PMI office’s coordinating role takes advantage of resources available within the network, rather than duplicating efforts. PMI is effective at fundraising and providing support to start-ups. Its role should continue to be providing indirect technical assistance and coordinating direct technical assistance. · Allowing flexible program methodology leads to innovation. The PMI office needs to document the variations that have taken place across the network for program learning. Significant advances will be made now that the Operations Officer is in place for documenting network activities. Program Recommendations · Clarify network/affiliate relationships. There is a need to clarify the partnership structure within the network and the roles of PMI office, PMI BOD, local BODs, and Executive Directors. PMI should work in conjunction with the country programs in the strategic planning process to decide what organizational and institutional structure makes the most sense for the network. They should evaluate various existing models, such as Women’s World Banking (local autonomous organizations) or FINCA (tightly controlled affiliates). PMI’s current organizational form and management practices apply various elements of each of these structures in different ways across the country programs. The current legal structure of the organization is not well understood within the network. The decision regarding institutional structure should be made with input from the country programs to insure their buy-in. · Establish and document policies and procedures, incorporating network wide learning. A systematic effort to define and document the PM program is critical for standardizing existing programs, implementing new programs, and facilitating PMI oversight and support of field 66 programs. These policies and procedures should be based on a clear set of Guiding Principles, which describe the unique characteristics of the PM program. The Country Director’s meeting in September 2003 made a significant contribution to the creation of the Guiding Principles. These Guiding Principles and an examination of current field program practices should lead to the documentation of policies and procedures, including financial management, grant management, credit methodology, internal controls, and human resources. New country programs require a start-up set of standard manuals. New and existing country programs would benefit from an analysis of the variations of the PM program, and timely access to information about on-going field program improvements. Since the establishment of the original PM program in Bolivia, there have been many operational and managerial innovations. This knowledge is shared informally, but these variations should be documented and approved by PMI and communicated across the network to support organization-wide standardization and learning. · Staff Development Activities. As noted by country directors in a survey about PMI, the PMI office has seen a dramatic increase in productivity since the full time Financial Officer was hired. They anticipate similar results with the recent hire of the Operations Officer. Staff turnover and the vacancy in the program officer position had prevented the office from achieving many of the network activities outlined in the DIP. Continuity in these positions is critical to the development of the PM network and should be supported by investment in the skill development of key personnel. Although five percent of the value of an employees salary may be used for staff development activities it is not evident that staff plans have been put in place to promote this development. This can be encouraged by making it part of the performance evaluation process. The network does not have consistent systems for staff evaluations. PMB developed a human resources plan with Price Waterhouse Cooper, but the templates and policies have not been distributed across the network. In addition, the new PMI Operations Officer has made available examples of the staff evaluation templates used by CRS and MEDA. The development of human resource policies and procedures is crucial for all organizations, but it is invariably the last priority. Each MFI “reinvents the wheel” on this topic. There should be standard recommended templates that MFIs can adapt to their organization’s specific needs. The PMI office should evaluate the three models and develop a consistent system for the PM network. 1.5 Program Management 1.5.1.4 PMI Management Approach Is the overall approach to program management flexible, appropriate and adequate? 67 PMI has an active, involved and qualified PMI BOD that works in conjunction with the PMI office to manage the activities of the network. The PMI BOD contributes to the management of the network through analysis and monitoring by the following BOD committees: Executive Committee; Development and Public Relations committee; Finance committee; Loan Fund committee; Governance committee; Task Force on Next Country. As described above, the PMI office has five important functions: oversight of programs; coordination of technical assistance (identifying resources within the network for inter-program technical assistance); documenting and sharing program innovations; start-up of new programs; and fundraising. The current program management approach is to use the PMI office as an extension of the PMI BOD to guide the network activities. This is appropriate for the cases of PMB and PMN. However, for the Peru and Mexico programs, this approach can undermine the authority of the local BODs, since the country programs report to or receive direction from the PMI office instead of their local BOD. This conflict could be resolved if the PMI BOD worked through its representation on the local BODs and if necessary through the assembly of the local BODs. Is the program cost effective and timely in converting inputs into outputs and outcomes? As discussed above, the PMI office is cost effective in the way it is structured as a support and coordination office. It takes advantage of resources within the network to provide outputs and technical assistance on a timely basis. 1.5.2.4 PMI Quality and Status of Detailed Implementation Plan (DIP) Discuss the quality of the DIP, i.e. the clarity and adequacy of the objectives, indicators, baseline studies and activities. The DIP for PMI contained only one general objective. In addition to not having a program officer in place to accomplish many of the activities outlined in the DIP, the DIP did not appear to provide an adequate guide for the PMI activities. Many of the indicators were quantitative check list items, i.e., number of private and corporate partnerships and number of proposals and grants received, as opposed to requiring qualitative analysis of progress being made in strengthening the institutional capacity of the organization. Comment on the utility of the DIP as a management tool for PM, their partners and PVC. – Are HQ and country programs using the same thing? PMI used the DIP to manage the progress of the country programs against the numerical objectives. However, it was stated that PM wanted to establish all locally autonomous affiliates. This objective does not seem to fit in well with the current legal structure of PMI, nor did it prompt the organization to 68 develop guidelines to clarify country program roles and responsibilities Has the DIP been of value to PM or was the DIP an annual reporting instrument for PVC? PVC wants to know how to modify the DIP to make it more useful. PMI commented that the DIP was a helpful instrument to start the planning process with the country programs, and provided a guide to monitor the indicators for the country programs. However, the usefulness of the DIP for the country programs diminished as their performance exceeded DIP goals. On the other hand, the DIP objectives and indicators for the PMI office were too general to be useful. It would be very useful that PVC require the institution to carry out strategic planning, if it doesn’t already have a recent strategic plan, and to then derive the objectives for the DIP from the institution’s strategic plan to ensure that the DIP is consistent with the organization’s strategic vision. In order to make the DIP more useful, organizations should evaluate at the midterm whether the indicators they chose have remained relevant to their operations. If not, they should report to USAID and there should be a process for altering indicators at this point. Indicators are selected on a best￾guess basis during the proposal process, and might turn out to be less relevant as the MG is implemented. There should be a concerted effort midterm – by the grant partner and USAID – to discuss with grantees the relevance of the original DIP indicators and activities. 1.5.3.4 PMI Financial Management Analyze accounting and internal control systems; are adequate financial control systems in place? The internal control system for the PM network is weak. PMI has primarily relied on external audits for this oversight. The external auditors, KPMG, identified areas for improvement with the internal control systems at PMP and appropriate actions are being taken (described above). In addition, the finance committee of the PMI BOD has proposed a three tiered internal and external control process for PM based on the CGAP manual and the Microfinance Network manual on internal auditing. The PMI BOD has requested input on this proposal from KPMG and the Bank of Boston and will implement it once approved. There are very few documented policies and procedures in place to guide the management of the accounting function. The PMI office uses the Fund-EZ accounting software. There is an accounting manual which outlines in very basic terms the major procedures for entering monthly data in to the system such as cash disbursements, cash receipts, credit card expenses, payroll, depreciation, grants, prepaid expenses and summary financial reports. Bank reconciliation is handled on a monthly basis, and Transfers, Accounts Receivable, Accounts Payable and Contributions are reconciled only on a quarterly basis. More detailed procedures for accounting and grants management should be developed for the PMI office’s transactions with the country programs. While there is a password to enter the accounting program, all PMI staff are able to access the system. PMI should further restrict access to the system.. 69 In 2002, when the previous accountant left PMI, the financial analyst who had been with PMI for 6 months, first as a volunteer and then as a full-time employee, assumed the accounting function and has maintained this position for a year. She is the only person who carries out the accounting and who knows how to use the software. There is an urgent need to have a staff member trained in this function as a back-up. On a monthly basis, the accountant produces financial reports, reconciles bank statements and PM books, prepares and sends invoices to each country, and pays bills. On a quarterly basis, she does an analysis of cash flow, reconciles accounts receivable and accounts payable with every country, and prepares financial reports for donors. She is also responsible for payroll, taxes, health benefits, pension, and sending loan interest payments. The Executive Director and the Financial Officer are the authorized signatories in the PMI office. Is the PVO leveraging additional resources (beyond the match)? Yes. PMI was required under the MG to match $1.5 million. During the course of the MG period, PM was able to raise $2.7 million in loans and donations to the loan fund, including the $1.5 million line of credit from Unitus for PMM. The total donations pledged to PM from 2000-2003 has been $4.1 million. 1.5.4.4 PMI Monitoring and Evaluation (M&E) System Does the M&E system supply accurate, reliable and timely performance data? Much of the findings in this section are reflected in the analysis in the PMP section above. As noted in the MEDA Internal Controls Report, it was recommended that PMI receive financial data on a monthly basis instead of quarterly. PMI receives monthly financial statements and variance reports and quarterly indicator reports. The data is analyzed and summarized more completely on a quarterly basis to prepare the financial reporting for the finance committee of the PMI BOD. How effective are the steps (if any), taken to institutionalize M&E at PM? Has the program undertaken to date, any special studies to assess program operations or impact? Comment on the quality and utility of these studies, in particular the impact evaluation funded through Summit and Ford The requirement of monthly financial information is an important first step in more effectively monitoring the financial performance of the country programs. The financial officer is producing quality analysis in her quarterly reports to the PMI BOD. In addition, the finance committee is prompted by information in the reports to conduct one-off studies. For example, PMI conducted an analysis of PM interest rates and those of its competitors for review by the PMI BOD. 70 In addition, the up-coming network accreditation process will be an important step to institutionalize M&E within the network, not only from the PMI office to the country level, but also from country program to country program. The country programs will be accountable to the PMI office and PMI BOD as well as to each other through their responsibility to perform as part of the network. There is also very important work on M&E taking place in PMP and PMB programs, through the PLP program on client assessment and through the Imp-Act/Ford project in Peru. These advances are expected to be embraced at the network level. PMP is creating a process for collecting, monitoring and evaluating client information. This will lead to improved financial and social performance of the programs as it better understands the clients and can respond to their needs. The analysis conducted by PMP goes far beyond monitoring and evaluating financial performance indicators of the program and its clients, and has been important for the program’s excellent performance. The work in Peru is seen as a pilot for developing a framework that can be used across the network. It will then be the responsibility of the coordination and support office to work with PMP to incorporate that learning and process to the other country programs. Assess if the partner organizations have increased their capacity to monitor and evaluate their work, document program achievements, and use data for decision-making and program advocacy. The financial officer has worked closely with each of the country programs to make sure they understand and know how to use the CCREP reports. This is part of the financial training that PMI provides to each of the country programs. The reporting and analysis of data by each of the country programs has improved since PMN detected fraud in its program in 2000. There has been a systematic effort by the financial officer to stress the importance of quality analysis and monitoring. The assistance that the country programs receive from the financial officer is highly regarded according to surveys conducted during the September 2003 director’s conference. What more could be done to improve the M&E systems and use data for decision-making and program advocacy? Because of the existing MIS system, data for consolidated reports needs to be extracted from several databases into an Excel spreadsheet, which increases the chances for error. This should be considered in the assessment of PMI network MIS needs. It is preferable for the MIS system to upload financial inputs rather than requiring data export. In addition, the MIS system should be designed to incorporate the client monitoring data collected by PMP. Their experience has shown that monitoring client information is important for program performance, and the ideal MIS system would allow for the collection and integration of this data. 71 1.5.5.4 PMI Information Systems Has the program increased in-country partners’ access to information technologies? No, this goal was not realized during the execution of the program. What have been the major obstacles which have prevented PM from completing its objective under the MG to implement a new management information system for all of its programs? · The attempt to customize the software without clearly identifying the mutual responsibilities of the provider and client. · The attempt by the software provider to work remotely. This impeded SYSDE’s ability to verify the problems that PMB attempted to explain by telephone, email, etc. This was an obstacle throughout the period, despite the fact that SYSDE initially utilized local personnel and representation. PMB followed all the best practices recommendations for choosing and implementing a new MIS. What went wrong? · Inopportune direction and follow up on the project, placing excessive confidence in the software provider despite their repeated failure to meet deadlines. · Failure to complete promises on the part of the software provider · PMB did not incorporate assistance from PM programs in the other countries in coordination, direction or technology. · The provider’s initial bid did not present realistic total costs, and there was no contractual limitation on cost increases. · The needs assessment failed to incorporate operational requirements (see comments in PMB section). The document PMB utilized to identify the MIS system registered important requirements for the system at the management level; however, it lacked a concrete description of the operational capacity required of the MIS by the institution. What steps have been taken by PM and its partners to share program information and learning? Since the implementation of the new software is incomplete, PMB has not continued with the planned tasks associated with the replication and implementation of the SAF2000 software in other countries. 1.5.6.4 PMI Staffing and Supervision Does PMI and partner organizations have an adequate number of staff with relevant expertise for supervising/backstopping the program? 72 PM has exceptionally dedicated staff which has been fundamental to its success. The program has been built on mutual trust and respect, and this has served the organization well thus far. Difficulties in PMN with fraud, however, pointed to the necessity of shifting the culture from that of a ‘family owned business’ to that of an institution with organizational needs for formal supervision and backstopping. This need has become more urgent as the program has grown to a four-country network with future expansion plans. The MG came at a critical time to begin this institutionalization of PM. The process has only just begun, however, and it will be critical in the upcoming year to continue evaluating the personnel needs of the institution. The most important step in this institutional process will be identifying and/or developing strong leadership. The PMI BOD should evaluate the roles and performance of the Executive Directors on an annual basis33. As the institution’s needs change, it is important to review, and re-define if necessary, these roles of PMI’s top leadership. PMI must acknowledge that there is a need for both operational management and for long-term strategic vision and leadership. It is important to clarify who bears these responsibilities and to ensure that all parties have the necessary skills and experience. Although both of the Executive Directors have played critical roles in the growth of the overall organization, PM is evolving and so must the role and responsibilities of the Executive Directors. They are both deeply involved in daily management decisions, which has left little room to focus on the long-term vision and strategic planning for the organization. In addition, the reported reluctance of PMB to participate in information sharing and capacity building across the network demonstrates a conflict in one Executive Director’s commitment to the development of the overall network and the success of that individual program. The necessary leadership could take different forms, and PM must choose which makes the most sense for its development needs and institutional culture going forward. It is possible that the current Executive Directors can be trained and supported by the PMI BOD to accomplish the new roles required by a larger network, such as in strategic planning and leadership. This would be in keeping with PM’s commitment to investing in the people who have been so critical to the development of the organization. However, the changing structural and financial requirements of the institution might require the addition of management with different skill sets. Alternatively, PM might consider developing a core group of BOD members to work closely with management that helps build a clearer vision and coherent strategy for the network. In terms of the rest of the PMI staff, an excellent team is being built to carry out the program activities. The Operations Officer and the Financial Officer are critical additions to the PMI office which are a result of the MG program. A Development Officer has also been hired in the last year to help the Executive Director in fundraising duties, which will be critical in raising the needed resources to fuel growth. The office assistant is over qualified and contributes beyond the duties of her job. This is a benefit to the organization now, but could be a problem in the future in terms of her motivation to do 33 There is no formal process to evaluate the performance of the executive directors. This should be carried out on an annual basis like other performance reviews of Country Directors and staff. 73 administrative tasks. The missing pieces are in MIS and internal auditing. PMI recognizes and is addressing these two critical areas where improvement is needed to supervise/backstop the programs. An analysis of network needs is currently being evaluated in both of these areas, and a strategy will be implemented to address the needs of the network as a whole instead of by individual country. In MIS, a terms of reference to identify an MIS consultant for both short- and long-term needs was written and reviewed by all of the country directors. Several candidates had been considered at the time of the evaluation. In November 2003, PMI hired Raul Tapia, a Bolivian MIS expert for this position. In addition, as described above, an internal auditing system is being developed by the country directors with input from KPMG and the Bank of Boston. Given PM’s desire to continue to support the growth of its programs in Latin America, is its current organizational structure appropriate, especially in light of PM’s goal to keep a lean infrastructure in headquarters (so that most of the resources go to the field)? What additional staff is necessary at headquarters to provide adequate support? In order to give more room to strategic planning and vision for the network, the Executive Directors need to spend less time on day to day management of the programs. Under the current structure, this would shift too much management responsibility to the Operations and Financial Officers and they would not be able to focus on their important deliverables/outputs for the network. It would be ideal, therefore, to have a Chief Operating Officer to handle the overall management of the partner programs and the PMI office. As noted above, the MIS and internal audit functions are also needed to backstop the network. Finally, if the Loan Fund continues to grow it will require a separate person with specific skills, as mentioned above, to manage it. 1.5.7 PVC Program Management ß Assess PVC’s oversight and backstopping of the cooperative agreement. PVC program management has primarily been conducted by review of annual reports and periodic meetings between the PVC Program Manager and the PMI office. The PVC Program Manager visited the PMI office in 2002. PMI found the USAID reporting requirements to be somewhat unclear in terms of the people to whom reports needed to be submitted. PM also pointed out that it would have been useful to have training or guidance in creating the DIP in order to prevent having objectives that were too vague. 74 1.5.8.4 PMI Program Management Recommendations 1. Define the management roles of PMI office, PMI BOD, local BODs, and Executive Directors. There is no clear consensus across the network about these roles. In particular, since the Executive Directors also serve as PMI and local BOD members, there is confusion among some BOD members, country directors, and PMI staff as to whom country directors are ultimately responsible for reporting. PM should evaluate various existing network models, such as Women’s World Banking (local autonomous organizations) or FINCA (tightly controlled affiliates). PM’s current organizational form and management practices apply various elements of each of these structures in different ways across the country programs. Finally, the PMI BOD should evaluate the performance of the co-Executive Directors. Confusion in the network might be clarified if roles were more clearly defined and monitored in performance reviews. 2. Strategic Planning: a. Rewrite strategic plan with country program input. A new strategic plan should be developed with full participation by the country programs. The input gathered at the Country Director meeting in September 2003 could form the basis for writing this plan. The analysis should not only set out targets as in the current strategic plan, but should also include strategies and implementation steps for meeting the goals. b. Use a rolling strategic planning system. PM’s current planning horizon, as reflected in the strategic plan, is 5 years. However, PM does not have a longer-term vision for the network. The lack of a long-term vision presents challenges to the validity and usefulness of the 5 year plan. A rolling strategic planning system, which incorporates medium-term (5 year) goals and objectives but is reviewed and extended bi-annually, would benefit PM in two ways. It would allow flexibility to periodically revise strategies as the country programs develop and new countries are added. It could also be a tool that PM uses to develop strategic planning capacity throughout the network, with the goal of creating a 10 year vision that then could inform short and medium term planning. c. Coordinate strategic planning, annual planning and budgeting. As discussed above, the network strategic planning process should gather input from the country programs, which is then consolidated by the PMI office and used for international level planning, and ultimately presented to the PMI BOD. The BOD-approved strategic plan should form the basis for country programs’ annual operating plans, which in turn should determine annual budgets. d. Include the Loan Fund in strategic planning initiative. PM should determine their goals for the Fund, considering whether they want to grow the loan fund, or to access other existing loan funds such as Blue Orchard. A decision to grow the Loan Fund should include hiring a Loan Fund manager with appropriate experience in investments, liquidity and foreign exchange management, and credit analysis. In the short term, they should evaluate options for optimizing use of excess liquidity across country programs. 3. Establish internal controls. PM recognizes and is addressing the need to establish an internal 75 controls system. The country programs have clearly identified their need for the PMI office to monitor field performance and to maintain quality control, and they have expressed support for an internal audit function to be run out of the PMI office. The Finance Committee of the PMI BOD is developing a proposal for an internal audit system. The field programs will review this proposal before the PMI BOD approves it. 4. Complete decision making for country program and network MIS. A decision has to be made soon on an MIS system for the network. It is crucial to have an external MIS needs assessment for the country programs and the PMI office, so that any system that is developed or adapted will efficiently supply the needed information to relevant parties. PM is in the process of hiring an MIS consultant to complete this needs assessment and to make a recommendation for a software program that will grow with PM into the future. As of November 2003, Raul Tapia has been hired. 5. Increase investment in human capital. PMI should include human resources in the review and documentation of policies and procedures. PMI should also work towards establishing formal training agendas and career paths for employees. 1.6 Evaluation Methodology Briefly present the methodology for assessing various aspects of the program. ß Describe the evaluation team members roles and responsibilities; -Till Bruett visited PMP and wrote PMP findings in this report. -Laura Foose visited PMI and wrote the PMI section of the report -Reuben Summerlin conducted phone interviews of PMB and PMN to discuss network partnership questions and to gather more detail on the indicators and activities in the DIP. -Raul Tapia conducted the analysis on the MIS system in PMB to investigate the problems that were encountered and to gather lessons learned. This work was supervised by Reuben Summerlin. ß Outline the methods of data collection and analysis to be used and indicate why these methods have been selected. Briefly state constraints of these data collection methods as well as data limitations; The ACT team communicated with each other before and after each stage of information collection. Prior to the visits/interviews, ACT prepared a list of questions relevant to the MG evaluation so that staff could prepare. The interviews were tiered so as to confirm, verify, and build on the information collected at each stage. The PMP interviews were first, the PMI interviews second, the MIS interviews third, and the PMB and PMN interviews conducted last. In Peru, Till Bruett of ACT, Ana Escalona of PMI and Katy de la Garza of PMM conducted structured meetings with PMP’s senior management, regional coordinators, and field staff in Puno and Juliaca. In addition, they met with the Chairman of PMP’s BOD, COPEME, and a number of PMP’s partners. A focus group was conducted with a group of clients in Puno and informal interviews were conducted with 76 clients and loan officers during visits to village bank meetings at focal centers. Time was spent reviewing documents on site, observing procedures, and reviewing the MIS system. The evaluation team was given full access to PMP’s offices and materials. ACT collected and reviewed all items listed in the planning matrix and assessed the reliability of client and portfolio data by reviewing the source data and overseeing the process of report creation. Given the time constraints, the team did not visit PMP’s new branch in Tacna. In New York, Laura Foose of ACT met with and interviewed all of the staff of the PMI office, and conducted phone interviews with the financial advisor, and three members of the PMI BOD- including the Chair and Vice Chair of the PMI BOD. In preparation for these interviews, Ms. Foose prepared a questionnaire focused on the achievement of indicators and activities contained in the DIP and forwarded it to the PMI office in advance. In New York, time was spent reviewing documents on site and primarily in interviews with the Financial Officer reviewing reporting procedures and with the Operations Officer discussing program planning. Two full binders of background materials were provided for the evaluator, including all of the PMI BOD minutes for the last three years, quarterly program reports, budgets, etc. Ms. Foose was given full access to PMI’s financial statements, reports and materials. The PMI team was very open, transparent and eager to learn/evaluate their experience. Ms. Foose provided a debrief of initial findings at the end of the visit to allow for feedback. In Bolivia, Raul Tapia conducted an on-site analysis of the design and implementation of PMB’s MIS in order to investigate the problems encountered and to gather lessons learned from the process. In preparation for the visit, PMB management was provided with a questionnaire to serve as a guide for the onsite interviews. After conducting his analysis, Mr. Tapia provided ACT with a report of his findings, which ACT then incorporated into the PMB section of the final report. According to the terms of reference agreed upon by USAID, no on-site visit was required to assess completion of programmatic objectives for either PMN or PMB (apart from the MIS implementation.) Instead, Reuben Summerlin of ACT conducted phone interviews with the executive directors of both PMB and PMN from the US. In preparation for these interviews, Mr. Summerlin prepared questionnaires focused on the achievement of indicators and activities contained in the DIP, as well as each program’s participation in the PM network partnership. Questionnaires were shared with members of the evaluation team as well as with Ana Escalona, for input on form and content. Once completed the questionnaires were forwarded to the executive directors of each program in preparation for the interviews. Each phone interview consisted of one hour-long session. In addition to answering all the evaluator’s questions, each executive director provided Mr. Summerlin with electronic copies of their CCREP reports to provide back up of reported indicators. 77 Attachment 1 Pro Mujer (Programs for Women) Scope of Work for Final Evaluation Cooperative Agreement FAO-A-00-99-00056-00 I. Program Context A final evaluation of Cooperative Agreement FAO-A-00-99-00056-00 is being conducted to review Pro Mujer’s progress/success and obstacles/setbacks in implementing its MG Program. Pro Mujer is an international women’s development organization, founded in 1990 to empower women to improve their social and economic status. Pro Mujer accomplishes this by establishing sustainable microfinance institutions that provide the credit and training in business development that women need to improve their small businesses and increase their incomes. Pro Mujer also provides health education, and links women and their families to health services. In 1999, when the Matching Grant (MG) was awarded, Pro Mujer had operations in Bolivia and Nicaragua and was serving a total of 26,100 clients with a loan portfolio of $2.5 million. Pro Mujer received a MG of $1.5 million from USAID to: · Strengthen Pro Mujer’s financial, managerial, technical and analytical capacity to develop and manage field operations in Bolivia, Peru and Nicaragua. · Establish a sustainable microfinance organization in Peru · Increase women’s access to credit in three countries · Increase the self-sufficiency of the credit programs in three countries · Develop a new management information system Pro Mujer’s match to USAID’s funding was $1.5 million. II. Evaluation Objectives The Cooperative Agreement between Pro Mujer and USAID/PVC requires an external evaluation of the MG program. For USAID/PVC the evaluation will answer basic questions to determine whether the program was implemented as designed and what results were achieved. The primary objectives of the evaluation for Pro Mujer are: 1) To evaluate Pro Mujer’s operations in Peru, key success factors in achieved results and to identify the institutional systems that it needs to put in place to continue to expand (i.e., identify threats to continued growth); 2) To identify lessons learned during Pro Mujer’s replication process to Peru to be able to apply them in future replications; 3) To 78 assess the processes that Pro Mujer has in place for gathering and measuring impact on clients; and 4) To evaluate Pro Mujer’s current organizational structure and design in light of its desire to be effective in supporting the growth and impact of its programs while at the same time maintaining lean operations at headquarters. III. Evaluation questions The evaluation should address Pro Mujer’s entire MG program including the PMI office as well as the programs in Peru, Bolivia, and Nicaragua. However, because most of the MG program activities took place in Peru and New York, USAID/PVC has agreed that on-site evaluations only need to take place in Peru and New York. The MG activities and results in PMB and PMN will be evaluated through document review and phone interviews. The Detailed Implementation Plan (DIP) is the USAID-PRO MUJER document that contains the agreed upon objectives and indicators forming the basis for an evaluation. It is important to keep in mind that final evaluations are comparative, not descriptive. The evaluation should contrast baseline and end of project data regarding the status of the beneficiaries and interventions. Evaluations should shed light on progress made and problems faced by the program, and explain why the project met or failed to meet its objectives. The evaluator must cite evidence to support conclusions. Key elements of a final evaluation are overarching lessons learned, particularly those that are relevant beyond the immediate program sites, best practices or program guidance. The following sections describe the key questions and issues to be addressed, in the order they must be answered in the final evaluation report. 1.1 Evaluation Profile Sheet The final evaluation report must include an Evaluation Profile sheet that provides the following information: · PVO name and Cooperative Agreement Number Pro Mujer (Programs for Women), Inc. FAO-A-00-99-00056-00 · Country Program Sites and Names of Principal Partners Peru – PMP Bolivia – PMB Nicaragua – PMN · Duration of Grant (Month/Year) September 1999 to September 2002 On August 2, 2002, a one-year no-cost extension was requested to extend the project 79 to September 2003 to complete the implementation of the MIS system, to complete the process of establishing a local NGO in Peru, and to begin the process for establishing a local NGO in Nicaragua. On June 19, 2003, a second extension was requested for three months, to December 2003 to complete the final evaluation of the MG and full debriefing of the findings with USAID/PVC and Pro Mujer. · Beneficiary Populations (by age/sex): As of March 31, 2003: Bolivia: 43,375 Nicaragua: 15,588 Peru: 15,221 · PVC-PRO MUJER match totals ($) and PVC-PRO MUJER match funds disbursed to date ($): Budget Actual Expenditures (as of 12/31/02) USAID 1,500,000 1,337,760 PRO MUJER 1,500,000 1,486,465 · Date DIP was first approved by PVC and changes made to DIP: Approved: April 2000 Changes: none · Evaluation Start Date (start implementing SOW) and End Date (submission of final report to PVC) Start: July 30, 2003 End: no later than December 31, 2003, including acceptance by USAID/PVC and debriefing 1.2 Summary of Conclusions and Recommendations Provide a summary of the evaluation’s main conclusions and recommendations 1.3 Program Background Provide a summary description of the program to be evaluated: 80 · Brief history of the MG Program; · Rationale for the program; · Situation on the ground and status of interventions at the beginning of the program and relevant baseline data; · What the program seeks to achieve; · Principal partners; · Current implementation status; and · Briefly comment on Pro Mujer’s overall development plans. 1.4 Program Effectiveness Program Model or Approach · Briefly describe the program’s general approach/model(s) and hypotheses; · Determine if the hypotheses and assumptions under-pinning the program model were sound. Achievement of Objectives This section should identify the program objectives and assess progress made to date toward achieving each major objective defined in the DIP, for the OMI office and for each country program. PMI Objectives: Objective 1: Enhance Pro Women’s managerial and technical capacity to establish and support local autonomous affiliates and coordinate activities of the network. PMP Objectives: Objective 1: Establish a local MFI Objective 2: Implement credit program PMB Objectives: Objective 1: Modify and test the new MIS, SAF2000 Objective 2: Expand training and credit program PMN Objectives: Objective 1: Establish a local MFI Objective 2: Expand credit program Under each main objective: Assess progress toward each major objective. Provide evidence, criteria for judgment and cite data sources. 81 · Identify major successes, challenges and constraints in achieving each objective. Include a chart that summarizes the program’s successes and weaknesses (see suggested format in Annex A). Attach a summary of the program’s DIP results status using template given in Annex B; · Discuss the impact of the program on target populations; · Assess the impact of the program on strengthening the capacity of the PVO’s operations at HQ and in the field to deliver sustainable services; and · Discuss any unintended impacts to date – positive or negative. Assess if the main strategies and activities employed to achieve results were effective. Identify any changes made during implementation. · Have the strategies and activities been effective in achieving mid-term/end of project (EOP) targets? · Did Pro Mujer conduct a mid-term evaluation or data review? Was program implementation on-track? Were mid-course corrections made and what are the effects of these changes? Cross-cutting Issues Partnerships: Assess the status and outcomes of Pro Mujer’s partnerships with local organizations, NGOs, networks, businesses or intermediate support organizations: · Assess the effectiveness of Pro Mujer’s approach to building in-country partnerships. Identify the key elements that contributed to success. · Identify local partners and cite changes in local partners’ institutional capacities that occurred as a result of the program partnerships. Identify what activities were most instrumental in strengthening local partners; · Cite changes in Pro Mujer’s institutional capacities, as a result of learning from in-country partners; · Discuss if these partnerships are mutually satisfactory and beneficial. Identify and discuss major constraints to developing durable and productive partnerships; · Identify outcomes of program links with any coalitions, networks or associations; and · Attach Partnerships Table (see template in Annex C) New Tools, Guidance Or Standards: Identify if new tools/guidance, approaches, or program standards were developed under this program: · Assess if these tools, approaches etc. were effective and merit broader distribution or application. · Assess the new products that were developed by PMP (seasonal loans and educational loans) to evaluate whether these new products are meeting client needs; how have these products been received by clients? Has the transfer of knowledge about the seasonal loan product to PMB been successful and has PMB introduced these new products? · What changes to the methodology were made in PMP that were successful/unsuccessful which could be applied in other Pro Mujer programs or other village banking programs? 82 · What are the key criteria of Pro Mujer’s methodology, as applied in PMP, that contribute to success (as measured by 0% delinquency, scale of operations, self-sufficiency and efficiency)? · What makes PMP so efficient compared to other PM programs and other village bank programs? · How effectively is PMP leveraging resources within the community to achieve the greatest impact to its programs? · How effective has the Pro Mujer loan fund, managed out of headquarters, been in leveraging existing resources and funding the growth of PMP? Advocacy: Determine if the program has engaged in policy or program advocacy: · Describe the advocacy efforts and assess if they contributed to strengthening the policy environment or promoted the program in any way; · How does PMP encourage women to be advocates for better health services (advocacy through action)? Conflict Mitigation/Prevention: If any of the MG country programs operate in conflict situations, provide the following information: · Determine if Pro Mujer has conducted any conflict vulnerability assessments; · Discuss the usefulness of the conflict assessment in tailoring the program design to address factors contributing to conflict; and · Briefly discuss relevant activities and outcomes. Sustainability and Scale-Up: Assess program sustainability and scale-up plans of operations: · Magnitude of the program: size of beneficiary population by age and sex; · Progress and potential for scale-up or replication; · Progress made by the program toward achieving self-sufficiency and if Pro Mujer is planning to measure post-grant self-sufficiency; · Progress program is making toward diversifying resources, for example building alliances with in-country business organizations. Lessons Learned and Program Recommendations · Outline the main lessons learned from the MG program that would be applicable beyond the program sites. · Provide recommendations for Pro Mujer, partner organizations and USAID/PVC 1.5 Program Management 83 Management Approach · In the context of the program model and the changing country situation, is the overall approach to program management flexible, appropriate and adequate? · Is the program cost effective and timely in converting inputs into outputs and outcomes? Quality and Status of Detailed Implementation Plan (DIP) · Discuss the quality of the DIP, i.e. the clarity and adequacy of the objectives, indicators, baseline studies and activities. · Comment on the utility of the DIP as a management tool for Pro Mujer, their partners and PVC. Financial Management · Are adequate financial control systems in place? · Is Pro Mujer leveraging additional resources (beyond the match)? Monitoring and Evaluation (M&E) System · Does the M&E system supply accurate, reliable and timely performance data? · How effective are the steps (if any), taken to institutionalize M&E at Pro Mujer HQ? · Assess if the partner organizations have increased their capacity to monitor and evaluate their work, document program achievements, and use data for decision-making and program advocacy. · Has the program undertaken to date, any special studies to assess program operations or impact? Comment on the quality and utility of these studies, in particular the impact evaluation funded through Summit and Ford. · Assess Pro Mujer and partner use of data to make management decisions. · What more could be done to improve the M&E systems and use data for decision-making and program advocacy? · Verify data pertaining to a random sample of indicators. Information Systems · Has the program increased in-country partners’ access to information technologies? · What have been the major obstacles which have prevented Pro Mujer from completing its objective under the MG to implement a new management information system for all of its programs? 84 · Pro Mujer followed all the best practices recommendations for choosing and implementing a new MIS. What went wrong? · What steps have been taken by Pro Mujer and its partners to share program information and learning? Staffing and Supervision · Do Pro Mujer and partner organizations have an adequate number of staff with relevant expertise for supervising/backstopping the program? · Given Pro Mujer’s desire to continue to support the growth of its programs in Latin America, is its current organizational structure appropriate, especially in light of Pro Mujer’s goal to keep a lean infrastructure in headquarters (so that most of the resources go to the field)? What additional staff is necessary at headquarters to provide adequate support? PVC Program Management · Assess PVC’s oversight and backstopping of the cooperative agreement. Program Management Lessons Learned and Recommendations · Discuss the management lessons learned and recommendations to Pro Mujer, in-country partners and PVC. · Discuss key success factors in Pro Mujer’s achieved results and identify institutional systems that it needs to put in place to continue to expand (i.e., identify threats to continued growth); · Discuss lessons learned during Pro Mujer’s replication process to Peru to be able to apply them in future replications · Discuss key criteria of Pro Mujer’s methodology, as applied in PMP, that have contributed to its success and that should be applied beyond PMP and beyond Pro Mujer. 1.6 Evaluation Methodology Briefly present the methodology for assessing various aspects of the program. · Describe the evaluation team members roles and responsibilities; · Outline the methods of data collection and analysis to be used and indicate why these methods have been selected. Briefly state constraints of these data collection methods as well as data limitations; · Schedule (see “Part 1:Evaluation Procedures”); · Attach a list of cites visited, persons/groups interviewed for the evaluation; and 85 · Attach a list of documents analyzed. 1.7 Evaluation Report The report should address evaluation questions in the sequence presented in the SOW. Use a 12-point font for the report’s narrative sections and a 10-point font for any tables or charts. Cross-referencing should be used judiciously to minimize duplication and redundancy. The list of attachments for the final evaluation report include: · The final SOW · DIP Matrix and Results Status Table (see Annex B for template) · Partnerships Table (see Annex C) · List of persons/groups interviewed · Select bibliography PVC’s approval of the evaluation report will be based on the evaluation quality standards cited in the text (see “Evaluation Quality Standards,” p. 5). The final report should be submitted to the relevant PVC program officer on or before the date for submission scheduled. Pro Mujer should submit to USAID/DCHA/PVC, one (1) unbound single-sided copy with all attachments, one (1) bound double￾sided copy and a 3 ½” diskette with an electronic copy in Microsoft Word97. Also, a complete copy should either be emailed or mailed (if the evaluation report and annexes are difficult to transmit electronically), to USAID’s Center for Development Information and Evaluation (CDIE) addressed to: The Acquisitions Coordinator Document Acquisitions USAID Development Experience Clearinghouse 1611 N Kent Street Suite 200 Arlington VA 22209-2111 USA docsubmit@dec.cdie.org IV. Budget and Timeline Annex D is the budget proposed by the evaluator, Alternative Credit Technologies (ACT) and accepted by Pro Mujer. The following timeline has been established for the final evaluation: 86 Activity Date Select the evaluation team members, including the external evaluator Done SOW developed by Pro Mujer and approved by USAID July 25, 2003 Contract signed between ACT and Pro Mujer July 25, 2003 Team Planning Meeting in DC July 30, 2003 Headquarters visit to NY September 15-17,2003 Field visit to Peru August 24-31, 2003 Draft of Preliminary Findings to Pro Mujer October 1, 2003 Presentation of findings to Pro Mujer October 8, 2003 First draft of Evaluation report November 7, 2003 Comments due from USAID & Pro Mujer November 14, 2003 Final Evaluation Report due November 26, 2003 Debriefing with USAID & Pro Mujer TBD, by December 31, 2003 V. Evaluation Methodology Please refer to Section 1.6. (ACT to provide information on the methodology) The evaluator will visit the headquarters office to gather and review information and interview key staff. During this visit, the evaluator will conduct phone interviews with Pro Mujer staff in Bolivia and Nicaragua as well as with Pro Mujer’s financial advisor, Nancy Natilson. The evaluator will request information from Pro Mujer’s headquarters staff as needed to perform the evaluation. The evaluator will also visit field operations in Peru because this is the site where most activities funded by the MG program took place. The evaluator will interview the PMP BOD president in Lima and will travel to Puno for 4 full days to evaluate the program’s results, outreach and impact. The evaluator will visit Pro Mujer’s national office in Puno, and at least 3 focal centers (branches) between the cities of Puno and Juliaca. The evaluator will randomly select the branches to be evaluated, will interview the head of the focal centers and a sample of clients (at least 25) either individually or in groups. Before leaving both the headquarters office and the PMP office, the evaluator should present its preliminary findings. Pro Mujer will provide additional information as needed, depending on the findings. VI. Roles of Evaluators and other stakeholders Alternative Credit Technologies will be responsible for writing all the reports, analyzing data and 87 comparing baseline vs. achieved targets. Pro Mujer will be responsible for providing the following information to ACT: · Grant Proposal, Cooperative Agreement and DIP · MG Annual reports · External Evaluation of PMP prepared by Joyce Lehman · Financial Statements and Indicator reports for PMB, PMP and PMN · Other, as needed Ana Escalona, from Pro Mujer headquarters will be the Evaluation Manager who will coordinate the evaluation implementation and deliverables. She will accompany ACT on the site visit to PMP. Pro Mujer will also invite a PMM representative to serve on the evaluation team during the visit to PMP. VII. Evaluation Report Format Please refer to section 1.7 above. VIII. Strategies for debriefing, sharing and using information The first draft of the evaluation will be sent to Pro Mujer’s headquarters and field staff for review and comments. Once the final report and recommendations are accepted, Pro Mujer will develop an action plan to address recommendations. The report and action plan will be submitted to USAID/PVC and a final debriefing meeting with USAID/PVC will take place by December 31st, 2003. Pro Mujer will also seek to publish a technical note, in collaboration with ACT, in a microfinance publication (Journal of Microfinance, Small Enterprise Journal, Microbanking Bulletin) to disseminate the lessons learned from Pro Mujer’s replication in Peru and specifically the key factors that have contributed to its success as measured by 0% delinquency, scale of operations, self-sufficiency, and efficiency. The note will include a description of how PMP leverages its resources in the community to achieve the most impact for the women it serves. Finally, Pro Mujer will seek to disseminate findings at the 2004 SEEP conference. Attachment 2 List of persons/groups interviewed Peru Gerardo Pejerrey, President of PMP BOD Armando Pillado, COPEME director Carlos Rios, COPEME manager Naldi Delgado, PMP country director Tania Menendez, Financial Manager Yenni Cordero, Accountant Juana Coya – Puno Region Credit supervisor Lorena Yturri – Juliaca Region Credit Supervisor Patricia Alvarez – Administrador Marlene Chura – Health Project Supervisor Demecia Benique – Business Development and Client Impact Supervisor Guido Penaloza & Joel Ponce – Systems supervisors Approximately 20 loan officers, & 4 focal center managers PMB Carmen Velasco, Directora Ejecutiva PMB Hugo Bellott, Gerente Financiero Roberto Cornejo, Responsable de sistemas Maria Elena Machaca, Técnico en sistemas Gonzalo Esprella, Operador de sistemas PMN Gloria Ruiz, Directora Ejecutiva, PMN PMI Peter Johnson, Chairman of PMI BOD Elaine Edgcomb, Vice Chair of PMI BOD Greg Hess, Executive Committee of PMI BOD Lynne Patterson, Co-Founder and Executive Director Ana Escalona, Financial Officer Jenny Dempsey, Operations Officer Sue Mark, Development Officer Cristina Del Molino, Accountant Julia Love, Office Manager Nancy Natilson, Financial Advisor 89 Attachment 3 List of documents reviewed Peru Credit · Methodología · Manual del programa de crédito Asociaciones Comunales (abril 2003) · Documentos que se manejan para un Comité de Asociación neuva o Solicitud de crédito regular Asociacón Comunal o Solicitud de crédito regular Grupo Solidario o Informe de verificación de negoicios y domicilios o Recibe de desembolso o Informe de ciclo o Movimiento de socias para el neuvo ciclo o Planilla de control de pagos y movimiento interno o Control interno de la Asociación Comunal o Instrucciones para realizar últimos pagos o Plan de pagos por socio · Documentos que se manejan para un crédito estacional o Solicitud de crédito o Informe de verifcación de negocios y domicilios o Plan de inversión de crédito estacional o Informe sobre el piloto de créditos estacionales · Crédito Escolar (descripción del programa piloto) Training · Destrezas básicas en negocios 1 (PMB, 3rd edition) · Asociaciones Comunales (PMB 3rd edition) · Mi negocio y yo (Desarrollo Empresarial 2003) · Manual del promotor del salud y desarrollo humano Governance · Actas de reunión de junta directiva (abril 2002, enero 2003) · Incripción de personals juridica extranjeras Pro Mujer · Inscripción de asociaciones Asociación Promujer Peru · Constitución de Asociación Civil Sin Fines de Lucro (Pro Mujer) · Registración de ENIEX de Pro Mujer · CVs and biographies of BOD Financial and Planning 90 · PMP Plan Estrategico 2002-2006 · 2003 Budget · Plan Operativo 2003 – Activities · CCREP June 2003 · Activity variance report June 2003 · Consolidated Financial Statements June 2003 · Balance Sheet for USAID funds, June 2003 · Audited Financial Statements (KPMG) 2002 PMB · “Archivo de documentación de selección, contrato y seguimiento del proyecto”. · “Requerimientos generales y específicos para la implementación del sistema integrado de información gerencial”. · Archivo de correspondencia del proceso de personalización e implementación de SAF2000 (ALOs). · CCREP reports, June 2003, December 2002 PMN · CCREP reports, June 2003, December 2002 PMI · By-laws of Pro Mujer (revised 11/03) · Biographies of PMI BOD members · Pro Mujer BOD Committee Descriptions, 5/03 · PMI Organizational Chart, 9/03 · Organigrams for PMP, PMM, PMN, PMB · Loan Fund Report, 6/03 · Description of Pro Mujer Loan Fund · Loan Fund Policies and Procedures · Pro Mujer Strategic Plan 2003-2006 · Strategic Plan for Peru, 2002-2006 · Strategic Plan for Nicaragua, 2003-2007 · Annual Operating Plan for Nicaragua, 2003 · Pro Mujer Organizational Chart · Pro Mujer Financial Policies, 11/02 · Quarterly CCREP reports for all country programs Country Director’s Reports to the PMI BOD · PMB, 11/02, 5/02, 9/01, 10/00, 5/00 · PMM, 11/02, 4/02 · PMP, 11/02, 4/02, 10/01, 4/00 91 · PMN, 11/02, 12/01, 10/00, 4/00 Minutes – PMI BOD meetings · 6/03, 4/03, 2/03, 11/02, 4/02, Jan/02, 10/01, 8/01, 5/00 Executive Director Trip Reports · PMP, 9/03, 1/02 · PMP and PMB, 1/03 · PMN, 9/00 Minutes from Annual Directors Meeting · Consolidated Country Directors’ survey of PMI (Respuestas Consolidades a la Encuesta de Percepciones y Satisfaccion Pro Mujer Internacional), 6/03 · Report for Country Director Meeting (Resumen de la Reunion de Directores 2003), 9/03 · Memo outlining follow up to Country Director meeting (Seguimiento a la Reunion de Directores), 9/03 · 11/02 · 4/02 · 10/00 Reports/Evalations · First Annual Report for Pro Mujer MG, 11/00 · Second Annual Report for Pro Mujer MG, 9/01 · Third Annual Report for Pro Mujer MG, 9/02 · Detailed Implementation Plan, 5/00 Pro Mujer Peru, A Report on a Review of Internal Control Systems to determine Vulnerability to Fraud, by Joyce Lehman, MEDA,12/01 · Pro Mujer Peru, Specific Findings of a Review of Internal Controls, 12/01 · KPMG Report, Pro Women Combined Financial Statements, 12/00-01 · KPMG Report, Pro Women Combined Financial Statements, 12/99-00 · Pro Mujer Interest Rate Analysis Report, 11/03 Budgets · MG FY98 Application, Reallocation Requests 2/02, 9/02 · Quarterly Budgets for PMI office, PMB, PMN, PMP, PMM – 2003 · Projected Cash flow statements PMI office, PMB, PMN, PMP - 2002 Funding · Development Plan – strategy to mobilize resources for the network, 2003 · General Reporting and Payment Calendar, 2003 · Chart of all outstanding Loan Fund proposals and grants received · Chart of Direct Funders to Country Programs · Chart of Funders and Initiated Proposals for PMI General Operating Support 92 · Chart of Non-Financial Services Funders · Chart of Grants Received, 2000-2003 Accounting · Accounting Procedures Manual · Chart of Accounts · various accounting forms Human Resources · Employee Files for PMI office staff · Job Descriptions · Resumes · Work plans · Employee Reviews · List of Staff Development Activities · Proposal for long-term MIS network consultant, 9/03 · Pro Mujer Employee Handbook, 8/02 · Proposal for Internal and External Audit System of PMI, 9/03 Publications/Newsletters/articles · Pro Mujer Annual Report, 1999-2000 · Pro Mujer Annual Report, 2001-2002 · Newsletters, 2000, Fall 2002, Winter 2002, Fall 2003 · PMP Internal newsletter, 6/03 · Loan Fund general pamphlet · Pro Mujer general pamphlet · Pro Mujer Bolivia case study, MBB, 9/00 · “Building Self Sufficiency”-Article in National Council of Jewish Women, 6/03