USAID/PVC Matching Grant Evaluation Series Improving Maternal and Child Health Impacts of Credit with Education Freedom from Hunger and USAID/PVC February 2002 Conducted under USAID/Evaluation Indefinite Quantity Contract #AEP-I-00-00-0024-00 Evaluation team members: Ellen Vor der Bruegge, Freedom from Hunger Laurie Zivetz, MSI, Team Leader i H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Acknowledgements I would like to thank the staff of Freedom from Hunger and its partners CARD and CRECER for their contributions to this evaluation. Ellen Vor der Bruegge gave generously of her time and insights, and drafted several of the key tables. Flor Sarmiento and the staff of CARD were warm, informative hosts. CRECER/Bolivia Executive and Training Directors Evelyn Grandi and Isabelle Rueda provided candid and helpful insights over the phone—the next-best-thing to a field visit. Thanks also to MSI for cheerful and efficient support, and to PVC for this opportunity, once again, for an inside look at an outstanding program. Needless to say, although members of the FFH team have been generous in their contributions to this report, errors and recommendations remain this author’s. Laurie Zivetz Washington, D.C. February, 2002 ii H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Young Credit with Education loan officer in Takawayan, Philippines, explains condom use to village bank members. Humor is an important part of the session. Group participation, shared credit liability and fiscal discipline are important aspects of CwE. Male field agents also deliver messages about maternal and child health and reproductive health. important aspects of CwE. iii H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Loans through CARD are highly sought after because interest rates are lower than the local moneylender. The village banking system is a cost-effective way of reaching remote areas with credit. One group member is designated to repay at the branch office on a regular basis—sometimes weekly. iv H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Students lined up to greet the evaluation team in Takawayan…….. turned out to be for President Gloria Macapagal Arroyo who praised the work of CARD in her address. v H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Evaluation Identification Sheet PVO name Freedom from Hunger Matching Grant Title Improving Maternal and Child Health Impacts of Credit with Education Cooperative agreement number FAO-A-00-99-00046-00 Amount of Grant $1,800,000 Period of Grant 9/20/1999-9/9/2002 Any cost/no cost extension No Current status of MG Month 28 of 36 completed USAID/PVC Grant Officers Joy Pentecostas (previous), Tom Kennedy (current) Technical area of grant MCH/RH Date of evaluation January, 2002 Countries of program activity USA, Bolivia, Philippines Country programs evaluated USA (HQ visit), Bolivia (by phone), Philippines (field visit) Evaluation Team Members Laurie Zivetz, Management Systems International, Team Leader Ellen Vor der Bruegge, Freedom from Hunger vi H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc List of Acronyms AED Academy for Educational Development BCC Behavior Change Communication BF Breastfeeding (Module) CA Credit Association CARD Center for Agriculture and Rural Development CARE-MoRR CARE International– Management of Reproductive Risk CBD Community-based Distributor CBD System Community-based Distribution System CORE Child Survival Collaborations and Resources Group CwE Credit with Education DIP Detailed Implementation Plan FANTA Food and Nutrition Technical Assistance FFH Freedom from Hunger ICF Infant and Child Feeding (Module) ILO International Labor Organization IMCI Integrated Management of Childhood Illnesses IUD Intrauterine Device JSI John Snow International LAM Lactational Amenorrhea Method LTSP Local Technical Service Provider MCH Maternal and Child Health MFI Microfinance Institution MG Matching Grant MOU Memorandum of Understanding NGO Non-governmental Organization PROCOSI Programa de Coordinación En Salud Integral (Bolivian Health network) PST Practitioner Services Team PVO Private Voluntary Organization R&D Research and Development RH Reproductive Health R&I Research and Innovation Team VB Village Bank BOT Board of Trustees BOD Board of Directors SEEP Small Enterprise and Education Promotion Network TA Technical Assistance TO Technical Officer TOT Training of Trainers USAID United States Agency for International Development BHR Bureau of Humanitarian Response PVC Office of Private and Voluntary Cooperation MED Microenterprise Development IGP Implementation Grant Program MG Matching Grant WOCCU World Council of Credit Unions UNDP United Nations Development Project CGAP Consultative Group to Assist the Poorest FINRURAL Asociación de Instituciones Financieras para el Desarrollo Rural Freedom from Hunger’s Current and Former Partners CwE Collaborating NGOs ACLAM – Haiti Action Contre La Misère CARD – Philippines Center for Agriculture and Rural Development COD/EMH – Haiti Comité Oecuménique de Développement-Eglise Méthodiste d’Haiti CRECER – Bolivia Crédito con Educación Rural CRS – Catholic Relief Services FAMA – Familia y Medio Ambiente (Honduras) FITSE – Malawi Financial Trust for the Self-Employed FOCCAS – Uganda Foundation for Credit and Community Assistance FUNDAP – Fundación Para el Desarrollo Integral de Programas Socioeconómicas (Guatemala) Haingonala – Madagascar WV Indonesia – World Vision Indonesia OICI - Opportunities Industrialization Centers International Volamahasoa – Madagascar Vola Mahasoa African Credit Unions FECECAM – Benin Fédération des Caisses d’Epargne et de Crédit Agricole Mutuel du Bénin FUCEC – Togo Faîtière des unions coopératives d’épargne et de crédit Kafo Jiginew – Mali Nyèsigiso – Mali OTIV – Madagascar Ombona Tahiri Ifampisamborana Vola RCPB – Burkina Faso – Réseau des Caisses Populaires du Burkina TIAVO – Madagascar Tahiri Ifamonjena Amin’ny Vola CUES Credit Union Empowerment and Strengthening (CUES)/Philippines BCS - Bansalan Cooperative Society FICCO - First Community Cooperative Kapalong KING-SF - King-San Francisco MMPC - Mediatrix Multi-Purpose Cooperative NICO - Nabunturan Integrated Cooperative PMPC - Panabo Multi-Purpose Cooperative SIMC - Samal Island Multi-Purpose Cooperative SMPC - Silangan Multi-Purpose Cooperative TC – Tagum Cooperative USPD - United Sugar Planters of Davao viii H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Ghana Rural Banks Afram Bonzali Brakwa Breman Builsa Lower Pra Nandom Nsoatreman Current and former FFH partners/Credit only Practitioners ACME Haiti– Association pour la Coopération avec la Micro Entreprise FONKOZE Haiti–Fondasyon Kole Zepol SED Thailand - Small Enterprise Development New Freedom from Hunger CwE Collaborators FINCA Peru–The Foundation for International Community Assistance SEWA India –Self-empowerment Women’s Association WOCCU/Ecuador–The World Council of Credit Unions Potential FFH Collaborators WTC – Mexico World Trade Center PCI – Nicaragua Project Concern International PACT – Nepal Private Agencies Collaborating Together WEP – Nepal Women’s Empowerment Program CARE – India Cooperative for Assistance and Relief Everywhere World Vision/Mauritania LTSPs - Local Technical Service Providers CAFEM – Centre d’Appui et de Formation en Management EAM – Entreprendre à Madagascar Freedom from Hunger/Ghana ix H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Executive Summary Since 1988 Freedom from Hunger has been refining an integrated approach designed to improve the livelihood of poor, rural women and their families by increasing income and access to business and health education. Through careful research and local adaptation of the credit component, many of FFH’s partners have achieved financial self sufficiency using Credit with Education (CwE). USAID has supported parts of this evolution through a series of Matching Grants to Freedom from Hunger. The current MG focused attention on the health education aspect of the model, proposing to add to and reallocate staff time towards upgrading a series of modules designed to help partner institutions provide generic information on maternal and child health, nutrition and reproductive health information during village bank meetings. Innovations in the management of supervision and monitoring information, health insurance, community based contraceptive distribution systems and adult education were also supported under the MG. These products—which garnered input (and financial support) from a variety of leading agencies—are of a high quality. They anticipate adaptation in a variety of institutional and country contexts, and include not only learning sessions, but trainers guides, training of trainers guides, training aids, adaptation guidelines and monitoring checklists. The CBD guide will be a comprehensive roadmap for implementation in the context of CwE. Progress tracking—FFH’s signature monitoring and supervision methodology—is a toolbox of simple, cost effective, adaptable methodologies, outstanding because they put data collection and analysis capacity in the hands of managers at any level of an institution. Although field testing occurred in collaboration with a number of FFH’s traditional partners, there were two major partners who received considerable capacity strengthening in the process of field applications. CRECER/Bolivia, formerly a subsidiary of FFH, achieved independence in 2001. FFH’s input during the MG helped establish management and planning systems at all levels of CRECER—anticipating growth in this highly successful CwE program which reaches some of Bolivia’s poorest women. CARD/Philippines, a leader in the Micro Finance sector in Asia, welcomed the CwE approach as a way to refocus on their development mission. CARD piloted the model in one branch office and has plans for scale up in the context of significant institutional growth over the next decade. FFH’s assistance to both of these institutions went far beyond training trainers and testing modules. The level of customized mentoring from a team of FFH staff provides a model of partnership. FFH stresses learning from action, careful research and documentation. It has focused on innovating around a single product rather than constant diversification. FFH has moved from being an implementing agency with subsidiaries to a catalytic agency working through partnerships and networks. The period of the Matching Grant coincided with considerable internal soul searching in the organization about the product, market position and partnership focus. Having commissioned a survey of its clients, the FFH Board issued directives to the staff which agree to de-linking and disseminating individual products (the health modules, for instance), while continuing to promote and advocate for the integrated model. The title of FFH’s new MG submission—Breaking Through Barriers to Growth—reflects a renewed emphasis on diversifying FFH’s partnership network to maximize scale. It would establish an international network of CwE Local Technical Service Providers, partnering with MFIs, NGOs and PVOs with existing reach, and continuing to adapt the model in places where hunger is most widespread. Major conclusions, challenges and recommendations follow. A. Conclusions: 1. Over the life of the MG, FFH has successfully balanced an institutional emphasis on credit with health education. The new MCH modules are excellent soup to nuts guides for institutionalizing maternal and child health and nutritional and reproductive health information provided in the context of village banking meetings. x H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 2. FFH partnerships create a respectful, intense learning environment in which both agencies are given space to question, listen, share and grow. Technical and management support to CARD and CRECER has been of high quality. Both agencies, as well as a number of other FFH partners, contributed significantly to field testing of modules and systems. 3. Progress tracking breaks new ground in management information and supervision systems with highly cost effective, user-friendly tools. 4. Because of significant changes in the way FFH has done business in the past organization-level strategic and business plans to guide sustainability, structure, scale up and partnership have lapsed. 5. FFH has established productive partnerships with a number of USAID-sponsored and Foundation donors who provided technical and financial support to the current MG process. B. Challenges: 1. Although the common wisdom is that the integrated approach requires “intersectoral compromise” in terms of resources and attention to each component, there is evidence (albeit largely anecdotal at the moment) that education actually enhances the sustainability of the credit program by increasing client satisfaction, group cohesion and lowered client attrition and default. 2. One of the major challenges for scaling up is finding cost- and human resource-effective strategies for transferring the CwE technology and growth management systems. FFH has developed state of the art training and information management and supervision products and systems. It has also developed the beginning of an international nexus of CwE Local Technical Service Providers, providing the more intangible, but critical mentoring and trouble shooting assistance to numerous partners in the process of rapid scale-up. This presents a significant challenge to scale. All of these components have been essential and synergistic aspects of FFH’s capacity building for MG partners CARD and CRECER. 3. Because the modules are delivered by loan officers who typically have no health background, and because they stimulate group discussion on topics which typically array a significant rumor burden, the potential for misinformation is a risk. FFH is seeking to address this. 4. Reflecting its history in the MFI sector, FFH reports “clients reached” in terms of those participating in village banks. However, typical clients are women in the latter part of their childbearing years; the assumption being that health education received in village bank meetings will be shared with family, friends and neighbors. By measuring results this way, FFH may be shortchanging itself. 5. Related to this, and despite the detailed nature of FFH finished products, FFH maintains that optimum use of tools and adaptation of CwE requires hands-on technical assistance, at minimum to introduce the technologies. While this evaluation confirmed the desirability of training and mentoring, the limits such a prerequisite places on dissemination and scale is self-evident. Local NGOs, in particular, may benefit most from this type of input, but typically do not pay for it. In the past, such partnerships have relied on third party funding. C. Recommendations: 1. FFH and partners are encouraged to build formal and informal links with public and private health agencies and providers at national and local levels in order to maximize congruency of messages and identify points of reference for loan officers who have technical questions or need to refer clients for more information or services. CRECER provides a good model of how this can be done. 2. The LTSP network is an essential part of FFH’s bid to take CwE to scale. In addition to individuals and institutions capable of providing training of trainers and advice on systems, FFH may wish to establish linkages with agencies which can provide inputs in adult education, management mentoring, specific health interventions and other context specific issues. 3. FFH needs a strategic cum business plan which sets partnership, regional and other targets and which articulates a plan for its own sustainability. The strategy should also articulate an advocacy strategy that leverages FFH’s tools and research to promote the integrated model. FFH needs to pay more attention to major donors, academic institutions and other key gatekeepers to influence on the development dialogue. As a long time supporter of FFH’s tool and research development, USAID/PVC has an important role to play in enabling this process. xi H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 4. Consistent with its internal rebalancing of focus, and as FFH expands its partnership network to include health agencies, it is appropriate to apply a multiplier to the client base when measuring results. The impact of CwE on income and health education is designed to impact on household livelihoods and should be counted as such. 5. Lessons learned from the adoption and use of the new MCH/RH modules may provide a profile of the ideal field agent—a person who can enforce the fiscal disciple of the village banking model, and provide effective facilitation of health education sessions. This profile will assist implementing agencies in hiring new staff. xiii H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Table of Contents Acknowledgements.................................................................................................................................................i Evaluation Identification Sheet............................................................................................................................... v List of Acronyms .................................................................................................................................................. vi Freedom from Hunger’s Current and Former Partners........................................................................................... vii Executive Summary .............................................................................................................................................. ix 2.0 Evaluation Methodology and Team Composition .............................................................................................. 1 3.0 Program Background........................................................................................................................................ 2 3.0 Program Background........................................................................................................................................ 2 3.1 Historical and technical context and partners......................................................................2 3.2 Project goal, objectives and major hypotheses to be tested .................................................3 3.3 Detailed Implementation Plan............................................................................................4 4.0 Purpose of the Evaluation ................................................................................................................................. 4 5.0 Program Implementation Evaluation Questions................................................................................................. 5 5.1 The Detailed Implementation Plan (DIP) ...........................................................................5 5.1.1 Meeting DIP targets and data accuracy.........................................................................5 5.1.2 Quality of DIP and degree of success in implementation..............................................5 5.1.3 Familiarity with DIP and design...................................................................................6 5.1.4 Major successes and shortfalls in implementation ........................................................6 5.2 Assessment of project model and hypotheses.....................................................................7 5.3 Advocacy under the project..............................................................................................21 5.3.1 Advocacy activities and impact..................................................................................21 5.3.2 Partner/PVO roles in advocacy ..................................................................................22 5.4 Implementation Lessons Learned .....................................................................................23 6.0 Partnership Questions..................................................................................................................................... 23 6.1 Analysis of Partnership Schemes (Annex E) ....................................................................23 6.2 Measuring Institutional Capacity......................................................................................25 6.3 Constraints to partnership ................................................................................................26 6.4 Information Technology...................................................................................................26 6.5 Use of local networks and service organizations...............................................................27 7.0 Program Management..................................................................................................................................... 27 7.1 Strategic Approach and Program Planning .......................................................................27 7.2 Country Initiatives ...........................................................................................................28 7.3 Conflict Management.......................................................................................................28 7.4 Monitoring and Evaluation...............................................................................................29 7.5 Overall Management........................................................................................................29 7.6 Sustainability (Annex F) ..................................................................................................30 7.7 Financial Management.....................................................................................................31 7.7.1 Effectiveness of financial management ......................................................................31 7.7.2 Leveraging other donor funds ....................................................................................31 7.7.3 Cost effectiveness of technical approach ....................................................................32 7.8 Information Management.................................................................................................32 7.9 Logistics..........................................................................................................................33 7.10 USAID Management......................................................................................................33 8.0 Lessons Learned............................................................................................................................................. 33 ANNEXES Annex A: Key Events Timeline............................................................................................................................ 34 Annex B: Detailed Implementation Plan Table...................................................................................................... 37 xiv H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex C: Scope of Work...................................................................................................................................... 45 Annex D: Advocacy Implementation Chain* ........................................................................................................ 54 Annex E: Partnership Table.................................................................................................................................. 57 Annex F: Sustainability Analysis.......................................................................................................................... 59 Annex G: List of people interviewed .................................................................................................................... 65 Annex H: Debrief with CARD, January 17, 2002 San Pablo, Philippines............................................................... 67 1 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 2.0 Evaluation Methodology and Team Composition This evaluation was carried out during January 2002. Team Leader Dr. Laurie Zivetz, reviewed documents and conducted phone interviews with several of FFH’s technical partners during the first week in January. Freedom from Hunger headquarters interviews were carried out January 7-9 in Davis, California. In addition to one-on-one interviews conducted with the professional staff listed in Annex G, group conversations were carried out with senior staff to explore issues of scale up. A debrief on the final day allowed for group discussion and further clarification of key issues. Funding limitations did not allow a visit to Bolivia, although it is included in this evaluation. Phone interviews with the Executive Director and Training Director were conducted with translation assistance at Freedom from Hunger on January 9. A follow up interview with the Executive Director was conducted on January 24. The Team Leader and Ms. Ellen Vor der Bruegge, Senior Vice President, Freedom from Hunger, visited the Philippines January 13-18, hosted by FFH’s partner, the CARD (Center for Agriculture and Rural Development) Bank. The team was joined by CARD Training Director Mrs. Flor Sarmiento. In-depth interviews were conducted with senior staff of CARD in the CARD Headquarters in San Pablo, Laguna on January 14. Interviewees are listed in Appendix G. The team proceeded to the project site in the afternoon of January 14, and spent the next several days in the field interviewing CARD field staff, supervisors, clients, non-clients, as well as local health workers and health officials. During this time the Team Leader observed four different village bank center meetings in which components of the Family Planning module developed under the MG were being presented. This was an opportunity to observe not only the education intervention in action, but also to observe the FFH/CARD methodology for field testing the module. The Team Leader debriefed CARD senior staff on January 17, offering an additional opportunity to explore and validate findings. The team met with USAID on January 18 to ascertain awareness and views about the MG, and informally debrief the USAID officer on the evaluation. Interviews used an iterative and triangulation approach to inquiry, checking and cross checking perspectives at various stages of the evaluation, and following new issues as they emerged. FFH’s extensive cataloging of processes and the emphasis inherent in the MG on training module production made document review an important source of information in the evaluation. The Team Leader is an experienced consultant and NGO professional with many years of experience in Asia (including the Philippines), extensive evaluation expertise, and a background in health and reproductive health. The PVO representative is one of the creators of the Credit with Education model, with 15 years of innovative experience on the model as a senior member of FFH staff. In addition to directing the module production effort, she has authored several of the modules and managed training and testing for several as well. The findings are limited by time as well as the fact that 9 months remain in the MG and many deliverables are still in process. 2 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 3.0 Program Background 3.1 Historical and technical context and partners Freedom from Hunger first pioneered Credit with Education—a model that incorporates business and health information dissemination into the village banking model—in 1988. Since that time, FFH has devoted itself to refining and disseminating the methodology, reaching some 200,000 clients in a variety of country and institutional contexts. The current matching grant (MG) is one in a series of MG’s which have enabled FFH to refine its approach to a point where a majority of its key partners have achieved financial self-sufficiency based on the CwE approach to microlending. The current MG sought support to further enhance the health education components of the model. This was to be achieved by adding health expertise to the staff, upgrading health modules, and adding new modules and field testing with partners. Also, proposed in the MG was the further development of an innovative MIS system—progress tracking—tailored to the CWE model, as well as capacity building for field partners. A dissemination strategy which would make the CwE system and products more widely available was also anticipated. The MG came at a time of some important transitions within the organization. Having operated through a network of subsidiaries’ local offices for many years, FFH was in the process of transitioning its last partner—CRECER/Bolivia—to local management. Looking to a future as a provider of technical assistance in CwE, FFH commissioned a client survey1 to take the pulse of existing partners. Results of this study precipitated directives from Freedom from Hunger’s Board enabling the provision of CwE in a more flexible way, responding to client demands for components of the model, rather than requiring wholesale adoption of the system. To achieve scale (FFH maintains as its vision the eradication of hunger for the 1.3 billion people without enough to eat), FFH has turned more focused attention on mature Micro Finance Institutions and networks—including credit unions, development banks and large NGOs – through which it hopes to reach many more women with its integrated package of credit and health education. This adjustment in modus operandi coincides with the recognition that the services provided to a variety of agencies through the Practitioner Services Team, while offering a sizable income stream to the organization, were not having the impact at scale relative to staff resources being invested in client contracts. At the same time, the MG comes at a time of seemingly greater receptivity to the integrated approach within the microfinance and health communities. In sum, the re-emphasis on health has come at a time when FFH is also focusing on new ways on dissemination and scale. The original submission proposed pilot testing/capacity strengthening with two FFH partners— FOCCAS/Uganda (which became an independent NGO in 1998, having been established by FFH in 1995) and CRECER/Bolivia, already scheduled for independence in July 2001. However, because of geographic discrepancies in-country, USAID/Uganda declined the MG request, and PVC requested FFH to identify an alternative partner. In hindsight, this was a fortuitous series of events as it led FFH to CARD—the Center for Agriculture and Rural Development—a well regarded MFI in the Philippines and the Asia region. FFH, CRECER and CARD share a commitment to fiscal discipline, using a village banking approach, reaching poor, remote, rural women, and incorporating development services and messages in highly cost effective ways. CECER, like FFH, delivers only CwE. It currently has 30,000 clients in the remotest parts of Bolivia, and while not a registered development bank, is a leader in the constellation of MFI stars in the country. FFH has had a long and at times challenging history with CRECER and has stuck with it through management, programmatic and financial ups and downs. FFH retains a seat on CRECER’s new Board of Directors, and, as described below, has played an important role in supporting stronger systems to support institutional sustainability and quality for CRECER. Much of this was done with MG support. 1 Collaboration Assessment for Improved Collaborative Relationships, Mark Leach and Darcy Ashman, Institute for Development Research, May, 2001 3 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc The CwE model was new for CARD, but consistent with its mission and structure. Founded in 1986, CARD has grown to a portfolio of $7,580,000, reaching 50,000 women in remote parts of Luzon with microcredit and related products, based on a modified Grameen bank replicate approach. CARD has a registered development bank, an NGO and an insurance and social security facility under its institutional umbrella. At the time Freedom from Hunger approached CARD, the organization was urgently looking to reclaim its original commitment to social development and avoid being pulled into becoming an entirely commercialized operation. CwE fit the bill. All three agencies anticipate significant scale up over the next 5-10 years—FFH 75,000 by 2006, and CARD from 50,000 to 500,000 by 2010. The MG (and for FFH and CARD the next generation MG— now under review at PVC) was seen by all three agencies and an opportunity to move towards these goals, and, in important ways hone the technologies and management systems necessary for operationalizing the education component to scale alongside microlending. 3.2 Project goal, objectives and major hypotheses to be tested The following table summaries the hierarchy of objectives, as presented in the Detailed Implementation Plan (DIP): Table 3.2 Project Hierarchy of Objectives Goal: To build the capacity of Freedom from Hunger to strengthen integrated microfinance-education practitioners to: ß Deliver high-impact health/nutrition-promoting education; ß Track and analyze indicators of their MCH-education status services and other impacts; and ß Use this information with financial performance indicators to guide their programs toward both financial sustainability and production of important benefits for very poor households, especially in rural and peri-urban areas. Objective 1: FFH Int. Center Balance microfinance expertise with increased staff time and expertise devoted to maternal/child health-promoting education within Credit with Education programs. Objective 2: FFH Int. Center Improve, document and disseminate methods and systems for delivery of high-impact MCH-promoting education within Credit with Education. Objective 3: FFH Int. Center Develop, document and disseminate system for monitoring education performance and impact with Credit with Education program. Objective 4: CwE Practitioners CRECER, CARD and Others Increase capacity to maintain and expand outreach of high-impact MCH-promoting education within CwE without compromising financial service-delivery quality or performance. Key hypotheses to be tested include2 : 1. Achievement of scale and sustainability of CwE institutions depends on · Products that are widely demanded and can be adaptable to local contexts · Systems that utilize cost-effective, management information · A capacity for transferring the CwE technology to all staff in the implementing organization. 2. Enhanced modules, incorporating learning’s from FFH’s prior series and state of the art adult education training techniques, will make them more user-friendly to facilitators and easier to adapt and implement widely at minimized cost. 3. A variety of dissemination platforms are appropriate for CwE. 2 These were not articulated in any of the early documents, and have been developed by the evaluation team with input from a number of FFH staff. 4 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 4. Each platform requires a unique configuration of tools, skills and systems to launch and sustain CwE. This requires products that are sturdy and at the same time adaptable. 5. High quality products and tools introduced by a catalytic team will maximize dissemination in a cost￾effective way. 6. An information system that is cost effective, flexible, highly participatory and embedded in an institutions’ management system is essential for monitoring quality, optimizing management decision making and multiplication of the model. 7. FFH’s organizational capacity to enhance the CwE product and systems will improve if it hires more staff in health and adult education. Because of its partnership structure, Freedom from Hunger was able to garner input and cross-pollinate considerable learning on all of these hypotheses, not just with MG partners CRECER and CARD, but with other NGOs with whom various modules and methodologies were tested, as well as technical partners LINKAGES, CAREMoRR, FANta and the Summit Foundation which provided technical and financial support. 3.3 Detailed Implementation Plan Annex B provides an updated DIP with achievements as at this evaluation. It also includes outcomes projected by FFH for the end of the project. The shorthand necessitated by this table does not do justice to the painstaking process FFH has set for itself of finely tuning ten modules, updating nine existing manuals and developing four discrete new products and tools.3 Nor does it adequately convey the level of high quality technical assistance provided to partners. No changes in knowledge, behavior, income or health outcomes were anticipated by this MG—thus making standard quantitative measures of performance impossible. The demand for the modules and new tools, even before they have been released, and an eagerness to continue cooperation on the part of CARD, CRECER and other agencies speaks to the quality of FFH’s technical assistance. For this reason, the updated DIP presented in Annex B, may be more detailed than what was required. 4.0 Purpose of the Evaluation4 The final evaluation fulfills the requirements of the USAID/BHR/PVC Matching Grant (MG) Program. The MG program will use the information to: assess how well the MG is meeting its objectives; determine patterns and emerging issues across all MG funded programs; determine technical support needs for grantees, shape new RFAs, and to review of any follow-on proposals; develop internal and external documents to demonstrate the effectiveness of the MG program and to share lessons learned with the entire PVO community. PVC will use information outlined in the SOW template in its annual Results Report and in USAID's annual report to Congress. Achievements cited in the evaluation need to be supported by evidence and should be verifiable. Freedom from Hunger and its collaborators, CRECER and CARD, will use the Matching Grant information to: receive an objective assessment and feedback on the products and activities implemented during the grant period, obtain documentation by an external evaluation of the quality and utility of the tools and products intended to be provided to potential users; and provide USAID with a report of Freedom from Hunger’s capacity to manage a Matching Grant as it considers FFH’s current application for another grant from USAID. 3 See Objective 2 in Annex B for a complete list. 4 This section is excerpted directly from the original Scope of Work for the evaluation. 5 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc A copy of the Scope of Work for the evaluation is included in Annex C. 5.0 Program Implementation Evaluation Questions 5.1 The Detailed Implementation Plan (DIP) 5.1.1 Meeting DIP targets and data accuracy A. Findings: 1. The project was not designed to affect measurable field level changes. Thus “data” and data accuracy in the DIP is less relevant than accomplishment and quality of deliverables. 2. Annex B details the status of DIP deliverables. This table was prepared and extensively reviewed by FFH staff, with final edits and validation by the Team Leader. 3. Given the sheer volume of documents (there are half a dozen components of each module and myriad other documents detailing workshops, several generations of tools, drafts and final professional papers etc. prepared during the project) the Team Leader reviewed a sample of modules and other documents. 4. As indicated in Annex B, documents are at various stages of completion. Module testing depends on the willingness of an implementing partner to co-field test a given module, and a “fit” with the timing of education cycles. 5. DIP development helped FFH identify and cost activities and outputs necessary to strengthening the MCH component of its education interventions. 6. FFH expects to complete most deliverables by the end of the project. B. Conclusions: 1. The DIP accurately reflects key objectives, activities and deliverables to meet the MG goal. 2. Annex B provides a status report on deliverables to date and projects accomplishments to the end of the MG. 3. While the DIP does not chart measurable outcomes, it is a useful checklist of the status of key deliverables, and gave FFH a roadmap of milestones towards reaching it goal. C. Recommendations: 1. A breakdown of module components in the DIP would help track specific deliverables and better reflect the level of effort involved in the production of each module. 2. FFH may wish to use a DIP or similar format in future planning for next generation modules, dissemination and technical assistance targets and activities etc. 3. Targets stated in terms of minimal and optimal numbers of testing sites, agencies demanding services, clients reached (albeit indirectly), modules purchased etc. would strengthen planning by establishing measurable objectives. 5.1.2 Quality of DIP and degree of success in implementation A. Findings: 1. Almost all targets have been or will be met by the end of the project. 2. The DIP provided a deliverables checklist, but did not detail quantifiable outputs, as mentioned above. 3. The one quantifiable indicator—grant support for MCH education strengthening—has been exceeded. 4. The DIP does not spell out specific module components, some of which were determined in the process of module development and field-testing. 5. The DIP does not provide details, targets or steps of the dissemination strategy. 6 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 6. Key outcomes, in terms of methodological innovation are lost in the DIP format. B. Conclusions: The DIP format may not be appropriate for the type of product development and capacity building undertaken by this MG. While a useful tool for framing goals, objectives and deliverables, the DIP does not anticipate or capture the type of innovation that characterized the process or products FFH used the DIP format as a framework for its R&D activities, leaving enough margin for creativity. Recommendations: · PVC consider a revised DIP format which allows space for PVOs to a) describe methodological/process aspects of similar R&D and capacity building projects; b) give some indicators of quality. · FFH seek to anticipate measurable parameters of future R&D proposals, as possible. 5.1.3 Familiarity with DIP and design Findings: · Staff directly involved in MG activities reported using the DIP as a management tool. This included all staff on the Research and Innovation Team as well as others involved with CRECER and CARD activities. · The original document identified specific professionals against each module, and proposed a timetable for development. Relevant staff were able to track their progress against the DIP. · Partner agencies were provided with a copy of the DIP. · CRECER reported using other management planning tools, introduced by FFH. · The CARD MOU and work plans worked as a subset of the DIP, based on more or less the same objectives as the DIP. Conclusions: · The DIP was a useful management tool, albeit with the caveats mentioned above. Recommendation: · See 5.1.2 5.1.4 Major successes and shortfalls in implementation Highlights of implementation experience, based on a review of the DIP are summarized in the table below. 7 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Table 5.1.4 Major Successes and Shortcomings in Implementation Implementation Experience at a Glance Major Successes Major Shortcomings 1. High quality, generic CwE MCH modules complete with field-tested learning sessions for village bank clients, training sessions for loan officers undertaking CwE, accompanying TOT guidelines and ancillary handouts and materials for trainers and adaptation guidelines at various stages of development and publication. 1. Limited outreach and alignment with other local MCH/RH programs, providers and messages may miss opportunities for resource synergies, referral networks and reinforcement of relevant messages5 . 2. Innovative, cost-effective, user-friendly MIS system—progress tracking--for monitoring and supervision, designed and adapted in two institutional settings. 2. Limited health background, age and sometimes gender of field agents may constrain the impact of key MCH/RH messages. 3. Results of CwE piloted in a major MFI in the Philippines will lead to institution wide adoption, potentially reaching some 50,000 women by 2005, and an opportunity for showcasing the model and establishing a technical resource for extension and replication in other institutions in the country and region. 3. Dissemination strategy more opportunistic than planned and proactive. 4. Programmatic, technical and growth management assistance to a major MFI in Bolivia consolidates and upgrades CwE in a former FFH subsidiary, currently reaching some 30,000 women; pilots growth management systems for wider application. 4. Absence of an FFH business plan to guide revenue generation in the context of program and sustainability objectives. 5. Links with U.S. and overseas technical implementation partners multiply learning and dissemination opportunities. 5. Reluctance on the part of FFH to disseminate products without accompanying TA may hold back widespread distribution. 5.2 Assessment of project model and hypotheses This section discusses each hypothesis separately and considers the extent to which it was tested in the MG, relevant outcomes, and what can be learned from the process. 1. Achievement of scale and sustainability of CwE institutions depend on · Products that are widely demanded and adaptable to local contexts · Systems that utilize cost-effective, management information · Cost of delivering a quality integrated product kept to a level that can be covered through interest earned on loans · A capacity for transferring the CwE technology to all staff in the implementing organization · Institutional leadership—board and management—share a vision and commitment to scale, sustainability and delivery of credit and education. 5 This item relates only to CARD, which is a newer partner which has not yet established itself in the health arena. CRECER—where CwE is more established—has strong links to health NGOs and government agencies. FFH points out that modules give field staff methods for linking with local providers, assessing local knowledge and incorporating messages. While some of this was evidenced in the Philippines context, it is likely that it has not been as comprehensive as in Bolivia, because of the relatively early stage of the program in that context. 8 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc A. Findings: 1. The CwE model strives for institutional cost effectiveness—and ultimately financial self￾sufficiency. 2. The underlying paradigm relies on two proven avenues for impacting on health and hunger-- raising women’s income and raising women’s access to health information. 3. To maximize impact on development outcomes, credit is combined with health information dissemination: the village banking structure and bank field staff deliver education to women clients attending village bank meetings. 4. FFH has worked hard to develop the interlocking pieces of the CwE model that are sturdy, elegantly simple (without being simplistic) and designed to anticipate differences in institutional requirements and country contexts. 5. Many of these components can be de-linked and used in non-CwE contexts, though FFH considers this a less desirable application. 6. In the past, FFH has taken a more purist approach to CwE, maintaining that the model requires an institution-wide paradigm shift and wholesale adoption of the integrated model in order to be effective. 7. While FFH and partner agencies shy away from assigning a cost to the education component, estimates range from 4.7 to 10%6 . 8. In the context of the CwE model, sustainability of the health education component depends in important ways on the health of the portfolio—if the MFI functions, education continues to be delivered. 9. Scale similarly depends on the institutional reach of the loan program. 10. Both sustainability and scale require systems that keep pace with and are appropriate to growth. FFH has worked closely with CRECER over the life of the MG to develop systems that will work for an MFI in a rapid growth spurt. 11. CRECER has benefited from several generations of innovations of the CwE model. It has achieved financial self-sufficiency working with a challenging population— poor women in geographically remote and economically marginalized situations. 12. CARD, a mature MFI using a modified Grameen approach, absorbed the CwE model and is in the process of upgrading training, supervision and MIS systems as a result of the partnership with FFH. 13. Senior management and board buy in was a critical first step in this process, which has now filtered down to the rank and file loan officers in at least the pilot site. 14. Both CARD and CRECER have designated training units, with Training Directors part of the senior management team. 15. These units have come into their own as a result of the MG, and have taken the lead in field testing new modules and training dissemination. 16. The CwE paradigm requires an institution-wide commitment to learning—using the field to understand and adapt the model—and to adult learning—respecting field agents and clients’ knowledge as the starting point for education. B. Conclusions: 1. When all of the conditions listed under Hypothesis 1. apply, CwE can be a powerful, sustainable model, which can go to scale. 2. While there are definite advantages and synergies in achieving all aspects of the CwE model, it is not absolutely clear that CwE, or even a good development program, require that all of the points listed under this hypothesis strictly apply. 6 Cost of Education in the Freedom from Hunger version of Credit with Education Implementation, Ellen Vor der Bruegge, Joan E. Dickey and Christopher Dunford, Freedom from Hunger, Research Paper, No. 6, updated 1999. 9 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 3. FFH has carefully considered, researched, field-tested and continues to refine all aspects of the CwE model, using its own standards of cost effectiveness, empowerment and adaptability as yardsticks. 4. CwE’s solid credit program is the foundation upon which all other activities rest—it is possible to have sustainable credit without education, but the reverse is not true. 5. Buy-in from senior management within an existing MFI is prerequisite to wholesale adoption of the paradigm and/or the education part of CwE. 6. FFH has done a very effective job in piloting CwE in the well-established village banking context of CARD. In the process, most of the preconditions of the hypothesis have been born out. Plans to go to scale within CARD are under discussion (and form part of the current MG submission). They coincide with internal plans to increase CARD’s client base a hundredfold over the next 8 years. 7. All of the preconditions listed under this hypothesis apply to CRECER which has a highly successful, widely recognized CwE program which is both sustainable and scaling up. 8. FFH has built on its long history with CRECER to address systemic issues of growth management for CwE using FFH’s modus operendii of learning by doing, reflection, and documentation. 9. The success of the CARD partnership substantiates contentions about the adaptability and viability of the model, and has pointed the way to a new partnership model for FFH. C. Recommendations/Thoughts: 1. The synergies from achieving all aspects of CwE mentioned under this hypothesis are greater than the achievement of one or a few. This is analogous for the components of the CwE model. However, one ore more of these components can have an impact on income, knowledge, and behavior though cost and impact may vary. More on this in following sections. 2. Enhanced modules, incorporating learning’s from FFH’s prior series and state of the art adult education training techniques, will make them more user-friendly to facilitators and easier to adapt and implement widely at minimized cost. A. Findings: 1. In the early stage of module development, FFH experimented with a number of approaches to module development, settling finally on a process which brought external expertise to the table for content and technical review, but retained control over module writing. 2. The education modules developed under the MG distill key messages on major MCH topics in a generic form, for use by loan officers from typical MFI partner agencies. 3. FFH has gone to great lengths to anticipate user requirements including highly detailed trainer’s guides and training aids (for both TOTs and loan officers), and adaptation guidelines. 4. Health modules have been scripted closely to ensure that key information is provided and sharing time takes place within a 30-minute period.7 5. The MCH Curriculum Package, supported under the MG, will include all of the Infant and Child and Breastfeeding Learning Sessions in publications, which can be used to mother’s clubs, self￾help groups, CHWs, as well as CwE agencies. 6. Adult learning principles underpin the design of all modules which stimulate group discussion and draw on client’s knowledge, while at the same time seeking to address misconceptions in the context of learning sessions (background notes for loan officers provide a list of possible rumors and provide responses). 7. Business-related modules have also been updated. 7 Time discipline is closely related to fiscal discipline, both hallmarks of the CwE model. 10 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 8. Training Directors in both CRECER and CARD were highly appreciative of the materials, which are also in demand with other PVOs and NGOs. 9. In their application in the Philippines, it was not apparent that assessment had been made of current client knowledge, research existing government programs information messages, or identify health provider or volunteer resources in the target communities (although FFH has incorporated this in training curricula for field agents). 10. Loan officers were provided with background reading materials on relevant topics, and appeared to make good use of them. However, when asked where they would go with further technical questions, there was no clear back up. 11. Importantly, loan officers were coached in their training to say “I don’t know” and “The health expert says….” Rather than pretending to be able to address all questions or issues. 12. Similarly, because field supervisors are bank managers, it is unrealistic to expect them to assess the accuracy of messages delivered. 13. In both CARD and CRECER, limits to loan officer health knowledge and lack of technical back￾up are recognized implementation challenges. 14. CRECER has gone some way towards addressing this by placing trainers with health background in each region who also double as quality supervisors on education. 15. FFH also recognizes the “intersectoral compromise” which the CwE appears to demand. 16. FFH is focusing now on subsequent cycle modules, which take village bank groups who have completed the information sharing modules into second and third generation modules that encourage family-based problem solving and community action. 17. It is also looking at how to give partners the necessary skills to enable them to develop their own modules. This has already happened with CRECER. B. Conclusions: 1. The new MCH modules are excellent stand-alone sessions which can reinforce correct information on important MCH topics. 2. The participatory emphasis is a strength and potential weaknesses of the modules in the context of CwE. 3. Because the modules are delivered by individuals with no health background, and because they stimulate group discussion on topics like contraception, breastfeeding, child health and nutrition, which typically carry some if not a significant rumor burden, the potential for misinformation is a risk. 4. MFIs and other credit-oriented agencies may not be familiar with public health programs or policies which would enhance education messages and follow up. 5. Second and third generation modules represent an exciting next step to move cohesive village banking groups institutions towards social mobilization and advocacy. C. Recommendations: 1. MFIs and other CwE agencies without a health orientation may need further guidelines and guidance on adapting modules to the local context. 2. Specifically, FFH partners should be encouraged to build formal and informal linkages with public and private health agencies and individual providers at national and local level in order to maximize congruency of messages and identify points of reference for loan officers who have technical questions or need to refer clients for more information or services. 3. If Village Health Workers or midwives are members of a particular village bank, they should be encouraged to provide perspectives and input. 4. Similarly, loan officers and bank managers should seek opportunities for inviting local health providers for a Q&A at village bank meetings (albeit maintaining the important time parameters). 11 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 5. Accuracy needs to be an important measure of performance. Assessing supervisors—who themselves may have the same level of knowledge about health issues as loan officers—to monitor accuracy will be important, particularly as programs scale up. 6. The implementation of second and third generation modules would benefit from linkages with other social action agencies and programs. There are several other hypotheses implicit in the education aspect of CwE which bear mentioning here8 : 2.2 Loan officers, who may be young, unmarried, of both genders with little or no public health background, are able to provide sufficient and credible information on MCH topics to clients who are primarily women in the latter part their childbearing years. A. Findings: 1. Many grassroots MCH and RH programs rely on volunteer peer networking to disseminate information and model new behaviors 2. CwE loan officers do not typically fit the age or marital status profile of village bank members, and neither group falls into the typical target group for MCH/RH messages—women early in their reproductive years. 3. Observation of village bank center meetings in the Philippines and reports from CRECER interviews substantiates the hypothesis that age, marital status and even gender need not be a barrier to delivering facilitated education sessions in the context of such meetings. 4. Loan officers carry considerable credibility with clients because of their association with the bank, education and mobility. 5. In addition, local bank and client informants indicated that the significant foreign presence over time in the pilot site in the Philippines also endowed the education modules with additional credibility. 6. Rumors are powerful, particularly in remote settings where mass media is less available as a countervailing voice.9 7. Modules provide a list of common rumors with responses typically starting with “The Health Experts say……….” B. Conclusions: 1. Loan officer credibility enables them to deliver messages to clients with greater real-life experience on sensitive subjects in cultural contexts where age is often used as a measure of wisdom. 2. Although CwE modules anticipate many common rumors it is impossible (and unfair) to expect loan officers (or the modules) to anticipate every rumor or question that arises. 3. Although loan officers are coached to be able to say “I don’t know”, it can be challenging to do so in a situation in which their position of authority is what gives them license to present information to more experienced clients. 4. In an environment where rumors are common, misinformation or mistakes attributed to a loan officer or the implementing agency could jeopardize the credibility of the education program as a whole. 5. As CwE goes to scale, the importance of accuracy is tantamount. 8 These have been added by the Team Leader. 9 During family planning sessions observed in the evaluation, numerous, strong and incorrect assertions of the negative side affects were put forward in all four contraceptive-related sessions observed. In some cases-—such as a much touted rumor about a woman with an IUD which caught the penis of her (illicit) lover causing death to both— even strong testimonials by current users did not appear to be enough to sway the common opinion. 12 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc C. Recommendations: 1. FFH may wish to address issues of gender, age, marital status in advising MFI and other non￾health agencies on criteria for field worker selection, particularly in situations like CARD and CRECER where large numbers of additional staff are anticipated in a time of rapid expansion. 2. While it is possible that older, female loan officers would be able to manage sessions on reproductive, maternal and child health, younger field staff may be more receptive to new information and new learning/facilitation techniques. More research could shed light on the alternatives, at least from a generic perspective. 3. Progress tracking systems that enable supervisors to carefully monitor message content are critical. 4. The availability of health providers, as resource persons (and potentially monitors), would take the onus off of loan officers and branch managers, and provide them with additional technical support. 5. Adaptation guidelines should include methods for researching and incorporating to government messages to reinforce consistent information, and encourage health-seeking behaviors. 6. A large enough MFI may be able to hire a health education professional to backstop training and supervision of the education component. 2.3 Clients will convey messages to daughters, neighbors and others to encourage attitudinal and behavior change. A. Findings: 1. The average age of village bank clients in both CARD and CRECER programs is 34-35. These women are able to undertake small enterprise activities because they are no longer overly burdened with young children. 2. Modules encourage sharing of information with husbands, family, and neighbors by assigning homework—talk about using condoms with your husband or a couple you know. The follow-on sessions begin with a report back—a chance to reinforce key information. 3. Clients reported—in sessions and interviews—that they do share information outside of the group. B. Conclusions: 1. Anecdotal information validates the hypothesis that clients share education messages with others. 2. Whether accuracy is maintained, as messages trickle out is unknown. C. Recommendation: 1. Since the target audience is one step removed from the immediate beneficiaries of CwE, the assumption that messages are accurately trickling out to them may bear further, systematic exploration. 3. A variety of dissemination platforms are appropriate for CwE. A. Findings: 1. In the past, FFH was engaged in direct implementation of CwE through subsidiaries. This allowed it to test and refine the model through active engagement and in-depth research. 2. Today CwE is being implemented through NGOs, PVOs, credit unions and development banks (see list of current partners following the acronyms table). 13 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 3. For historical and programmatic reasons, the starting point for implementation has been the credit side of the CwE model which determined choice of implementation partners. The village-banking infrastructure is necessary as a conduit for education, but the reverse is not true. 4. However, recent interest by health agencies—PVOs and NGOs—will allow FFH to test whether it is possible to embed a village banking infrastructure in a health delivery program (and institutional culture). B. Conclusions: 1. FFH has identified an array of appropriate partners and catalysts for CwE. 2. New technologies developed under the MG will enable this experiment insofar as they have upgraded and aligned credit and health education implementation, supervisory and monitoring systems. 3. FFH hopes to push the boundaries of the diversity of platforms for CwE even farther in the coming years, as outlined in section 6.1 below. C. Recommendations: 1. FFH continue to diversify its portfolio, maintaining its learning approach to new partnerships. 2. Application of components of or the whole CwE model to health agencies bears particular scrutiny. 3. An intentional strategic plan which is explicit about the types, numbers, location and nature of partnerships FFH seeks to cultivate over the next 5-10 years would enable more proactive and targeted leveraging of FFH’s resources, as well as a roadmap against which insiders and outsiders can monitor progress. D. Conclusions: 1. FFH continues to carefully monitor CwE implementation through different mechanisms and to learn what works, and to fine-tune the model to suit. 2. After several years of offering services to whomever was willing to pay, FFH is reassessing this approach in favor of a tact, which will leverage its human resource more strategically—i.e. to maximize scale using CwE in total wherever possible. 3. One of the constraints to this approach is the cost of FFH services, which precludes partnership with local NGOs and MFIs in the absence of third party funding. 4. Each platform requires a unique configuration of tools, skills and systems to launch and sustain CwE. This requires products that are sturdy and at the same time adaptable. A. Findings about the product: 1. Cognizant of the rich variation in institutional and cultural contexts, FFH has gone to great lengths to anticipate the steps and questions that will arise as institutions adapt the CwE model. 2. A hierarchy of manuals and tools is worth mentioning in this regard: a. The CwE package—published under the MG—is a menu of services and a generic timetable for implementation of CwE. This is more of a marketing tool from which clients can select the types of services they wish FFH to customize. b. The Field Agent Operations Manual is a reference guide for the field staff that deliver the credit and education services. It features information on the promotion, formation, training and monitoring of the Credit Associations as well as basic information about the delivery of the education component. c. The Learning Game is a complementary document to the Field Agent Operations Manual. It is a four-hour simulation of key events in the promotion, organization, training and 14 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc implementation of Credit Associations. It is used in the initial training of the field staff as a new CwE practitioner organization launches the credit and education services. d. The Coordinator Operations Manual—in production with MG support—is a reference guide for the supervisor (branch manger) to address implementation issues particularly related to the selection, support and signaling to field staff—considered an essential element of the CwE approach. e. Tools also anticipate adaptation. For instance: · Progress tracking begins with a mapping of existing information content and pathways, and an assessment user needs; · Each module contains an adaptation guide that is a process for transforming generic learning sessions into locally appropriate presentations. · FFH has piloted the addition of a CBD component to its family planning module— largely in response to unmet demand for contraceptives by CRECER clients in remote areas of Bolivia. · Still a work in progress, the CBD module promises to be a valuable step by step guide for planning and implementing all aspects of a CBD program which compliments the CwE approach and is congruent with its main principles of women’s empowerment, income generation and health education. 3. Background for MCH modules contains a list of rumors, with correct, “The Health Expert says…” scripts for loan officers to deliver. This provides a format which can be revised based on the local situation. B. Conclusions: 1. FFH has developed a generic CwE toolkit which anticipates the technical, management and implementation needs of an institution. 2. These tools have benefited from and grown out of years of working with the model in diverse contexts and the special attention FFH takes to reflect on and document learning. 3. Tool development has assumed some level of FFH input at the very least in introducing or adapting their use to individual clients. C. Findings about the process: 1. During the MG period, FFH commissioned an external study of its client base10 to enable a better understanding of its current and potential users. This study was not funded by the MG, but impacts upon dissemination strategies that are. 2. The study indicated to FFH that they were perceived as sometimes too rigid in terms of the way they promoted the model as an all or nothing proposition. 3. It also suggested to FFH that it was perceived to be sheltering the “rights” to the intellectual property developed around the CwE model. 4. The study catalyzed considerable soul searching within FFH, resulting in a directive from the Board of Directors that de-linking components of the CwE model was acceptable in cases wherein doing so would attract new collaborators, provide new opportunities to innovate and learn, enhance FFH’s reputation, provide an advocacy opportunity, generate income to support FFH’s core business or have a minor impact on staff time.11 10 Ibid, footnote 1. 11 Draft memo from Chris Dunford to FFH International Center Staff, October 31, 2001. In an important rethinking of FFH’s mission as synonymous with CwE, this memo states: “We promote and support Credit with Education as a bundle of services distributed by other organizations (they “retail” our “product”)…. The strategic question before the Board of Trustees was ‘How much should we be prepared to ‘unbundle’ these services in order to do business with other organizations?’” 15 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 5. While this may seem common sense to the outsider, it represents a significant (and pragmatic) compromise by FFH which has agreed, albeit reluctantly, that module and other tools may be used without direct FFH input. 6. In addition to agencies that have been involved in field testing new modules and methodologies, FFH is beginning work and negotiations with a number of other agencies—sometimes introducing a piece of CwE as a (hoped for) catapult into mainstreaming the model. For instance: a. FFH is providing assistance to SEWA/India—a major bank in a key country--on measuring client satisfaction, using progress tracking. b. Several health agencies, including Andean Rural Health Agency (Curamericas) and PCI (Project Concern International) have approached FFH about integrating microlending into their program. c. FFH will be presenting the new modules at an upcoming meeting of the CORE (Child Survival) group of PVOs in an effort to expose more agencies to the model. 7. In addition, FFH has engaged with the Hesperian Foundation (publisher of Where There Is No Doctor) to help format the health modules for publication. D. Conclusions about the process: 1. FFH demonstrated its commitment as a learning and client-oriented organization by commissioning the study of client perceptions. 2. Issues of ownership and internal integrity of the product are real, valid and likely to be on-going debates within FFH and with its client base. 3. The dialogue and decision making process that the IDR study precipitated reflect an engaged board, with close staff alignment, genuine partnership relationships (report findings were shared back with study informants), an agile organization, and a continued emphasis on learning and intellectual growth that is one of the hallmarks of the organization. 4. Given its historical tenacity to the integrity of the model, this decision is a bold step forward. All concerned appear aware of its risks. 5. The ramifications of these decisions on CwE methodologies and tools is significant, as it demands that these products be able to perform in a greater diversity of contexts in the hands of a greater diversity of users, and potentially to function independently, outside of the CwE model. 6. It also recognizes that inevitably many of the products will be and are being co-opted independent of FFH input or the CwE model. (This issue is taken up in the following section). 7. This may be the single most important challenge of the next generation of FFH products— beginning with those developed under the MG. E. Recommendations about the process and products: 1. FFH begin to think of giving away (and selling) its products as a marketing tool which will leverage greater demand for its services—but not in every case. 2. In this regard, the website and other PR portals should be restructured to draw attention to the quality products and services available through FFH.12 3. Now that a policy decision has been made, FFH should seek to set more specific targets and parameters for “roll out”, via a strategic and/or business plan (a roadmap with clear signposts) to ensure that marketing and advocacy remain consistent with the agency’s core mission and goals. 12 It appears that a new website ffhtechnical.com, will have all of the products, once they are finalized. 16 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 5. High quality products and tools introduced by a catalytic team will maximize dissemination in a cost-effective way. A. Findings: 1. As mentioned above, most of the CwE tools have been designed to be introduced by FFH. 2. FFH staff feel strongly that orientation by a FFH or LTSP professional on adult learning principles and techniques followed by a walk through of every module for senior organization level trainers is a necessary prerequisite to proper use of the modules. 3. FFH’s mentoring as well as training has been significant in helping its partners come as far as they have. 4. If this hypothesis is correct, and if interested agencies abide by it, cost and scale up implications are significant. 5. FFH’s Local Technical Service Provider strategy—described in greater detail below—will establish a CwE training capacity in most regions. 6. However, it is not anticipated that all LTSP’s will be able to provide the level of business development, mentoring and troubleshooting support that has been so important to the MG partnerships. 7. With regard to access, it is realistic to assume that where there is a demand people will find ways of putting their hands on products with or without permission or technical assistance. This has already happened. 8. FFH recognizes all of this, and is grappling in the glow of growing demand for its products with attempting to maintain the integrity of the CwE model. B. Conclusions: 1. The outcome of this struggle will probably fall somewhere between purity and pragmatic reality, with FFH unlikely to be able to completely control what happens once its products “hit the street”. 2. FFH is like to have to continue to deal with the problematic area of intellectual property and corralarily the tension between its goal of taking the model to scale, quality, attribution, and an understandable entrepreneurial desire to glean some remuneration from its efforts. 3. With respect to adult learning: although school systems in most developing countries do not develop the fundamentals of positive inquiry in students, many agencies have successfully integrated this approach in their programs and methodologies. It is not unique to CwE. 4. The MG allowed FFH to provide partners with new tools, training in their use as well as mentoring and trouble shooting—these synergistic inputs, while placing limits on scale—are clearly more powerful than one element on its own. C. Recommendations: 1. FFH continue to balance concerns of quality with scale in disseminating products with and without technical assistance. 2. In particular, the development of the LTSP network of cost effective, language- ready trainers in priority regions who are not just skilled in the CwE methodology but also empowered to actively market their services will help to maintain the balance in favor of a more quality outcome.13 3. Consistent with them, closer attention to the individual or institutional attributes of a trainer or training agency which can also provide the mentoring and leadership support to new CwE clients needs further attention. 13 FFH points to some of its partners—notably FFH/Ghana and KAFEM/Haiti who have been highly entrepreneurial in promoting the model. Whether FFH can train others to do so, is the challenge. 17 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 4. Without expending large amounts of money, a comparative study of implementation in a “with and without TA” context could inform both future TA (what is absolutely essential?) and strengthen modules and manuals (what would they have to know if we weren’t here?). 5. FFH may wish to include a disclaimer in the front of published (and non published) tools and manuals explaining the type of “living” TA that is a desirable companion to the documents, and providing reasons and examples. 6. FFH may also wish to research and short list agencies in each region which offer adult learning facilitation training, and even to proactively seek out partnerships with them as part of its LTSP network building. 6. An information system that is cost effective, flexible, highly participatory and embedded in an institutions’ management system is essential for monitoring quality, optimizing management decision making and multiplication of the model. A. Findings: 1. Progress tracking, an FFH original, is an innovative menu of generic tools that can be adapted and used for supervision and program impact assessment. 2. With methodologies borrowed from PRA, business, and management, progress tracking offers a fast turn-around alternative to cumbersome MIS systems and KAP surveys. 3. The objective of “zero sum impact” on field staff work, and a cost consciousness that weighs the addition of every data item against the proposition that borrowers are paying for it, have gone into careful and creative construction of the CwE progress tracking approach. 4. Progress tracking is part of the CwE paradigm, which values information as power in the hands of staff and client. 5. It is another of FFH’s tools that is deceptively simple, but not simplistic. 6. Enthusiastic about the kind of information and staff participation progress tracking offers, CARD has already absorbed progress tracking tools and approaches into existing MIS systems and is enthusiastic to do more. 7. In particular, the use of mini-surveys to assess client satisfaction and checklists to monitor loan officer performance in education sessions have been enthusiastically adopted by CARD. 8. Progress tracking, and FFH’s infectious emphasis on learning by doing has imbued the Research Team of CARD and CRECER with new skills, enthusiasm and status within the organization. A recently conducted client satisfaction survey, carried out by CARD’s Research Unit resulted in changes in loan policies. 9. CRECER also expects to use these tools to measure client satisfaction. 10. As with the modules, the introduction of progress tracking has to date required intensive hands on training. 11. The human resource on both ends of the partnership is limited—while more staff at FFH are familiar with in progress tracking, there is really only one person—who pioneered the system-- with total fluency in all aspects of the system. 12. Likewise, few if any development agencies have research arms large enough or MIS units flexible enough to adapt this type of innovative system in one gulp. 13. FANta (Food and Nutrition Technical Assistance project) has provided support in addition to the MG to develop the progress tracking tools. FFH has produced a series of papers on various tools, which it continues to test and develop with FOCCAS, CRECER and CARD. 14. FANta is also assisting FFH with dissemination to the larger NGO/PVO community. B. Conclusions: 1. Progress tracking has significant potential for a variety of types of development agencies. 2. In particular, health agencies which struggle with mountains of unusable data may find applications for the system—or components of the system—even outside the CwE model. 18 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 3. Appropriate application of progress tracking may require significant changes in the way data is managed within an organization. 4. It may also demand a philosophical shift to support an organizational culture, which empowers managers and staff with collection, analysis and use of information, and takes a minimalist approach to data items and sample size in favor of cost efficiencies. 5. Once again, the issue of quality vs. cost and capacity is relevant—to do it right, progress tracking may in fact need training and mentoring to get it right. C. Recommendations: 1. FFH needs to look for cost effective and practical ways of introducing progress tracking into organizations that may not have the staff, training or absorptive capacity to incorporate it at one time. 2. A manual which systematically lays out the system, describes the use (and abuse) of individual tools, and includes incremental learning sessions for field and supervisory staff, is critical to allow multiplication of the use of progress tracking through the CwE partnership network and beyond. 3. LTSP’s should be trained in progress tracking. 4. At the same time, FFH may wish to consider developing another cadre of LTSP that specializes in progress tracking—professionals with qualitative research expertise present a potential pool of candidates. 7. FFH organizational capacity to enhance the CwE product and systems will improved if it hires more staff in health and adult education. A. Findings: 1. Before this MG, only one staff person was designated to the health education aspect of CwE. 2. Although FFH only anticipated hiring one health professional under the MG, it hired two – a tripling of expertise and human resource to the subject. 3. Both staff bring special technical and regional expertise which has added fresh and complimentary perspectives to the existing staff. 4. The new staff also bring a particular expertise and commitment to adult learning a la Jane Vella. 5. The presence of the new staff and activity surrounding work with the MG has opened new doors for FFH in the health community.14 6. Other staff have contributed to module development and integration considerations. 7. While the emphasis in FFH remains on the credit aspect of CwE, hiring of the two staff has infused the dialogue and emphasis within the organization with greater balance vis a vis the model. 8. The outputs in terms of sheer volume and quality of products and technical assistance based on this rebalancing—and the new staff—is significant and impressive. 9. FFH is committed to retaining these staff who are now seen as essential members of the team. B. Conclusions: 1. FFH has proactively “righted the ship” in the words of a senior staff, in bringing on more health expertise. 2. Their presence has also re-energized a focus on the health education aspect of the CwE model. 3. It has also drawn in a host of new and potential clients and partners. 14 Senior staff Ellen Vor der Bruegge is on the Board of CORE and very active in the Child Survival community. The MG enabled new partnerships with LINKAGES, CAREMoRR, FANTa and a host of agencies related to these and other projects. 19 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc C. Recommendations: 1. FFH find ways of partnering with health PVOs, NGOs and others, although this may be more challenging than the traditional MFI partners. Aside from use of the modules, progress tracking offers an area of potential technical assistance. 2. FFH will have to balance the potential demand from health agencies for technical assistance and products concerning one slice of the CwE repertoire with its desire to transfer CwE whole. 3. FFH may wish to develop a roster of health consultants who can be drawn in to develop new modules, advise partner agencies, or conduct subject specific training as necessary. 5.2.2 Replication and scale up of approaches in project area or elsewhere A. Findings: 1. Replication and scale up are central to FFH’s mission, not just this MG. 2. This section will consider replication and scale up within the two partner agencies and country contexts and the issue of replication and scale up more broadly for FFH (which has already been touched on in the previous section). 3. Partnerships and partnership targets established during the life of the MG are discussed in the next section (6.1). 4. CRECER’s original raison d’etre was to test and replicate the CwE model. 5. Today, as an independent NGO, it reaches 30,000 clients. By moving out in concentric circles from areas of current service delivery, CRECESR hopes to reach 75,000 clients by 200615, with 70% of clients in rural areas. 6. CRECER’s replication and sustainability strategy cross-subsidize income from more well off, urban clients with revenues from its primary target group—poor, rural women. 7. In a crowded microfinance environment, and in the absence of government registration (which limits access to soft loans)16, CRECER has managed to scale up by: a. Focusing on very remote and difficult areas of the country where MFI and health services are scarce or absent;17 b. Maintaining positive relationships with clients through “extra” services, including education, health-related benefits (CRECER members get special privileges at NGO and government health facilities); c. Establishing strategic relationships with other MFIs, NGOs and PVOs. 8. CARD has set itself a very ambitious scale up target. Currently serving some 50,000 mostly landless, rural women, it expects to reach 500,000 by 2010. 9. This target is based on the financial health of the organization, projections based on a dependable pipeline of funding for expansion (reliant, primarily on internally generated funds), and assuming extension of the CwE model to 50,000 of its clients by 2008.18 10. As a leader in the MFI movement in the Philippines and Asia, CARD will also serve as a showcase for the model, and plans to provide technical assistance in CwE to other institutions nationally and in the region (this is spelled out in the new MG proposal). 15 Interestingly, CRECER’s 5-year plan (2001-2005) lacks targets for this period. This number was provided by FFH and appears in the Business Plan, not yet out. 16 In order to attain registration, CRECER would have to change the types of services it provides, and specifically eliminate the integrated approach. 17 In addition to areas already covered by CRECER, the NGO was invited by USAID to join a project targeting some of the remote, dangerous, narcotics growing areas of Bolivia—this represents not only a physical risk for CRECER staff, but a fiscal risk to the organization as crime and theft are commonplace, and attrition anticipated to be higher. 18 “Breaking through barriers to Growth”, Application for USAID Matching Grant, December, 2001 20 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 11. For Freedom from Hunger, replicability and scale are the burning issues of the moment—the recent Matching Grant proposal, which brings together the collective forward thinking of the organization is entitled “Breaking Through Barriers to Growth: Expanding the Scope and Scale of Credit with Education in Two Regions”. 12. Scale and replicability have always driven FFH’s mission, which is the eradication of hunger on the planet. 13. At this particular moment in the agency’s history, FFH appears poised to leverage its accumulated experience, track record, partnerships, networks and new products to break through its own barriers to growth—some of which are discussed above. 14. Strategic alliances formed with credit unions and credit union affiliates in West Africa and CARD in the Philippines, as well as emerging partnerships with several other major MFIs and PVOs offer additional potential for scale. 15. Local Technical Service Providers are being groomed to multiply the model. Freedom from Hunger/Ghana, for instance, has provided technical assistance to ActionAid, Plan International, UNICEF, CARE as well as a local rural bank—Brakwa Bremen—which is in turn training other rural banks in CwE with support from DANIDA. 16. Anomalously, while targets have been set in the MG proposal, senior FFH staff seem less sure about overall targets for the next 5-10 years. 17. The current strategic plan projects reaching 330,000 clients through direct partnerships and 660,000 for agencies applying the integrated approach by example by 2003. 18. Immediate projections for the last several years indicate a lag. Projected borrowers for 2002 were 261,781. Actual members totaled 201,423, with 156,094 borrowers. 19. On the other hand, the internal loosening up and cautious de-linking of the CwE components may allow FFH to multiply its impact more rapidly to more partners—albeit not always with the more powerful Credit with Education package. 20. FFH is committed to remain at approximately its current staff size, so replication breakthroughs will necessarily come through strategic partnerships, LTSP networks, demand for products and training. 21. Limits to scale may be challenged by the level of institutional input FFH requires, particularly for new partners. 22. FFH’s focus on growth management—and particularly adjusting CwE management systems for scale up—is a good example of its careful, holistic approach to sustainable scale up. These growth management strategies have been pioneered with CRECER and will no doubt be adapted with CARD and other partners. 23. A note on measuring scale is relevant here: FFH counts its clients as those women who are active members of village banks which utilize the Credit with Education methodology. To date, they total 200,000 members (156,094 borrowers), associated with nearly 9000 credit associations, with a total of $12m loans outstanding. 19 24. When compared to FFH’s target of reaching 1.3 billion hungry people, its accomplishments are modest, almost embarrassingly so. 25. There are several reasons for this: one is that FFH has been more focused on refining the model than on replication and scale—until very recently. 26. The other has to do with the way clients are counted. 27. By restricting the number of tallied beneficiaries to village bank members, the numbers ignore the family members, friends or neighbors who may benefit directly or indirectly from income generated, business advice, or health education. 28. The design of the project assumes this spread effect, as discussed above but does not take credit for it. 29. FFH is aware of this measurement option and has made an intentional decision to count clients this way—probably because of its strong affiliations with the MFI community. 19 Ibid 21 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc B. Conclusions: 1. FFH’s strategy for forging strategic alliances with a number of large, established MFIs and the positioning of LTSPs in key regions offer the potential for rapid scale up of CwE. 2. FFH may be short-changing itself by counting beneficiaries in terms of banking clients only. 3. Scale up within CARD is limited more by the lack of human resources with a good grasp of CwE methodologies, than existing systems. C. Recommendations: 1. FFH needs to develop more specific targets for scale up which include regional, institutional, client, and impact targets. 2. This should be done as part of a business plan that looks at the benefit/risk of various investment options for FFH’s scarce human resource—measuring each against the potential for achieving scale with quality of Credit with Education. 3. Given the strength of the FFH-CARD partnership, growth management strategies piloted in CRECER should be further tested, refined, adapted and documented with CARD. CARD's capacity in this area will be an invaluable resource in the region for scale-up. 4. More research needs to be done on the trickle out effect of education information (which will, inevitably be difficult to measure and different in each setting and most likely to increase with time and exposure to incremental generations of modules). 5. Based on averages from a number of sites, FFH would be able to assign a multiplier to its client base to estimate actual numbers of people reached. 6. A disaggregation of this in terms of how many women and men in their early reproductive years an average client talks to about information she has learned in CwE would be helpful. 7. This is a tall order, but perhaps possible with proxy indicators and using assumptions from the Behavior Change Communications field. 5.3 Advocacy under the project 5.3.1 Advocacy activities and impact A. Findings (see Annex D): 1. Advocacy for the integrated approach to credit and health education is a key aspect of what FFH does. 2. Careful analysis of various aspects—longitudinal, case studies and financial analysis of CwE experience in the field has given FFH an arsenal of research which it draws upon in its advocacy work. 3. FFH has engendered and nurtured this in its partners and models it in its own activities. 4. Partnerships with AED-sponsored projects, the CORE and SEEP networks, a workshop of credit unions sponsored by FFH in West Africa, and other types of interface have provided FFH with many opportunities to continue to promote the synergistic, cost and developmental benefits of the integrated approach. 5. Careful and sometimes challenging advocacy with important players in the credit union— Desjardins from Quebec and the World Council of Credit Unions (WOCCU) are beginning to pay off in partnerships in West Africa—a major target area (where there is the deepest hunger). 6. Despite these regional achievements, there appears to be a general disappointment within FFH with the lackluster response from the international MFIs. 7. On the other hand, quiet advocacy with CARE/India and SEWA/India has begun to bear fruit, opening FFH’s second major target area (where there is the greatest number of hungry people live). 22 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc B. Conclusions: 1. FFH stands out from many other NGOs in terms of the thoughtful, well-documented research it has undertaken over the years. 2. This has given the agency—and its partners--a firm platform for promoting the integrated model. 3. Thorough research by external agencies lend significant credibility to these efforts. 4. FFH has undertaken its advocacy work, oftentimes against rather resistant odds. 5. However, by its own admission, FFH has not impacted on the major players—donors, MFIs— some of whom are unaware of its work. C. Recommendations: 1. Study tours for senior management of target agencies could be an effective advocacy tool. 2. FFH can use its new modules to further advocate for CwE by including brief testimonials, compelling data and short case studies in published documents. 3. USAID, a major direct and indirect investor in FFH and CwE needs to bear some responsibility for advocacy. If CwE merits on-going support, then perhaps it merits wider exposure, for instance with CA’s, other parts of USAID, and other donors. 4. FFH may wish to explore ways of measuring the impact and reach of its advocacy work for the integrated approach, in order to justify additional funds to this aspect of its work. 5.3.2 Partner/PVO roles in advocacy A. Findings: 1. During the MG both CRECER and CARD both exploited similar opportunities for advocacy within their own peer networks. a. In CRECER’s case this included active participation in both a network of MFIs (FINRURAL) as well as PROCOSI, a PVO/NGO Child Survival network. b. Several studies, past and current, have helped CRECER tweak its program and advocate for it. c. CRECER openly invites researchers to study its achievements, mirroring its mother agency as a receptive, learning institution. d. CARD, a leader in the MFI community in the Philippines, “shocked” other MFIs20 by announcing its intention to integrate education with its highly successful village banking services. e. Peer agencies are observing with interest as CwE unfolds; the pilot area has already attracted visitors, despite its distance from Manila. B. Conclusions: 1. Both of the main MG partners as well as many of FFH’s other partners routinely advocate for the integrated approach—with peer agencies, donors and government. 2. FFH partners exhibit an enthusiasm for sharing and promoting CwE. 3. Most local advocacy appears to be carried out independent of FFH, in the context of CARD’s regular meetings with peer agencies, government and donors. 4. FFH partners have benefited from research carried out by external investigators—some of it under FFH auspices, some not. 5. CRECER and CARD appear to be primarily focused on internal programmatic and institutional issues and putting the foundations for rapid scale up in place. 6. The level of advocacy currently being carried out is appropriate. 20 Interview with Aris Alip, Founder and Acting CEO of CARD NGO and President of the Microfinance Council. 23 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 5.4 Implementation Lessons Learned A. Findings: 1. Much has already been said about the benefits of the MG on the quality of tools and methodologies used in implementing Credit with Education—FFH’s core business. 2. The process of developing, testing, refining and disseminating these tools has engaged a diversity of institutions and individuals in a multi-layered dialogue, orchestrated by FFH. The benefits of this dialogue are important, as they have catalyzed learning and innovation within participating agencies, in addition to strengthening products and methodologies. 3. Of special note here is the voice field agents and clients have had in module production: FFH has developed ways of systematically but sympathetically soliciting input from end users. B. Conclusions: 1. FFH has used the opportunity of the MG to upgrade many aspects of the CwE model, while at the same time continuing and expanding its networked learning approach. 2. FFH is a good example of a learning organization. C. Recommendation: 1. The approach adopted by FFH in product development, though laborious and detailed, bears observation by development agencies still stuck in a top-down technology transfer orientation to “innovation”. 6.0 Partnership Questions 6.1 Analysis of Partnership Schemes (Annex E) A. Findings: 1. Since FFH is no longer an implementing agency, partnerships are essential for achieving its mission. 2. FFH’s partnership networks can be depicted as a series of concentric circles, with now￾independent subsidiaries—CRECER, FOCCAS, FFH/Ghana, as well as close allies such as CARD on the inner circle; on-going CwE partners and LTSP institutions—many of whom agreed to have the modules tested in their programs (CRS/Madagascar, CUES/Philippines, KAFEM/Haiti); clients past and present who have incorporated or piloted CwE in one geographic area, or adapted a part of the CwE approach (WV/Indonesia, WV/Malawi, Plan International/Ghana, Plan/Senegal, Plan/Togo, Plan/Bolivia, Plan/Mali, Plan/Philippines, ActionAid Ghana donors, supporters and technical colleagues such as USAID, LINKAGES, a host of loyal, private donors; and on the outside the greater PVO, MFI and NGO community. 3. Annex E shows the number of networks in which FFH participates, both directly and indirectly— a key aspect of its approach to leveraging advocacy. 4. FFH articulates its partnership agenda in terms of learning together, open dialogue and shared vision. 5. FFH’s scale-up goals rely in important ways on partnership. In particular the following types of agencies, some of which are already in FFH’s orbit, are the focus of past, present and planned courtship efforts: a. Credit unions, which already provide services to large numbers of clients all over the world offer a potentially sizable, existing infrastructure for delivering credit and education to the poor. 24 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc · FFH has been attempting to engage with major credit union confederations over the last several years with mixed results (resistance to adding other non-financial services and a poverty focus). · In particular, attempted cooperation with Desjardins—a major Canadian Credit Union federation and the World Council of Credit Unions (WOCCU) have not brought the alliances FFH had hoped for. · A West African summit convened by FFH under the MG brought together new and existing CwE partners and re-energized a focus on poverty lending and the principles of CwE, leading to planning for the future (and the next generation MG). b. Local MFI partners such as development banks, Grameen replicate MFIs offer fertile ground for CwE replication. c. CARD, CRECER, FOCCAS, FFH/Ghana have been able to showcase the model, provide opportunities for local advocacy on the integrated approach, and offer an incubator for institution-based local technical service providers (LTSPs). d. In addition to these existing partners, FFH has begun working with some other major local MFIs during the MG period, including: · SEWA, India · FUNDAP/ Guatemala · FINCA/Peru · FINCA/Tanzania · Fonkoze, COD and Aclam in Haiti · Tiave (IRAM) and Vola Manahasoa (CIDR) in Madagascar · FITSE / Malawi. e. Local Technical Service Providers (LTSPs) are individuals and organizations which have staff who have been trained and mentored in the CwE approach, and are able to provide training in CwE start up and each of its components. · FFH trained LTSP providers in Haiti and Madagascar under the UNDP Microstart program. · FFH is currently developing a strategy for LTSPs which will describe: · Capacity development for an LTSP · What services they will provide · How to mobilize them · Certification process · FFH anticipates having at least one LTSP able to provide training in each major language. f. Health PVOs such as PCI (Project Concern International), Helen Keller International, Andean Rural Health ((Curamericas). g. Self-help groups such as women’s, youth, civic or other networks with established infrastructure and a development focus may also provide a platform for launching CwE. · FFH has not explored this as extensively as the other channels, although it financed participation in an assessment of Pact’s Women’s Empowerment Program in Nepal—a credit and literacy program of women’s self help groups which went to scale quite rapidly—in order to learn lessons for the future. 6. Matching Grant partnerships with CARD and CRECER are reviewed here. 7. CRECER, formerly a branch office, has persevered through various management challenges, fiscal troubles etc. FFH has stood by CRECER. a. Under the MG, FFH has worked with CRECER to establish a new Board, hire new management, address major programmatic and systems issues. b. It has done this in such a way that CRECER appears to have emerged empowered, energized and ready to expand. c. Importantly, the MG did not underwrite any operating costs, only the cost of technical assistance to CRECER. 25 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 8. In contrast to CRECER, CARD came to the partnership with FFH fully formed. a. The negotiations for participation in the MG set a tone for the partnership which both agencies valued: CARD agreed to participate, if FFH shielded the organization from onerous reporting requirements. b. Within the partnership, FFH has built staff capacity at all levels, through training and mentorship; it strengthened systems and enhanced the education approach. c. CARD allocated human and logistical resources; it brought an active, engaged intellectual and technical expertise to the table that is highly valued by FFH. 9. FFH shares a vision and commitment to similar values with both agencies. B. Conclusions: 1. FFH has focused on cultivating strategic partnerships with institutions that can maximize scale and multiply the model. The results have been mixed, largely because of resistance in the MFI sector to the integrated approach. 2. While there has been a genuine reciprocity in both MG partnerships, it is also the case that FFH has invested significant staff resources in both agencies. 3. While this investment has already reaped significant and sustainable dividends in terms of field testing a variety of tools, enabling the establishment of LTSP's in two regions, and establishing a showcase for CwE in a prominent MFI in South East Asia, these types of partnerships may not be replicable from a cost or human resource point of view.21 4. FFH is acutely aware of this fact and the reality that third party donors are essential for allowing this type of partnership with local NGOs. 5. Having said this, from both partners’ point of view, FFH inputs have been of a quality and consistency that the impact has been, by all accounts significant, and sustainable. C. Recommendations: 1. FFH’s focus on leveraging partnership for scale is appropriate. 2. While FFH staff appear to hold a shared vision for targeted partnership, a clearer articulation of strategies and an estimate of resources necessary to achieve various partnership configurations in a working business plan would help clarify the roadmap and resource requirements. 3. FFH’s LTSP strategy needs to clearly identify the reciprocal responsibilities of FFH and LTSP’s, including marketing of services, quality monitoring certification, and fees. 4. The LTSP strategy should be a component of FFH’s overall business plan with specific institutional client or revenue targets for each LTSP—if FFH invests in them, there should be an explicit expectation of return. 5. Reciprocal visiting staff arrangements with LTSP partners would deepen learning opportunities and could provide additional advocacy opportunities. 6.2 Measuring Institutional Capacity A. Findings: 1. FFH did not do a formal capacity assessment of either of the MG partners. 2. FFH has employed a variety of organizational capacity assessment tools including the SEEP Institutional Development Guide and framework, and UNDP’s Sum Institutional Development 21 From October,1999 to December 31, 2001, Freedom from Hunger provided ~$59,000 in TA ($149,410 including travel, communications and other direct costs) to CRECER and ~$47,000 in TA to CARD. At least 4 professional staff have had input to each agency over the life of the MG. 26 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc checklist, and MEDA’s organizational assessment tool (Mennonite Economic Development Association). 3. In addition FFH does cycle assessments and product performance assessments with partners on a routine basis. A set of checklists and reports guide a participatory “audit” which looks at quality in the delivery system and general management of the CwE product. Management can use the feedback to set training and supervision priorities for the organization. 4. Part of progress tracking includes a mapping of information flows and data needs in the organization which provides the baseline for prioritizing information users and information requirements, and assessing where changes (and efficiencies) can be made using the new tools. 5. FFH plans to explore cooperation with Pact to consolidate capacity building aspect of its work with local organizations. B. Conclusions: 1. Although many of the activities carried out under the MG undoubtedly strengthened the capacity of the two partner agencies (and required informal assessment to identify areas for strengthening), the fact that no baseline of capacity was undertaken is understandable given the focus of the partnership on product development and field testing. 2. Also, given the nature of the relationships with both institutions, a formal capacity assessment may have been considered inappropriate unless requested by the local agency. C. Recommendations: 1. FFH’s mapping tool for progress tracking may provide a template for other types of institutional diagnostic tool. 6.3 Constraints to partnership A. Findings (see Appendix E): 1. It was challenging for FFH partners to identify constraints to partnerships even after significant probing. 2. FFH’s greatest constraint to partnership probably lies in a lack of unrestricted funds which would allow it to partner with more local, likeminded agencies. This has certainly been a constraint in their courtship of credit unions. 3. FFH is seeking to address this through its new venture philanthropy strategy (discussed below). B. Conclusions: 1. FFH brings a lot to its partnerships in terms of technical excellence and a respectful, open learning approach. 6.4 Information Technology A. Findings: 1. As MFIs both partner agencies rely on information technology to maintain ledgers and monitor their portfolios. 2. CRECER has received considerable technical advice and hired new staff to address their hardware and software needs over the life of the Matching Grant. TA has supported systems upgrading with respect to portfolio management, communications, maintenance systems, and staff training. FFH has facilitated aspects of this process, but not did not provide technical assistance or funding. 27 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 6.5 Use of local networks and service organizations A. Findings: 1. As mentioned in previous sections, participation in networks is an important advocacy and dissemination strategy for FFH and its partners. 2. Both CRECER and CARD are active participants in technical MFI networks in their respective countries and CRECCER also participates in the USAID-sponsored Child Survival Network (PROCOSI), as mentioned above. 3. This participation provides both agencies with access to information, peer input and dialogue, a platform for dialoguing with external agencies—donors, government, funding (in the case of PROCOSI), and an advocacy opportunity. 4. CARD participates in ImpAct, an action research project of the Ford Foundation supporting the development of management learning sessions or impact evaluations. B. Conclusions: 1. Networks are an important aspect of FFH’s CwE replication strategy. 7.0 Program Management 7.1 Strategic Approach and Program Planning A. Findings: 1. The MG application process appears to have been an important catalyst for planning in FFH. 2. The application for the present and proposed MG’s reflects the collective vision of senior staff in terms of next step priorities in the evolution and multiplication of the CwE model. 3. The existing strategic plan (2003) and the 5-year PST business plan are both considered outdated. 4. As CEO Chris Dunford puts it: “We did a business plan for the Practitioner’s Service Team, but the real world factored in. Now we need to lick our wounds and see where the real market is.” 5. In contrast, a detailed plan for FFH’s newest fundraising initiative—venture philanthropy, which seeks to attract social venture capital to underwrite start up costs for promising new CwE partnerships—has been approved by the Board. 6. It is ironic that FFH’s capacity building for partners relies so solidly on good planning systems, but that FFH itself has no firm plan in place. A lot of effort has gone into assisting CRECER with its strategic and business plans, for instance, but FFH’s are out of date. 7. CEO Chris Dunford suggests that business plans are quickly outdated and the test of a good business is its agility and responsiveness to new market opportunities. B. Conclusions: 1. Although planning is an important part of what FFH does and promotes, it has not been a recent priority within the organization. 2. One reason may be the significant changes inside FFH’s described in previous sections, which have rendered previous plans outdated. 3. Another may be that the small staff appears committed and clearly focused on a collective vision embodied in a single product—leaving only small margins for individual deviation in interpretation at the margins. 28 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc C. Recommendations: 1. FFH needs to develop a business plan which identifies programmatic and fiscal targets, strategies and a clearly articulated statement for its expanding partnership strategy. 2. Importantly, in addition to anticipating regional, institutional and financial targets and strategies, the business (and future strategic) plans need to incorporate the LTSP plan in terms of its pivotal role in extending FFH’s reach. 7.2 Country Initiatives A. Findings: 1. FFH considers USAID a development partner and seeks to share information as appropriate. In Washington DC, this is largely via network meetings such as CORE and SEEP (not to mention annual, MG reports). 2. The only information the USAID mission in the Philippines had about the MG project was from briefings initiated by FFH representatives as well as the recent MG proposal which was circulating for mission comment. 3. No other donors were contacted in the course of this evaluation, though it is clear from the number of Foundation and UN donors who have provided support to FFH and affiliates over time that the organization does talk to donors about its program. B. Recommendations: 1. FFH may wish to update its website to highlight tools and innovations, and provide links to new products and services. 2. Insofar as the integrated approach and many of the innovative systems developed by FFH are adaptable at scale, USAID may wish to be more proactive in identifying opportunities for FFH to share lessons and tools with CAs and even overseas government agencies. 7.3 Conflict Management A. Findings: 1. Bolivia is prone to civil unrest which impacts on development programming. In particular, blockades and strikes cause office closures, disturb the movement of people and goods, and have resulted in organized loan defaults. 2. These problems are exacerbated in narcotics production and trading areas where crime and law and order uncertainties prevail. 3. The CwE model, and CRECER’s special relationship with its clients have cushioned it somewhat from these potentially fatal realities. 4. Even when staff cannot reach an area for 4-5 months, or offices are shut down, village banks— which are located and controlled in communities—continue to function. 5. Confident that the model is sturdy enough to function in even more challenging areas, CRECER has agreed to join USAID to establish the program in narcotics production areas. 6. Luzon in the Philippines—where CARD has its program—is more prone to natural disaster, and particularly typhoons. 7. CARD has instituted a “credit rehabilitation program” wherein repayment requirements are suspended for a short period of time following a natural disaster. It also extends emergency loans. 8. Interestingly, CARD reports that very few women suspend repayments and few have availed the loans even in 1999/2000 in which the area was declared a calamity area. 9. CARD attributes this to the fiscal disciple that its model instills in clients—they know they will have to repay eventually, and they do not want to overly burden their cohorts. 29 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 10. Another issue in this regard is the extent to which CwE may impact on civil cohesion or dissonance. 11. A major cause of civil unrest is poverty. To the extent that CwE positively impacts on household and community livelihoods, it can be expected to have positive or at worst neutral impacts on civil society. 12. Some credit programs target more wealthy, collateralized or well-connected women--thereby potentially reinforcing existing gaps (and tensions) between economic or ethnic groups. The fact that the village bank is formed through a process of peer selection22 is likely to obviate against that potential. 13. Third generation CwE modules which will focus on social action may have a positive impact on civil society in that they will seek to mobilize the power of group solidarity on issues of major concern to women. B. Conclusion: 1. Both MG partners deal with the fallout from conflict and disaster situations in their programs and both have developed “coping mechanisms” that protect their clients and their program. C. Recommendation: 1. This may be an aspect of village banking that bears further consideration in terms of implementation of the approach in post conflict situations 7.4 Monitoring and Evaluation A. Findings: 1. Because this was not a field program, impact indicators were not articulated in measurable terms. 2. Measurable indicators for the number of MOU’s with MFIs and the amount of additional grant money anticipated for MCH promotion were provided. 3. In general, the DIP provides intermediate indicators of immediate outcomes, which are more tasks than they are indicators in most instances. 4. M&E tools used for progress tracking of CwE programs have been discussed in other parts of the report. B. Recommendations: 1. In an R&D project like this, qualitative indicators may be a reasonable substitute for quantitative indicators. 2. Proxy indicators like market demand, invitations to present, speak, test products and services etc. are useful in this regard. 7.5 Overall Management A. Findings: 1. The FFH President was proposed as the Project Director, but in reality project management and liaison was carried out by the person with most health expertise--Sr. Vice President Ellen Vor der Bruegge. 22 CARD also has an income threshold to ensure its loans are going to poorer women. 30 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 2. No individuals were designated team leaders for specific countries, although work plans developed with CARD and CRECER specified inputs and individuals who would provide those inputs. 3. One person coordinated overall module production, and there was a designated author for each module. 4. Because of the size and collegial work environment of FFH staff this approach seems to have functioned reasonably well. 5. However, FFH lost some time in developing a module development approach that felt comfortable—part of the learning by doing style of the organization. 6. Also, since the timetable for module testing was not worked out during the DIP process, and since it relied on partner agencies agreement as well as coinciding education cycles for specific topics, FFH lost some time waiting for opportunity to coincide with draft module readiness. 7. As their names would indicate, the Research and Innovation Team is primarily responsible for product development while the Practitioner Services Team is responsible for dissemination. 8. The absence of a clearly articulated (and written) dissemination strategy is interesting in light of the highly detailed documentation flowing from the R&I Team. B. Conclusions: 1. The rather flat management structure of the project reflects FFH’s organizational culture and is conducive to its size. 2. FFH may have lost implementation time by not planning module production and field testing in advance. 3. The absence of a written dissemination strategy may be a result of changes in FFH service provision directions, described above, which impact on the nature of dissemination and are likely to impact on internal structures as well. C. Recommendations: 1. FFH should develop a dissemination strategy (as mentioned above). 2. Some reflection on what worked and what didn’t in planning product production and managing the input of partners in the process might assist in managing analogous future multi-function activities. 7.6 Sustainability (Annex F) A. Findings: 1. Financial sustainability is a fundamental tenant of the CwE model. 2. Both MG partners have reached a greater than break even point with respect to their credit programs. 3. Sustainability relies on cost consciousness in staffing and administration, and particularly on retaining high repayment rates in a portfolio of a certain size. 4. The question posed most often to FFH is about the added cost of the education component. a. Research, referenced above places it at 4.7-10%, although credit without education is not that much cheaper. b. The model doubles up on the infrastructure and staffing required to support the village bank, to provide education. c. There is a cost to the education component (training, supervising, materials etc), but it is not nearly what a stand-alone health education program would be. 5. There is some intriguing evidence, that rather than drain resources, the education component is actually cost effective, in that it appears to enhance retention rates, client satisfaction and group cohesion—and thereby sustainability of the program/institution. 31 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 6. As interesting as programmatic sustainability, is consideration of the sustainability of FFH as an NGO. 7. FFH, unlike its partners, has no interest income stream to depend on. 8. Its sustainability to date has relied on quality programming and products, loyal donors and maintaining costs (size) at manageable levels. 9. FFH’s new venture philanthropy initiative, though arguably a year too late, is an innovative strategy to draw socially minded clients into its program. It also challenges FFH to begin to quantify returns in social capital terms—which will be the actual return (not dollars) to donors. 10. With disappointments in what Practitioner Services was able to achieve in terms of impact at scale (financially FFH appears satisfied), the organization is preoccupied with what next, and particularly: how? 11. Issues of continued revenue streams seem to take a back seat to these more heady issues—a refreshing luxury for an NGO. B. Conclusions: 1. Institutional sustainability and programmatic sustainability are part and parcel in CwE institutions. 2. FFH strives to assist partners with institutional sustainability but does not have a sustainability plan for itself. C. Recommendations: 1. Research on the financial benefits of the education component in the CwE model would be useful especially for advocacy to commercially-driven MFIs. 2. FFH needs to bring together various strands of planning and thinking—about LTSPs, venture philanthropy, the Practitioner Services Team into an institutional sustainability/business plan. 7.7 Financial Management 7.7.1 Effectiveness of financial management N/A 7.7.2 Leveraging other donor funds A. Findings: 1. As indicated in Annex B, FFH had 4 major funding partners over the life of the MG (the LINKAGES project, the CAREMoRR project, the FANta initiative, and the Summit Foundation). 2. Though only one was a non-USAID donor, all deemed FFH’s MG activities worthy of additional support. 3. LINKAGES and CAREMoRR have extended funding for module dissemination and related aspects of FFH’s MG program—some of it transcending the life of the MG. 4. Two of the four agencies were interviewed for the evaluation and gave FFH very high marks for the quality of their products and partnership with the respective agency. B. Conclusions: 1. FFH successfully leveraged the MG activities to engage other players, particularly in the MCH and RH communities in providing not only funding but critical technical assistance to the process. 2. An important by-product has been entrée into new networks, as discussed above. 32 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 7.7.3 Cost effectiveness of technical approach A. Findings: 1. The total project cost was: USAID $ 900,000 FFH 600,000 Summit Foundation* 300,000 LINKAGES** 264,956 CARE MoRR** 262,784 FANta** 154,397 $2,482,137 *Summit Foundation grant contributed to FFH’s match **Not included in FFH match 2. To date, FFH has expended $1,500,000 of this amount ($1,032,000 of MG support). 3. Of this amount, approximately: a. $600,000 from the MG has gone into module production ($500,000 for FFH technical staff costs, the remainder for travel, consultants, translations etc) b. An additional $500,000 of funds contributed by CAREMoRR, LINKAGES, and FANta covered similar costs of module production and expenses related to specific deliverables c. $180,000 has been used for TA to CRECER for both the credit and education parts of the program, as well as broader management support d. $57,000 was used for TA to CARD e. $100,000 was earmarked for CARD operating costs of which only about half has been expended 4. FFH anticipates spending the remaining funds on headquarters costs, refining and testing modules, TA to partners, additional work on the LTSP and progress tracking systems as well as documentation, translation and support. B. Conclusions: 1. The cost of module production is relatively high, although clearly the level of effort into each of the products has been significant 2. While the agencies who have piloted and will continue to use the modules have benefited in the immediate term, the real yardstick of value for money will come with widespread demand and utilization 3. The project is 78% complete and only 62% of project funds have been expended 4. FFH anticipates expending all project funds by the end of the project period. 7.8 Information Management A. Finding: 1. Reporting to USAID and reporting from the field seemed to have been prompt and trouble free. 2. Both partners reported that they felt comfortable emailing FFH with questions, and there was a visible comfort level in communications. 3. CRECER voiced a desire to have more Spanish-speaking trainers, but acknowledged this was minor in comparison to the quality of the English speakers it had received. 33 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 7.9 Logistics N/A 7.10 USAID Management A. Findings: 1. As mentioned above, FFH considers USAID a genuine partner in its activities. 2. FFH felt supported by PVC during the life of the MG. 3. The MG preparation and feedback at various points, particularly in DIP preparation, was taken seriously and in the spirit of learning. 8.0 Lessons Learned A. Findings: 1. This matching grant is slightly different than many others in its focus on product development rather than direct program implementation. 2. FFH also views itself in a more catalytic, technical partnership role. 3. For both of these reasons, the evaluation necessarily took a slightly different tact, and relied on different types of indicators and data. B. Conclusion: 1. FFH’s evolution from direct implementer to a technical partner offers many rich lessons for other agencies seeking to move in this direction. C. Recommendations: 1. Aspects of its approach and “style” which merit special attention include: a. Its partnerships style and commitment b. Network participation c. Its attention to research and documentation as the foundation for program excellence and advocacy d. Its focus on honing a single approach rather than constantly searching for the next “innovative” model e. Its commitment to a development agenda in the context of a part of the industry that is increasingly focused on the bottom line—and its ability to nurture sustainable, competitive programs reaching the poorest anyway. f. Its self-imposed limits to growth in favor of scale through empowering local technical capacity and partners. 34 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex A: Key Events Timeline Freedom from Hunger International Center Date Event October 99 LINKAGES MOU - ongoing November 99 Dunford presentation: The Holy Grail of Microfinance: Helping the Poor AND Sustainable? November 99 APHA Presentation - Integration of Health Education and Microfinance Services: the Impacts, Challenges and Costs January 00 FOCCAS Progress Tracking System initial design February 00 CRS/Madagascar Breastfeeding Module Test March 00 CARE MoRR MOU – ongoing June 00 Francophone Trainers’ BF Workshop in Togo Jan 01 Global Learning Partners Workshop and MOU February 01 FANta MOU – Three Task Orders April 01 West African Credit Union Conference May 01 Presentation by MkNelly and Dunford: USAID Assessing the Impact of Microenterprise (AIMS) May 01 Paper commissioned by Microcredit Summit: Building Better Lives: Sustainable Linkages of Microcredit and Education in Health, Family Planning and HIV/AIDS Prevention for the Poorest Entrepreneur June 01 Collaboration Assessment Report by IDR to FFH Board of Trustees July 01 LQAS Workshop with CARD and CRECER in Davis September 01 Presentation by Dunford: Village Banks Networks as Governance Mechanisms in Development Finance October 01 Health Insurance Report by Jyothi Thrivikraman October 01 Progress Tracking network (PLAN International and SEEP Client Impact Working Group) meeting January 02 MG Final Evaluation (9 months early) CARD Date Event May 00 MOU between CARD and Freedom July 00 DIP Development October 00 Breastfeeding and Manage Your Business Money Education Training February 01 Infant and Child Feeding & Increase Your Sales Education Training February 01 Progress Tracking Introduction Meeting July 01 Family Planning and Plan for a Better Business Education Training July 01 Progress Tracking Tools Development Oct 01 Annie Alip participated in SEEP Client Impact Working Group (CIWG) action research project annual meeting (first of three planned) to share experiences and plans for systems development. Also participated in PLAN international Microenterprise Impact Working Group (MIWG) for similar purpose particularly in the area of impacts on children. November 01 Progress Tracking Staff Training and implementation of mini-survey using LQAS and participatory focus group discussions to get feedback on CARD’s services. December 01 CARD-wide Expansion Decision January 02 Women’s Health Training 35 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc January 02 Training System Design May 02 (scheduled) Quezon Area-wide Expansion Training CRECER Date Event December 99 Progress Tracking begins with tools development on an incentive system and qualitative evaluation February 00 Breastfeed and Infant/Child Feeding topics training February 00 Chemonics Institutional assessment using CGAP tools May 00 Women’s Health topic finalized for local use August 00 Strategic Plan approved by Board August 00 Progress Tracking on Qualitative Evaluation tool design August 00 Self-Esteem topic design September 00 Microfin training in Davis September 00 Ohio State University Poverty Outreach Study launched October 00 CBD System Participatory Assessment with CHANGE December 00 PROFIN Study on the Group Instrument in Financial Intermediation February 01 CBD System enhanced action plan launched July 01 CRECER formally established as an independent NGO October 01 Latin America MicroCredit Summit presentation by Executive Director November 01 Education and Training System presented to CRECER Board for approval November 01 Field Agent and Credit Association orientation training guides standardized February 02 Presentation of finalized Business Plan to Management Team for approval 37 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex B: Detailed Implementation Plan Table Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? Goal: To build the capacity of Freedom from Hunger to strengthen integrated microfinance-education practitioners to: ß Deliver high-impact health/nutrition-promoting education; ß Track and analyze indicators of their MCH-education status services and other impacts; and ß Use this information with financial performance indicators to guide their programs toward both financial sustainability and production of important benefits for very poor households, especially in rural and peri-urban areas. FFH IC Objective 1: Balance microfinance expertise with increased staff time and expertise devoted to maternal/child health-promoting education within Credit with Education programs. 1. Restructure IC teams ß Organization Chart Practitioner Services and Research & Innovation Team formation complete. Team formation completed Y NA Yes 2. Recruit and hire new MCH staff ß At least one senior full-time MCH expert added to the R&I team ß Staff time sheets show 50% of key R&I time allocated to MCH promotion activities. Vicki Denman and Robb Davis hired as full-time experts on the R&I team. Y NA Yes 3. Solicit new grant support from donors for MCH development ß Secure at least $100,000 per annum in new grants for development of MCH education strengthening. LINKAGES ($264.956) CARE MoRR ($262,784) FANta ($154,397) Summit Foundation ($300,000) Total:$ 982,137 Y NA Yes 4. Develop DIP with specific targets and indicators ß DIP submitted DIP submitted and reviewed. Completed Y NA Yes 38 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc FFH IC Objective 2: Improve, document and disseminate methods and systems for delivery of high-impact MCH-promoting education within Credit with Education. 1.Review literature and update and expand CwE MCH curricula NA All scheduled to be met by end of grant. Health/Nutrition/Population a. Breastfeeding Completed with LINKAGES and CRS/Madagascar NA b. Infant/Child Feeding Completed with LINKAGES and CARD/Philippines NA c. Family Planning Testing Phase with CARD/Philippines NA d. HIV/AIDS Testing Phase with World Relief and FFH/Ghana NA e. Women’s Health Testing Phase with CARD/Philippines NA f. Child Health (IMCI) Design Phase with JSI and CRECER NA g. Diarrhea Simplified design and testing TBD NA h. Immunization Simplified design and testing TBD NA i. Malaria Negotiations with technical experts underway for design work NA Self-Confidence Revised modules and trainer’s guides designed, tested, published and disseminated for each designated topic. Tested “modules” that include learning sessions and resource pack; trainer’s guide with resource pack and tool kit; adaptation guide; community exploration guide; and supervision tool finished, published and available for dissemination. Designed with CRECER and ready for test phase NA · All design work scheduled for completion by end of grant. · Module field testing relies on field opportunities: d.,f.,g.,h.,i. Pending. · Publication format is still in process. · Basic procedures and initial dissemination will be completed by end of grant. Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? 2. Upgrade and expand training system and manuals NA All scheduled to be met by end of grant. a. Trainer’s Guide for Orientation Guide completed Y NA Products in various stages b. Coordinator Operations Manual Development in progress Y NA of development or completion. c. Management Committee Training Manual Manual completed Y NA Dissemination strategy under development d. Credit Association Training Manual Publication disseminated Y NA e. Monitoring and Reporting System Initial work with FOCCAS. Additional work in process in Mali. Y NA f. Second-Generation Modules Concept designed, first topic in process Y NA g. Training System Design In process with CRECER and CARD Y NA h. Adult Learning Primer Primer completed with additions as needed Y NA i. Facilitation Skills Primer Tools and products, training systems and manuals are designed, tested, published and disseminated for designated topic. All tool, products, systems and manuals designed, tested, published, available for dissemination. In process Y Added 39 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? 3. Research and innovate MCH-enhancing service delivery tools and products NA a. Health-Service Linkage Protocol Guidelines for linking health and MFI sectors to be finalized for broader use, especially with IMCI. Prototypes established with CRECER and FUNDAP. Y NA Wider distribution by end of grant. b. Health Insurance Strategic decision made to focus on “health savings” model with CwE, self-help groups or other savings groups. Literature review and field research report complete. Analysis of opportunities and future direction determined. Y NA Yes c. Progress Tracking Tools Manual(s) for range of cost￾effective progress tracking tools selection/adaptation, staff training, participatory analysis and how to use the information for program strengthening. Methodologies for tracking client satisfaction, quality of service delivery, and impact tested with CRECER, CARD, and FOCCAS. Y NA Scheduled to be completed by end of grant. d. CBD System MCH-enhancing products and tools are designed, tested, published and disseminated for designated topics. Testing and module development for CBD with CwE. · Assessment and strengthening of CRECER CBD system complete. · Draft protocol for CBD system planning and design, training, logistics and reporting management in the test phase with ACCLAIM in Haiti · Design of enhanced system for MFIs complete and test scheduled for Feb. Y Extensive field testing and detailed documentation requires more time Finalized manual may be published after end of grant. e. Master Trainer Dissemination Strategy TOT workshop format for disseminating education materials will be finalized and documented for large-scale application. · Testing of new TOT prototype in Togo with francophone CwE practitioners completed in July 01. · Test in Ghana with CwE and non￾CwE practitioners planned for March 02. N NA Scheduled to be met by end of grant. f. MCH Curriculum Package Basic modules finalized and ready for publication. · Cluster of enhanced MCH education modules finalized for use by group￾based programs. · Module publication design with Hesperian to be finalized March, 02. · Dissemination strategy design underway with LINKAGES and CORE to be launched April.02. N NA Scheduled to be met by end of grant. 40 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? 4. Develop and implement action agenda with CRECER, CARD and other R&D and CwE practitioners NA a. Breastfeeding, I/C Feeding and LAM modules with LINKAGES Breastfeeding, ICF and LAM modules competed. Methods for training tested and in-process. On-going relationship for further design, testing and collaborative work. MOU with LINKAGES for TA and support in the development and testing of BF, ICF and LAM education materials. TA and support for design and testing of dissemination strategy. Y NA Scheduled to be met by end of grant. b. CBD System development with NETMARK and/or CARE CBD System developed with NETMARK and/or CARE · Determined NETMARK approach not applicable to CBD system with MFIs. · See 3.d above Y Greater demand and broader application than originally anticipated. Test of new CBD model in Haiti may not be finalized by the end of the grant, but it will be by the end of the calendar year. c. Progress Tracking with in CRECER, CARD, FOCCAS with support from FANta CRECER, CARD, and FOCCAS using progress tracking tools and using information for program strengthening. MOUs with CRECER, CARD and FOCCAS include progress tracking systems development, testing of tools and staff training. Y NA Anticipate all three agencies will be using progress tracking tools by the end of the project. d. Health insurance with Emory University, RCPB and/or ILO Health insurance with Emory University, RCPB and/or ILO · Literature review and field research report completed/presented by Emory graduate at SEEP Annual Meeting-Nov 01. · ILO train seminar attend-ed with RCPB. FFH TA with design requested. · FFH strategic decision made to focus on “health savings” model with CwE, self-help groups or other savings groups. Y NA On going work with RCPB, CARD and others anticipated in months to come. e. IMCI linkages with CORE, BASICS and/or Emory Univ. IMCI linkages with CORE, BASICS and/or Emory Univ. · FFH is a member of the IMCI Working Group of CORE and participated in the design of the C￾IMCI framework. Dissemination team. · IMCI module to provide entry point for linkages with health service providers. N NA On going work with CORE and others planned beyond grant f. Adult Education with Global Learning Partners At least four signed collaborative agreements and/or projects to strengthen MCH Technical Assistance to microfinance organizations. Adult learning principles applied to all education and training materials. · All FFH staff attended GLP methods workshop. · MOU signed for on-going collaboration with GLP. Y NA Yes g. Action Agenda with CRECER and CARD At least four TA trips per year to CRECER or CARD dedicated to MCH education. Participatory act agenda resulted in enhanced tools and materials in use by CRECER, CARD as well as other Action agenda developed with CRECER and CARD for MCH material design and testing. Yes 41 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? organizations. 5. Revised Practitioner Services business plan to market and disseminate MCH￾related TA ß PST business plan reflects MCH market strategy. ß MCH services delivered. ß Revision of existing business plan not yet accomplished. ß Multiple new CwE practitioners attracted to use the model. ß Business plan revision delayed. ß MCH services expanded to reach 200,000+ clients. ß New CwE practitioners added in: Benin, Haiti, Guatemala, India, Mauritania, Indonesia, Peru and Ecuador Y NA Business plan will be developed in the next fiscal year (July 02) Goal/Objective/ Activity Indicator End of Project Target Accomplishment (to date) Data Verified? Explanation for Change Target Met? FFH IC Objective 3: Develop, document and disseminate system for monitoring education performance and impact with Credit with Education program. 1. Review relevant and promising work on monitoring education quality and impact evaluation for food security, nutrition and MCH Y NA Yes 2. Build collaborative action agenda with other action R&D groups working in this area Y NA Yes 3. Develop and implement work plan with CRECER and CARD for building impact monitoring into their M&R systems ß One or more formal collaborations with other action research and development groups working on impact evaluation and monitoring for microfinance and education. ß Documentation of recommended impact￾monitoring protocols. Significant work on tools and systems design and testing of progress tracking implemented. ß Series of design and training workshops for CARD and CRECER to launch and embed progress tracking into their management systems. ß CARD, CRECER, FOCCAS and FFH participate in R&D groups working in this area (e.g. for FFH: SEEP, ImpAct, MIWG, FANta) Y NA Yes 42 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Goal/Objective/ Activity Indicator End of Project Target [Current Status] Accomplishment (to date) Data Verified? Explanation for Change Target Met? CwE Practitioners CRECER, CARD and Others Objective 4: Increase capacity to maintain and expand outreach of high-impact MCH-promoting education within CwE without compromising financial service-delivery quality or performance. 1. Develop and test protocols for adapting revised curricula to local needs Protocol for adaptation process included in each enhanced topic. Curricula adaptation protocol developed and tested for each enhanced topic. Y NA Yes 2. Develop and test protocols for training and supervising field agents Participatory process for development of training system and supervision tools available for use with CwE practitioners and other organizations. ß Tool developed with CRECER for participatory evaluation also served to supervise on-going CBD activities. Training system expansion on design incorporating regional trainers. Supervision training for field agents. ß Methods developed with CARD and FOCCAS for participatory design of supervision tools. Y NA Documentation of protocol to be finalized by the end of the grant. 3. Develop and test protocols for linkages to local health-service providers Linkages to health-services providers established. made ß Models for linkages to health￾service providers tested with CRECER and FUNDAP. ß Linkages to be built with CARD prior to the end of the grant. Y NA Documentation of protocol to be finalized by the end of the grant. 4. Develop and test education performance and impact-monitoring system See progress tracking system See progress tracking system Yes 5. Update business plan to balance financial and impact goals ß Use updated MCH curricula. ß Use protocols for adapting curricula to local needs. ß Use protocols for training and supervising field agents. ß Use protocols for linkages to local health-service providers. ß Use management to monitor education performance and impact for input to decision-making. ß 100% self-sufficiency with evidence of MCH impacts (behavior change, increased use of existing services, better health status). CRECER Business Plan to maintain self-sufficiency while delivering balanced credit, saving and education services scheduled for completion prior to end of the grant. Process could be applied to additional MFIs. Series of planning and preparation meeting for the development of a business plan for CRECER on track. Scheduled to be met by end of grant. 43 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Partnership Table FANta CAREMORR LINKAGES Summit Foundation Technical Networks: CORE, SEEP Credit Union Networks: CIF, Desjardins, WOCCU Implementing Partners* * Funding for CRECER covered TA only. CARD funding also covered operating costs. **LTSPs: CUEs/Philippines, FFH/Ghana, KAFEM/Haiti, CARD/Philippines, Individual Associations (5) ***Implementing Partners tested modules: FFH/Ghana, CRS/Madagascar, CARD, CRECER Note: modules were tested in these agencies, but over 40 other CWE practitioner agencies Technical Networks: PHILNET, CASHPOOR CARD CRECER LTSP Associates** Technical Networks: PROCOSI FINRURAL FFH Key/Notes: Direct Funding Funding from other sources Technical Cooperation 45 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex C: Scope of Work For the Final Evaluation of the Freedom from Hunger Matching Grant, Improving Maternal and Child Health Impacts of Credit with Education I. PROGRAM IDENTIFICATION ß PVO name: Freedom from Hunger ß Cooperative agreement number: FAO-A-00-99-00046-00 ß Date of the evaluation: January-February 2002 ß Country programs evaluated: Bolivia and the Philippines PROGRAM BACKGROUND History of the Program Having developed Credit with Education as a financially sustainable strategy for practitioners based on our best understanding of recommended practices for adult education and microfinance-service delivery, Freedom from Hunger’s dual goal now is to deepen and broaden the potential of Credit with Education for substantial positive impacts on maternal and child survival and health within the families of very large numbers of the poorest women possible and to develop the capacity of Freedom from Hunger’s International Center to be a more cost-effective catalyst for adoption and adaptation of Credit with Education by many more development practitioner organizations. Freedom from Hunger will accomplish this by balancing the financial service-building capacity of its International Center with increased capacity for developing the education component of Credit with Education, both in terms of methods, systems for sustainability, and documentation for dissemination to other organizations (current and prospective practitioners of Credit with Education) and in terms of developing a dual system for financial performance monitoring and periodic impact tracking that can be used by practitioner organizations for guiding decision-making toward a dual bottom line of financial sustainability for the implementing organization and impact for the poor. The planned strategy is to reallocate the time of key senior and new technical staff and their assistants based at the Freedom from Hunger International Center toward much greater involvement in further developing the education component for improved MCH impacts and service-delivery sustainability, particularly with the Matching Grant partners in Bolivia and the Philippines. Other Credit with Education practitioners and R&D groups will also be involved in the design and test phases of development. In sum, the focus of this Matching Grant is fourfold: 1. A realignment of staff and programmatic priorities to focus on the “Education” aspect of Credit with Education; 2. The development, testing and promotion of tools and systems for training, managing and tracking MCH education; 3. Implementation of Credit with Education, including the new tools, in two sites; 4. Documentation, dissemination, promotion of Credit with Education. These foci represent a “paradigm progression” for the Credit with Education model—a rounding out of years of thinking, testing and refining. The results which the evaluation needs to address, therefore, derive as much (if not more) from transformations that have taken place at Freedom from Hunger International Center—new perspectives, talents, tools, strategies, and partnerships—as they do from results reflected in the programs and capacities of the two partner agencies. Current Status A brief overview of the status of the objectives and activities planned for implementation during the first two years of the Matching Grant is Attachment 1. 46 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Partner Organizations Freedom from Hunger collaborates with two organizations in the implementation of the objectives and activities of the Matching Grant. · CRECER – Bolivia CRECER’s mission is to enhance the food security and well-being of its clients, their families and their communities by providing high-quality, affordable financial and educational services primarily to women and especially to those in rural areas where access to financial and educational services is limited or nonexistent. For many years CRECER has been a subsidiary of Freedom from Hunger. However, with the help of Freedom from Hunger, CRECER aims to achieve financial self-sufficiency and institutional sustainability by becoming a rural financial institution of national scale in Bolivia. Progress toward this aim reached a major milestone during the second year of the Matching Grant. In July 2001, CRECER became a legally established NGO. CRECER is working to establish an in-house capacity to develop and adapt systems, techniques and materials for staff and client training and education and to manage Credit with Education as a viable line of business that yields positive MCH impacts. The Matching Grant supports Freedom from Hunger to assist CRECER in building its capacity to: 1) deliver higher-impact MCH-promoting education, 2) track and analyze indicators of their program’s MCH and other impacts, and 3) use this information in combination with financial performance indicators to guide their program toward both financial sustainability and production of important benefits for very poor households. The technical capacity of CRECER has been enhanced through regular interaction with Freedom from Hunger’s International Center staff. · CARD – Center for Agriculture and Rural Development – the Philippines The Center for Agriculture and Rural Development (CARD) is a non-government organization (NGO) implementing a replication of the Grameen Bank model in the Philippines known as the Landless People’s Development Fund (LPDF). CARD was organized in December 1986 with it headquarters located in San Pablo City, Laguna in Southern Luzon. Its mission is to build a sustainable financial institution owned, managed and controlled by landless rural women. CARD is committed to provide continued access to financial services to an expanding client base, by organizing and empowering landless rural women, and by instilling the values of discipline, hard work and saving in an atmosphere of mutual respect. The Matching Grant supports Freedom from Hunger to assist CARD in building its capacity to: 1) deliver higher-impact MCH-promoting education, 2) track and analyze indicators of the education and other services impacts, and 3) use this information in combination with financial performance indicators to maintain financial sustainability and important benefits for very poor households. The technical capacity of CARD is enhanced through regular technical assistance trips by Freedom from Hunger’s International Center staff. A collaborative agreement and regular negotiation of joint activities guide the work. Planning Matrix The Planning Matrix for the Freedom from Hunger Matching Grant gives the objectives and indicators for the grant is Attachment 2. Attachment 1: Matching Grant First- and Second-Year Objectives and Indicators Status of Implementation briefly reports on the status of each activity. Baseline Studies or Description of the Status at the Beginning of the Project Freedom from Hunger implements integrated microfinance-education services, generically known as Credit with Education, with the goal to increase the income and assets, self-confidence and status, and appropriate knowledge and practice among large numbers of very poor yet economically active women, in order to improve household food security and individual health and nutrition, particularly for mothers and young children. With Matching Grant support from USAID, Freedom from Hunger’s 47 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc pilot work demonstrated that Credit with Education is a feasible strategy for integrated delivery of microfinance and education services to groups of women in rural communities. Additional Matching Grant support from USAID allowed Freedom from Hunger to test and refine the replication of the basic services offered, and demonstrate their flexibility to be adapted to a variety of institutional structures. Our work with credit union federations and community banks in West Africa and with Bolivian and Ugandan NGO partners demonstrated that Credit with Education could reach large numbers of women and attain or approach financial self-sufficiency. Following the invitation of USAID to use assessment techniques or existing data to verify that the objectives are on target, three sources of information were used to shape and confirm the rationale or basis for the work outlined in the Matching Grant proposal and the Detailed Implementation Plan: the impact studies implemented in Ghana and Bolivia, the performance reports from Credit with Education practitioners and the previous Matching Grant final evaluation results. During the replication phase of implementation, in-depth, controlled, longitudinal studies of Credit with Education impacts were conducted by Freedom from Hunger from 1993-97 with one of the Ghana practitioners and with a local partner in one region of Bolivia. The Ghana results showed impressive impacts on income and assets, self-confidence and social status, knowledge and self￾reported practice of recommended health behaviors, and nutritional status of young children. The Bolivia results were less impressive, reflecting management problems, especially in the field supervision of the education service. However, in communities where education was of greater quantity and better quality, maternal and child health (MCH) impacts were greater. Important conclusions drawn from analysis of the data follow: 1. Evidence that Credit with Education can achieve its intended impacts. 2. Positive impact is seen on infant and child nutrition. 3. Important health/nutrition behavior change can be influenced; for example, a shift to exclusive breastfeeding for the infant’s first six months in Ghana and maintaining hydration of children suffering from diarrhea in Bolivia. 4. Less dramatic impacts in Bolivia indicate that education matters and that participants benefit from more and better-quality education. The analysis and comparison of the Ghana and Bolivia impact studies, interaction and discussions with practitioners of Credit with Education and recommendations from external evaluation of the strategy strengthen an appreciation for the need to commit more attention to the education component and the management of its delivery. Therefore, the purpose of the Matching Grant is to strengthen Freedom from Hunger’s capacity to enhance the Credit with Education technology, especially the education component, and to better transfer the technology needed by integrated microfinance-education practitioners to deliver high-impact health/nutrition-promoting education, to generate and use information about service delivery, quality, client satisfaction and impacts together with indicators of financial performance to guide their programs. The tools, products and lessons learned in the work with CARD, CRECER and others will be disseminated and used to enhance the services being offered by Credit with Education practitioners and additional organizations delivering similar services. Information and Data for the Evaluation All documents, reports, tools, products, agreements and records of the work supported by the Matching Grant will be made available to the evaluator. Changes since the Initiation of the Program In the early months of the grant, Freedom from Hunger was instructed to identify another local country partner when the Uganda activities were not able to continue. The history of this development 48 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc is documented in the DIP and the Matching Grant First and Second Annual Reports. Since then, no changes have occurred in the implementation of the planned activities. PURPOSE OF THE EVALUATION The final evaluation fulfills the requirements of the USAID/BHR/PVC Matching Grant (MG) Program. The MG program will use the information to: assess how well the MG is meeting its objectives; determine patterns and emerging issues across all MG funded programs; determine technical support needs for grantees shape new RFAs and to review of any follow-on proposals; develop internal and external documents to demonstrate the effectiveness of the MG program and to share lessons learned with the entire PVO community. PVC will use information outlined in the SOW template in its annual Results Report and in USAID's annual report to Congress. Achievements cited in the evaluation need to be supported by evidence and should be verifiable. Observations on data quality or constrains to interpretation should be stated as data from these evaluations is used for USAID reporting purposes and is subject to audits. Technical/program opinions and observations are an important element of the evaluation --- but should be stated as the evaluator estimate, opinion or forecast. Freedom from Hunger and its collaborators, CRECER and CARD, will use the Matching Grant information to: receive an objective assessment and feedback on the products and activities implemented during the grant period, get documentation by an external evaluation of the quality and utility of the tools and products intended to be provided to potential users; and provide USAID with a report of Freedom from Hunger’s capacity to manage a Matching Grant as it considers our current application for another grant from USAID. THE EVALUATION QUESTIONS ß PVC Evaluation Questions Program Implementation 1. Assess progress towards each major objective (Note: This final evaluation of the Freedom from Hunger Matching Grant is being implemented 9 months before the end of the grant period. This should be taken into consideration with regard to the completion of the objectives outlined in the planning matrix.) ß Based on the program planning matrix, determine if the program objectives have been met, partially met or were unattained. This is the single most important element the evaluation must document and discuss. In addition to the discussion of project results in the text of the evaluation, this information should also be put into matrix format. List each objective, and key outcomes at the effects and/or impact level. In the text: ÿ Identify major successes and constraints in achieving objectives and unanticipated effects. As part of this discussion, comment on the PVO and their local partners’ capacity to do program monitoring and evaluation. Note any constraints that prevented the PVO from measuring achievement of program objectives. In the case of this project, look especially at the use of the progress tracking approach and use of data. ÿ Identify if the project had a detailed implementation plan and the familiarity of field and International Center staff with the project design, implementation plan process and products or outcomes. ß Assess effectiveness of models, approaches or assumption that underlie the project. Does the approach have the potential for been scaled-up in the project area or replicated elsewhere in country or in other countries? What are examples of these activities? 49 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc ß Assess progress on dissemination of the Credit with Education modules or tracking system, based on activities carried out under the MG. · Has the PVO engaged in program or policy advocacy? What was the focus of the advocacy and effects? Focus here on local partner advocacy, as well as FFH’s advocacy with other microfinance organizations—for instance that integration of the health education component can be cost effective. ß Discuss what the PVO and local partners have “learned” implementing this project. Identify if these “lessons learned” have been applied elsewhere (other projects or countries). 2. Assess the status of partnership(s) with NGOs, community based organizations or local level government. (Note: While developing strong collaboration with local practitioners is critical to the way Freedom from Hunger operates, this was not one of the objectives in the Matching Grant.) ß Include a chart that: ÿ Categorizes local level partners. Are the partners: NGOs, affiliates of the PVO, private or commercial groups, cooperatives, community-based organizations, regional or local governments or intermediate service organizations? ÿ Identify the type of mechanism employed with each partner, i.e. MOU, sub-grant, contract. ÿ Outline the roles, responsibilities and decision-making responsibilities of the partners. ÿ Identify the fiscal autonomy and amount of grant funds directly managed in past year. ß Assess the process that the PVO used to build and maintain local partnerships. ÿ Does the PVO have a partnership policy and approach to assess potential partners? ÿ Did the PVO do a formal assessment of local partner capacity and develop plans to build their capacity? ÿ Document change in local partner capacity. ÿ What were the major constraints to effective partnerships? ÿ Has the project increased the local partners’ access to information technology? How? ß Assess the local level partners’ satisfaction with the partnership. ß Assess the PVO and their local partners’ involvement in local networks or with intermediate service organizations. 3. What potential does the organization have through participation in networks or service organizations to impact operational or technical thinking and strategies? What would make it more effective? Cite the major implementation lessons learned and recommendations · Assess the PVOs linkages with CAs, PVOs and other agencies for developing, testing and refining tools and tracking systems B. Management Capacity/Institutional Strengthening The objective of the MG is to build PVO headquarters and field organizational and technical capacity. This section of the evaluation should assess change in the PVOs operational and management capacity (organization, structure or quality of planning and management) as a result of PVC grant. ß Strategic Approach and Program Planning Have changes occurred in PVO headquarters capacity to: ÿ manage the planning process --- program renewal, strategy integration, project design; ÿ address over-arching program issues of replicability, scale-up, sustainability. 50 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc ÿ Use performance data to forecast emerging trends and develop strategic plans. ß Country Level Initiatives Identify and assess (if relevant), PVO contributions in the following areas: ÿ PVO cooperation and coordination with the USAID mission and other development partner programs including natl./local government agencies; ÿ PVO advocacy activities: issues, goals, partners and results (Has the PVO used project data for advocacy with the public sector or consistently shared lessons learned with other PVOs in country or with non-partner NGOs?); ÿ If the country or program area has a history of violent conflict, other man-made/natural disasters or food insecurity: a) PVO activities in conflict prevention, mitigation, resolution or post-conflict transition; b) PVO’s contingency plan to ensure the safety of program staff and program continuity. Note: this can address several issues: a) whether FFH or partners’ programs program is designed to prevent, contribute to or is neutral in terms of civil strife and/or food insecurity? b) what would be the implications of a natural or manmade disaster to the implementation program? ß Monitoring and Evaluation Has the project implemented a process for developing sustainable and management-oriented systems to monitor project performance and client outcomes? Provide evidence that the project: ÿ Defined areas of informational needs, developed and tested cost-effective and management oriented methods for generating, analyzing and using this information to inform action. ÿ Transferred monitoring and evaluation skills to local partners? - What changes have occurred in the capacity of the local partners to define informational needs, measure performance and analyze and use this information to improve service quality and the likelihood of positive impacts. - Have local partners increased M&E in their own activities (non-PVC-funded programs) as a result of skills gained through this project? - What would accelerate the capacity of the local partners to document and pro-actively respond to performance? Determine if the PVO has used the MG to develop capacity at headquarters and in the field offices to monitor project performance and client outcomes. Has the PVO headquarters: ÿ fostered analysis and self evaluation in country programs to conduct quantitative or qualitative analysis to refine interventions; ÿ developed, tested and documented performance monitoring and client outcome tools and systems with MG funds into other non-PVC grant funded programs, and; ÿ institutionalized performance monitoring and impact evaluation systems developed with MG funds into other non-PVC grant funded programs, and; (Note: has FFH institutionalized this? Level of resources? ) What were the biggest constraints to improving project monitoring and evaluation and what are the recommendations for PVC and the PVO? ß Sustainability ÿ Does the project have a system for addressing financial or operational sustainability? ÿ Does the project have a business plan? ÿ Describe the program elements, financial or operational, that are intended to be sustained (objectives); the means for judging if the sustainability objectives have been achieved (indicators); and sustainability achievements and prospects for post-grant sustainability. 51 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc ÿ Identify if the project has any cost-recovery mechanisms, i.e., local level financing or approaches to generate resources to support project operations. Describe the achievements of these mechanisms and provide an estimate of the magnitude of the system, for example, provide a ratio of costs recovered to operational expenses. Other Management Systems 5 Financial Management ÿ Are adequate financial monitoring systems in place? ÿ Has the program leveraged additional resources (beyond the match)? ÿ How cost-effective is the technical approach? Information Management ÿ Comment on the utility and timeliness of PVOs required reports. ÿ Has the PVO developed, disseminated and used “lessons learned” from the project? ÿ Information Technology Logistics ÿ Comment on the adequacy and timeliness of PVOs material inputs. 6 Supervision/HRD ÿ Assess if there were sufficient staff with the appropriate technical and management skills to oversee program activity at both headquarters and in the field program 7 USAID Management Comment on USAID’s oversight and backstopping of this cooperative agreement. Cite the major management lessons learned and recommendations V. EVALUATION METHODOLDOGY The evaluation will seek to address the questions posed in this SOW through a process of discovery, using qualitative tools and seeking to match performance and outputs with objectives. The Team Leader will conduct a series of one-on-one, in-depth interviews with key staff at the FFH HQ, over the phone with key CRECER staff and in the field at CARD HQ. She will observe CARD training in the field and interview clients, participants and community leaders using focus group and other PRA techniques. Staff from CARD will also be involved in parallel data collection to give greater scope to findings. The TL will debrief CARD following fieldwork, and FFH HQ by phone upon return to the U.S. (and prior to finalizing the draft report). An exit meeting will be held with USAID in Manila to solicit input on the CARD program and provide a briefing on findings from the field. In addition, the TL will talk with PVO partners (CARE and AED) who have provided technical and financial support to the project. A. Approach The PVO’s program was developed and funded prior to the Agency's emphasis on results￾oriented program designs and the development of PVC’s Strategic Plan. The data from all PVC￾funded programs is critical to PVC's ability to report on achievements against the Office's Strategic Plan. Until all current PVC-funded programs have made the transition to a more results￾oriented project plans, it will be necessary for the evaluator to conduct a team-planning meeting with the PVO and local partners to: ® refine and consolidate the purpose-level objectives and outputs into a set of results￾oriented objectives; and ® Agree upon a set of appropriate indicators against which the evaluation will assess the achievement of project results outlined in the SOW and will be judged. And 52 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc where necessary, identify criteria for judgment. B. Methodology The Evaluator will: ® explain the appropriateness of using the data collection approaches; ® use the Agency's microenterprise (ME) indicators to assess the status of the ME intervention; ® document data sources (data constraints, quality, etc.); and ® Provide, a copy (electronic or paper) of all primary data collected and analysis performed. TEAM COMPOSITION AND PARTICIPATION Team Leader: - Coordinate and lead data collection activities - Undertake interviews with senior staff at Freedom from Hunger, CARD and CRECER (by phone) - Draft major findings sections of the report - Submit and revise report to MSI Senior PVO representative on evaluation team: - Coordinate scheduling, meetings and liaison with Freedom from Hunger and partners - Provide background and contextual briefing to Team Leader - Draft background sections of the report and prepare relevant contextual narrative Language requirements: FFH will provide a Spanish translator to allow the Team Leader to conduct (phone) interviews with key management staff of CRECER. MSI will also provide a translator for follow up interviews (mid/late Jan). CARD will provide translators for the field interviews from among its staff in the Philippines. No translators are needed for CARD senior staff. Technical expertise, or country experience: both team members have Philippines, MCH and microenterprise expertise. - Evaluation methods and data collection expertise: the Team Leader has conducted similar evaluations for the PVC office in the past, and has many years of rapid turnaround evaluation experience. VII. SCHEDULE The initial assessment work with the Freedom from Hunger staff is scheduled between January 7- 9, 2002. In addition, the initial phone interviews with CRECER will be scheduled during this time. The details of the daily agenda will be developed in coordination with the needs and request of the team leader. The field assessment with CARD will occur between January 14-18, 2002. The team will leave the U.S. on Friday, January 11 and complete the fieldwork and meetings with the USAID/Philippines mission by Friday, January 18. Interviews with CARE (MoRR) and AED (LINKAGES) will be conducted prior to departure for the Philippines. Follow up phone interviews with CRECER, if necessary will be facilitated with the assistance of an MSI translator. One additional phone interview and debriefing with Freedom from Hunger Davis staff will be scheduled during the workweek after returning from the Philippines. 53 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc VIII. REPORTING AND DISSEMINATION REQUIREMENTS - The SOW will serve as the outline of the report - Delivery schedule - First draft of the report to be submitted no later than February 3, 2002 - Review/revision policy - Comments will be incorporated within 5 working days from receipt of the last set of comments from USAID, MSI and FFH. 54 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex D: Advocacy Implementation Chain* Responsible Organization (Type) Advocacy Capacity Improvement Activities Org’s targeted for improvement Advocacy policy targets and (institution targeted) Actual Advocacy Events Policy changes realized Comments National Level CARD: (MFI: development bank and NGO) Other MFIs; development NGOs members of Philnet Presentation of CwE to Philnet members; discussions with other MFIs TSPI, another MFI leader in the Philippines requested more info Many MFIs and NGOs asking for information Asia regional Grameen Bank replicate MFIs members of the CASHPOR network Presentation of CwE to CASHPOR members TSPI, another MFI leader in the Philippines requested more info Donors Have discussed CwE approach with AusAid, Oxfam America, CIDA and PLAN International Donors more comfortable with supporting integrated strategies National Level CRECER: (LNGO) N/A N/A Other MFI’s targeted via participation in the FINRURAL network. Exposure to the benefits of the integrated approach CRECER GM Board member of FINRURAL. Participation and presentations at meetings. Participating in C￾GAP/Ohio State University study of major Bolivian MFI’s regarding poverty outreach. Part of global 3- instituion study on sustainability by Results. One of 2 institutions studied in PROFIN / COSUDE study of methodological factors for providing financial services. Three regulated MFIs have approached CRECER for strategic partnerships. Recognized as one of the most successful MFIs in Latin America at regional Microcredit Summit Conference. Nominated for Excellence Award by Grameen Foundation (awards to be determined April 02). N/A Other PVOs and LNGOs working in health. USAID via the PROCOSI network: exposure to the benefits of the integrated approach “ “ Invitations to work with FUNDAP, Andean Rural Health and USAID to integrate / replicate the CwE approach in remote area—all taken to be acceptance of CwE approach. Peer LNGOs offering special services to CRECER clients. MOH: access to services via CwE and CBD links Cooperation on CBD implementation 55 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Responsible Organization (Type) Advocacy Capacity Improvement Activities Org’s targeted for improvement Advocacy policy targets and (institution targeted) Actual Advocacy Events Policy changes realized Comments Local Level Providers of health services at local level CRECER has established relationships with local health services providers for referrals of CRECER clients Improved facilitate access to quality health services for clients. FFH: (PVO) National Level N/A N/A Public Donors Participation major conferences (Microcredit Summit, Frankfurt Development Finance Conference, etc.), publication of papers and articles, participation in the Microenterprise Coalition None in the last 5 years N/A N/A MFI and NGOs doing microenterprise activities— Practitioner Community Formation of CwE Learning Exchange. Participation in SEEP committees. Using DEVFINANCE and other list serves, publications and conferences (Microcredit Summit CGAP Brownbag Series, etc. Greater receptivity to integrated message in PVO and credit union communities resulting in more integrated strategies adopted more and more use of impact related tools and systems. N/A N/A Child Survival PVOs participating in the CORE network; other affiliates such as LINKAGES and other CA’s FFH staff on CORE Board. Active in committees, discussions, and presentations Growing interest in use of adult learning methods and behavior change methods for Child Survival education topics N/A N/A USAID & Congress on targeting of microcredit money to the poorest. Chris Dunford proposed language for poverty levels and indicators. Discussions among key decision￾makers have begun. N/A N/A Measuring impact: Practitioners and Donors Creation of the Impact Working Group, conferences, publications; participation in USAID’s AIMS and Ford Foundation’s evaluation initiative “Impact evaluation has moved from being an expletive to the realm of polite conversation in the industry”, FFH Note: advocacy in this case is primarily related to the promotion of an integrated credit and health education approach, and more specifically Credit with Education 57 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex E: Partnership Table Level Organization Org. Type Agreement Type Role/ Responsibility Funding Quality/Outcome of Partnership Primary Local Partners CRECER NGO MOU CwE TA Formerly an FFH organization, FFH has supported a successful transition to an independent, self-supporting org. Links and reciprocal commitment remains strong. FFH has representation on BOD. CARD NGO/Dev Bank MOU Modified Grameen replicate Op. Costs: $100,000 TA High levels of reciprocal respect, mutual learning based on shared vision and complimentarity of skills and needs. Affiliates/ Funders LINKAGES USIAD-funded project (AED) MOU Tech. Input on breastfeeding, infant feeding, LAM and maternal nutrition plus dissemination LINKAGESinkages provided significant input to Breastfeeding and Infant and Child Feeding modules. Now supporting dissemination of CwE to health community. FANta USAID-funded project (AED) Task Orders Documentation of 3 progress tracking methods Documentation of: · Participatory/qulaitative monitoring · Applying LQAS to client knowledge and practice surveys · Observational checklists for education delivery CAREMoRR USAID MOU Strengthen technical capability for RH services Summit Foundation Foundation Grant Grant making $300,000 Supported the development of the CBD system and family planning module. Important Partners that do not receive funds -Networks CRECER PROCOSI Network of Child Survival PVOs in Bolivia Member Info sharing and technical guidance; funding conduit NA Good sharing of technical info and materials. Breastfeeding module used by member orgs (LINKAGE’s participation) CARD Philnet Membership org. for Grameen replicator MFI’s in Philippines Member Info sharing NA Network of Grameen Bank replicators in the Philippines dedicated to lateral learning and technical assistance. CASHPOR Membership org of MFI’s in Asia Member Information sharing NA Network of Grameen Bank replicators in Southern Asia dedicated to lateral learning and technical assistance. FFH CORE Network of Child Survival USPVOs Member Info sharing; tech updates NA Dissemination channel; FFH planning major presentation to CORE USPVOs: 58 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Level Organization Org. Type Agreement Type Role/ Responsibility Funding Quality/Outcome of Partnership SEEP Network of microcredit USPVOs Member Info sharing; tech updates NA -ditto￾Individual or agency affiliates LTSP Associates Tech assistance providers/Individ ual consultants Contracts Provide business management services to NGOs etc. Fee for service contracts Important dissemination and technical assistance mechanism CwE Implementing partners NGOs, PVO, Development Bank MOU (CARD), Grant agreement (CRS/Madagas car), Affiliate relationship (FFH/Ghana) Deliver services to clients NA “ “ Can serve as demonstration sites 59 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex F: Sustainability Analysis Supporting factors Inhibiting factors Conclusions CARD Political National Government Support ß CARD Bank accreditation allows legal operation to mobilize savings ß CARD has been approached by a number of Ministries interested in Cooperation ß CARD Bank must adhere to stricter requirements in taxation and personnel management ß CARD is well regarded and in good standing with GOP. Local Government Support ß CARD always obtains Local Govt. Agency support before opening a bank. ß Resistance to family planning in one municipality ß CARD is in good standing with LGAs. Institutional Networks ß Membership in Philnet and CASHPOR provide opportunities for networking, training, dissemination with other Grameen replicate agencies ß Focus on rural and poor reduces potential for competition with other MFIs ß CARD trains many MFIs and NGOs to do microfinance ß Potential for competition among MFIs. ß CARD maintains a leadership role in the MFI movement in the Philippines and Asia. Management ß Very stable, cohesive management team ß Engaged, diverse BOD (includes members) ß Each senior staff has a personal plan for each year (includes center visits) ß Culture of agility enables rapid uptake of new approaches ß Progress tracking as catalyzed a significant rethinking and simplification of data management ß Time to maintain traditional practices—connection with clients—threatened as organization grows. ß CARD NGO’s CEO is acting ß CARD has become more inclusive of staff and clients in decision making ß FFH has given CARD methods for accessing and affirming the value of this input. ß High level of staff commitment and satisfaction at all levels. Structure ß Different financial centers spreads risk ß Management is appropriately decentralized ß Programmatic focus on saturation in target areas is cost-effective ß With changes in key positions, various cost centers could become competitive. ß CARD has established the foundation for replication and growth. Technical Development ß CARD is adept at identifying needs and organizations to provide TA ß Timing of FFH input coincided with felt need of CARD to reconnect with its development focus ß CARD continually modernizing its systems (computerized) to be more responsive ß Resources ß Lack of in-house health expertise ß MIS system based on MFI requirements only. ß CARD’s core technical capacities and ability to identify and absorb TA are strong. 60 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Supporting factors Inhibiting factors Conclusions HR Development ß CARD put all Area and Branch Mangers through MBA program, paid by CARD. ß Rapid scale up in mid-90’s led to hiring of very young, less technically qualified staff ß HR guidelines still formalizing. ß CARD invests in staff. Quality staff and staff-friendly organization. Financial ß CAARD has successfully diversified and maintained its donor base ß Economic slump impacts all MFIs ß Drop in donor support to the Philippines ß CARD has remained attractive to clients and donors despite contextual pressures. CRECER Supporting factors Inhibiting factors Conclusions Political Natl. Govt. ß CRECER is recognized as a Bolivian civil association ß Have agreements and increasing cooperation with MOH and municipality health offices ß Government has created national fund, FONDESIF, to channel bilateral support in the form of loans to non-regulated microfinance institutions ß Have been nationally and internationally recognized ß Requirements to become a FFP are very high; as a financial NGO, CRECER is limited in services it can provide and access to resources. ß Government feels pressure from debtor associations and other pressure groups to intervene in sector. ß For the foreseeable future, CRECER will not change its legal status. ß CRECER needs to build on its relationship with the MOH and better leverage it at local levels. ß As one of the leading non-regulated financial institutions, CRECER has had good access to loans made through FONDESIF. ß CRECER’s growing reputation nationally and internationally should increasingly yield results in terms of access to additional resources and other opportunities. Local Govt. ß Good relationships with local government and clients ß CBD and referral services help to build relationships with local health services. ß Blockades and strikes stall flow of goods, services ß Availability and quality of health services varies ß Throughout the periods of difficulties, CRECER’s staff were able to move about due to good local relations, and CRECER’s clients continued to make repayments (some clients attributed this to a feeling of loyalty engendered by the fact that CRECER “cares” about them, because it not only provides financial services, but also education. ß CRECER is continuing to work on reinforcing and expanding these relationships. 61 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Supporting factors Inhibiting factors Conclusions Institutional Networks ß Longstanding member and leader of PROCOSI, a national health network (CRECER has held positions on the executive board and continues to play a leading role in development of educational materials) ß Playing leadership role within FINRURAL (CRECER currently is on executive board and is treasurer) ß Leadership at the regional offices actively participate in regional networks. ß National and regional leaders use professional networks to stay informed and have influence in the microfinance and health sectors ß CRECER is being approached by regulated institutions to form strategic alliances to reach rural clients. ß As a provider of integrated microfinance and health education, CRECER feels pressure from both sides to conform completely to the norms of each industry. ß There are increasing pressures to regulate and tax even the financial NGOs. ß CRECER is being more strategic about cultivating and maintaining its relationships formally through networks and informally ß CRECER is gaining a higher profile through participation in national (Santa Cruz workshop on savings mobilization in June 2001), regional (presentation at Latin America Microcredit Summit meeting in Pueblo, Mexico in October 2001), and global meetings and workshops (scholarship for General Manager to attend Frankfurt Conference on Microfinance in Frankfurt, Germany in September 2001; paid trip by PROCOSI to Boston, USA in January 2002) ß CRECER is exploring alliances to expand services to clients and increase its access to funds for on lending. ß CRECER is using its role within FINRURAL to influence regulatory developments. ß CRECER is helping PROCOSI ensure the materials and training guidelines developed are more practical. Management ß Strong, technically capable, well respected, and engaged BOD ß Capable General Manager and use of management team ß FFH demonstrated confidence in CRECER governance and management of CRECER by transferring all assets at time of independence, resulting in mutual goodwill and a productive partnership, as embodied in strategic alliance ß Critical systems, such as management information systems and internal control systems, are being upgraded to enable CRECER to meet growth targets. ß Strategic and business plans provide vision and strategy for moving forward ß Low turnover of staff, even through transition to independent organization ß Renewed commitment to education and understanding impact and client satisfaction ß Development of human resources may not keep pace with requirements of organization. ß Not clear to what degree CRECER depends on continued involvement of FFH in governance bodies. CRECER and FFH need to continue to establish new methods of collaboration and partnership. 62 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Supporting factors Inhibiting factors Conclusions Technical Development ß CRECER is flourishing in highly competitive microfinance environment ß Education and Human Resources Manager now part of management team ß Incorporated upgraded education CwE components ß Improved training and supervision systems are yielding results in service quality, particularly of education component ß Enhanced MIS-progress tracking is being developed for monitoring quality at scale. ß Lack of access to financial resources (even commercial loans) to fund portfolio growth ß Need to identify additional sources of support for development of educational materials and special initiatives ß Need to develop new financial products for longstanding clients ß Security risks of carrying cash. ß ß CRECER will continue to build on relationships with donors, technical agencies, other MFIs, and government to develop products or provide services in alliance with other entities. Financial ß Has exceeded operational self-sufficiency (102%) and is close to financial self-sufficiency (98%) ß ß Excellent portfolio quality (0.4% portfolio at risk and 0.2% long term loss rate) and culture of repayment amongst staff and clients ß High client satisfaction and retention, attributable in part to education services and to internal account lending ß ß Mix of sources of concessional and commercial loans and diverse sources of grants for educational activities and special projects ß For CRECER’s portfolio to triple during the next five years, it will need access to long term loans and increase its equity base if possible ß ß Inflation and particularly devaluation continue to negatively affect CRECER’s financial situation ß CRECER needs to more fully develop financing strategy and do advocacy work on behalf of well-managed and well-governed non-regulated institutions to access diverse sources of funds. Freedom from Hunger Supporting factors Inhibiting factors Conclusions Govt. Support ß Progressive MG’s gave FFH opportunity to develop model, strategy, and services ß Changing USAID/congressional agendas ß While not overly dependent on USAID funding, FFH regards USAID as a development partner. Institutional Networks ß Growing network with an interest in the application of CwE services in their operating environments ß FFH’s role in several key health and microfinance networks provides a diversity of dissemination and demand for services channels ß Some MFI resistance to integrating non financial services with financial services ß FFH’s focus on the very poor, which sets it apart from other MFIs that are more focused on commercialization ß Life of networks unpredictable: based on funding availability ß FFH views its participation in and links with networks as an effective way to achieve scale for CwE. 63 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Supporting factors Inhibiting factors Conclusions Management ß Cohesive, stable team ß Strong and engaged BOD ß Dynamic, respected leadership ß Self-determined limits to growth ß Size and agility makes innovation easy ß Outreach grows while staff size remains constant ß Catalytic orientation supporting large impact ß Learning culture: open to self exportation ß Too much demand for core staff overwhelms availability ß Distance from DC may preclude attendance at seminal events ß Harder to recruit staff to Davis ß Little diversity among staff ß Too much soul searching? Not aggressive enough ß Management capacity and structure well suited to achieving FFH’s catalytic vision. Technical development ß Deep technical competency within staff in key areas (health, adult education, microfinance, capacity building, research) ß Staff feel comfortable with technical aspects of the model ß Culture of respect and integrity allows for cross fertilization of expertise ß FFH has consciously moved towards greater flexibility in the mix of services it will offer to collaborating organization ß Growing receptivity to the CwE model ß Vision has not materialized in terms of the way contracting agencies envisioned CwE (what they wanted to buy was not what FFH wanted to sell) ß Possibility that agencies will take modules and not ask for TA ß Use of CwE methods with low quality can impact on FFH’s reputation. ß FFH maintains a highly focused mission and commitment to pushing the boundaries of excellence within its mission and at the same time remain relevant to the development debate. This is likely to ensure it a sustainable niche in the PVO/NTO/MFI world. Financial ß Diverse donor base ß Loyal major private donors ß BOD actively engaged in fiduciary oversight ß Have an expectation of mixed revenue streams including service revenues, direct grants and social venture capital investments ß Small size allows for more flexibility ß Maintain reserve funds ß Vulnerable to changes in the economy ß Not a household name ß Lack of a business plan ß ß Integrated approach requires more marketing to sectorally-based sources of funds ß Financial sustainability relies on FFH’s self-imposed limit to growth, a diverse donor base, and growing range of marketable products and services. ß Need to develop a business plan, taking into consideration collaboration assessment, market assessment, and strategic direction 65 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex G: List of people interviewed Freedom from Hunger, International Center Christopher Dunford, President Joan Dickey, Vice President, Finance and Administration (by email) Claire Thomas, Vice President, Development and Public Affairs Didier Thys, Vice President, Practitioner Services Beth Porter, Director, Technical Operations Robert Richards, Technical Advisor, International Center Kathleen Stack, Senior Vice President Ellen Vor der Bruegge, Senior Vice President Robb Davis, Senior Technical Advisor, Maternal & Child Health Vicki Denman, Senior Technical Advisor, Maternal & Child Health Barbara MkNelly, Senior Technical Advisor, Evaluation Cooperating Agencies Kim Winnard, LINKAGES, AED Jamie Steward, CAREMoRR Project, CARE International USAID Tom Kennedy, PVC/USAID Nards Dayao, PHN, USAID/Philippines CRECER Evelyn Grandi, Executive Director Isabelle Rueda, Training Director CARD Aris Alip, Founder and Acting CEO, CARD NGO Flor Sarmiento, Training Director Dolores Torres, CARD Bank Executive Director Annie Alip, Research Director Dick Alvarez, Area Manager Quezon Elvira Kamayacayac, Branch Manager, Tagkawayan Somomon Aramil, Senior Technical Officer, Tagkawayan Aurea Dequito, Technical Officer Vivian Erivera, Technical Officer Sheilo Kamppitan, Technical Officer Edison Reyes, Technical Officer Tagkawayan Barangay Health Unit Rebecca Castillo, Baranguay Health Nurse In-Charge Also interviewed: Village Health Workers Approximately 10-15 CARD clients individually 66 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc CARD Financials INDICATORS NGO BANK TOTAL Date Established DEC-86 SEP-97 No. of Branches 18 7 25 No. of Active Clients 26,697 23,190 49,887 No. of loans disbursed 160,497 165,878 326,375 Amount of loans disbursed 762,741,168 1,043,759,550 1,806,500,718 Ave. loans disbursed 4,752.37 6,292.33 5,535.05 No. of loans outstanding 40,729 42,960 83,689 amt. of loans outstanding 157,634,377.77 221,357,060.29 378,991,438 ave. loan amount outstanding 3,870.32 5,152.63 4,528.57 total no. of savers 28,921 55,116 84,037 total savings balance 47,146,103.16 121,732,609.21 168,878,712 repayment rate 100.00% 99.90% 99.95% portfolio at risk 0.14% 0.63% 0.43% operational self￾sufficiency 135.95% 142.00% 139.23% financial self￾sufficiency 103.48% 115.00% 118.23% adjusted returns on assets 1.06% 5.00% 5.00% adjusted return on equity 1.80% 31.40% 15.37% asset to equity ratio 2.33 6.89 3 administrative cost ratio 28.08% 21.00% 24.00% 67 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc Annex H: Debrief with CARD, January 17, 2002 San Pablo, Philippines Thanks for hosting me. I know that visitors take away from your important work. These few days have given me a small taste: would like to reflect on what I’ve seen with the caveat that it is necessarily superficial. Use this session for clarification, feedback, and raise some Qs for the future. Want to talk about 1. The partnership 2. Specific aspects of the program 3. Future challenges and issues to think about. Partnership 1. Appears that it has been a mutually happy coincidence that you met when you did—love at first sight. 2. CARD looking for a boost on the development side—back to the original vision as a development agency. 3. FFH looking for a mature MFI partner to help build the next generation of education modules and think seriously about systems for scale. There is clearly respect and very good energy on both sides: good example of organizational synergy. 4. FFH approaches and methods have been absorbed and adapted based on your own needs as you deemed appropriate—the whole MIS area is a good example (Boss Aris told me that FFH is now an adjective that describes the way you do data collection)--in terms of the focus on simplification, the use of checklists. 5. FFH has benefited from your input—grounded in field realities-- on these systems, on the modules, on systems and approaches, as they attempt to develop generic systems and guidelines for wide dissemination and adaptation. 6. One thing that I perceive as having made the partnership work was a shared vision and commitment to business principles of fiscal discipline and adherence to policy at all levels—principles that in important and interesting ways make the education component operationally possible as well—more on this below. CwE: Systems Training: 1. This is the cornerstone of CwE. A cascade training approach supports the dissemination of key messages to the clients. 2. The methodology relies on State of the Art adult learning principles to which FFH has a solid commitment. 3. The transfer of technology is deceptively simple—modules distilled to essentials. 4. Challenges: ensuring accuracy and encouraging participation are both essential but competing objectives of CwE. · Reading a list of facts to a passive audience allows the trainer to remain in control of information. · But we all know people don’t learn as much this way. · Adult learning engages participants in the teaching process. · The challenge I saw in Tagkawayan is that you have facilitators who are skilled bank tellers, conveying public health information that is new to them. It would be easier if they could present it, but they must present and facilitate. 5. Rumors, fears, beliefs, husbands’ and other family members power are all factors that come into the mix. Clients know a lot and have strong beliefs about most of the topics CwE introduces. 68 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc · When 3 women speak forcefully about a couple that died because the woman was using an IUD which got caught on the man'’ penis, it seems to carry the day despite the fact that another member of the group gives personal testimonial about successful use of the IUD. · This kind of situation is bound to come up in many ways—condoms produce a rash, make men impotent, give women headaches, pills make women sterile. · With only a superficial understanding of the medical issues, it is impossible (and unfair) to expect TOs (or the curricula) to be able to anticipate and respond to every rumor or question that will come up. · And TOs must sort out what is true and what is myth on the spot. The danger here is that in a facilitated session, which encourages participants to share views and experience, the incorrect rumor will carry the day. · FFH and CARD will have to think together about how to address this—particularly in a scaled up situation where the 100sof TOs are conveying information to 1000s of women, where the level of supervision, day to day input and foreigner presence is not there. · Misinformation or mistakes have a funny way of multiplying or rebounding in big ways. While a lot of safeguards are in place already, this is a potential Achilles heel of the approach which but merits vigilance as you continue to test and expand CwE. Having said this, it is unrealistic to expect CARD to dispel all myths through CwE. If it plants a seed, reinforces correct information, and allows an opportunity for dialogue, it has made a contribution. 6. Going back to the challenges: I wondered before I came out how men could talk to women about such sensitive issues as family planning and when I met the TOs about how mature women would respond to young unmarried people could credibly present information about issues related to sexuality and childrearing. This is a tall order. In Tagkawayan you seem to have pulled it off, and it is working. When you’re addressing issues of scale, I think it’s very important to ask: why is it working? and be ever mindful of the critical factors when you look at scale up. I asked questions about this to everyone: Dory, Annie, Ellen, Flor, Dick, Elvie, the TOs and the clients. I’d like to share what I learned and see what you think: 1) credibility: these young people have already established a good bit of trust and confidence among the clients, so the information they bring carries weight no matter what the subject matter. 2) the visible input of foreigners into the process has leant the information considerable credibility in the Tagkawayan context. 3) The modules themselves—which script exactly what the TOs should, also anticipate questions or rumors 4) Permission to say: “I don’t know” when TOs feel a question or issue is beyond what they know. I would like to add 3 which did not come up but which also seem relevant to TO acceptance: 1) the homework TOs have done in mastering some of the information and continuing to update their knowledge is impressive. 2) the team spirit in which branch managers and area managers also have an active stake in education outcomes. 3) The fact that TO performance is now assessed both in terms of portfolio performance and education outcomes. I have to say that I was very impressed with the TOs--with their confidence and competence and with the dialogue they were able to stimulate in the groups. I no longer think age or gender is a central issues, though it is certainly worth monitoring that (more below). 69 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 7. Adult learning: just want to say that this is another, rather invisible building block of CwE which deserves attention. It is an integral part of the training which could easily be diluted in the rush to achieve scale. It is very clear that the attention to this issue has turned the TOs into real trainers rather than just conduits for information. And this is no easy feat. Again, something that bears vigilance in the scale up phase. Targeting 1. The other anomalous part of the education component is the audience for the CwE. Normally MCH information is targeted at women in their reproductive years. Early is better than late so your investment pays off. 2. In the case of CARD, and perhaps many MFIs, clients are more at the end of their reproductive years. So there is an assumption about dissemination at this level too—that older women will talk to daughters, sisters, neighbors etc. 3. Anecdotal information from this evaluation validates this assumption—which is further reinforced by homework assignments asking clients to talk to others about what they have learned. 4. However, since the target audience is essentially one step removed from the immediate CwE beneficiaries, this assumption probably bears further, more systematic exploration. Staff recruitment 1. As you expand your organization, bring on new staff, and disseminate the education approach thru the org., the profile of the type of field level staff you hire may shift somewhat. 2. Challenge: finding staff who can maintain the fiscal discipline of a loan officer but at the same time have the personality to train and facilitate sessions on rather sensitive subjects. 3. I think one question in all of this is whether it is easier to make a loan officer into a health trainer or vice versa? Perhaps this is something you and FFH can experiment with as you move forward. 4. In discussions at FFH there was a feeling that health workers may find it more difficult to adapt to the hard-nosed disciplines required by the loan officer, whereas loan officers would find it easier to learn basic health messages. 5. It is also possible that young people can better absorb new information on both fronts and make more malleable trainers/facilitators. In the case of Tagkawayan, this may be right. 6. In any case, you will no doubt want to look at issues of age, gender, marital status, training and esp. facilitation expertise, and academic work background related to both accounting and health as you recruit. Technical back up/cooperation with health providers 1. There is always an issue in doing health education or family planning promotion that you are raising awareness, motivation and encouraging health seeking behavior, but the services are not there to meet the demand you are generating. 2. I am concerned about this, and also about whether CARD feels it important to build bridges to providers—both to back up information but also provide services. The Barangay Health nurse in Tagkawayan only knew about CARD from her laundry woman. 3. Links with health professionals seem to rely on TO’s relationships (one has an uncle who is a Dr. etc). 4. I would encourage you to actively build bridges not only to government but also local private and NGO health providers—encourage TOs to bring them in as speakers, let them know what you are doing, use them as resources, figure out where to send women with questions or health needs. 5. FFH has prepared a useful investigative checklist for assessing health and nutritional behaviors and cost of services. Seems an appropriate tool for Branch Managers to start this outreach at that level. 6. You need to explore whether more formalized relationships would also make sense: in Bolivia CRECER members get ID cards that give them special privileges when they go to govt. clinics or hospital. Another point on this: 70 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 1. CARD is not delivering health information a vacuum. Women get information from a variety of sources. Government has messages that it promotes. It is important that CARD at very least inform itself about what these messages are, and insofar as possible seek to align with them. A confusion of messages may make it more challenging for clients and may negate any positive benefits of CwE. 2. Specific example: a VHW told us she was trained to advise women to breastfeed exclusively for only 3 months (you are recommending 6). While it is possible this is outdated or incorrect, it is important that CARD staff know what the government position is on this, on family planning, on iodized salt, on fortified foods etc. Even if a slightly different message is being delivered, it can be delivered in context and linked to government information and programs. 3. This investigation should be done as modules are adapted to the local situation (use FFH adaptation guidelines; expand FFH adaptation guidelines). Supervision and quality 1. Maintaining quality is key to successful CwE—accuracy of MSG and client participation major indicators. 2. From the small glimpse of CARD, it seems like you have a staff-centered supervision system in place, good job satisfaction, built on fairly strict performance measures. 3. The supervisor in Tagkawayan reported that she feels more bonded with her staff because they are all learning the new information together. 4. Checklists give her ways of assessing their performance, but she cannot be expected to either provide additional health information and can only assess when they go beyond the script—not the accuracy of any other info they may present. This is a challenge! 5. Wondering about your plans not only for introducing the checklists but using them institution wide— some qualitative assessment involved. · Training in use of checklists? · Triangulation of perception? · How you will aggregate data and make it useful for management to assess performance of field staff/branches on education? · Whether TOs need to demonstrate a basic level of comprehension of the materials as part of their initial training/orientation? 6. Also the issue of time: the education component makes additional demands on supervisor’s time. What gets sacrificed? MIS/progress tracking 1. Important step that research and dev. has come into its own—reflects a mature, learning org. 2. Rapid incorporation of learning’s and tools. 3. Seem to be making effective use of progress tracking tools for sr. management decision making re policy—e.g. client satisfaction led to the introduction of the “resting” concept. 4. Q: human resource w/in CARD during times of rapid scale up. I know this is a concern of yours’. This is not only to do the analysis, but train other staff in key tools, data collection, make sure managers understand how to interpret and use, and do the kind of one-off studies that help improve the model and facilitate dissemination. Measuring outcomes and impacts 1. Related to this: how do you know you’re having an impact on health behaviors and outcomes? These are a lot more squishy than financial outcomes to measure—can’t link mortality or morbidity— illness—with CwE and hard to measure behavior or attitudinal change. 2. Understand you have a grant from Ford to look at impact—does this include CwE? Dissemination to other MFIs good aspect of this program. Scale up 1. Have touched on issues related to this above but want to add a bit here: 71 H:\INCOMING\06-26-MSI-fina-FFH-evaluation report\PVC FFH Final Evaluation Report.doc 2. CwE and FFH partnership seem to have watered the seeds that were already planted and ready to sprout for a quiet cultural transition w/in CARD. Still in testing phase, but seem committed to institutionalizing and extending beyond CARD borders. 3. One of the things that FFH has sought to develop over the life of this MG are better tools for achieving scale. I’d like to hear from you to what extent you feel equipped by these tools to reach for your new heights? 4. Challenge to institutionalizing--retaining quality at all l3evels. Build confidence and capacity among staff w/out the very intensive input Tagkaywan has received. How will you take what you have learned with 5 TOs there and empower 172 TOs (and growing). 5. You all seem very empowered and confident about doing this. Great. 6. Caution against rapid scale up without really finding out what works in practice. One of the things FFH is modeling is this step by step action research approach, which particularly in these early days will serve well in the longer term. So: you are doing and at the same time you are observing and recording what you are doing, and learning at each step. 7. CARD is obviously a leader in microfinance not only in the Philippines but regionally and globally. Very exciting to hear your president mention you in speech. So what you do will send a lot of signals to the whole industry. 8. Also you and FFH will have to think together about how you make more essential the process you and they have gone through as you attempt to extend the technology and system to other agencies. Let’s say another mature MFI asks you to help “embed” CwE in their program. Do you need a week of training, a month of TA, a year of mentoring? What kind of package of skills and approaches are you offering? 9. Also need to look at your own human resource: balance scale up (500,000 new clients by 2010) and integration of CwE with extension outside. You have a commitment to retain your connection w/field—visit centers, personal touch w/staff. How to maintain this? See that you are continually trying to maintain. Thanks and very best wishes.