IIRR International Institute of Rural Reconstruction A Transregional Program to Build Capacity for Sustained Development Impact Final Evaluation September 2000 Project No. 938-0158 Cooperative Agreement No. FAO-A-00-96-960050-00 This report was prepared by AMA Technologies, Inc. under contract FAO-C-00-95-0036-00 with the Office of Private and Voluntary Cooperation, Bureau for Humanitarian Response, USAID. Evaluation Team: Dr. Constance M. McCorkle Team Member: Scott Killough TABLE OF CONTENTS General Acronyms Executive Summary i-xvi 1. Introduction 1 1.1 The Institute 1 1.2 The Matching Grant 3 1.3 The Evaluation 7 1.3.1 Tasks and Timeline 8 1.3.2 Methods and Subjects 8 1.3.3 Organization and Use of This Report 11 1.4 Summary of Findings on Progress Toward MG Objectives 12 2. Program Implementation: Publications and Communication 13 2.1 Overview 13 2.1.1 Capacities 13 2.1.2 Policies 14 2.1.3 Approaches and Procedures 15 2.2 Progress and Trends 17 2.3 The Writesop: A Participatory Publication Methodology 23 2.4 Lessons and Recommendations 31 2.4.1 Lessons 31 2.4.2 Recommendations 32 3. Program Implementation: Education and Training 35 3.1 Overview 35 3.1.1 Categories and Content 35 3.1.2 Policies 38 3.1.3 Approaches and Procedures 39 3.2 Progress and Trends 41 3.3 Alumni Associations: A Two-Way Tool 48 3.4 Lessons and Recommendations 50 3.4.1 Lessons 50 3.4.2 Recommendations 53 4. Program Implementation: Field Projects, Technical Assistance, and Partnerships 55 4.1 Overview 55 4.1.1 Field Projects 55 4.1.2 Technical Assistance 58 4.2 Progress by Region 59 4.2.1 Africa 59 4.2.2 Asia 61 4.2.3 Latin America 63 4.3 Partnerships: More Bang for the Buck 67 4.4 Lessons and Recommendations 74 4.4.1 Lessons 74 4.4.2 Recommendations 75 5. Program Management 77 5.1 Overview 77 5.2 Progress and Trends 82 5.2.1 Organizational Structure 82 5.2.2 Staffing 87 5.2.3 Human Resource Management and Development 93 5.2.4 Financial Management 96 5.2.5 Governance 103 5.3 Lessons and Recommendations 105 5.3.1 Lessons 105 5.3.2 Recommendations 106 6. Looking Ahead: Sustainability and M&E 110 6.1 Progress and Trends in Sustainability 110 6.1.1 The Institute and the Regions 110 6.1.2 Publications, Training, Field Projects and TA 120 6.2 Progress and Trends in M&E 124 6.2.1 The Institute and the Regions 125 6.2.2 Publications, Training, Field Projects and TA 132 6.3 Lessons and Recommendations 138 6.3.1 Lessons 138 6.3.2 Recommendations 140 ANNEXES A Evaluation Scope of Work B Findings from Focus Groups with IIRR Partners C DIP Schedule of Activity D Selected Raw Data Compilations D-1 IIRR Training and Related Data D-2 AFR Regional Staff D-3 Ethiopia Country - Program Staff D-4 LAC Regional Staff D-5 Grant Income Sources D-6 Earned Income Sources D-7 Notes on Data Analysis Process E Persons Contacted F Documents Consulted G Organizational Acronyms LIST OF TABLES 1.1 IIRR Functional Units and Program Thrusts 2 1.2 Milestones in IIRR Decentralization 3 1.3 Evaluation Interviewees 11 2.1 Numbers of Major Publications 17 3.1 IIRR Courses 37 3.2 Numbers of Training Events (and Participants) 42 4.1 1992-1995 Field Projects 56 4.2 1996-1999 Field Projects 57 5.1 Planning and Management Milestones 77 5.2 1999's Year-long Strategic Planning 80 5.3 1995 and 1999 Staffing 88 5.4 Staffing Distributions as of December 1999 89 5.5 MG Expenditures by Units/Regions (US Dollars) 100 5.6 MG Direct Costs as Budgeted versus Expended by Units/Regions ($000s, including procurement but excluding indirect costs) 101 6.1 Revenue Sources (excluding private donations) 111 LIST OF FIGURES 1.1 BHR/PVC Support to IIRR 4 2.1 Map of 1996-1999 Publications 18 2.2 The Writeshop Process 24 4.1-A Partnerships with GROs and NGOs in MG Focus Countries, as of 1995 (N=19) 68 4.1-B Partnerships with GROs and NGOs in MG Focus Countries, as of 1999 (N=46) 68 4.2-A Partnerships with PVOs in MG Focus Countries, as of 1995 (N=11) 69 4.2-B Partnerships with PVOs in MG Focus Countries, as of 1999 (N=27) 69 4.3-A Partnerships with Research and Training Institutions Worldwide, as of 1995 (N=31) 71 4.3-B Partnerships with Research and Training Institutions Worldwide, as of 1999 (N=107) 71 5.1 1999 Strategic Planning Process (March - November) 81 5.2 Organizational Structure as of 1995 83 5.3 Organizational Structure as of 1999 84 5.4 Proposed Organizational Structure as of 2000 87 6.1 IIRR Revenue ($000s) 112 6.2 Grant Income and Earned Income for AFR and LAC ($000s) 114 6.3 Computerized Resource-Generation Management System 118 LIST OF BOXES 2.1 Some B.I.G. Publications Payoffs 20 2.2 Runaway Publication Successes! 22 2.3 Meeting a Region-wide Information Need 25 2.4 Writeshop Multiplier Effects 27 2.5 Satisfied Customers GO Back for More 27 2.6 A Writeshop with a Whollop! 28 2.7 Innovating on the Writeshop 30 3.1 IIRR to Offer Accredited Training 43 3.2 Responding Sensitively and Flexibly to PVO/NGO Training Concerns 44 3.3 Training Builds Polyvalent Partnerships as Well as PVO/NGO Capacities 45 3.4 Cost-Effective and "People-Effective" Community Training 47 3.5 Cross-Fertilization in IIRR Training 48 3.6 Effective Training is Put Straight to Work 51 3.7 Effective Training Replicates Itself 52 3.8 Effective Training Wins Funds 53 4.1 AFR Gains Repute by Cleaving to IIRR Vision-Mission-Values 61 4.2 A New Take on Environmental Health 63 4.3 Farmer Participatory Research 64 4.4 Farmer-led Extension 65 4.5 The Power of IIRR Partnerships 72 5.1 Creative Staffing: A Win-Win Proposition 95 GENERAL ACRONYMS AFR Africa Regional Office/Center (of IIRR) Ag Agriculture/agricultural AI Appreciative Inquiry ASI Asia Regional Office/Center (of IIRR) ATIK Agroforestry Technology Information Kit BHR/PVC Bureau of Humanitarian Response/Private Voluntary Cooperation Office (of USAID) B.I.G. Bio-intensive gardening project/course/publication (of IIRR) BOT Board of Trustees CFO Chief Financial Officer CMIS Central Management Information System CSS Computer Systems Specialist DIP Detailed implementation plan (of MG) FG Focus group FIS Financial information system FPR Farmer participatory research FTE Fulltime employee units FY Fiscal year GO Government organization GRO Grassroots organization ha Hectares HQ Headquarters IARC International agricultural research center (of the CGIAR) IT Information technology LAC Latin American Regional Office/Center (of IIRR) LOE Level of effort LOP Life of project/program M&E Monitoring & evaluation MG Matching grant (of BHR/PVC) MIS Management information system MOA Ministry of Agriculture MOU Memorandum of understanding MSO Microsoft Office N/A Not applicable NGO Non-governmental organization (national or regional, in developing countries) NRM Natural resource management OCAT Organizational Capacity Assessment Tool OJT On-the-job training PM&E Participatory monitoring and evaluation PVO Private Voluntary Organization (international, typically headquartered in developed countries) RC Regional Center (of IIRR) R&D Research and development RR Rural reconstruction SMART Simple, measurable, accurate, reliable, timely [indicators] SOW Scope of work SP Strategic planning TA Technical assistance TOT Training/trainer of trainers TSG Technical Support Group (of IIRR) VP Vice President US United States (of America) USAID-Matching Grant Evaluation Report i EXECUTIVE SUMMARY 1. Introduction Established in 1960, the International Institute of Rural Reconstruction (IIRR) is a US-registered private voluntary organization (PVO) with headquarters (HQ) in the Philippines and an ancillary, two-person office in New York, NY. The Institute's mission centers on sustainable rural development through local self-help and mutual aid backed by support from other organizations, all with a highly people-oriented and participatory approach. Institute programming has three substantive thrusts: environment, natural resource management, and agriculture; institutional capacity-building; and community health, reproductive health, and nutrition. These are cross-cut by two themes: local knowledge and gender. IIRR acts primarily as an intermediary PVO. It provides development information, training, and technical assistance in the foregoing arenas to non-governmental and grassroots organizations (NGOs, GROs), the regional and country chapters of other PVOs, and government organizations (GOs). However, IIRR also collaborates in field￾based projects for applied research and development purposes, as these pertain to its thrusts and themes. Until little more than a decade ago, IIRR was essentially a Filipino NGO with some activities in other Asian nations. With substantial matching grants (MG) from USAID's Bureau of Humanitarian Response, Office of Private Voluntary Cooperation (BHR/PVC), however, IIRR successfully extended its operations beyond Asia (ASI) into Africa (AFR) and Latin America (LAC). In the process, the Institute restructured itself -- first, by establishing AFR, ASI, and LAC Regional Centers respectively in Kenya, at the HQ site in the Philippines, and in Ecuador; and second and still ongoing, by endeavoring to decentralize the bulk of programmatic and related managerial and financial decisioning and action to these Centers. To reinforce IIRR restructuring and decentralization (among other things), in 1996 BHR/PVC awarded the Institute a follow-on grant to its 1990- 1995 MG of $3.5 million. The follow-on award was for $1.5 million and 3 years, with a no-cost extension of several months in order to accommodate a final evaluation. The new grant designated certain countries in which each Regional Center was to focus its MG activities: Ethiopia, Bangladesh and Nepal, and Ecuador. The MG's primary objectives can be paraphrased as follows: USAID-Matching Grant Evaluation Report ii I To strengthen the capacity of IIRR's NGO and other partners to implement participatory, people-centered local development programs. II To increase the capacity of IIRR's Regional Centers to deliver and sustain services to partners, undertake process documentation, promote inter-institutional learning and networking, and do impact evaluation. III To build Institute capacities as a whole, the better to achieve 1 and 2 (an implicit purpose). The present report constitutes a final, external evaluation of IIRR progress on these objectives under the 1996-1999 MG. The evaluation was conducted by an independent senior scientist during October through December 1999. Analysis was done on three levels: Institute-wide; regionally; and where appropriate and feasible, nationally by focus country. Note, however, that while IIRR's AFR and LAC Centers achieved a modicum of the operational independence and maturity envisioned for them under decentralization, to a large extent ASI was held hostage, budgetarially and managerially, by various HQ units. Consequently, most financial and program data for ASI are inseparably aggregated with HQ data. This made analysis of many ASI-specific activities and achievements impossible. IIRR activities in pursuit of the above objectives spanned: publications and communication; education and training; field projects, technical assistance, and partnerships with other development organizations; and Institute and program management, including sustainability and monitoring and evaluation (M&E) initiatives. 2. Publications and Communication With few exceptions, IIRR's formal publications today are handsome, well-edited, professionally produced, and field-sturdy books and spiral-bound manuals. They capture best practices from IIRR's and other agencies' work, proffer often-original field-based technical information, give methodological guidance, explore emerging development issues and approaches, and more. The Institute's publications program could well be termed "the shining star of IIRR" for reasons outlined below. Institute-wide Achievements. By nearly 100%, this category surpassed the target of 17 formal publications set in the MG's detailed implementation plan. During 1996-1999 IIRR produced and distributed a total of 32 publications. This equals more than twice the 14-item output of the 1992- 1995 baseline. Nor does this 32-count include many other, less formal but equally valuable manuals, guides, handbooks, workshop reports, and extension USAID-Matching Grant Evaluation Report iii and training materials generated by the Regional Centers and HQ during the MG. Moreover, another 13 formal publications were in press at the time of evaluation. Content naturally reflects the Institute's programmatic thrusts and themes. Compared to the baseline, institutional capacity-building and gender received much more attention in IIRR publications during the MG. Health subjects also came more to the fore. Other baseline-to-MG trends included a sharp increase in the number of non-English-language publications and in the percentage of publications with global, as versus only regional, relevance. These figures went from 0 to 11 and from 28% to 47%, respectively. Relatedly, cross-regional collaboration in publications increased, as did collaboration with other development organizations generally, and specifically with higher-order research and training institutions worldwide, such as Northern universities and the World Bank's system of international agricultural research centers (IARCs). Part of the secret of the publications program's success is IIRR's innovative and highly cost-effective "writeshop" methodology. This participatory technique ensures that, from the concept stage, publications garner widespread financial support and notice by other organizations and that they speak to development practitioners' and/or rural peoples' real needs. Consequently, when they reach print, IIRR publications are already in demand and promptly "get used," likely with broad-based impacts. For example, publications generated under the MG have been adopted as university textbooks, as NGO/PVO training manuals, as reference works distributed by umbrella NGOs to their institutional membership, and as models for these and other groups to do publications of their own (including GOs and a major development press in Europe). Indeed, the writeshop methodology is so powerful that, once exposed to it, other organizations worldwide quickly take it up and re-apply it themselves, with or without further IIRR assistance. Another part of this program's success stems from a combination of aggressive marketing and altruistic dissemination during the MG. Examples are putting the Institute booklist online, planning for credit-card purchases, and making published IIRR development information and "best practices" available gratis to CD-ROM development libraries and on the Internet. Also contributing to this "shining star's" brilliance is IIRR's unique policy of not copyrighting publications. It permits even greater distribution and impact of IIRR-generated development information via other organizations' translating, copying, or modifying and re-publishing IIRR documents -- often based on disc-copies supplied gratis to IIRR writeshoppers. Likewise, IIRR itself often revises, updates, translates, and re-packages existing publications for new audiences. USAID-Matching Grant Evaluation Report iv Coupled with the Institute's collaborative and participatory approach, this strategy makes IIRR's publications program highly cost-effective. Regional Highlights. Nearly 50% of the 32 publications were generated by IIRR's new Regional Centers, as compared to only 14% during the baseline. Particularly noteworthy was LAC's rocketing from 0 to 9 books. Also during the MG, AFR added a fulltime, permanent Communication Specialist to its staff. Moreover, during the MG several landmark publications by AFR and LAC won these young Centers wider professional recognition and regard within their regions, as well as significant earnings and new clienteles for writeshops and other IIRR services. All these findings attest to the Regional Centers' growing strength and maturity. Issues and Recommendations. Without a doubt, the MG substantially strengthened IIRR's publication program across-the-board and, with it, the AFR and LAC Centers' reach, renown, and sustainability. However, rapid decentralization may have sacrificed some quality- and policy-controls over publications. These need to be formally addressed now, before they become a problem. Other recommendations include the following. IIRR should consider how best to recoup the audiovisual capabilities it lost in an HQ fire, for their continued use in training, service marketing, and public awareness. Conversely, the Institute should not resuscitate its research journal. Poor English-language editing skills at HQ need to be remedied. Likewise for the lack of communication staff and desktop-publishing capability in LAC. Finally, a major issue is the failure to monitor and evaluate the spread and multiplier effects and any attributable impacts of publications. 3. Education and Training IIRR offers a somewhat confusing panoply of course types at its HQ training unit, in the Regional Centers, and within its collaborative field projects. It classifies courses as international, regional, national, customized, community￾level, peer-group, and at HQ/ASI only, study programs. Also, although heretofore unrecognized by IIRR, the Institute provides significant informal training via its numerous writeshops and conferences. Institute-wide Achievements. By all available internal and external evaluation measures, there is little doubt about the high quality and, say trainees, "the practicality" of the training delivered by all IIRR units. The Institute cleaves to all the best principles of interactive, adult education, with USAID-Matching Grant Evaluation Report v added field-based and participatory features. Trainees can hardly praise IIRR training enough. And they and their organizations regularly "come back for more." As with publications, another "proof of the pudding" appears to be other organizations' enthusiastic modifying, copying, and re-using of IIRR training materials and techniques. Nor is the quantity of IIRR training in doubt. IIRR exceeded its implementation plan's target of 100 formal training events by an additional 151. If writeshops and conferences are included, this 251 swells to 280, as compared to the baseline's 203. In total, during the MG 5194 people received some form of IIRR training, as versus the baseline's 4321. Like publications, course content follows programming. However, particularly in AFR and LAC, the MG witnessed a strong trend away from biological/technical/sectoral subjects toward participatory methods and institutional capacity-building skills. The latter included, e.g.: feasibility studies, proposal writing, project planning, logframing, gender analysis, financial management, leadership, organizational development, development communication, participatory M&E, and management of rural development projects/programs, co-ops, or microcredit. An important new initiative is HQ/ASI's plan to offer government- and university-accredited courses and field experiences. Across all IIRR units, training in the form of coursework and writeshops has been a springboard to increased client demand for other IIRR services and, ultimately, to new and more comprehensive inter-agency partnerships for IIRR. Regional Highlights. LAC and especially AFR trended away from supply-led, standardized courses toward more demand-driven, customized and modular offerings. This move represented an increased responsiveness to NGO/PVO clients' expressed training needs and available funds. In both Centers, the numbers of training events and trainees increased slightly (AFR) or massively (LAC) between the baseline and the MG. In contrast, HQ/ASI courses and enrollments dwindled, despite the fact that only HQ/ASI enjoys a well-established Alumni Association. A signal feature of LAC training is its tight integration with LAC publications and field projects. This strategy ensures that region- and language-appropriate training materials are ready at hand, that field learnings are compiled and communicated, and that more IIRR publications are sold! USAID-Matching Grant Evaluation Report vi Issues and Recommendations. It is recommended that IIRR: assess the relative costs and benefits of more formal forms of trainee follow-up in AFR and LAC, such as Alumni Associations; do a finer, internal-comparative analysis of why ASI/HQ enrollments are slipping; re-establish the scholarship fund for women trainees -- especially for AFR, where female enrollment has stagnated; following LAC's lead, better integrate training with publications and field projects wherever feasible; set Institute-wide policies and quality controls on training management, fee collection, etc.; and above all, establish a clear "division of labor" such that HQ and regional courses do not compete with each other. Lastly, a major issue is the lack of M&E for tracking and reporting training outreach and impacts, as versus merely training processes. 4. Field Projects, Technical Assistance, and Partnerships IIRR collaborates in field projects, with primarily two ends in view. One is to access natural social laboratories in which to garner the on-the-ground experience and development information necessary for testing out IIRR products and services, so as to ensure their practicality, robustness, context￾appropriateness, cost-effectiveness, etc. and their credibility to other development organizations. The other is, via links with implementing organizations and their projects, to promote or scale-up the use or dissemination of validated IIRR products, services, methods, etc. Institute-wide Achievements. Evaluation of field projects was limited primarily to MG focus countries. Globally, however, between the baseline and the MG, the numbers of rural people reached via IIRR collaboration in field projects burgeoned as follows: AFR 1000 → 6200, ASI/HQ 7700 → 13245, and LAC 200 → 2344! Taken together, these figures represent a 144% increase in rural beneficiaries. As per IIRR's traditional sectoral strengths, agricultural and environmental projects predominated during the MG. In particular, LAC consolidated and greatly extended its agroecological work in farmer participatory research and farmer-to-farmer extension. However, the MG was distinguished from the baseline period by the addition of women- and health￾oriented projects (mainly in AFR and ASI, respectively) and by the award to both AFR and LAC of new microcredit projects, with an emphasis on women borrowers in the latter region. Besides technical assistance via publications and training, IIRR lent many other kinds of assistance to NGOs/PVOs during the MG: facilitation of special workshops and seminars; needs assessments, gender audits, USAID-Matching Grant Evaluation Report vii participatory and conventional evaluations; strategic planning, logframing, proposal writing; M&E and rural appraisals; some biological/technical/sectoral advising on curricula and project design; and in LAC, survey design. Across the MG, IIRR significantly increased its roster of partners in all arenas except ASI focus countries. AFR and LAC grew their GRO/NGO partners in Ethiopia and Ecuador by approximately eight- and three-fold, respectively. PVO partners nearly quadrupled in Ethiopia and doubled in Ecuador. And the Institute partnered with 107 international research and training institutes -- 350% more than during the baseline. As with other IIRR activities, IIRR's policy of partnering with other organizations in its fieldwork makes Institute projects more cost-effective. Regional Highlights. AFR deserves kudos for successfully piloting an innovative project of peer-counseling in reproductive health among rural Ethiopian women. Another noteworthy achievement was AFR's modification of the Organizational Capacity Assessment Tool to fit GROs and small NGOs. MG field projects planned for ASI focus countries did not materialize, but ASI laid significant groundwork for an innovative environmental health project in Nepal, in collaboration with two NGOs there. In Ecuador, LAC is to be applauded for at once refining and simplifying, and then greatly scaling-up implementation of an IARC-generated model of farmer participatory research. Also laudable is the way LAC interlinks its field projects. For example, farmer￾tested technologies from the farmer participatory research program are fed into the farmer-to-farmer project. Further, findings from all fieldwork are then fed forward into LAC publications and training. Some differential trends in technical assistance between the baseline and the MG were the following. AFR work in this category fell to nearly zero as this Center shifted into a more fully projectized mode of service delivery. Conversely, LAC consciously increased its technical assistance as part of a marketing strategy to garner new clients by first giving them a "taste" of IIRR capacities. As a result, LAC added several higher-order institutions to what was heretofore only an NGO/PVO client list. ASI levels of technical assistance remained roughly the same as during the baseline. Across the MG, all three Centers and HQ built some particularly strong and mature working relationships with what came to be special, "buddy" partners. These spanned umbrella NGOs, PVOs, GOs, private foundations, and IARCs. This kind of buddying reflects a natural complementarity that promises continuing mutual benefits for all concerned, e.g. in rural people's participation, institutional strengthening, sustainability, and program impacts, USAID-Matching Grant Evaluation Report viii depending on different partners' mandates and needs. To take one example, both AFR and HQ/ASI have established a close relationship with the Ford Foundation in their regions. Indeed, in AFR, Ford is as likely to solicit assistance from IIRR as IIRR is to submit unsolicited proposals to Ford. To take another example, throughout southeastern Africa and in Bangladesh and Ecuador, CARE offices now consistently draw on and/or collaborate with IIRR's Regional Centers in a gamut of activities; and CARE routinely directs other organizations to these Centers. At a larger level, such regionally forged relationships open the door to mutually profitable global ones, as between the Institute and CARE International. Issues and Recommendations. IIRR should continue and reinforce its policy of doing field projects collaboratively; and all such projects should be administered through the Regional Centers rather than by HQ units. IIRR needs to do more by way of Institute-wide and hence more cost-effective outreach to and nurturing of its partners. Currently, HQ services its own and ASI's needs in these regards, but gives scant support to AFR and LAC. In like vein, IIRR units and centers need to look to one another as premiere partners in all Institute endeavors. At present, there is a not-unwarranted perception of competition rather than collaboration, especially in all three Centers' view of HQ. Thus, fair-minded structures and internal incentives to promote intra￾Institute collaboration are wanted. More integrative efforts make for regional cross-fertilization of ideas, techniques, tools, etc., and they produce more generalizable development lessons. They can also bring in larger, institutional￾level grants. Finally, IIRR needs an M&E system that can capture and synthesize project and technical-assistance impacts Institute-wide. Without such a system, IIRR cannot report its institutional achievements additively. Nor can it analyze its comparative advantages by region, sector, partner, etc. so as to make informed management decisions about whether to engage in a given field project or technical assistance activity. 5. Institute and Program Management Across 1999, as per the MG implementation plan, IIRR undertook an intensive, Institute-wide strategic planning process, following up on decisions taken and directions set in a similar exercise in 1997. The longterm goal of these exercises -- and, indeed, of IIRR's entire MG proposal -- was to "Fully shift IIRR's center of gravity from HQ to regional and country-based support initiatives, in order to bring the Institute closer to the rural people it serves." Consequently, the strategic planning process touched upon every aspect of USAID-Matching Grant Evaluation Report ix Institute structure and functioning. Institute-wide Achievements. A major achievement of 1997's and, even more so, 1999's strategic planning was the rationalization of Institute structures such that by 2000, superfluous HQ units will be eliminated while others (i.e., publications, education and training) will be bundled together in a "central center" of their own. Relatedly, ASI is to be vouchsafed the same status as AFR and LAC. The center of gravity for programmatic and managerial decisioning can then fully and appropriately shift from HQ to the regions. This is to be facilitated by a new, integrated Management Team, tentatively consisting of the IIRR President, her No. 2, and the three Regional Directors, with Heads for finance, personnel, and M&E serving the team in an advisory role. In this process, HQ administration is to be streamlined. If implemented, these plans should successfully conclude the restructuring and decentralization the Institute began under this and the previous MG. Between the two strategic planning efforts, IIRR made some important gains in staffing flexibility while keeping gender ratios virtually equal. It also increased national diversity among staff, as befits a transnational PVO. Highlights of staff development activities included an innovative Trustee-led mentor program, much-needed staff training in proposal and budget preparation, and 3 staffers' MA and PhD studies. All these staff development opportunities were confined to HQ, however. The MG supported several advances in administrative and financial management, e.g.: elaboration of an Institute-wide Program Procedures Manual; consolidation of all major financial functions at HQ; installation of a new financial information system at HQ; and a renewed policy of transparency in all fiscal matters. The result has been accelerated financial reporting, improved financial controls, better annual budget-building, and increased potential for integrated financial+programmatic analysis and forward-planning at all levels and in all units. IIRR took giant steps in governance throughout the MG. New members with solid development credentials, PVO management backgrounds, financial savvy, and years-long Africa experience joined the Board of Trustees under the leadership of a dynamic new Chair. Moreover, Trustees contributed generously of their money, time, and professional skills far above and beyond the call of duty. In Jan 1999, one of the Board's most internationally distinguished Trustees agreed to fill IIRR's vacant presidency. Finally, in consultation with the evaluator during 1999's strategic planning, the Board created a 6-point action plan for proactively streamlining Board costs, formalizing Trustee tenure, USAID-Matching Grant Evaluation Report x putting Trustees in closer touch with the Regional Centers, and more. Regional Highlights. Thanks in large part to MG salary supports, AFR and LAC staff were beefed up in critical areas. AFR was able to add much￾needed permanent positions in training and communications (1 each) to its core staff. Another 3 to 4 new technical positions and various support staff allowed AFR to consolidate its nascent Ethiopia Country Office and successfully undertake major activities there. LAC increased its technical staff from 2 (the Director and one other person) to 3.5. Even this tiny increment allowed programs in Ecuador to take off in a big way. At the same time, HQ/ASI and especially AFR made wise, cost-effective, and sometimes innovative use of volunteers, interns, and secondees; LAC did likewise with part-time hirees. AFR eagerly and expertly adopted the new financial system. And LAC devised some astute self-financing and financial management mechanisms. Issues and Recommendations. Despite the foregoing advances, AFR and especially LAC were starved of key technical staff, and ASI remained overly dependent upon HQ. Meanwhile, HQ became bloated with all sorts of personnel who, despite their numbers, rendered few services to regions other than ASI. At the time of evaluation, the HQ-to-regional staff ratio stood at 2:1. Further, HQ appears to hire and/or retain staff with skill levels and management styles that are inadequate to the demands of a modern, transnational PVO. For instance, during the MG, IIRR experienced serious skill lacks in top-level financial, personnel, and program management. Despite considerable staff development (at least at HQ), the Institute also suffered from acute shortages in a number of key operational skills, e.g.: professional-level proposal￾preparation, budgeting, and business acumen; basic financial analysis; writing and multilingual communication; and M&E. To survive in today's highly competitive PVO environment, IIRR must quickly find the "political will" to fire and hire wisely and to invest in staff development judiciously so as to correct these shortcomings. To this end, the Institute needs to professionalize its recordkeeping, management, and search procedures for personnel. Among other things, immediate attention must be paid to: completing or updating personnel files; revising job descriptions to reflect decentralization needs; standardizing performance reviews; setting rules for fair and well-reasoned deployment of staff-development opportunities; not bidding or contracting-out >100% of Institute personpower or, relatedly, expecting employees to work excessive night and weekend hours; incorporating writing and language requirements into all job announcements and hiring decisions; and contracting out some of the low-level physical-plant services at HQ. Professionalization is also sorely USAID-Matching Grant Evaluation Report xi needed in financial management. To wit, IIRR requires a chief financial officer rather than a comptroller; and it must put in place an internal auditor. A related issue is the ability to interlink finance with strategic planning and program implementation. The MG proposal logically allocated 25% of BHR/PVC funding to each of the three RCs and HQ+NY. But analysis of IIRR activity charge-code data at the MG's end revealed that as much as 50% of BHR/PVC's $1.5 million match may have been consumed by HQ+NY operations. While AFR managed to garner 20%, ASI and LAC were each allotted only 12% of MG monies. Moreover, MG monies represented virtually the only funds to flow from HQ to the regions throughout the MG period. It is therefore little surprise that ASI met few of its MG goals. And, how much greater might LAC's already impressive MG achievements have been had this Center not been deprived of desperately-needed staff, information technology, professional development, and even office space? As with staffing, such financial imbalances must be corrected if IIRR is serious about becoming a transnational PVO with strong Regional Centers that can function effectively "close to the people." In fact, HQ miserliness has left all three Regional Centers in what is ultimately a non-sustainable position at MG's end. Simply put, AFR and LAC do not have the critical mass of senior staff strength, and in LAC's case the English-language abilities, to long survive in the absence of their PhD-level Directors. All other permanent staff are at best national-level BAs. The solution is creation of a strong No. 2 position in each Regional Center, filled by a mid-level staffer. These individuals should be competent to take charge of most matters during Directors' travel, illness, or resignation; routinely assume a portion of the daunting administrative burdens of these Centers' burgeoning success; train independently; and so forth. These No. 2's would leave Directors free to focus more on what only they can do, e.g., negotiate and sign new contracts, fundraise, represent the Center at high-level international meetings, attend and host Management Team meetings, and so forth. ASI's case is different but equally untenable: it lacks a PhD-level Director. In part, this explains why ASI was colonized by HQ units. Obviously, the top-most financial and program management systems that gave rise to such a shaky situation must be re-vamped. As noted earlier, also at issue are policy and quality controls by competent and even-handed HQ Heads for personnel, finance, publications, and training. Besides re-staffing as needed in order to meet the demands of decentralization and to bring IIRR into the modern world of PVO management, IIRR needs participatorially to write a "charter" clearly stipulating the relative roles, rights, and responsibilities of HQ USAID-Matching Grant Evaluation Report xii versus regional management. With such a charter, the newly proposed Management Team should have little trouble enforcing it, albeit perhaps with some targeted cross-training in certain subjects for different Regional Directors. 6. Sustainability and M&E Two elements in particular are key to the Institute's future: financial sustainability; and systematic, Institute-wide M&E that builds and informs the finance-to-program links upon which sound strategic planning and management decisioning for longterm sustainability must be based. Institute-wide Achievements. In addition to intensifying its strategy of cost-effective partnering, as part of its financial sustainability drive, IIRR pushed to diversify its funding sources throughout the MG. The Institute made particular progress in this regard among NGO and PVO sources of both grant and earned income. The number of private-sector donors also increased slightly. In the process, the Institute reduced its dependence on USAID monies from 23% to only 13%. An explicit MG objective was for IIRR as a whole and the Regional Centers individually to derive 50% of revenue from earned (as versus grant) income by MG's end. Both the Institute and AFR reached this 50% target. For LAC, this figure was 30.5%. Desegregate financial data for ASI were not available. Unfortunately, the "big picture" for IIRR was one of shrinking income overall. Across the MG, total revenue dropped by $4 million from the baseline's $15.2 million. IIRR therefore took Herculean steps to strengthen its fundraising and public awareness capacities. For example, (mainly only HQ) staff were trained staff in fundraising, proposal writing, budgeting, marketing, and PVO financial management generally. Project budget formats were revised. Consultants were contracted to write business, financial, and public-awareness plans and to produce public-relations materials. A number of promotional events in the US and elsewhere were held. A Public Awareness Officer position at HQ was created and filled at HQ. A SWOT analysis of Institute revenues was conducted. Computerized systems for tracking fundraising efforts and outcomes were improved. And in 1999, an 8-point action plan for the Institute was written. Meanwhile, HQ stepped up its marketing of IIRR publications and of HQ/ASI (but not AFR and LAC) courses using conventional media plus increasingly sophisticated information technology. A signal achievement during USAID-Matching Grant Evaluation Report xiii the MG was the creation of an informative and professional-quality IIRR website for these and other purposes. Of course, the ultimate key to fundraising success and thus PVO sustainability is tangible, measurable, and cost-effective development impacts, to show would-be donors and clients that their money will be well-spent. For this, an integrated Institute-wide system of M&E is required. MG achievements on this front centered on bringing IIRR into the modern information age: improving phone and internet connectivity, and thus HQ-to-region communication; hiring the Institute's first Computer Systems Specialist; upgrading computer hardware and software at HQ/ASI and AFR (but not LAC); building and testing a computer networking system at HQ; translating eroding DOS-based data into Windows-compatible files; fashioning systems to interface these with financial information; enriching data entry and analysis potentials via adoption of a relational database; creating an archive of M&E￾related hard-copy documents; and writing guidelines, policies, and procedures manuals for all the above. At the time of evaluation, however, actual program data available for the new information technology to operate on were largely limited to descriptive and process information on IIRR training and extension. Gross data on publications, field projects, technical assistance, and partners/donors also existed, albeit in a scattered and mainly hard-copy form. In the course of the evaluation or shortly thereafter, all such data were up-dated at the Regional Centers or other points of origin; entered into the HQ relational database where possible or, alternatively, manually tabulated; and then compiled by the HQ data analyst according to evaluator suggestions, for use in the present report. With evaluator guidance, some anecdotal impact data were also gathered and reported. Of course, hardware, software, raw process data, and spur-of-the￾moment analysis hardly constitute an M&E system. During the MG, the HQ manager responsible for M&E oversight generated an important "think piece" and a number of insightful memoranda plus in-house discussions on how to design and utilize a comprehensive M&E system. These efforts included attention to almost all necessary components of such a system, e.g.: definition of impact as well as process indicators; integration of data on personnel, physical plant operations, and of course finance; complementary evaluation studies; individual staff M&E assignments and related hiring or skill-upgrading needs; Institute-wide policies and procedures for collection and submission of all types of M&E data; reporting schedules and formats, such that M&E data USAID-Matching Grant Evaluation Report xiv are available in meaningful and timely "packages" for their effective use by managers and donors; and funding for M&E. However, action on most of these fronts stalled. Regional Highlights. Both AFR and LAC intensified efforts to advertise their income-earning products and services during the MG. As noted above, AFR met its 50% target for earned income, while LAC reached near-self￾sufficiency in funding. Significantly, LAC pioneered a self-funding/-sustaining endowment mechanism that turned what was initially only a project into a lasting LAC program. Also, it established a self-sustaining seed fund for its publications. And it successfully experimented with flat-task-rate and product￾deliverable contracts. The latter strategy had several advantages. It provided a source of non-earmarked income and permitted some pricing flexibility such that LAC could accommodate "poorer" clients. It did not tax GROs or unsophisticated NGOs with accounting headaches beyond their management abilities. And it made LAC's administration of very small grants and contracts more cost-effective. Meanwhile, AFR made major strides in information technology and M&E. It gained good phone connectivity plus modern computer hardware and software on a par with HQ's; and it established its own website. AFR staff became exceptionally well-trained and adept at the meaningful use of these new capabilities for manipulating both program and financial data and sharing information generally. Furthermore, early in the MG, AFR won agreement from the German Development Services to second an M&E Officer (a German national) to the Center for up to 6 years. As a result, AFR produced the first in what it envisions as a series of M&E training manuals, workbooks, and tools for NGOs. AFR also won a number of grants and contracts to provide training and technical assistance in M&E to regional NGOs and PVO chapters, especially in the realm of organizational capacity-building. Issues and Recommendations. In addition to the many and excellent Institute-wide plans for sustainability that were developed during 1999's strategic planning, Center-specific business plans need to be devised. This task, which was slated to be done under the MG, now needs to move forward. Particular attention must be paid to budgeting for HQ support to the Regional Centers in terms of: first, properly staffing, equipping, training, and housing LAC; second and as already noted, strengthening regional leadership structures; third, widely publicizing AFR's and LAC's existence and aggressively marketing their publications and services; fourth, capturing larger, longer-term Institute-wide grants (like the MG) in addition to the smaller, shorter-term grants the Centers win on their own; and fifth, fairly sharing out grant monies USAID-Matching Grant Evaluation Report xv (including indirect costs) across all three regions. Although both AFR and LAC did an outstanding job of designing and expanding their delivery of high-quality, client-sensitive development programming during the MG, it is doubtful they can maintain this impetus or consolidate their MG gains with the minimal home￾office backing they have so far received. Yet within IIRR, AFR and/or LAC appear to be producing some of the more cutting-edge financial-management ideas, forging brand-new partnerships and donor linkages (and hence funding possibilities), generating some of the most dramatic development results, and leading the way in M&E. These are precisely the sorts of outcomes IIRR needs to analyze and reinforce internally and to document and showcase externally in order to regain its former revenue status and ensure Institute sustainability. Unfortunately, IIRR did not follow through on its MG plans to build an M&E system to detect and document such organizational outcomes and development impacts. Indeed, a critical HQ position for an M&E Specialist that was budgeted for in the MG proposal was never funded, advertised, or filled. This situation was paralleled in Regional Centers' lack of counterpart M&E personnel, except for AFR's secondee. Thus little of HQ's solid conceptual work on an M&E system was ever implemented. Moreover, HQ inaction on an Institute-wide framework blocked further progress AFR might otherwise have made in M&E. Overall, despite considerable advances in information technology, at the time of evaluation, only process data were systematically collected. But there was no established framework of indicators (with their accompanying baselines, targets, and timelines), stipulated methods of analysis, or regular schedule of results reporting for these data. Moreover, as of Oct 1999, computerized entry of even process data for AFR and LAC training had fallen 12 months behind at HQ. In sum, at present IIRR has little, if any, impact data to use as ammunition in its fundraising sallies. Yet impressionistic evaluation findings (imposed by the lack of indicators and impact assessment) suggest that the Institute has produced many and important impacts across all its programmatic and thematic thrusts and in almost all non-administrative units. But in order credibly to demonstrate this to donors and responsibly to share any resulting lessons with the development community at large, a comprehensive M&E system is imperative. Absent such a system so far, for immediate sustainability purposes IIRR urgently needs to recuperate as much impact information as possible on its work in recent years, before institutional memory is lost. USAID-Matching Grant Evaluation Report xvi 7. Summary of Progress Toward Objectives With regard to MG Objective I, IIRR strengthened the capacity of a greatly increased number of NGOs and other localized partners to implement development activities. Strengthening was achieved via: IIRR publications -- whose numbers, local-language versions, subject matters, and dissemination modes all increased immensely during the MG; IIRR delivery of many kinds of high-quality training, follow-up, and technical assistance, including writeshops and conferences -- all of which increased significantly during the MG; IIRR collaboration with NGOs and other partners in field projects -- the rural beneficiaries of which also increased; and IIRR strengthening of its own Regional Centers, the better to serve NGOs and other localized partners. This last item constitutes Objective II, which zeroed in on Centers' ability to deliver and sustain services to partners, undertake process documentation, promote inter-institutional learning and networking, and do impact evaluation. Thanks almost solely to MG inputs, IIRR made tremendous progress toward strengthening its AFR and LAC Centers on all these fronts but the last. In addition to greatly increasing their publications and other best-practices documentation, AFR and LAC grew their partners manyfold and, with this, their inter-institutional learning and networking. AFR and LAC also made substantial strides toward self-sustainability. Progress by the ASI Center is difficult to determine due to the conflation of ASI activities, budget, and management with HQ's. Sustainability at all three Centers is still a question-mark in terms of staff skills and personpower, notably for regional leadership structures. On the implicit Objective III of building IIRR capacities as a whole, significant progress was made in virtually all programmatic realms as well as in global partnerships, financial information and reporting systems, governance, and information technology. However, HQ experienced considerable difficulties in managing finances in relation to the Institute's overarching programmatic goals and also in decentralizing decision-making authority from HQ to the regions. All Regional Centers plus HQ fell down on the job of doing impact assessment and, indeed, M&E of other than training process data. Hence the impossibility of providing more definitive or quantitative evaluation here of IIRR's progress toward its MG objectives, or a more comprehensive evaluation of its development programs. USAID-Matching Grant Evaluation Report 1 1. INTRODUCTION 1.1. The Institute The International Institute of Rural Reconstruction (IIRR) is rooted in the Mass Education Movement founded by Dr. Y. C. James Yen in China in the early 1920s. Based on the values, development framework, and outreach approaches he developed there, in 1952 Yen helped establish the forerunner to IIRR -- the Philippines National Rural Reconstruction Movement (NRRM) -- on a 50 ha campus in Silang, Cavite, the Philippines. Named after its founder, this "Yen Center" campus today houses IIRR's global headquarters (HQ). IIRR was formally constituted in 1960, when it incorporated as a US PVO and opened a supporting office in New York, NY. Staffed only by a director, an administrative assistant, and a part-time bookkeeper (as of 1999), the NY Office essentially serves as an offsite extension of the HQ administration unit. It fulfills various legal, fundraising, public-relations, and international logistic and banking functions. NY also services IIRR's Board of Trustees (BOT) while generally backstopping the HQ in proposal writing, grants management, networking, and partnership development (IIRR/NY 1999). For the past 8 years, IIRR's average annual revenue has hovered around $3 million. IIRR's mission has never wavered from Yen's original concept. As per a 1994 formulation of the Institute's mission that still holds good today, IIRR aims "To improve the quality of lives of the rural poor in developing countries through rural reconstruction: a sustainable, integrated and people-centered development strategy generated through practical field experiences" (IIRR/HQ Dec 1996:B10). Until a decade ago, Institute activities were almost exclusively restricted to Asia, as directed from the Philippines HQ. However, thanks to substantial support from USAID's Bureau for Humanitarian Response/Office of Private and Voluntary Cooperation (BHR/PVC), IIRR has since become a truly transnational PVO. It boasts vigorous regional centers in Africa (based out of Nairobi, Kenya) and Latin America (based out of Quito, Ecuador) in addition to an Asia Regional Center (based out of the HQ campus, in the Philippines). Each of these centers operates internationally within its respective region. Under this new, decentralized structure, each center is envisioned as taking the lead in all programmatic decisioning and operations, with the Philippines HQ ideally being limited mainly to administration and some basic support roles in education/training, publications/communication, and technical advice. Worldwide, IIRR today acts primarily as an intermediary PVO -- an "NGO for NGOs," it says -- to strengthen the capacity of other rural (and now also periurban) USAID-Matching Grant Evaluation Report 2 development organizations via information dissemination, training, and technical assistance (TA). These efforts build on two broad activities: • IIRR's gathering and packaging of development information generated worldwide by its own and other development organizations' experiences; and • "action research" in field-based projects and "social laboratories" -- i.e. longterm field-activity sites established by IIRR, mainly in the Philippines -- whose aim is to generate and test development information, ethnoscientific (local/indigenous) and scientific technologies, participatory methods, training approaches, and so forth. IIRR does not see itself as a major leader in largescale field-project implementation but rather as an information provider, trainer, and facilitator for other organizations whose primary focus is implementation. As of Dec 1999, HQ is organized into three operational units. These units are not structurally duplicated in the three regional centers; they remain as-yet undifferentiated in internal structure. Institute-wide, however, all programmatic work is grouped into three thrusts, with two cross-cutting themes. Table 1.1. summarizes the foregoing, working programmatic structures. [For a current organigram by operational + administrative + geographic units, see Chapter 5.] Table 1.1. IIRR Functional Units and Program Thrusts In final production of the following table, please make double or extra-dark lines between each of the 3 columns to indicate that they are not inter-related along the horizontal axis. Regional Centers HQ Functional Units Institute-wide Program Thrusts Africa (AFR) Publications and Communication Environment, Natural Resource Management (NRM), and Agriculture Asia (ASI) Education and Training Institutional Capacity Building Latin America (LAC) Technical Support Group Community Health, Reproductive Health, and Nutrition --- Administration Cross-cutting: Local Knowledge, Gender Source: IIRR/HQ May 1998. Briefly, IIRR programming is directed mainly to development groups that operate at or near the field level or that directly or indirectly support such groups. These include not only grassroots organizations (GROs) and national or regional USAID-Matching Grant Evaluation Report 3 NGOs but also the gamut of local- to national-level government agencies with some development mandate and, increasingly, the national offices or country chapters of PVOs. This is in addition to IIRR's historical "heritage" partners, i.e. the NRRMs established in strategic countries throughout the world. IIRR also collaborates with a wide variety of international research and training institutes in activities of mutual interest, often assisting such organizations to translate their learning into practical and participatory field-level approaches and publications. 1.2. The Matching Grant IIRR has enjoyed a long and productive relationship with BHR/PVC. In fact, BHR/PVC has been the lead supporter of the Institute's globalization and decentralization efforts, as per various IIRR strategic plans (e.g., IIRR/HQ 1993). As Figure 1.1. (next page) indicates, IIRR won a substantial 1990-1995 BHR/PVC matching grant (MG) of $3.5 million in order to operationalize these plans. Along with many other positive outcomes, a final, external evaluation reported "Impressive achievements in internationalization of IIRR programs" as a result of the 1990-1995 MG (O'Brien 1994:25). The evaluation concluded that: "...unmistakably...IIRR has been institutionally strengthened most significantly in the course of the past four years. The AID Cooperative Agreement has had a major influence and impact in the achievement of this institutional strengthening. IIRR has been extremely successful in the planning and implementation of its 'Internationalization' strategy" (O'Brien 1994:26). The general transcourse of this strategy is traced in Table 1.2's milestones. Table 1.2. Milestones in IIRR Decentralization HQ 1989-1990 With BHR/PVC support, conceptual creation of AFR, ASI, and LAC Regional Offices at the Philippines HQ, under the administration of the Department of International Training and Outreach. 1994 Based on a strategic planning (SP) exercise, administrative re￾assignment of the regional offices to the Office of the Vice President for Program [hereafter, just VP], with theoretically equal administrative standing for each region. Africa 1989 Hire of Dr. I. Bekalo (an Ethiopian) as AFR Regional Director. 1990-1993 Extensive travel and site-surveys in Africa by the Director (40% of each year). 1993 Out-posting of the Director from the Philippines to Kenya. 1994 Establishment of the Regional Office in Nairobi, Kenya under a space￾sharing arrangement with another PVO country-office. 1995 Designation of the Regional Office as a Regional Center, and rental of separate premises. 1 Jan 1984 31 Dec 1986 1 July 1990 30 June 1995 1 Oct 1996 30 Sept 1999 6 Dec 1980 5 Dec 1983 1 July 1987 30 June 1989 1 July 1995 31 Dec 1995 Title “International Sharing in Rural Reconstruction” (BHR/PVC) “International Outreach in Rural Reconstruction” (BHR/PVC) Other USAID (not BHR/PVC) “Strategic Options for International Technology Transfer and Institutional Development” (BHR/PVC) Bridging Grant to “Strategic Options….” “A Transregional Program to Build Capacity for Sustained Development Impact” (BHR/PVC) USAID Amount $1,410,000 $2,300,000 $1,680,000 $3,200,000 $300,000 $1,500,000 Project No. AID/PDC-0151-G-00 1020-08 No.938-0150 AID/PDC-0248-G￾SS-4071-00 AID/W-OTR-0286- A-00-7132-00 AID/OTR-0158-A- 00-0084-00 Same as previous MG FAO-0158A-00- 6050 Figure 1.1. BHR/PVC Support to IIRR USAID-Matching Grant Evaluation Report 5 Africa 1995 Establishment of a Country Office in Addis Ababa, Ethiopia. 1997 Legal registration of the Ethiopia Country Office. Asia 1989 Appointment of Mr. D. Imperial (a Filipino) as ASI Regional Director. 1992 "Inauguration" (IIRR/HQ Dec 1996:E44) of the ASI Regional Office at HQ. 1996 Addition of a second ASI staff member, Ms. L. dela Vega, as Program Officer. 1997 Establishment of a Nepal Country Office in Kathmandu, to coordinate activities in South Asia. With restructuring, transfer of some staff from other HQ units to the ASI Office at HQ. 1998 Designation of the Regional Office as a Regional Center, and physical re-location and amalgamation of its HQ-based staff in a single building on campus. Also, reorganization of all ASI program activities as more administratively independent, akin to AFR and LAC. 1998 Halt in full-scale implementation of the Nepal Country Office, due to legal registration difficulties. 1999 Retirement of the Director and appointment of the Program Officer, Ms. L. dela Vega, as Acting Director. 1999 Formal country agreement with IIRR approved by the Nepali Parliament (but still awaiting IIRR signing). Latin America 1990 Appointment of Mr. S. Killough as interim, 30%-time LAC Regional Director. Development of the first regional strategy for LAC. 1993 Hire of Dr. D. Selener as fulltime LAC Regional Director, posted at HQ for his first 9 months in order to become familiar with IIRR philosophy, programs, etc. 1994 Out-posting of the Director from the Philippines to Quito, Ecuador and establishment there of the LAC Regional Office. 1995 Designation of the Regional Office as a Regional Center, based on hire of first technical staff. Source: IIRR documents and evaluator interview data. Although the evaluator found room for improvement along several fronts, he made the following, top evaluation recommendation. The highest priority should be given to extending the AID-IIRR Partnership for another 3-5 year period. Much has been accomplished in a relatively brief time span. Much remains to be done. ...Deadlines of three and five years are worthwhile planning devices, but must be recognized as artificial, at best, when it comes to evaluating results and judging if the endeavor warrants longer-term support... This USAID-Matching Grant Evaluation Report 6 "Partnership" definitely warrants continued support...[to]...guarantee...the 'payoff' of hoped-for results (O'Brien 1994:28). Thus, in 1996 IIRR proposed to BHR/PVC what was to be a final "exit" MG of 3 years' duration [Figure 1.1.] to complete and consolidate its globalization and related efforts (IIRR/HQ Dec 1995). This MG was awarded in the amount of $1.5 million, with an IIRR match of $2,327,228 (Marshall 1996). It forms the subject of the present, final evaluation report. The proposal for this new MG emphasized sustainable improvements in income, health, and food security for the poorest of the poor in selected "focus" countries: Ethiopia, Bangladesh and Nepal, and Ecuador (AMATech 1999:2). Entitled A Transregional Program to Build NGO Capacity for Sustained Development Impact, the MG was envisioned to support two main Institute-strengthening aims. • Fully shift IIRR's center of gravity from HQ to regional and country-based support initiatives, in order to bring the Institute closer to the rural people it serves. • Make IIRR's regional centers self-sustaining. These aims built upon the well-known and longstanding people-centered dimensions of IIRR's NGO capacity-building, including IIRR's strong base in action￾learning processes, its pragmatic fusion of practice-based with science-based knowledge, and its reliance on proven tools and methodologies for democratizing the development process through participatory approaches. The underlying strategic rationale for the MG program centered on the role that civil society and non-state public organizations will play in the 21st century given the limitations of state institutions. This rationale was translated into two explicit, and one implicit, MG purposes. Purpose I: To strengthen the capacity of IIRR's NGO partners to implement participatory, people-centered local development programs; and Purpose II: To increase the capacity and the sustainability of IIRR's regional offices (later, "centers") to: (a) deliver and sustain services to NGOs, (b) undertake process documentation and impact evaluation, (c) promote inter-institutional learning and networking. Purpose III: To build capacities of the Institute as a whole, the better to achieve (I) and (II) above. IIRR took a four-pronged approach to Purpose I. USAID-Matching Grant Evaluation Report 7 • In collaboration with leading research and training institutions, organize regional and transnational training courses and workshops on critical development issues for senior PVO/NGO managers from focus and other countries in each region. • Provide training and TA to groups of 10 to 20 partner PVOs/NGOs in each focus countries, to upgrade their capacity in participatory and sustainable rural development. • Implement collaborative field programs in each focus country, to demonstrate successful rural development strategies, and to ensure and document people￾level impact. • Develop and share participatory evaluation systems so that partner PVOs/NGOs may know, and demonstrate, their impact. In furtherance of these activities and of IIRR's own capacity-building under Purpose II (and implicitly, Purpose III), the MG also sought to achieve the following, concrete outputs. • Build the capacity of IIRR's regional offices via modest expansion of their technical staff and support from HQ technical staff. • Reinforce regional operations in AFR and LAC by stationing program support staff in Ethiopia and Honduras. • Open a subregional ASI office in Nepal. • Establish regional training facilities in each region, in partnership with PVOs/NGOs there. • Manage IIRR's regional activities so that they will become self-sustaining based on the income they earn from clients for IIRR training, publications, TA, and other services. Overall, the MG program as stated in the proposal was and is consistent with IIRR's mission and strategic goals and with BHR/PVC and USAID's New Partnership Initiative. Also, the proposal incorporated appropriate attention to all the previous MG's final evaluation recommendations. The program promised good performance in building longterm partnerships between IIRR and other development organizations (especially local and regional ones) to strengthen the latters' institutional capacity to plan, implement, and measure sustainable development. 1.3. The USAID Evaluation The independent external evaluation reported here was conducted by a senior scientist with extensive monitoring and evaluation (M&E) experience, a longstanding and firsthand knowledge of BHR/PVC's MG program, and some prior familiarity with IIRR. However, the evaluator was greatly assisted in the evaluation by the data￾gathering and -compilation efforts of IIRR staffers. USAID-Matching Grant Evaluation Report 8 Evaluation preparations, travel, data analysis, and report drafting took place between the first week of Oct 1999 and the end of Dec 1999. As noted in Annex A's Scope of Work (SOW), the overarching evaluation mandate was not only to gauge IIRR progress on its MG objectives but also to "provide an overview and specific recommendations that will feed into IIRR's program priorities and strategic planning process" for the next 3 years (AMATech 1999:3). Thus, the task amounted to far more than the usual end-of-project evaluation. At every turn, the evaluator was asked to contribute substantively to IIRR's ongoing SP. This touched on all aspects of institutional functioning: from the Institute's stated vision-mission-values to their realization in reality versus rhetoric, as reflected in program and budget; and from governance and management all the way down through personnel, physical-plant and support services, and equipment. 1.3.1. Tasks and Timeline The evaluation took place in 10 steps. 1. Start-up activities in the Washington DC area for both evaluations: preliminary meetings and innumerable coordinating phone calls and e-mails, SOW refinement, document collection and review, and structuring of reporting format and evaluation methods (1-8 Oct). 2. Travel to HQ for a kick-off meeting with top IIRR global staff, followed by work with HQ staff to begin gathering and compiling the necessary quantitative evaluation data; also, interviews to evaluate ASI programs (9-15 Oct). 3. Travel to LAC (Ecuador) to evaluate the program there (15-23 Oct). 4. Turn-around in the Washington DC area, and phone meetings there with the IIRR Director for Programs and the BHR/PVC MG Project Manager (24-25 Oct). 5. Travel to AFR (Kenya) to evaluate its programs (26 Oct-3 Nov). 6. Return travel to HQ to complete data collection, observe and contribute to SP, and give initial evaluation feedback (4-18 Nov). 7. Return to Washington DC area for data analysis, assisted during part of this time by an IIRR staffer (19 Nov-11 Dec). 8. Drafting of evaluation report (12 Dec 1999-7 Jan 2000), in tandem with ongoing IIRR efforts to locate and compile missing data. 9. Submission of draft final report to AMATech. 10. Report revisions, preparation of executive summary, writing of an additional annex on methods at USAID request, and submission of the final draft for professional production at IIRR (intermittent in Jan 2000). 1.3.2. Methods and Subjects A major start-up task was to touch base with as many HQ staff as possible in order to garner firsthand, up-to-date information on the MG program and the Institute, beyond just documentary data. In these initial, unstructured and open-ended USAID-Matching Grant Evaluation Report 9 discussions, interviewees also shared their thoughts on evaluation priorities and hoped-for feedback. A kick-off meeting with top IIRR managers at HQ clarified the evaluation approach and methodologies, and the complex corresponding logistic and scheduling arrangements. At that time, as per the SOW the evaluator emphasized the importance of "providing evidence, criteria for judgment and citing data sources" for "material that is quantifiable and measurable as BHR/PVC is held accountable for results from the programs it funds" (AMATech 1999:3,2). In response to this mandate, the following evaluation methods were employed. Unless otherwise noted, all methods were applied across all IIRR units and regions. • Visual inspection of Institute offices and equipment, vehicles, guesthouses, cafeterias, and other facilities. • Critical review of some 150 formal Institute reports, publications, PR materials, videotapes, website content, etc. [See Annex F's bibliography.] • In selected cases, review of such materials for client organizations of IIRR. [Only a few are listed in the bibliography, however.] • At HQ, review of BOT communiqués, key management memoranda, and the vast number of interim strategic-planning (SP) documents and memoranda generated throughout 1999. • Direct examination of personnel and financial records and reporting systems. • Critical review of samples of raw program data such as: course curricula and evaluations; trainee action plans; inventories of field projects and countries served, etc.; TA and other contracts between IIRR and partners; and so forth. • In lieu of a functioning M&E system, evaluator design of data tables, graphs, and figures on numerous variables, for IIRR staff to generate in order to provide the necessary before-and-after data for analysis of progress on MG objectives. • Relatedly, testing/observation and assessment of the Institute's existing data￾collection and -handling equipment, methods, and computerized+hard-copy systems for gathering, storing, manipulating, and reporting program process and (theoretically) impact data at various levels of aggregation. • In the course of the two foregoing tasks, firsthand observation and assessment of staffers' professional/technical competencies. • Group interviews with: all available BOT members; organized subgroups of IIRR staff; and staff of supporting organizations like the Asian Institute of Management (AIM). • One-on-one interviews with key BOT members, IIRR staff, and senior managers of client/partner organizations. • Direct observation of interactions between IIRR staff and Institute partners and between HQ and regional staff. • In AFR and LAC, review of e-mail traffic between HQ and the regions, as a check on HQ-to-region coordination and communication. • In ASI, a group interview of all trainees attending an ongoing course. USAID-Matching Grant Evaluation Report 10 • In LAC, field-site visits. Note: In AFR all fieldwork is currently located in Ethiopia, but the evaluator's visit was limited to Kenya; and in ASI, currently there are no field projects in the focus countries while field sites in the Philippines were not pertinent to the MG. • Focus Group (FG) interviews with IIRR partners and stakeholders by region. This last method (cf. Krueger 1994) was employed as an impartial way of assessing strategic partnerships between IIRR and GROs, PVOs/NGOs, and local￾level government organizations (GOs) across regions, as per SOW Item IV.A.3. Along with a brief discussion of the FG methodology, Annex B displays details of FG composition and findings, plus the FG interview guide and an illustrative FG schedule. Readers are urged to refer to Annex B throughout this report as the FG findings constitute a particularly reliable and candid dataset, used to inform all subsequent analysis. A second methodological thrust was to quantify and compare the targets set in the MG's Detailed Implementation Plan or DIP (IIRR/HQ Spring 1997) with the results actually achieved across the MG. This exercise produced Annex C's DIP Schedule of Activities. Its data are drawn upon throughout this report in assessing IIRR progress toward MG objectives. The DIP was chosen over the MG logframe as the basis for a more operationally thorough evaluation. In addition to DIP data, for selected activities the evaluator asked IIRR to provide aggregate datasets arrayed according to a baseline period (1992-1995) matched in length to the MG period of 1996-1999. By smoothing out inter-annual bumps and dips, these arrays permit more confident analysis of IIRR progress and its attribution to MG support. Especially for overall institutional strengthening, they also facilitate appreciation of trends, which are particularly useful inputs into SP. More specific information on methods, data sources, evidence adduced, and criteria for judging each evaluation topic is provided in the relevant chapters. For convenience, however, Table 1.3. summarizes the numbers and types of evaluation interviewees. For greater interviewee detail, see Annex B's table of FG member types (who must otherwise remain anonymous) and Annex's E's list of other persons contacted. USAID-Matching Grant Evaluation Report 11 Table 1.3. Evaluation Interviewees Types of Interviewees Female Male Total BHR/PVC Staff 2 0 2 IIRR Professional Staff and Trustees 27 25 52 Staff of Partner/Donor Organizationsa 4 8 12 FG Members (exclusive of the above) 13 28 41 Students in an HQ Course 3 9 12 Ecuadorian Farmers (exclusive of FG members) 6 5 11 Total 55 75 130 Source: Evaluator notes, FG lists. a Includes only senior staff for AFR and ASI organizations. 1.3.3. Organization and Use of This Report This report is organized in 5 chapters, as per the general structure laid out in the SOW. Chapters 2, 3, and 4 describe IIRR programs and achievements by three main activity categories: publications/communication, education/training, and field projects/TA. Also included in Chapter 4 are data on the nature and growth of IIRR partnerships, illustrating from the MG focus countries. The order of each chapter is: overview of activity types and associated procedures and approaches; tabulation of progress toward MG objectives, comparison of MG and baseline outputs, and trend analysis; special topics; lessons for other development organizations; and recommendations. Throughout, qualitative evaluation data are presented in "boxes" containing mini-case-studies and other kinds of supporting qualitative evidence. Chapter 5 turns to the many and complex issues that arose during the evaluation concerning the Institute's management, governance, finance, and personnel systems. Along with sustainability issues, M&E is tackled in a separate final chapter, as this vital task has proved particularly daunting for IIRR. A word on acronyms is in order. These are broken into general versus names of organizations and projects. The former are listed at the beginning of the report, as per standard practice. But the latter are appended as Annex G because of their immense number, thanks to the many and far-flung partnerships IIRR has forged across the years. Also because of their number, organizational acronyms are not spelled out in the text except where particularly pertinent to the topic under discussion. In the interest of a smoother, tighter text, the reader is asked to consult Annex G for any unfamiliar acronyms. USAID-Matching Grant Evaluation Report 12 1.4. Summary of Findings on Progress Toward MG Objectives Very briefly indeed [see also Executive Summary], the evaluation found the following with regard to IIRR progress toward its MG objectives. For Objective I, the Institute strengthened the capacity of a greatly increased number of NGOs and other localized partners to implement development activities. Strengthening was achieved via: IIRR development publications -- whose numbers, local-language versions, subject matters, and dissemination modes all increased immensely during the MG; IIRR's delivery of many kinds of high-quality training, follow-up, and technical assistance, including writeshops and conferences -- all of which increased significantly during the MG; IIRR collaboration with NGOs and other partners in field projects -- the rural beneficiaries of which also increased during the MG; and IIRR's strengthening of its own Regional Centers, the better to serve NGOs and other localized partners. This last item constitutes Objective II, which zeroed in on Centers' ability to deliver and sustain services to partners, undertake process documentation, promote inter-institutional learning and networking, and do impact evaluation. Thanks almost solely to MG inputs, IIRR made tremendous progress toward strengthening its AFR and LAC Centers on all these fronts but the last. In addition to greatly increasing their publications and other best-practices documentation, AFR and LAC grew their partners manyfold and, with this, inter-institutional learning and networking. AFR and LAC also made substantial strides toward self-sustainability. Progress by the ASI Center is difficult to determine due to the conflation of ASI activities, budget, and management with those of HQ. Sustainability at all three Centers is still a question￾mark in terms of staff skills and personpower, notably for regional leadership structures. On the (implicit) Objective III of building IIRR capacities as a whole, significant progress was made in virtually all programmatic realms as well as in global partnerships, financial information and reporting systems, governance, and information technology. However, HQ experienced considerable difficulties in managing finances in relation to the Institute's overarching programmatic goals and also in decentralizing decision-making authority from HQ to the regions. And all Regional Centers plus HQ fell down on the job of doing impact evaluation and, indeed, M&E of other than process data. USAID-Matching Grant Evaluation Report 13 2. PROGRAM IMPLEMENTATION: PUBLICATIONS AND COMMUNICATION 2.1. Overview A signal area of IIRR capacity, earnings, and excellence has long been its publication and (formerly) communication program, designed to capture best practices and to share learnings from Institute and other development agencies' activities as widely as possible. Indeed, in this evaluator's opinion, the publication program could justifiably be termed the "shining star of IIRR," for reasons detailed throughout this chapter. 2.1.1. Capacities Before decentralization, all publications emanated from the HQ publication unit, which previously also produced an Institute research journal. Resuscitation of this journal is now unwisely being considered. As a general rule of thumb, such journals are expensive to produce, have a narrow audience, are inappropriate for organizations with limited scientific research capacity, and have low credibility for the foregoing reason and also because they can be subject to internal politiking. That said, the HQ unit was, and is, well-equipped with state-of-the-art desktop publishing capacity. It boasts a small (N=3 professionals) but impressively skilled production staff and a unit head who doubles as an editor. It also manages a countertop bookstore that sells IIRR and partner publications. Previously, the unit also had photo and audiovisual capabilities. But in 1996, the building in which these facilities were housed burned down, taking warehoused copies of earlier IIRR publications with it. Both the building and its contents were uninsured. Fortunately, all but the oldest stock of publications lost in the fire have now been recouped and reprinted from disc copies stored elsewhere. However, the non-print-media capacities have not been restored. This is unfortunate given that media such as videotapes of IIRR activities have proved useful training and service-marketing tools. For instance, a videotape of an IIRR writeshop [Section 2.3.] in ASI was instrumental for "selling" this concept in AFR. As decentralization took hold, the regions began generating their own publications in situ, albeit initially with some on-the-job (OJT) training and TA from senior IIRR technical staff [see boxes below]. The physical quality and visual presentation of all IIRR publications today is highly professional and attractive. An examination of earlier versus later publications reveals their increasing sophistication in these regards, as both desktop publishing technology and IIRR staff skills advanced. The regions have also done an admirable job in terms of the timeliness, USAID-Matching Grant Evaluation Report 14 usefulness, professionalism, and often globally relevant content of their publications. The latter is signalled by the diversity of purchasers and users around the world [see boxes]. These achievements have come about despite AFR's lack of a communication specialist until 1999 and LAC's obsolete computers and tiny technical staff (N=3.5) with no special expertise in communication. For reasons explained in Chapter 5, most Asia-oriented publications were produced by the HQ publications unit. Strangely, by comparison to AFR or even to LAC's English-language outputs, publications from the longstanding HQ unit are the most likely to miss the mark in English-language editing. Some important volumes edited by the present unit head have been produced in embarrassingly broken English. One such is the proceedings of a major international conference on participatory monitoring and evaluation (PM&E) mounted by the Institute (IIRR, IIED, IDS, et al. 1998) [see Box 4.5]. 2.1.2. Policies IIRR has a truly unique policy of not copyrighting its publications, and clearly so states on the inside front cover of most IIRR publications. This policy permits their ready duplication in whole or in part, and thus their wider dissemination and use worldwide. For the same reason, the Institute even encourages a sort of plagiarizing or "copy-catting" of its publications. For example, in its justly famous writeshops [Section 2.3.], upon request author￾contributors are given disc copies of the resulting publications. They can take the discs back to their home organizations or countries and revise and/or translate and re￾publish there for new or larger audiences, with no IIRR strings attached [see boxes below]! Sometimes the Institute even foregoes mention of its name on the cover of publications for which it has garnered the funding, the better to allow contributing organizations to take credit for the work. An example is the 10,000 information kits produced with the Philippines government's Department of Environment and Natural Resources (DENR) [see Box 2.2.]. With decentralization and its challenge of building transnational coordination and communication systems [Chapters 5 and 6], however, publication policies and procedures have not been consistently applied, enunciated, or evolved to fill an evident need for cross-Institute integration and M&E. The Institute could benefit from more uniform policies and oversight mechanisms to ensure the continued high quality, audience-responsiveness, and maximum accessibilty of IIRR's publications and development communication services across regions. Such policies and procedures might involve, e.g.: • verifying that the non-copyright policy is followed -- e.g., LAC failed to do so in one instance; • setting minimum standards for English-language editing; USAID-Matching Grant Evaluation Report 15 • issuing guidelines on gender-sensitive terminology; • suggesting minimum physical production standards (paper, ink, binding) according to the different uses envisioned for publications -- this suggestion comes from the LAC Director; • as per an AFR policy, requiring a literature review before embarking on major, IIRR-driven publications to check that they do not duplicate existing or forthcoming materials by other development organizations and IIRR units; • ensuring in-print acknowledgement of all relevant donors -- for instance, BHR/PVC is mentioned in only a handful of 1996-1999 publications, even though MG salary supports for expanded AFR and LAC staffing directly contributed to these efforts; • requiring systematic citation of all IIRR offices in all publications so that when potential future clients happen upon them anywhere in the world, they are provided region-appropriate contact information; • relatedly, making clearer in each publication where and how readers from different nations can purchase it most conveniently; • in every IIRR volume, including a list of other available Institute publications; • determining when, or if, it is cheaper or faster to prepare camera-ready copy in￾or out-of-house at HQ versus the regions; • devising publication cost- and revenue-sharing, overhead assignment, and other such financial formulae as needed to promote greater cross-Institute collaboration on publications; • regularly monitoring, reporting, and analyzing vital M&E data on publications Institute-wide. A normal part of the growing pains of decentralization, initial omissions in setting and enforcing unifying publication policies and procedures should be corrected now, taking into account varying regional considerations such as language(s), programmatic emphases, staffing, audiences/clients, etc. 2.1.3. Approaches and Procedures IIRR publications result from varied activities: conferences; workshops; special commissions; occasional scholarly publications by senior IIRR staffers as an outgrowth their own or IIRR-related efforts -- e.g., Selener 1997 or Scarborough, Killough, Johnson, et al. 1997; and above all, the participatory, grassroots-oriented, and multi-institutional "writeshop" process [Section 2.3.]. The publications themselves consist of handsomely printed books, equally handsome spiral-bound documents, and "information kits." The latter are something of an IIRR innovation. Kits typically focus on scientific and/or ethnoscientific/indigenous technical information. Often, the information is organized in several subtopical volumes packaged together in an attractive library-shelf box. USAID-Matching Grant Evaluation Report 16 Publications are funded through many different, and often highly leveraged, mechanisms. Almost invariably they involve contributions from multiple donors and PVOs/NGOs as well as self-paid participation by PVO/NGO staff as technical resource persons and author/contributors [see boxes below]. Coupled with a new policy of contracting editorial and graphics support staff for offsite publications out-of-house, this strategy makes IIRR publications very cost-effective. It also ensures that they are largely demand-driven or, at the very least, responsive to topics and issues that many other development groups deem timely and important. Furthermore, it makes for healthy book sales. During the MG, IIRR has moved aggressively to market its publications and, equally important, to disseminate gratis the best practices and other development information therein more widely, via steps like the following, plus still others discussed in later chapters. • In 1997, the HQ unit began forwarding copies of all IIRR publications for scanning and inclusion in the CD-ROM Humanity Development Library for Sustainable Development and Basic Human Needs. Supported by such agencies as GTZ and GATE under a Belgian NGO Global Help Project, the CDs are distributed gratis worldwide. To date, 14 IIRR publications have been included. • All IIRR units are generous with courtesy copies of their publications. • In 1999 AFR began experimenting with making the entire text of its most popular publication available on-line at its website [see Boxes 2.2. and 2.5.]. A great deal has been done during the MG to disseminate IIRR publications and the development messages therein. Still, FG commentary suggests that AFR and LAC could profit from additional ideas and further HQ support in this regard. In AFR, former IIRR trainees and writeshoppers are so pleased with Institute publications, yet at the same time concerned about dissemination, that they pack copies to show off at conferences and meetings. On her own initiative, one AFR FG member even carries extra copies to sell, and remits the monies to AFR thereafter. As a still-new organization in Africa and Latin America, IIRR is not yet familiar to many development actors there. But to judge by FG and other commentary, IIRR's quality publications are perhaps the single best way to advertise its existence, philosophy, and service availability in these regions. One last activity merits mention. Since 1994 the HQ unit has participated in, and often hosted, the annual meetings of an informal inter-institutional Communication Workgroup that embraces the ADB, two IARCs (ICLARM and IRRI), and various NRM and agricultural R&D centers in the Philippines. The workgroup's themed meetings address contemporary methodologies, technologies, and issues in development communication, such as: different levels and types of editing; desktopping technology; kit design; audio-visual services; computer and other graphics; website content; CD- USAID-Matching Grant Evaluation Report 17 ROM use and digital archiving of media assets; production economics, copyrighting, distribution and marketing schemes, including on-line and credit-card sales; publication impact assessment; and more. IIRR should try to take greater advantage of the workgroup network by accessing it more often for assistance and ideas on Institute publication/communication needs. The most immediate of such needs are marketing to new regional audiences and M&E/impact assessment. 2.2. Progress and Trends The number of IIRR publications more than doubled between the baseline and MG periods, from 14 to 32; and an additional 13 works are well underway [Table 2.1.]. Vis-à-vis the DIP, all IIRR units and regions met or surpassed their publication targets. During the MG, all also exceeded their own baseline output. In the aggregate, DIP targets were surpassed by close to 100%, with 17 publications targeted and 32 produced. Table 2.1. Numbers of Major Publications Unit/Region Baseline Output MG Output (& in press) DIP Target Target Variance HQ Publications Unit 12 17 (9)a 3 +14 Africa 2 3 (2) 3 0 Asia N/Ab 3 (0) 3 0 Latin America 0 9 (2) 8 +1 Total 14 32 (13) 17 +15 Sources: Annex C's DIP Schedule of Activities plus two tables of baseline/MG publication outputs compiled by IIRR at the evaluator's request. a Numbers in parentheses indicate additional works in progress but not yet published during the MG. b Numbers for ASI-only as distinct from HQ publications are somewhat arbitrary, for reasons explained in Chapter 5. Publications appropriately center on the Institute's program thrusts and cross￾cutting themes. Ag, environment, and NRM -- which historically have constituted IIRR's prime sectoral strength -- naturally take pride of place. As evidenced in tabular data compiled at the evaluator's request, these subject matters dominated IIRR's book list across the baseline and MG periods alike. Institutional capacity-building is reflected in works on planning, participatory methods, education, training, and extension. All such topics received growing attention in IIRR publications during the MG as compared to the baseline. Likewise for farmers' agroecological and local/indigenous technical and livelihood knowledge and practices. Health subjects also came to the fore during the MG, along with more women-oriented publications. USAID-Matching Grant Evaluation Report 18 Figure 2.1's map graphically summarizes publication outputs during the MG. Aside from more diversified but programmatically apt subject matters, several other positive baseline-to-MG trends are visible in the tabular data compiled for this Figure 2.1. Map of 1996-1999 Publications USAID-Matching Grant Evaluation Report 19 evaluation. [Because of their length, these tables are not included in the present report.] One is, despite decentralization, an increase in publications with global as well as regional relevance. These went from 4 to 15, representing 28% and 47% of baseline and MG output respectively. They include many works produced regionally but that have applications worldwide. Particularly noteworthy are the English versions of LAC publications on participatory methods [see again Figure 2.1]. Another is a strong trend toward publishing in situ in greater numbers of countries and local languages. During the baseline period, nearly all publications were produced at HQ, and all appeared only in English. In contrast, 11 other-language works were issued in more than half a dozen countries during the MG. Besides LAC volumes in Spanish, works were produced in Bengali (for the focus country of Bangladesh), Chinese (with solely ASI backing), Vietnamese, and (in progress) Swahili. A brand new trend is toward cross-regional publications in which at least two IIRR regions participate. The sentinel case is 1998's Reproductive Health and Communication at the Grassroots: Experiences from Africa and Asia. A related trend is to replicate successful technical publications from one region in another. For instance, AFR's 1996 Ethnoveterinary Medicine in Kenya followed the lead of HQ/ASI's 1994 Ethnoveterinary Medicine in Asia (reprinted in 1998). All but 50 of the 2,000 copies printed of the AFR volume had been distributed as of Nov 1999, so a reprinting in planned for 2000. It merits mention that printing was contracted with a Kenyan NGO (the African Center for Technology Studies) thus representing a sort of epiphenomenal NGO strengthening on IIRR's part. Another and very strong trend is to translate and/or re-tool publications generated in one language and/or region for one audience into languages and formats useful for other regions and audiences. Indeed, to this end LAC has set a policy of always translating all its publications into English. Box 2.1. presents an even more dramatic example of this astute strategy, which allows IIRR greatly to extend its publications' reach and impact in an extremely cost-effective manner. USAID-Matching Grant Evaluation Report 20 Box 2.1 Some B.I.G. Publication Payoffs Fondly known as B.I.G., the Bio-intensive Approach to Family Food Gardens information kit was first produced in 1987 as an IIRR in-house compilation of literature￾based vegetable-raising technologies, with support from the Asian Vegetable Research and Development Centre (an IARC). With funding from Australian sources and a US foundation, in 1993 the kit was revised to feature tested gardening practices based on the field experiences of IIRR and other organizations. Also, new technical topics and themes were added. The kit was then re-issued as a book entitled Bio-intensive Approach to Small-scale Household Food Production. Since then, the revised kit has been adapted and translated in Bangladesh, Cambodia, Guatemala, India, Indonesia, Kenya, Laos, Nepal, and Thailand. For use by farmers in the Philippines, it was translated into 5 local dialects and re-formatted in a comic-book￾like series, coordinated by the Philippines Department of Agriculture. In 1997, AVRDC collaborated with LAC to adapt the B.I.G.'s content to Latin America and publish it as Guía Práctica para su Huerto Familiar Orgánico ('Practical Guide for Your Family's Organic Garden'). In 1998, due to continuing demand, the 1993 version of the B.I.G. went into a second printing. USAID-Matching Grant Evaluation Report 21 Another salubrious trend is for IIRR to collaborate on publications as a co-equal with top-flight international scientific research centers, such as FAO or the IARCs of the CGIAR system. To take just one example, FAO has asked IIRR and ICLARM to revise and expand the technical content of their joint 1992 publication, Farmer-proven Integrated Agriculture-Aquaculture, so that FAO can re-issue it. Referring again to Table 2.1., note that it does not include reprintings of pre-MG works (N=8) in order to meet continued demand during the MG. Nor does it capture additional handbooks and non-formal but nevertheless-valuable information and training materials. A few examples are AFR's Monitoring Made Easy: A Handbook for Development Practitioners (Schroll 1999), AFR/Ethiopia's Gender Training Handbook (1999), and ASI's Village Health Worker Training Manual in Tagalog (1996-1999). Likewise for important workshop publications, like the 100-page Research Partnerships: Issues and Lessons from Collaboration of NGOs and Agricultural Research Institutions (CGIAR and IIRR 1999). Neither does Table 2.1. include the many "plagiarized," copy-cat, or follow-on publications, produced with and without IIRR participation or imprimatur, and sometimes even IIRR knowledge. These are generated by PVOs, NGOs, and GOs who may have sent staff to one of IIRR's famous writeshops [next section] or simply by enthusiastic individuals who admire IIRR's work. Box 2.2. is illustrative of such phenomena, as well as of IIRR flexibility in information formats so as to respond to different end-user needs. USAID-Matching Grant Evaluation Report 22 Box 2.2. Runaway Publication Successes! With funding from the Ford Foundation/Philippines and DENR collaboration, in 1989 IIRR held a writeshop to produce what eventually became the Institute's most widely circulated publication, the Agroforestry Technology Information Kit or ATIK. Based on user needs and preferences at the time, the kit was printed in color-coded, loose-leaf sheets with a hard-cover carrier. In a 1992 workshop to update ATIK content, extension workers asked if the kit could also be made smaller so they could carry it into the field. IIRR therefore created 4 pocket-sized volumes, with a sturdy library-shelf storage box. This popular version of the ATIK has since been reprinted twice. In 1996, the loose-leaf ATIK was independently translated into Vietnamese by a faculty member at Vietnam's Thu Duc College of Agroforestry, who learned of the kit while participating in a different IIRR writeshop. Similarly, a Vietnamese NGO translated and re-published the ATIK's Volume 1, on "Soil and Water Conservation Technologies and Agroforestry Systems." Further, in 1999 the ATIK was one of 3 book prizes given away in a drawing by The Overstory, a US-based interactive electronic newsletter on agroforestory. USAID-Matching Grant Evaluation Report 23 Produced with FAO support in a 1994 writeshop, IIRR's kit on Resource Management for Upland Areas in Southeast Asia underwent similar appropriations. In 1997, several Indonesians from the writeshop adapted, translated, and published the kit in Bahasa Indonesian, with support from several national NGOs and GTZ. A Thai writeshopper likewise produced a related publication in Thai, in collaboration with the Royal Forest Department of Thailand, FAO, and other groups. Still other organizations plan to translate and release other IIRR publications in Bahasa, including the 1994 kit on Ethnoveterinary Medicine in Asia (by HPI) and the 1998 Participatory Methods in Community-based Coastal Resource Management (by the Forum for Coastal Community Studies, hosted by the University of Indonesia's Anthropology Department). Such follow-on outcomes and, indeed, much of the success of IIRR's publication program as a whole, are directly due to the Institute's non-copyright policy plus a unique and highly participatory approach to the generation of development information materials: the writeshop. 2.3. The Writeshop: A Participatory Publication Methodology Beginning around 1987, as a way of rapidly and inexpensively producing practical yet high-quality information materials that give voice to multiple stakeholders and their best-practices savvy, the Institute pioneered an exciting, participatory publication method known as the "writeshop." For readers unfamiliar with this process, it is graphically and textually summarized in Figure 2.2. Also consult King and Shakya 1994 and Mundy and Mathias 1999. USAID-Matching Grant Evaluation Report 24 a Writeshop managers begin preparations up to one year ahead. a A few months before the writeshop, a multi-stakeholder steering committee is formed to develop detailed guidelines for authors. a Authors bring their manuscripts to the writeshop on disc and in hard copy, plus reference materials. a Participants brainstorm any missing topics. a At each draft/present/critique/revise stage, authors work one￾on-one with (mostly local) editors and artists. a Also available to authors are translators (ideally persons with firsthand knowledge of the writeshop subject). a Second-draft critiques are done in participant breakout groups by subtopical expertise, geographic area, or other variables. a Authors return home carrying their third and final drafts with them, to do with as they please. a In a month or so, the publication is done. a Besides receiving complimentary copies of the final publication, writeshoppers may request it on disc if they wish to revise and/or translate and re-publish it in their own countries and languages for new or larger audiences -- with no copyright restrictions whatsoever. Figure 2.2. The Writeshop Process Illustrating from just a few among many such writeshops during the MG, the boxes below illustrate the richness and the multiple and varied outcomes and impacts of this participatory method. USAID-Matching Grant Evaluation Report 25 Box 2.3. Meeting a Region-wide Information Need AFR's first major publication was the 241-page Sustainable Agriculture Extension Manual for Eastern and Southern Africa (IIRR/AFR 1998). The Ford Foundation/Kenya had received a dozen different proposals from local PVOs and NGOs to produce much￾needed materials on this topic. But Ford deemed these groups insufficiently experienced in development communication. Moreover, according to the evaluator's interview with the Ford program officer in charge of this effort, none was willing to attack this evident need in a collaborative mode that would draw upon the other groups' learnings. Aware of IIRR's long history in the professional yet participatory generation of technical informational materials for agriculture, Ford therefore turned to AFR for help. According to the Ford program officer, as a largely non-implementing organization but one with demonstrated publication expertise plus strong participatory skills, IIRR was uniquely positioned to overcome the rampant inter-group jealousies in the region's development community and get the job done. AFR accepted the challenge, and wrote to 25 PVOs/NGOs working in sustainable ag to invite them to document their experiences as a team. Fifteen responded favorably. Ford thus awarded AFR a start-up grant of $45,000 to hire a project coordinator and to set up a multi-institutional steering committee consisting of representatives from Ford, CARE/Kenya, ITDG/Kenya, the SIDA-funded Regional Land Use Unit (RELMA) based in Nairobi, and a local NGO specializing in sustainable ag. Under AFR's leadership, the committee wrote a successful proposal to garner the additional $93,000 needed. These funds came from the Centre Technique de Coopération Agricole et Rurale (CTA) in the Netherlands, Germany's GTZ, and again, Ford and RELMA. In addition, the internationally renowned agroecologist Miguel Altieri was recruited to write a foreword. The 2-week writeshop was held in early 1998 at the Kenya College of Communication Technology. In attendance were 67 participants from secular and religious PVOs and NGOs, ag colleges, national ag research institutes, independent research agencies, and Ministry of Agriculture (MOA) entities plus, of course, farmers. Writeshoppers came from 8 countries Africa and were overwhelmingly African nationals. The AFR Director facilitated the writeshop. Editors included: the AFR Training Officer; HQ's Deputy VP; the editor of ALIN, a pan-African drylands information network; and the head of an Southeast African network of participatory development organizations. This group was headed by a former IIRR staffer who helped pioneer the writeshop concept. Artists and desktop-publishing experts were all contracted locally. Briefly, the resulting volume covers participatory technology development and extension techniques generally, plus their application to credit and marketing, resource use and conservation, cropping and stockraising systems, and gender issues, as illustrated from participants' field experiences. As in most IIRR writeshop products, appendices list many networks, associations, organizations, books, manuals, newsletters, videos, slidesets, and additional sources of extension materials on sustainable ag. USAID-Matching Grant Evaluation Report 26 Within one year, 2,150 of the 3,000 copies printed were distributed or sold. Besides writeshoppers and sponsoring agencies, particular target groups have been government extension workers and farmer training centers. Other telling examples of the book's dissemination and use are its purchase: • as a working manual for extensionists of the Adventist Development and Relief Agency (ADRA) in Ethiopia (14 copies); • as a textbook in the Extension Department of Ethiopia's second-largest university (17); • as a manual for training slum dwellers who relocate to periurban areas with help from the Italian PVO Sucos (no data); • as a reference for UNHCR workers in their refugee rehabilitation programmes in Uganda (150); • for dissemination by the Harare-based umbrella organization, PELUM, to its Africa-wide membership (1,000); • for international redistribution by CTA (1,000). Still other significant purchases were made by perhaps a score of disparate groups, of which a few examples are: • in Kenya alone -- Baraka Agricultural College, the Development Programme of the National Council of Churches of Kenya, the Institute of Organic Farming, the national ag research institute, and ILRI (a livestock IARC); and • in other nations -- CARE/Uganda, the University of Maiduguri in Nigeria, various FAO projects in Africa, a European ag information center (ILEIA), UK and US universities, and even Asian NGOs. Like most IIRR publications, the AFR manual has been widely reviewed in both Northern (e.g., the ILEIA Newsletter, Spore, The Indigenous Knowledge Monitor) and Southern (e.g., ALIN, Baobob) development magazines. In the words of evaluation interviewees, this handsome volume has been a major force in "putting IIRR on the map in Africa." It has also been a major income-earner for AFR. But the story of the institution-building and information-sharing power of such efforts does not stop there [Box 2.3.]. USAID-Matching Grant Evaluation Report 27 Box 2.4. Writeshop Multiplier Effects As a result of the resounding success of its ag extension manual, AFR has been approached with a number of follow-on requests and inquiries. • A Mozambican organization has written to ask about producing the manual in Portuguese. • A Kenyan association has requested TA to produce a seed security manual for development workers in 2000, and AFR has already helped the association mount a preparatory workshop for the manual. • PELUM has approached AFR to help it writeshop a manual, based on the enthusiastic recommendation of one of its staff who participated in the extension-manual writeshop. • The famous "Flying Doctors of Africa" (AMREF) have contracted AFR to co￾publish a volume on Best Practices in Community-Based Health Care, for which 75% of the funding was raised as of Nov 1999. • Likewise for a volume on Agricultural Technologies for the Drylands of Africa, supported by CTA and RELMA. • In Dec 1999, CTA expects to sign a contract for AFR to facilitate a writeshop for an advocacy manual for ag organizations in Africa, possibly to be published in French as well as English. This will be CTA's first writeshop experience. • Moreover, reportedly CTA may shift its own publications strategy to emphasize IIRR's participatory methodology, in order to make CTA technical information more simply and widely accessible. The foregoing case is by no means unique. It could be duplicated many times over in Asia, where the writeshop concept was birthed. Box 2.5 offers one such example. Box 2.5. Satisfied Customers GO Back for More DENR participants in the ATIK writeshop were so impressed with the process that in 1997 this GO asked IIRR to do a similar writeshop for its staff nationwide to compile their past 10 years of experiences with environmentally friendly livelihood strategies, both traditional/indigenous and modern-scientific. This resulted in the 3-volume kit, Sustainable Livelihood Options for the Philippines. For a special 1999 launching hosted by DENR's national secretary and with heavy media coverage, 10,000 copies of the kit were printed. DENR distributed 7,500 of these to all 15 Philippines regions (500 per region). USAID-Matching Grant Evaluation Report 28 Particularly telling testimony to the robustness and power of the writeshop process is Box 2.6's description of its translation to a new continent and language. Box 2.6. A Writeshop with a Whollop! LAC's first writeshop product was the Manual de Prácticas Agroecológicas de los Andes Ecuatorianos (1996). It arose from two observations by the LAC director. First, like most people, he had been greatly impressed by the writeshop process he witnessed while at HQ. Second, upon taking up his post in Ecuador, he identified an Andean-wide need for accessible technical information on agroecological practices appropriate to this complex ecozone. Thus, on a return trip to the Philippines he sought OJT in writeshopping from the HQ publications unit. Next, he elaborated a proposal for an integrated project to produce the manual. The project was designed in four phases. I. A year of participatory field research on Ecuadorian farmers' useful local practices in agroecology, followed by the drafting of case studies on these practices. II. Three months' organizing for a 2-week writeshop at which farmer-authors would present their case studies, with development personnel also attending. III. The writeshop itself. IV. Final editing and printing of the manual. The director corralled $69,000 for the project from the Ecuadorian branches of CARE and CRS plus the regional office of LWR, along with perhaps some $40,000 of cash+time match from HQ and LAC. Forty-four farmer-authors from 79 communities of highland Ecuador contributed to the resulting manual. Also involved were 19 development technicians, educator/trainers, and researchers from a wide variety of institutions: some 10 national GROs, NGOs, and farmer groups; 3 PVOs with national offices in Ecuador -- CARE, the Bern-based SwissAid, and WN; 2 rural colleges; the Ecuadorian MOA and various of its national￾level programs; and 1 international research center (ORSTOM) and 1 UN (FAO) project. Also in attendance were key HQ staff experienced in both agroecology and writeshops. As part of IIRR's capacity-building strategy, primarily Ecuadorian technical editors and exclusively Ecuadorian copyeditors and artists were hired. In this way, a "bonus" transfer of knowledge took place. Moreover, all production and printing were done in￾country. The grand total of participants summed to 72. An express part of this or any IIRR writeshop was/is to train other PVOs/NGOs and GOs in this participatory publication method. Among the writeshoppers were staff of three such organizations: a project coordinator of CIDICCO, a Honduran NGO; 7 senior managers of CARE/Ecuador's project on sustainable ag; and the country director of SwissAid. USAID-Matching Grant Evaluation Report 29 Inspired by the writeshop, the Honduran attendee asked LAC to help do another agroecological manual in Honduras (CIDICCO et al. 1997). This independently￾conceived and -driven follow-on effort was implemented in a participatory conference on cover crops and green manures in Mexico and Central America. It brought together representatives of 14 NGOs, universities, or development projects from Costa Rica, Guatemala, Honduras, and Mexico, thanks to funding from a variety of religious and secular PVOs and NGOs, the WB, and still other organizations and projects. Meanwhile, the CARE attendee decided to mount a writeshop that resulted in a 293- page publication on CARE experiences in Andean NRM (CARE/Ecuador et al. 1998), closely modeled on the LAC manual. The SwissAid country director realized the need for likewise documenting and disseminating his organization's work in sustainable ag. So he contracted with LAC to assist SwissAid with the now-in-press, 120-page volume La Granja Biológica Campesina [lit. 'The Biological Peasant Farm'] plus an accompanying poster printed in Kichwa as well as Spanish. Approximately 500 Ecuadorian farmers contributed their agroecological wisdom to this initiative. As of the time of this report, the 1,000 LAC manuals were essentially sold out. A reprinting of at least 500 is envisioned shortly. Sales have so far amounted to $10,800.61, excluding 203 courtesy copies distributed. Purchasers and courtesy recipients of this Spanish-language-only volume span a vast variety of PVOs, NGOs, farmer associations, individual farmers, and rural schools in Ecuador and other Andean nations. To take just one example of the manual's dissemination, and thus its potential impact, the Ecuadorian Ministry of Basic Education purchased 450 copies for distribution to all bilingual teachers in the nation's rural elementary schools. These cases illustrate the proactive and rapid response of IIRR's new regional centers to a real need for vital development information in their service areas. Moreover, in each region just a single writeshop awakened latent PVO/NGO desires to reflect on and communicate about their development experiences as validated in and facilitated by the writeshop process. Indeed, AFR and LAC have responded so rapidly and capably that both are inundated with requests for more or similar publications and facilitation services in more languages, and/or for mentoring other institutions in the writeshop methodology. Further, to the extent that "imitation is the highest form of flattery," the excellence of this IIRR information-sharing method is reflected in the interest of a major Northern publisher of development books (CTA) to emulate it. Moreover, as the writeshop moves from its Asian birthplace into new parts of the world, it has begun to evolve new forms and applications [Box 2.7.]. USAID-Matching Grant Evaluation Report 30 Box 2.7. Innovating on the Writeshop Because of the overwhelmingly positive response to AFR's ag extension manual, the Ford Foundation/Kenya asked AFR to present a Phase II proposal (IIRR/AFR May 1998) to disseminate the manual's information more widely. Funded in 1999 to the tune of some $50,000, under this grant AFR intends to take the writeshop in some fresh directions by: • translating (completed) and re-publishing the manual in Swahili; • releasing the entire manual on-line on AFR's website (completed); • building an interactive e-mail network for region-wide dialogue on sustainable ag and its extension (underway); and • disseminating selected manual topics via audio cassettes so as to make the information less prone to miscommunication by low-skilled extensionists and more accessible to illiterate farmers and other poorly served audiences via radio and listener groups. Furthermore, AFR has deployed the writeshop as an evaluation tool, for documenting development experiences and lessons learned for project evaluations. This application resulted from a CARE/Kenya and /Tanzania request for AFR to help them document 29 cases from their work on a basic-education project, as part of the project's midterm evaluation. LAC has used a modified version of the writeshop with GOs. One example comes from Ecuador's national ag research agency, INIAP. With LAC TA and workshop facilitation, INIAP scientists wrote and published a pair of extension booklets based on their recent research findings in potato-raising (Gallegos et al. 1997 and Merino et al. 1997). Thereafter, INIAP was able independently to produce more such materials for practical use by extensionists and literate farmers. Meanwhile, back in AFR, the Rockefeller Foundation has challenged AFR to evaluate how different media and applications variously convey the kind of technical and other information captured in publications like AFR's ag extension manual. A proposal for this activity is in the pipeline for 2000. USAID-Matching Grant Evaluation Report 31 2.4. Lessons and Recommendations 2.4.1. Lessons Several major lessons for the PVO/NGO community emerge from the foregoing discussion -- which, by the way, appears to represent the first-ever analytic overview of IIRR's publication program. Most of the lessons are obvious from the discussion itself and, especially, from the boxed case studies. Nevertheless, they are briefly re￾capped here along with the corpus of data on which they are based. The first and overarching lesson is the inestimable value of the writeshop, IIRR's unique contribution to development communication. Writeshops serve myriad PVO/NGO and also GO capacity-building purposes, plus broader development aims. This statement and those to follow are supported by a suasive body of evidence: • commentary from writeshoppers themselves, as reported in print (King and Shakya 1994), in participants' post-writeshop evaluations (as collected and stored by various IIRR units), and in the evaluator's participant observation as a scientific resource person in a 1994 writeshop [her only previous contact with the Institute]; • an HQ videotape of an actual writeshop-in-progress (viewed by the evaluator); • the enthusiasm for writeshopped publications among FG members [Annex B], as representatives of the PVO/NGO community at large; • published reviews of writeshop products (compiled by the HQ unit); • healthy sales of writeshopped books; • the diversity of audiences and uses for writeshopped publications, including PVOs, NGOs, GOs, universities, extensionists, farmers, and likely also researchers [see last bullet below]; • other organizations' rush to duplicate the method once they have experienced it -- as illustrated in the boxed case studies above; • likewise, the translation and/or adaptation and re-publishing of writeshopped publications by other organizations and individuals at their own expense; • reports of plans on the part of development communication organizations in the North to emulate the process; and • probably significant numbers of citations in R&D literature for writeshopped works (a possible M&E impact indicator). Certainly, the capacities of PVOs/NGOs who participate in an IIRR writeshop are strengthened. After just one such event, many organizations are able to apply this rapid and cost-effective information-sharing methodology to their own ends, often with little or no further IIRR TA. Through learning-by-doing -- in effect, another form of IIRR training -- writeshops quickly and effectively build the capacity of participating organizations and individuals to prepare useful information materials of their own. Hence, therefore, the rich multiplier effects that can result for a sector, a region, a USAID-Matching Grant Evaluation Report 32 development method or issue, and for IIRR partner institutions. In turn, the latters' partners and beneficiaries are thereby able to codify, share, and use their own and others' development lessons, practical farming or healthcare techniques, and so forth. Of course, writeshoppers themselves are trained or empowered in many ways. Obviously, they learn something about how to share their knowledge in writing. Even among scientists, in the South an IIRR writeshop may afford many their first experience of writing for formal publication and/or for different audiences. Certainly, writeshops allow voices to be "heard in print" that might otherwise remain forever silent, such as those of the hundreds of illiterate men and women farmers who have participated in IIRR writeshops across the years. Most writeshops also bring together disparate voices -- scientists and farmers, Southerners and Northerners, NGOs and GOs -- in a non-threatening and mutually productive forum. Such "choirs" result in development information that is better grounded and validated than if only one group were consulted. As attested in multiple data sources, this participatory method fills an acute felt need among development agencies of all ilks to exchange and analyze their own and others' knowledge and experience and, moreover, to package and disseminate useful or needful technical, development, and now also evaluation information rapidly, practically, and cost-effectively. And again, the process makes room for special kinds of information that otherwise might never reach the rest of the world -- like rural peoples' technological, agroecological, and ethnomedical savvy. A second lesson lies in the way IIRR wrings multiple, continuing, and cost￾effective use out of its publication investments and promotes their widest possible access by updating, modifying, translating, or re-packaging them. Moreover, re￾packaging now includes CD-ROM, audiotape, and internet as well as print options. A third lesson is the value of publications generally as a way of "advertising" a PVO and almost literally "putting it on the map" in new regions or with new audiences. In this regard, other PVOs may find it useful to emulate many of IIRR's marketing and outreach strategies. By default, a fourth lesson is the importance of tracking what appear to be powerful multiplier, spillover, follow-on, and outreach effects from writeshops and publications. In lieu of systematic M&E, these can be only hinted at in the space of even the most thorough-going outside evaluation. 2.4.2. Recommendations Despite contretemps like fires, staff shortages, obsolete equipment, inadequate policies and M&E, there is no denying the many and impressive achievements, innovations, and apparently mounting impacts of IIRR's publication program across the USAID-Matching Grant Evaluation Report 33 MG period. To consolidate, expand, and fully capitalize on these successes, the following recommendations are offered. a Congratulate all relevant staff for the greatly increased output, the continuing overall excellence, and the growing -- albeit largely undocumented -- spread, multiplier, and impact effects both regionally and globally of IIRR writeshops and publications during the MG. a Do not resuscitate the old IIRR research journal. Instead, use existing development journals, magazines, and other (e.g., electronic) outlets to disseminate IIRR action-learning findings far more cost-effectively, broadly, and credibly. a Calculate the relative costs of restoring IIRR's in-house audiovisual capacities as versus hiring this service as-needed out-of-house. In either case, make greater use of these media for multiple communication purposes (e.g., marketing, training). a Remedy the problem of poor editing at HQ. a Review LAC needs for publication/communication equipment and staff, and promptly supply the most pressing such needs. a Establish clear Institute-wide (but also appropriately region-specific) publication policies and quality controls [see text]. Then designate a qualified staffer at HQ and a counterpart in each region to monitor and enforce them. a However, do not change IIRR's non-copyright policy nor the policy of producing publications collaboratively, since both these strategies contribute greatly to the wider distribution and relevance of IIRR publications and the information therein. a Especially for AFR and LAC, give greater HQ assistance to publicize and disseminate IIRR publications. a Draw more heavily on the inter-institutional Communication Workgroup and other such networks for ideas and assistance with IIRR publication and communication needs. a Continue and, as regionally and financially appropriate and feasible, intensify the salubrious publication trends that emerged during the MG, e.g. toward: greater global relevance; IIRR's and other organizations' modification and translation of existing publications; increased cross-regional collaboration on publications within IIRR; and partnerships with higher-order organizations, like IARCs and other USAID-Matching Grant Evaluation Report 34 research centers. All such strategies capitalize on publication investments already made and compound their global impacts cost-effectively. a Continue exploiting to its fullest what is one of IIRR's greatest strengths, the "writeshop"; and depending on how AFR experiences play out, disseminate more writeshop products via non-print media. a As per the next chapter's evaluation findings, increase integration between publications and training. a Promptly establish a systematic and Institute-wide program of M&E for IIRR's publication/communication program. USAID-Matching Grant Evaluation Report 35 3. PROGRAM IMPLEMENTATION: EDUCATION AND TRAINING 3.1. Overview IIRR modestly claims to have trained over 10,000 development practitioners from more than 100 countries and 2,500 development organizations since its inception (IIRR/HQ 1999:2). Given IIRR's genesis in the mass education movement, training constitutes the historic backbone of Institute programs, and it retains this position today. 3.1.1. Categories and Content Training is conducted by an HQ training unit that also services ASI needs and by the AFR and LAC Regional Centers. Training takes many different forms at IIRR, depending on the venue, event, participants, and whether the course is supply-driven or demand-led, i.e., promoted by IIRR or requested by a client organization. There appear to be the following general types of training, although IIRR staff use a confusing variety of terms for them, and some doubtless overlap. International Courses. Historically, these have consisted of between 3-and-6- week-long, supply-driven residential courses, with students of mixed (but mostly Asian) nationalities housed on the HQ campus. The sentinel example and also the flagship international course is Rural Development Management. Other examples of current courses are: Development Communication, Systems in Community-managed Health, and Training of Trainers in Sustainable Ag. Regional Courses. These are supply-driven courses offered by AFR and LAC to students of multiple countries from those regions. Many regional courses echo HQ curricula, but they tend to be non-residential and shorter. Some cross-site examples during the MG are: Farmer-led Extension, Participatory Rural Appraisal, and Strategic Management and Organizational Development, as well as regional versions of Rural Development Management. National Courses. These were grossly defined to the evaluator as supply￾driven courses offered at a supra-community level to nationals of a single country, on largely the same topics as the first two categories. For AFR and LAC, most such courses have been held in the focus countries of Ethiopia and Ecuador during the MG. [A LAC example is referenced in Box 4.3.] Customized Courses and Study Programs. These seems to be defined more by their funding mechanism than any other parameter. IIRR often provides tailor-made, in-service training as per client requests and special contracts. USAID-Matching Grant Evaluation Report 36 Usually, these demand-led "customized" courses are variants on, or modular selections from, other IIRR course content. [For one example, see Box 3.2.] Study Programs consist of customized but residential courses held at HQ. They generally involve small, GO groups. Some MG examples are study programs for: DFID staff from Bangladesh (a focus country) and India on project management; senior MOA officials from Bangladesh on farmer-led extension (Arevalo 1998); local GO administrators in the Philippines on facilitating participatory planning; SIDA community-health workers from Vietnam; and UNDP/China hirees on poverty alleviation. Community-level Courses. These consist of training conducted in communities and/or with GROs like farmer co-ops or women's groups. Often, this term seems to reference "projectized" training offered as an IIRR contribution to a joint endeavor in which other partners see to more biological/technical, research, or other implementation tasks. An example from each region is: • under the Learning Our Way Out Project (LOWO) in Ethiopia, training local men and women leaders to facilitate and take notes on discussions with people in their communities on the sensitive topics of family planning and reproductive health (Anonymous 1999); • in ASI, training Filipino village health workers, with provincial and municipal health workers plus traditional midwives as resource persons, under funding from the Philippines Region IV Department of Health; • training farmer promoters/extensionists in LAC in practical technical advice on sustainable agriculture. Peer Training. IIRR has employed some innovative forms of peer training, especially at the local level. One is the pasantía, elaborated by LAC in its Farmer-to-Farmer (F2F) program. It consists of IIRR-sponsored farmer internships, in which a farmer goes for a week or two to live and work with a farmer in another community who is experimenting with or demonstrating a useful ag or NRM technique. Another is the LOWO peer counseling mentioned above. Taxonomically, the foregoing categories span both formal and non-formal education. From the evaluator's point of view, however, IIRR also provides informal education in ways it does not always explicitly recognize. Writeshops. As noted in Chapter 2, these embody a strong hands-on training component. Writeshoppers learn this participatory method "by doing"; novice authors gain writing experience; scientists may learn to write for other audiences; and in AFR and LAC, local-hire or PVO/NGO editors, artists, and secretaries who participate in writeshops reportedly receive significant OJT from IIRR writeshop leaders. USAID-Matching Grant Evaluation Report 37 Conferences. This category is included for two reasons. One is that any good conference (or workshop or seminar) leads to conferee learning. The other is that, in the evaluator's opinion, conceiving of IIRR-sponsored conferences as a form of training is the most logical and efficient way to tally and integrate conferee data into an M&E system with coherent, Institute-wide indicators. Many IIRR training topics have been adumbrated above. However, Table 3.1. provides a rough overview of content across the baseline and MG periods as reflected in course titles. Note that the categories of general, biological/technical/sectoral, participatory methods, and institutional capacity-building are evaluator-imposed, as an analytic heuristic. IIRR appears to have established no curricular categories. Table 3.1. IIRR Courses 1992 - 1995 1996 – 1999 Africa HQ/ASI LAC COURSE CONTENT BY CATEGORY Africa HQ/ASI LAC General ✓  Rural Education and Reconstruction ✓  Integrated and People-Centered Development  Integrated Conservation and Development ✓ Biological/Technical/Sectoral ✓ ✓✓  Bio-intensive Gardening ✓ ✓  HH Food Security through Home Gardening ✓ ✓ ✓✓  Regenerative/Sustainable Agriculture ✓✓ ✓✓  Integrated Pest Management ✓  Community-based Integrated Watershed Management ✓ ✓  Food Security through Watershed Management ✓✓✓  Buffer-zone Core Group Development ✓ ✓  Agroecological Systems and Technologies ✓ ✓  Low External-input Rice Production ✓ ✓✓  Aquaculture ✓  Alternative Animal Healthcare ✓  Agriculture Commercialization and Information Systems ✓  Sustainable Agriculture Policies ✓  Small and Micro-enterprises  Gender Analysis in Agriculture, Forestry and NRM ✓  Home Health Care ✓ ✓  Family Planning ✓  Systems in Community Managed Health ✓  Indigenous Knowledge ✓ USAID-Matching Grant Evaluation Report 38 1992 - 1995 1996 – 1999 Africa HQ/ASI LAC COURSE CONTENT BY CATEGORY Africa HQ/ASI LAC Participatory Methods ✓  Community Organizing ✓✓ ✓  Farmer-led Extension ✓✓✓ ✓✓  Participatory Rural Appraisal ✓✓  Participatory Research ✓  Participatory Needs Assessment ✓✓  Participatory Monitoring and Evaluation ✓✓ ✓  Participatory Research in Community Medicine  Participatory Planning ✓ Institutional Capacity Building  Development and Management of a Training Program ✓✓  Training of Trainers for Training Program Development ✓  Training of Trainers in Sustainable Agriculture ✓✓  Training of Trainers in Gender ✓ ✓  Gender/Gender Planning ✓ ✓  Conducting a Feasibility Study ✓✓  Proposal Writing and Project Planning ✓  Project Design and Proposal Writing ✓✓  Logframing ✓✓ ✓  Project Management  Management of a Cooperative ✓  Micro-credit Management ✓ ✓  Financial Management/Sustainability ✓ ✓✓  Planning and Management of Rural Development ✓  Rural Development Management ✓✓ ✓  Leadership ✓✓ ✓✓  Strategic Management and Organizational Development ✓✓✓  “Systematization” ✓ ✓  Development Communication ✓ 3.1.2. Policies Supply-driven courses are not all necessarily offered every year on a regular schedule. Rather, they are keyed to subscribership, thus contributing to their cost￾effectiveness. Reportedly, however, not all units (notably HQ/ASI) have instituted the wise policy of collecting significant, non-refundable registration fees to avoid revenue losses to undersubscribed courses. An Institute-wide policy on this may be needed. That said, for course development and planning, staffing, cash advances, logistic matters, etc., across 1997-1998 the HQ training head developed a recruitment tracking system and exhaustive guidelines for improving internal management of training (e.g., Espineli 1998). These thoughtful and detailed materials were shared with all regions. But they did not find their way into the Program Procedures Manual USAID-Matching Grant Evaluation Report 39 (IIRR/HQ May 1998) elaborated during the MG in an effort to systematize policies and procedures Institute-wide and clarify unit roles across the now-far-flung organization. Compared to publications, regional training seems to have received greater HQ support. As per the MG proposal's budget, a portion of MG monies were spent on travel for HQ trainers to assist in AFR course start-up, and for AFR staff to attend international courses at HQ. Foiled by the dearth of Spanish-language skills at HQ [Chapter 5], LAC received no such HQ support. Still, de facto policies and procedures for training are fairly coherent across the regions, although they naturally differ by training category. AFR has set a conscious policy of targeting PVO/NGO "clusters," with the latter defined geographically, sectorally, or institutionally, as convenient. LAC has arrived at much the same policy in practice [e.g., see Box 3.3.]. But as with the publication/communication unit, an overarching set of training policies and oversight mechanisms is lacking -- most painfully so for M&E. 3.1.3. Approaches and Procedures IIRR can be justifiably proud of numerous key elements that characterize its in￾trainees'-eyes excellent training approach worldwide: • professionalism of trainers; • sensitivity to the needs of each particular group of trainees and, relatedly, a flexible approach to training techniques and methodological content; • a highly participatory stance, not only in the participatory methods taught but also in classroom interactions and the design of teaching materials and activities; • incorporation of hands-on field experiences and field-based case studies and R&D materials; • immediate practical applications in trainees' own work of the lessons, methods, and approaches learned; and • a perspective that is at once transnational yet adds "a touch of the South" [in one FG member's words], as versus more the more constrained perspectives of training institutes in students' home countries/regions or in the North; • considerable post-course follow-up. In the space of even a detailed evaluation, it is difficult to convey the depth of IIRR commitment to these principles. Likewise for the breadth of trainees' appreciation of them. But they are attested over and over again in at least four significant evaluation datasets. One is an ad hoc interview by the evaluator with the 12 students in a Rural Development Management class at HQ. They included men and women hailing from Bangladesh, Cambodia, Ethiopia, India, Indonesia, Kyrgyztan, and Laos, where they worked for PVOs and NGOs. A review of their responses to the query, "What are the USAID-Matching Grant Evaluation Report 40 strengths of IIRR training?" confirms all the above-listed principles. Trainees' only substantive critiques centered on a desire for more region-specific case materials (in this instance, from Africa and the Newly Independent States). A small but representative sampling of their commentary on IIRR training is: "It takes a professional approach, even in minor things." "It responds to people's real needs, and also teaches how to assess these needs." "It models true, in-depth participation, not shallow or pretend participation. The training method itself is participatory." "It provides a model of training that can be readily applied in our own countries." Second was a one-on-one interview in AFR with a senior manager of Farm Africa/Ethiopia and Chair of the Forum for Sustainable Land Use in Ethiopia and Eritrea who has taken a number of AFR courses. His comments paralleled HQ trainees'. In particular, he praised AFR for "not just talking about bottom-up and grassroots approaches, but really living them and showing others how to use participatory methods." He also appreciated AFR's close course-to-job links, such that trainees work on reports and proposals of their own during training, and leave with completed products as well as greatly improved skills at such tasks. This individual further described how after AFR and/or HQ training, his colleagues were all able to give home-institution courses in the subjects they had studied, using IIRR materials and approaches. A third dataset is Annex B's candid FG commentary on the high quality, relevance, and practicality of IIRR training regionwide. Reader are urged to study these data, as a majority of the 41 FG members were IIRR alumni. FG commentary on weaknesses/recommendations reflected mainly a desire for still more of the same principles enunciated above (e.g., more field-based materials and experiences) plus greater access to IIRR training (e.g., through scholarships, less expensive modular courses, more trainers). Fourth are mini-case-studies that emerged during or after the FG and other interviews, some of which are "boxed" below as illustrative of concrete and verifiable training impacts. Such mini-cases also illustrate the kind of qualitative, impact￾oriented M&E -- as versus merely monitoring process indicators -- that, to its own detriment, the whole of IIRR has failed to do. Contributing to the overall excellence and hence to (what impressionistically appears to be) the positive impact of IIRR training are a number of astute procedures. One is the maintenance of a well-run and efficiently organized two-person Learning Resource Center, which the evaluator toured and also tested by making use of its services and which, in 1999, became an accredited depository library of the ADB. Others include the following. • As noted above, thoughtful guidelines exist to direct the entire training process, from course conceptualization through delivery and final reporting. USAID-Matching Grant Evaluation Report 41 • Detailed course curricula, modular lesson plans, and objectives are drawn up beforehand, with client input. • For course design, piloting, and delivery, IIRR typically collaborates with other organizations such that, while IIRR contributes its expertise in training techniques, partners contribute resource persons in specific biological/technical or sectoral areas and sometimes also fieldsites for trainee study visits. Collaboration also helps with trainee recruitment. • Trainers conduct daily, participatory course evaluations to which they respond the next day, as well as systematic final evaluations. • Annually, IIRR reviews all course evaluations and trainer experiences for dynamic re-design, omission, or addition of curricula. • There is ongoing and extensive staff review and critique of all new and revised course curricula, lesson plans, and objectives. • Most formal IIRR education features a well-designed end-of-training action plan for all trainees to complete. • As noted earlier, all IIRR units do trainee follow-up, and some have also instituted Alumni Associations [Section 3.3]. As with publications, IIRR vigorously markets its training services, particularly in and for HQ/ASI. However, a puzzling evaluation finding is that, with the exception of LAC, IIRR rarely uses its own publications in training. LAC has functionally integrated these activities by producing manuals on many of the same topics featured in its courses, as well as in its field projects and TA [Chapter 4]. In this way, LAC ensures that it has appropriate training materials on hand and that it sells more books! Other units would be wise to adopt this strategy. 3.2. Progress and Trends As with Institute publications, when it comes to sheer numbers of training events, across all units and regions IIRR exceeded all its DIP targets (Table 3.2.). In the aggregate, targets were surpassed by 51%, i.e. 151 training events were held as against 100 targeted. If informal training (writeshops and conferences) is factored out -- since it was not expressly targeted in the DIP -- then this percentage is 29%. Also impressive is the >5,000 persons trained. USAID-Matching Grant Evaluation Report 42 Table 3.2. Numbers of Training Events (and Participantsa ) Unit/Region and Type of Eventb Baseline Output (and No. Pax) MG Output DIP Target Target Variance Africa Formal Informal 36 (929) 1 (37) 45 (958) 4 (177) 23 -- +22 -- Asia/HQ Formal Informal 119 (1,871) 25 (897) 90 (1,467) 21 (738) 49 -- +41 -- Latin America Formal Informal 21 (521) 1 (66) 116 (1,678) 4 (176) 28 -- +88 -- Total 203 (4,321) 280 (5,194) 100 +151 Source: Annex D's Table D-3, "IIRR Training and Related Data," compiled by IIRR at the evaluator's request. a No targets for numbers of participants were set in the DIP. b Here, "formal" incorporates all data on courses and study programs; "informal" captures those on writeshops and conferences. "Non-formal" training data presumably are incorporated in the next chapter's discussion of field projects. DIP targets were set only for formal training. Also bear in mind that this taxonomy of educational formality was elaborated solely for the evaluation analysis. As a 77-year historical overview documents (IIRR/ETD n.d.-c), IIRR training has been anything but static. Important changes can be detected between the baseline and the MG, and even during the MG. Two evident cross-regional trends are: away from very general courses [Table 3.1.] and towards participatory methods and institutional capacity-building skills; and at least for AFR and HQ/ASI, away from teaching primary technical skills and information -- such as farming techniques or medical procedures -- toward higher-order courses such as training-of-trainers (TOT) in sustainable ag or design of community healthcare programs. Other trends vary across regions, given differing development needs plus other factors like the Technical Specialist Group (TSG) at HQ [Chapter 5]. For instance, TSG influence may help explain why HQ/ASI has increased its biological/technical/sectoral offerings while AFR and LAC have moved toward participatory methods and institutional capacity-building. The latter emphasis is probably more consistent with IIRR's vision of itself as an intermediary PVO with an unwavering grounding in people-oriented development USAID-Matching Grant Evaluation Report 43 based on self-help, and with the Institute's avowed MG focus on NGO strengthening -- where, moreover, IIRR would seem to have a comparative advantage vis-à-vis other PVOs or national and international research and training centers that are better equipped and staffed to deliver in-depth courses on biological/technical subjects. Given a 22% slippage between the baseline and MG in HQ/AIS enrollments, IIRR may want to do a finer analysis of the trends suggested by Annex Table D-3 and then re￾examine its HQ/ASI offerings, especially in light of other emerging options. The latter include certificate and degree training plus accredited field experiences (Box 3.1.). Box 3.1. IIRR to Offer Accredited Training In 1998, ASI signed a 5-year MOU with the University of the Philippines' Open University to explore collaborative initiatives to train rural development workers. Possible activities are: cooperation in writing proposals, mounting field projects, and field-testing rural-development training programs; participation of UPOU students in IIRR field and research projects as part of their degree studies; joint elaboration and delivery of open-learning and distance-education courses, drawing upon IIRR's in-depth experience in community organizing and other rural development arenas; joint delivery of accredited short-courses leading to postgraduate degrees; and for these and other purposes (course marketing, information exchange), IIRR participation in the UPOU's network of Southeast Asian universities. Good candidates for accreditation (and thus increased training revenues) are IIRR courses for Filipino health professionals, who are required to upgrade their skills regularly by taking government-accredited courses. IIRR is also planning to link with the Global Partnership Program of the US's School for International Training, which already includes national NGOs such as BRAC in Bangladesh and ORAP in Zimbabwe. This link would allow MS students (including many from the South) to pursue credit-earning coursework and fieldwork in conjunction with IIRR training and other activities. In return, IIRR would garner additional "student personpower," fresh R&D perspectives and networks, intercultural inputs, and perhaps some new funding sources. Discussions are underway for similar linkages with Cornell University, Michigan State University, and the University of Wisconsin-Madison's Study Abroad Program. During the MG, HQ/ASI began holding some of its international courses offsite in other Asian countries. This strategy captures new audiences who cannot afford travel to the Philippines; and it provides more relevant field experiences. Perforce conducted collaboratively, some logistic and payment problems have arisen in coordinating with offsite partners; but HQ has now written guidelines to forestall such problems (IIRR/ETD n.d.-b). USAID-Matching Grant Evaluation Report 44 AFR is notable for its move toward more customized and/or modular courses. Indeed, the AFR director reports that in 1999 about 45% of all AFR training was customized. Based on commitments for 2000, AFR monitoring data show a fourfold increase in this category since 1995. This trend is exemplified in Box 3.2. Box 3.2. Responding Sensitively and Flexibly to PVO/NGO Training Concerns When AFR conducted a Rural Development Management course in Ethiopia in 1995, the national PVO/NGO umbrella organization there (the Christian Relief and Development Association) witnessed the high quality of AFR training. At first, CRDA worried a bit that AFR might supplant its own role as a provider of training and TA to its 150 member organizations. In response to this concern, AFR worked with CRDA to identify its in-house needs for educational materials and TOT. In the end, CRDA hired AFR to support it in these respects; and AFR agreed not to offer any competing courses in-country. With MG scholarship funds, AFR sent several CRDA trainees to some AFR and HQ TOT and facilitation courses in 1995. Thrilled with this training, across 1997-1998 CRDA itself began sponsoring staff from its member organizations to attend IIRR courses. Further, in 1997 it contracted AFR for a customized TOT course to be delivered to its members in-country. In 1998, this was followed by: AFR's delivery of two other customized trainings to CRDA partners; a contract for AFR to give three modular courses -- on Organizational Development (OD), Strategic Management, and M&E -- to senior managers of CDRA organizations across 1998-1999; and spontaneous formation of an OD Network by trainees in that module. In 1999, CRDA ordered a TOT course in sustainable ag from AFR; and AFR invited CRDA staff to participate in its workshop on assessing training impacts. For 2000, CRDA plans to contract another round of the modular courses, and AFR intends to produce formal training manuals for them. Thereafter, CRDA may be able to deliver such courses itself. Another baseline-to-MG trend in AFR has been away from mass training at the community level. This had been the norm before an IIRR office was opened in Africa, when work there centered on agroforestry and B.I.G. training in Ghana. As per the MG proposal, AFR has shifted to training clusters of PVOs/NGOs, in the process building strong and polyvalent partnerships with them. Box 3.3. offers another example in addition to Box 3.2.'s. USAID-Matching Grant Evaluation Report 45 Box 3.3. Training Builds Polyvalent Partnerships as Well as PVO/NGO Capacities A strong partnership has emerged between AFR and CARE/Kenya. It all began in 1993 when CARE sent some trainees to a B.I.G. course (delivered jointly with ActionAid). Pleased with the training approach, in 1994 CARE contracted AFR to conduct a 10-day, in-service version of its Rural Development Management course for 30 senior CARE managers. The following year, CARE asked AFR to facilitate its upcoming SP workshop, which AFR did in 1996. During the workshop, CARE identified capacity-building for local NGOs as a priority. So in 1997, CARE contracted AFR to do an organizational diagnosis of 18 NGOs in its Basic Education Fellowship (BEF) Project. The diagnosis revealed a need for fundraising skills, which led to an AFR workshop on this subject for all CARE/Kenya partners in 1998. In 1999, CARE turned to AFR for further capacity-building (this time for itself as well as its partners) in facilitating NGOs SP. Also in 1999, CARE had AFR lead it in a writeshop to document 29 BEF cases throughout Kenya and Tanzania. The writeshop produced the handsome volume Accessing Quality Basic Education through Fellows and Partners (CARE/Kenya-Tanzania-Atlanta 1999). In 2000, CARE/Kenya plans to use IIRR services to: deliver TOT training and develop a TOT manual using CARE cases; head up the midterm evaluation of CARE's Girl Child Project; lead CARE in revising its participatory capacity assessment tool for partners; advise it on developing an NGO financial management guide; and facilitate CARE's own longterm SP. Other outcomes of this growing partnership have been requests for AFR services from other CARE branches (e.g., in Zambia) and PVOs (e.g., Compassion International) as well as contact with CARE HQ in Atlanta, thus opening the door to possibly global IIRR-CARE partnerships in future -- especially given a similar history of collaboration between ASI and CARE/Bangladesh and also between LAC and CARE/Ecuador. In these relationships, CARE has turned into a regular, repeat "customer" for ASI and LAC training: staff from one CARE project refer those of another CARE project to ASI and LAC courses; alumni return to take additional classes; and CARE now confidently recommends IIRR training to its own and other PVO/NGO staff. A uniquely AFR trend is to train and mentor PVOs/NGOs and also GOs in gender analysis and planning. This content responds to the field projects in which AFR is involved, as well as the general need in Africa for greater understanding of gender in development. Indeed, according to HQ interviewees, AFR has become the Institute's "gender center of excellence." That said, AFR process data reveal that female course attendance has stagnated across the years. Although trainee gender lies outside IIRR's manageable interest, the HQ training unit has documented how female attendance increases when scholarships are available. Given its strong USAID-Matching Grant Evaluation Report 46 program focus on gender, AFR should consider establishing a women-only scholarship fund for its supply-driven courses. Along with the usual donors, organizations such as the US's American Association of University Women and counterpart groups in Europe might be willing to contribute to such a fund. Finally, AFR is considering adding a course in participatory environmental impact assessment and one or more courses in Appreciative Inquiry (AI) along the lines of BHR/PVC's Global Excellence in Management (GEM) Project (Anonymous n.d.-a&b, GEM Initiative 1996, Liebler n.d., Watkins and Cooperrider n.d.). The latter idea grew out of gratifying experiences with this participatory planning tool during AFR's 1999 SP meeting. AFR and GEM are now discussing how the latter might school the former in AI. Based on evaluator interviews with GEM clients in Africa, it seems the market for AI there is huge. AFR would be well-advised to pursue this idea, as might other IIRR units. Turning to LAC, in some ways it has gone in the opposite direction from AFR. LAC moved from zero community-level training during the baseline period to heavy involvement in this category during the MG. Indeed, this move is what accounts for Table 3.2.'s surprisingly large training figures in LAC. The shift is linked to the nature of LAC field projects, which emphasize farmer training and extension. Box 3.4. describes one type of LAC community-level training and, equally important, how LAC has astutely integrated training with publications and field projects. USAID-Matching Grant Evaluation Report 47 Box 3.4. Cost-effective and "People-effective" Community Training A recent case study indicates that the 140 farmer facilitators (120 men, 20 women) trained by LAC's F2F project have spread new ideas in ecologically sound ag to a total of 1,344 male and female farmers, in nearly equal gender proportions. The project has now held a total of 91 training events in 80 communities of highland Ecuador on 37 agroecological subjects, many of them drawn from local/indigenous knowledge. Farmers say they like the training because it is "given in our language, is practical, and helps us improve our lives." For example, one 13-year-old girl whom the F2F program taught to grow and market medicinal plants now pays for all her own clothing and educational costs; and in the F2F spirit, she graciously shares her knowledge with others who visit her on pasantías 'farmer internships'. Likewise, farmer facilitators report that they have won new respect and recognition within their families and communities. As some have mused to LAC staff, "We never thought of ourselves as having something to teach others." At least 30 farmer facilitators have gained renown for their practical farming and training skills beyond their home communities. GROs, NGOs, and LAC itself now hire them as "experts" for F2F and other events. Most F2F training is by community request, rather than supply-driven. Many requests flow through the 8 GROs and local NGOs who have affiliated with the program. Two of the 8 have incorporated it into their 10-year plans; and one has already established a team of 9 volunteer farmer-facilitators to carry the program forward independent of IIRR. These represent the first steps toward Phase III of the F2F program, in which farmer-led training will be institutionalized in GRO and NGO Farmer Agroecological Schools, with "schools" merely consisting of several trained farmer-experts in a community who receive and train other farmers at their homes and in their fields. [Phase I consisted of the field research for producing the agroecological manual described in Chapter 2.] All this has been achieved during the space of the 3-year BHR/PVC MG. Moreover, with only 2 LAC staff assigned to the F2F and with farmers and NGOs/GROs bearing most of the few expenses involved, each training event cost LAC only some $100 to $200. Beginning mainly during the MG and thanks to its support of IIRR decentralization, an enriching new training trend has begun: to wit, the increased globalization and cross-fertilization of courses, content, and cases across HQ and the regions [Box 3.5.]. USAID-Matching Grant Evaluation Report 48 Box 3.5. Cross-Fertilization in IIRR Training AFR started its training program with a version of HQ's flagship course. For the first three offerings (1995, 1997, 1998), an HQ trainer came to Africa to assist in course delivery and content modifications. As a result of this close HQ-AFR interaction, the trainer was able to feed fresh content and case studies back into the HQ course, further internationalizing it. HQ was the leader in organic gardening topics [Chapter 2], which AFR and LAC have since adapted and folded into their training on, respectively, food security and regenerative ag. But in 1994 LAC led the way in offering the first IIRR training concentrating on proposal writing, which AFR subsequently integrated into its program to good effect [Box 3.8.]. With its F2F program, LAC was also the leader in farmer-led extension courses for farmer leaders, which are now offered by all IIRR regions. AFR was the first region to identify a need for a Community-based Watershed Management Course. An HQ technical specialist was called out to Africa to help design and direct this course during its initial offerings in 1996 and 1998. He found the experience so valuable that, drawing upon his own long-time expertise in the subject in Asia as well as Africa, he mounted the first such international course back at HQ in 1999. 3.3. Alumni Associations: A Two-way Tool A signal feature of IIRR training is the pains it takes to sustain contact with and services to its PVO/NGO students. This intent is stated again and again in the MG proposal, which speaks of: following-up trainee action plans, facilitating communication and cross-site visits among IIRR alumni, promoting peer learning among clusters of organizations whose members IIRR has trained (AFR); carrying out regular follow-up interventions, engaging alumni as trainers in future IIRR training, and facilitating home-country alumni networking (ASI); and actively supporting trainees in applying the lessons and skills they learn from IIRR (LAC). This is not just rhetoric, as evidenced in appreciative FG commentary [Annex B] and other evaluator interviews. As one AFR FG member stated in no uncertain terms, "Were it not for IIRR's follow-up policy, I would not have taken the trouble to be here today." Indeed, in a country like Kenya -- where PVO/NGO workers expect to be paid "sitting fees" for attending any outside meetings or conferences -- gratis FG participation itself constituted testimony to the esteem in which alumni hold IIRR training and follow-up. AFR alumni were also impressed by AFR staffers' taking the time to visit them in their home institutions and fieldsites. And paraphrasing from alumni in the LAC FG, "I have never known IIRR to turn away a phone call or visit [from a former trainee or partner organization]" and "They [LAC staff] are always USAID-Matching Grant Evaluation Report 49 available to help when one...needs advice, support, or a critical eye; this is a natural part of IIRR's workstyle." In other words, IIRR training does not stop when trainees walk out the classroom door. With its longer history, HQ has evolved a more formal follow-up mechanism in the form of country- or region-based IIRR Alumni Associations (N=7, including one for eastern Africa). Every HQ course concludes with an Alumni Affairs Session that explains the aims and advantages of such associations: information sharing and development networking, including exchange visits between IIRR and trainees' organizations; joint creation and/or distribution of publications; collaborative training, field, and other projects; and for IIRR, impact evaluation of its training. In furtherance of these aims, every HQ trainee leaves his/her course with a handout on the alumni program, an alumni card, and a tidy booklet with full contact￾information for, and an individual color-photo of, every other participant in his/her course. Graduates also receive a "chain letter" form, asking them to contact IIRR within a year to report on their activities and whereabouts. Once every few years, HQ mails out an Alumni Update Form and then redistributes updated alumni contact information to all concerned. And annually, alumni receive HQ's attractive alumni newsletter, International Sharing, with articles on new IIRR courses and publications, activity reports from alumni, and so on. While all such efforts are laudable in terms of ultimate training and development goals, they are also institutionally smart. In truth, IIRR gets as good as it gives from alumni follow-up. Tangible Institute benefits from such continuing contact include: • alumni's referral of new trainees [in fact, this was the way many students interviewed in the above-referenced Rural Development Management course found their way to IIRR]; • access to the mailing lists of alumni's home institutions, for advertising IIRR training and other services -- as AFR now does with CRDA; • consequently, repeat "group" business from alumni's and affiliate organizations who "come back for more" (especially customized) training and also TA -- for example, upon return to his home institution in Ethiopia, the Farm Africa manager of Section 3.1.3. promptly began negotiating with his HQ to commission a series of modular AFR courses [also recall Boxes 3.2. and 3.3.]; • use of alumni organizations' field sites and projects for providing hands-on field experiences to subsequent IIRR trainees; • joint proposal-writing between IIRR and trainees' organizations, leading to new and mutually strengthening collaboration in training and other activities; and • a strong network of PVO/NGO/GRO partners that can be called upon in time of need -- as when a dozen CRDA member organizations voluntarily wrote to the Ethiopian government in support of IIRR's legal registration there. USAID-Matching Grant Evaluation Report 50 In light of all these benefits, particularly in the new regions of AFR and LAC IIRR should look for as many ways as possible to deepen its training follow-up. Essentially, AFR and LAC are already doing this; but they might consider adding some more formal linking mechanisms to their now-largely-informal ones. Moreover, the HQ training unit and HQ management should actively backstop AFR and LAC as well as ASI in trainee follow-up [see Section 4.3.1.]. 3.4. Lessons and Recommendations 3.4.1. Lessons An obvious lesson for other PVOs that deliver (or are interested to deliver) training is the value of alumni follow-up. Hence that subject's separate treatment above. A related lesson is that, with such follow-up, training can be the springboard for strong and growing partnerships in many kinds of activities among many kinds of development organizations. The foregoing analysis also points up the virtue of integrating training with publications and field projects, as LAC has done. This builds strong, from-the-bottom￾all-the-way-up stakeholdership in a unified program that furnishes technical information and training materials while constructing and modeling the multi-tiered organizational infrastructure necessary for successfully implementing, replicating, and scaling-up training lessons. Other organizations -- and indeed, other IIRR units and regions -- would be well-advised to emulate this astute strategy. Neither would the evaluator hesitate to recommend IIRR's general training approach to other PVOs, since IIRR training embodies the fundamental principles of effective adult education. Indeed, whether at HQ or in the regions, IIRR would be an outstanding mentor to other BHR/PVC PVOs in the area of training. Beyond the foregoing observations, however, it is difficult to draw more universal yet concrete lessons from IIRR training, for at least two reasons. One is the understandable diversity in training categories and content across regions. Illustrating from the two poles, HQ/ASI is moving towards international and national accreditation options while LAC is zeroing in on grassroots training. Yet each of these stances is institutionally and regionally appropriate in its context. The other reason is IIRR's failure to gather systematic outreach and impact data on its training, even though there seems little question that IIRR training has had and is having immense spread effects and positive impacts (although HQ enrollments have begun to slide). But at present, these outcomes cannot be credibly documented much beyond the information presented in this report. Indeed, the evaluator bent every effort to suggest ways in which IIRR can demonstrate the effectiveness of its training, USAID-Matching Grant Evaluation Report 51 beyond just process-level "body counts" -- although these, too, could be greatly improved [Chapter 6]. Several ways to assess training impacts for PVO/NGO capacity building are suggested in Boxes 3.6. through 3.8., by researching simple questions like the following. Is training effective enough that trainees can enunciate concrete changes in their work and/or their PVOs/NGOs as a direct result? Also, can trainees successfully pass along their IIRR lessons? And can some tangible impacts be detected using a simple measure like, say, PVO/NGO funds garnered as a result of training? Box 3.6. Effective Training is Put Straight to Work One case of real NGO capacity-building as a result of IIRR training was discovered in the AFR FG. This was the Nairobi Parents of the Deaf Blind Self-Help Group (NPG for short), a small partner NGO of CARE/Kenya. CARE supported two NPG staff through a series of IIRR courses that covered topics like SP, project design, proposal writing, fundraising, and organizational management. As a direct outcome of this training, the NPG has since: held its first strategy planning meeting; mounted a stakeholder and networking workshop; scheduled a team-building retreat; designed 3 projects and written proposals to fund them; developed several manuals for NPG operations, including one on financial management; documented some of the group's successful project work with CARE's Kenya Girl Child Project in a format that can be presented to other development organizations and donors; and drafted an M&E plan. Whew! As the NPG FG representative summed up, "In fact, we actually need[ed] them [the courses]" and the IIRR "training was very timely for our needs." USAID-Matching Grant Evaluation Report 52 Box 3.7. Effective Training Replicates Itself As a result of HQ/ASI's longtime training in the technology and extension of regenerative/sustainable ag, many alumni now teach such courses themselves in their own countries, sometimes under the aegis of an Alumni Association. An example is BARRA, the Bangladesh NRRM, which has begun delivering sustainable ag courses in situ with no IIRR assistance. With occasional TA from IIRR, BARRA has also taken over delivery of all Bangladesh-based training in sustainable rural livelihood technologies and community development (two earlier IIRR courses). Meanwhile, HQ/ASI has moved ahead to focus on TOT -- on how rather than what to teach in sustainable ag -- as per its assessment of major PVO/NGO training needs in this subject in Asia today. Similarly, the express goal of AFR coursework with CRDA [Box 3.2.] is to strengthen CRDA so it can itself provide basic kinds of training to its 150 member PVOs/NGOs. Likewise for LAC's F2F program, which is now being institutionalized in Ecuadorian GROs and NGOs [recall Box 3.4.]. In fact, in some subject matters with some clients, LAC has already "put itself out of business." In the LAC FG, for instance, 3 senior managers from a major Ecuadorian environmental foundation described how they have put their lessons from LAC courses on participatory planning, logframing, and "systematization" of development results to such good use that all these methods have now been firmly institutionalized within their foundation. Moreover, they have duplicated their LAC training with their national NGO partners and with a sister foundation in another Andean country with whom they have a major transboundary environmental protection project. USAID-Matching Grant Evaluation Report 53 Box 3.8. Effective Training Wins Funds The NPG representative of Box 3.6. went on to describe how, armed with AFR proposal-writing and fundraising training, the group put their new-found skills to work writing requests for services to various GO and other agencies. Results were immediate. "The government gave us a deaf-blind teacher, and we received [unspecified additional] training from a local organization." The representative of a Maasai GRO, Dupoto-E-Maa, told a similar story. Among other things, his GRO supports boarding schools for the daughters of this semi-nomadic pastoral people. After AFR training in proposal writing and fundraising, he and other Dupto-E-Maa trainees in turn coached the headmistress and two teachers of a girls' boarding school in the essentials of proposal writing (another instance of training replication). Later, they reviewed the schoolteachers' draft proposal. To make a long story short, the school's Ksh 30 million proposal was accepted within 2 months' of submission to a donor! Now the school has money to buy more books, a few computers, and some continuing education for its teachers. Again, there seems little doubt that IIRR training is of high quality, dynamic, and in the main responsive to PVO/NGO capacity-building needs -- and thus effective. As with publications, the "copycatting" of IIRR training is one positive indicator. In fact, as with publications, IIRR training materials have been copied, modified, and used far and wide. But the reach and effectiveness of IIRR training cannot be objectively demonstrated without systematic M&E, and specifically, impact assessment. Moreover, one cannot help but wonder about the quality of courses IIRR offers in M&E... 3.4.2. Recommendations With the glaring exception of M&E, because the evaluator is convinced of the quality and effectiveness of nearly all IIRR training, many of the following recommendations essentially boil down to "Keep up the good work."  Continue to implement and, where feasible, intensify the many excellent principles, approaches, and procedures behind IIRR training, especially: • the participatory, client-responsive, and in FG words, methodologically "non-dogmatic" nature of IIRR training; • the incorporation of case studies and hands-on field experiences; • sustained post-course contact, services, and encouragement for alumni and their organizations; USAID-Matching Grant Evaluation Report 54 • collaboration with other development organizations in almost every kind of training.  As for publication/communication, establish clear Institute-wide (but also appropriately region-specific) training policies and quality controls [see text]. Then designate a qualified staffer at HQ and a counterpart in each region to monitor and enforce them.  Continue and reinforce the many salubrious training trends that have emerged during the MG, e.g. toward: • content emphases on participatory methods and institutional capacity￾building skills (including the new-to-IIRR area of AI); • accredited training opportunities; • off-campus international courses; • customized, modular, and other kinds of flexible training formats; • polyvalent partnerships that often begin with training.  However, do a finer-grained analysis of why HQ/ASI enrollments are dropping, and re-evaluate course offerings and HQ staffing accordingly.  Especially for AFR, consider establishing a scholarship fund for women.  Increase programmatic integration among training, publications, and field projects.  Promptly establish a systematic and Institute-wide program of M&E for IIRR's training program. USAID-Matching Grant Evaluation Report 55 4. PROGRAM IMPLEMENTATION: FIELD PROJECTS, TECHNICAL ASSISTANCE AND PARTNERSHIPS 4.1. Overview IIRR engages in field projects, usually collaboratively, in a "social laboratory" spirit. For IIRR, the main aim is to generate and test useful development information, technologies, participatory methods and other tools, which it can then feed back into its other activities. Likewise, good TA always results in lessons learned, especially when it involves organizations or locations new to the Institute. IIRR publications and training are thus assured greater practical development content. And the Institute as a whole wins greater credibility in the development community, as a PVO that "does" as well as publishes and "teaches." Field projects may also provide sites for field visits and practice during training. Finally, in some cases (especially LAC), field projects also stimulate IIRR's and other organizations' scaling-up of participatory methods, capacity-building tools and approaches, etc. that IIRR has tested and proved out. 4.1.1. Field Projects Tables 4.1. and 4.2. summarize all IIRR field projects during the baseline and MG periods. As the tables show, IIRR upped its direct or indirect outreach to rural beneficiaries via project work by approximately 144%, from some 9,000 to nearly 22,000, between the two periods. USAID-Matching Grant Evaluation Report 56 Table 4.1. 1992-1995 Field Projects Project and Country No. of Rural Beneficiaries Africa Agroforestry in Ghana Capacity-Building of Ethiopian NGOsa (Ethiopia) Food Security Project: Phase 1b (Ethiopia) 200 --- 800 Subtotal 1,000 Asia/HQ Children's Program Bicol (Philippines) Local Resource Management (Philippines) Low External-Input Rice Production (Philippines) Negros Food Based Income-Generation (Philippines) Conservation-Oriented Livelihood Options in the Northern Negros Forest Reserve (Philippines) Marinduque Sustainable Ag Development (Philippines) Participatory Approach to Rural Economic Development: Phase IV (Philippines) Sustainable Ag in Rice Farming Countries of Southeast Asia and Indochina (multi-country) Watershed Development in Bangarpet District, India 800 1,000 100 1,000 >900 200 2,500 200 1,000 Subtotal 7,700 Latin America Farmer Participatory Research (Ecuador) 200 Subtotal 200 Total 8,900 Source: IIRR data compiled by IIRR's data analyst and Deputy VP at the evaluator's request. a Consists of training for 12 NGOs, as per the data displayed in Annex Table D-1, "IIRR Training and Related Data." Those data are not reported here so as not to double-count them. b Phase I is titled "Family Food Security through Bio-Intensive Gardening." Phases II and III [next table] and titled "Promoting Environmentally Sound Agriculture to Combat Hunger and Malnutrition in Ethiopia," and "Food Security in Kadida Gamela Integrated Watershed and Livestock." USAID-Matching Grant Evaluation Report 57 Table 4.2. 1996-1999 Field Projects Project and Country No. of Rural Beneficiaries Africa Capacity-Building of Ethiopian NGOs (Ethiopia) Food Security Project: Phase II and III (Ethiopia) Gender and Development: Capacity-Building -- implemented with DPPC through NGOs (Ethiopia) Gender and Development: Women in Politics and Decision Making: Phase 1 Planning Grant (Ethiopia) Gender and Development: Women in Politics and Decision Making: Phase II Implementation (Ethiopia) "Learning Our Way Out" Pilot Project on Reproductive Health (Ethiopia) Microfinance Project Fund for Kenyan NGOs (Kenya) --- 6,000 ND NA 000b 200 000b Subtotal 6,200 Asia/HQ Bicol Agroforestry: Phase II (Philippines) Children's Program Bicol (Philippines) Community-Based Conservation and Development Program for the Northern Negros Forest Reserve (Philippines) Conservation-Oriented Livelihood Options in the Northern Negros Forest Reserve (Philippines) Drylands Management (India) Eel Aquaculture for Food Security (Philippines) Integrated Farming in Rice-based Ecosystem Svay Teap (Cambodia) Low External-Input Rice Production for Bicol (Philippines) Marinduque Sustainable Ag Development (Philippines) Marinduque Watershed Development (Philippines) Mekong River Resource and Capacity Building Program (Vietnam) Multi-sectoral China Program Promotion of Environmental Health and Occupational Safety and Sanitation in China's Rural School System (China) Participatory Approach to Rural Economic Development: Phase IV, continued (Philippines) Smallscale Freshwater Aquaculture for Food, Income, and Resource Conservation in Mindoro: Phase I and IIb (Philippines) Sustainable Ag in Rice-Farming Countries of Southeast Asia and Indochina (multi-country) Towards Community-Managed Health: Phase II - Towards Appropriate Community Health Model (Philippines) Village-Level Post Harvest Handling Techniques: Phase I and II (Vietnam) Water Resource Equity (Vietnam) 75 800 >500 1,900 1,800 100 500 50 200 150 120 1,500 500 2,500 >450 350 250 1,200 Subtotal 13,245 USAID-Matching Grant Evaluation Report 58 Project and Country No. of Rural Beneficiaries Latin America Improving Food Security in the Highlands of Ecuador (Ecuador) Farmer Participatory Research (Ecuador) Farmer-to-Farmer Training Program in Andean Agroecology (Ecuador) Food Production in Honduras (Honduras) IIRR/ACT Microcredit for Women (Ecuador) 100 400 1,344 200 300 Subtotal 2,344 Totalc 21,789 Source: IIRR data compiled by IIRR's data analyst and Deputy VP at the evaluator's request. a Consists of training for 12 NGOs respectively, as per the data displayed in Table D-1, "IIRR Training and Related Data." Those data are not reported here so as not to double-count them. b Indicates projects that have only just begun or been approved. c Note that for phased projects, figures are often cumulative between this and the foregoing table, in that work continued with previous beneficiaries as new ones were added. Field projects targeted in the MG are briefly discussed below by region, along with post-MG project plans (where known) as an indication of program sustainability. Note that the evaluator visited only one focus country (Ecuador) where MG-supported fieldwork was ongoing and accessible. Thus this chapter is mainly document- and interview-based. Given these limitations and with only the titles in Tables 4.1. and 4.2. as data on non-MG-targeted field projects, it is difficult confidently to distinguish any overall trends in IIRR field projects. However, the evaluator's more in-depth review of AFR and LAC projects suggests that ASI/HQ projects may be oriented to more specific biological/technical/sectoral concerns (e.g., aquaculture, rice farming). As with training, this finding may relate to TSG influence. Also as with training, a more fine￾grained analysis of ASI/HQ field projects by topic and differential impact/outreach in relation to persontime and monies expended might prove instructive for strategizing future ASI projects. 4.1.2. Technical Assistance Discussion of TA is brief and limited to MG focus countries, since TA was not an MG emphasis and few DIP targets for TA per se were set. Often, TA is distinguished from other only by its funding mechanism and short duration. In fact, much of IIRR TA has already been captured in preceding chapters. Besides assistance or collaboration in publications and training, other common types of IIRR TA are: facilitation of special workshops and seminars; needs assessments, gender audits, participatory and conventional evaluations; SP, logframing, proposal writing, USAID-Matching Grant Evaluation Report 59 curriculum development, PM&E systems, and participatory rural appraisal (PRA); some biological/technical/sectoral advising on curricula and project design; and in LAC, survey design. Especially at HQ, IIRR has sometimes labeled such work as "consulting." Several broad baseline-to-MG trends in TA by region are seen in an HQ tally of raw data provided to the evaluator. In AFR, for instance, TA fell to nearly zero as AFR established itself and began to win project funding and longer-term contracts with "buddy" partners. ASI maintained its already high levels of TA (N=53 events). LAC TA increased slightly across all the variables examined, including TA events (N=22 during the MG), agencies assisted, and countries involved. Also, LAC added multilateral organizations like PAHO/WHO, UNICEF, and the WB plus Ecuadorian GOs to its formerly PVO/NGO-only client list. It merits mention that LAC sees TA largely as a strategic marketing tool, to "show off its stuff" to new clients, whom it hopes will then become regular, repeat "customers." 4.2. Progress by Region 4.2.1. Africa AFR field projects fall under three, sometimes overlapping rubrics: gender and development, food security, and NGO capacity-building. During the MG, all AFR projects were implemented in the focus country of Ethiopia. As per the DIP, AFR participates in the LOWO Project (Gurmu et al. 1999) in collaboration with an Ethiopian family-guidance NGO, a research center of Addis Ababa University, and a US university center. AFR's contribution consists of training family planning "multipliers" as peer counselors in reproductive health and family planning. A signal LOWO achievement is that, although it has no technical intervention component (e.g., distribution of prophylaxes), across its 8-month pilot phase women's contraceptive use rose by 1/3. In 1998 AFR signed an agreement with ICCO, with SNV funding, to give gender training to clusters of some 20 GROs, NGOs, and GOs. Also during the MG, AFR won a planning grant from the Dutch Embassy to develop a project on empowering women to become more active in politics and leadership/decision-making at all levels in Ethiopia. Under this grant, AFR conducted a 1999 needs assessment of gender awareness and the implementation of government policies about women among leaders in more than 30 Ethiopian NGOs and GOs. AFR's Food Security Project promotes socioeconomic development leading to livelihood security at household and community levels using environmentally USAID-Matching Grant Evaluation Report 60 sustainable technologies. The project has two components. One is a water￾harvesting effort, begun in 1997 in collaboration with ADRA under FICAH funding (IIRR/AFR n.d.). The other is jointly implemented since 1995 by AFR and PACT under the AID-funded NGO Sector Enhancement Initiative. On both, AFR provides training and TA in technical and extension subjects. Another key AFR contribution to the latter project -- and the focus of AFR fieldwork in NGO capacity building -- consisted of assisting in the design and diagnostic application of an Organizational Capacity Assessment Tool (OCAT) to a cluster of 12 rural NGOs using participatory methods (e.g., Ibus and Geleto 1996). The OCAT revealed that SP and M&E were the two most critical capacity-building needs (IIRR/AFR and PACT 1996). Others were suitable ag/watershed management (e.g., Ira et al. 1996), gender planning (e.g., Velasco and Wodemariam 1997), and micro-credit and financial management. AFR thus implemented a series of training, mentoring, peer-learning/sharing, and audit/assessment activities on these topics with the 12 NGOs [see again Chapter 2 for training outputs]. Directly or indirectly, AFR support to food-security activities reportedly benefited some 6,000 rural Ethiopians during the MG [Table 4.2.]. It bears noting that in subsequent TA for CARE/Kenya's Girl Child Project, AFR successfully adapted the OCAT for GROs and small NGOs (IIRR/AFR Nov 1997). AFR also facilitated a CARE workshop on partnerships (IIRR/AFR Apr 1998) to clarify, among other things, how CARE might best respond to its partner NGOs' needs. AFR has developed at least two proposals for post-MG fieldwork in the focus countries: in Ethiopia, the IIRR-Whitus Watershed Approach; and in Uganda, Food Security Through Watershed Management. While awaiting word on them, AFR won funding for at least two other projects, to start in 2000: a 5-year grant from the Royal Netherlands Embassy for capacity-building among women leaders in Ethiopian NGOs and GOs; and a DED-supported GAA microfinance project fund in Kenya. The latter illustrates how IIRR cleaves to its own vision-mission-values even when tempted by fat grants or TA contracts. [Other such cases were identified at HQ.] It also documents the solid reputation that AFR is building in the region [Box 4.1.]. USAID-Matching Grant Evaluation Report 61 Box 4.1. AFR Gains Repute by Cleaving to IIRR Vision-Mission-Values Despite its youth, the AFR program has won a reputation for scrupulous project accounting and all-around honesty -- commodities that, by all reports, are in short supply in development organizations in Kenya. As a result, German AgroAction/Africa took the initiative to approach AFR, requesting that it manage GAA monies for a new microfinance fund for food security work by GROs and small NGOs in Kenya. However, AFR pointed out that "just moving money" was not consistent with IIRR's people-centered and institutional capacity-building vision. The AFR Director further noted that both the development literature worldwide and recent experiences in Kenya predict a high failure rate for micro-project grants unless they incorporate organizational strengthening and capacity-building for the GRO/NGO recipients. He thus made a counter-proposal to GAA that AFR re-design the project, building in GRO/NGO training in SP, organizational management, accounting, relevant technical skills, and networking. GAA agreed to consider this proposal if AFR would first submit to an independent outside evaluation of its entire program: staff, training capacities and quality, equipment, you-name-it. AFR gladly acquiesced, always happy to get expert input on its operations. Based on a rigorous institutional review using, among other things, a scored assessment tool, the resulting report was nothing less than glowing, with observations like the following (Bachmann 1999:unpaginated). [AFR] "conducts very good training seminars. The methods and curricula used are fully up to the international standards of similar institutions. This is certainly a strength... [AFR is] ...very functional and with a good internal communication and working atmosphere. ...[It] uses very modern management resources... The fact that the main organization comes from the Philippines can...only be regarded as an advantage, because the African staff get a chance to see beyond the end of their own nose and to be confronted with different ideas and concepts. In Kenya, where corruption unfortunately is widespread, it is also certainly better to have an international status than the status of a national NGO. [Also AFR is] very well prepared in project planning and control because this is...an important part of its present work. ... In summary...[AFR]...can be expected to be very competent..." 4.2.2. Asia ASI/HQ implemented a great variety of field projects during the MG, mostly in the Philippines [Table 4.2.]. Although the Philippines were not included in the original proposal, several projects were later scheduled for support in the DIP as per some USAID-Matching Grant Evaluation Report 62 revisions noted in Attachment 2 to Marshall 1996. No projects were mounted in the focus countries of Bangladesh and Nepal. However, considerable training, TA, and special workshops did take place in both countries and/or with Bangladeshi and Nepali participants at HQ. In Bangladesh, field-project plans reportedly did not materialize because proposals developed during the MG did not win funding. These included one on aquaculture and another on rice production. One wonders whether greater progress might have been achieved with more cutting-edge proposal topics. One TA to the Bangladeshi NGO CDA is worthy of mention. This was a 2-week PM&E workshop delivered in 1997 to 26 CDA staff in-country. Founded in 1986 by a two-time alumnus of IIRR training, CDA works with 1,200 "para-level" interest groups of 10 to 15 members as well as 350 GROs. Nepal project plans were stalled because of difficulties in winning legal recognition for IIRR there. Nevertheless, ASI did complete a number of the project￾design and supporting educational and data-collection activities scheduled in the DIP. Most notable was a 1997 workshop that illustrates: once again, the effectiveness of IIRR training and the value of alumni links; the way that many diverse elements typically come together in the evolution of IIRR projects; and an innovative approach to environmental health beyond just "the same old water and sanitation stuff," as one HQ interviewee put it [Box 4.2.]. Indeed, historically ASI has been the IIRR leader in the health sector, much as AFR is now in gender. USAID-Matching Grant Evaluation Report 63 Box 4.2. A New Take on Environmental Health In 1997, with MG support, three ASI/HQ health staff held a 4-week workshop on Community Health in Rural Development in Nepal. It was designed for senior staff, area managers, and fieldworkers of two Nepali NGOs, both of whose heads were IIRR alumni. IIRR adapted its experiences from health projects and PRA training in the Philippines and Vietnam for this effort. The two NGOs were CWDS and Rural Reconstruction/Nepal (RRN). The workshop aimed to help them integrate health into their rural development programs and, along the way, look for collaborative field opportunities with ASI. The workshop included a field-training component on PRA -- a methodology new to both NGOs. The exercise also served to gather situational data for later joint design by IIRR and RRN of an innovative Ecosystems Approach to Health Project that melded agriculture, nutrition, and disease challenge by ecozone (uplands/lowlands) in a holistic health perspective. Without any further IIRR input, RRN wrote a successful pre￾proposal to IDRC, to whom ASI had just introduced it. The subsequent RRN/IIRR joint proposal was thus better informed, and there are hopes of fresh IDRC funding for the latter. Meanwhile, the RRN has sent about 60% of its technical staff to HQ/ASI courses in the Philippines. 4.2.3. Latin America During the MG, LAC implemented two major (targeted in the DIP) and three smaller (not in DIP) field projects, up from only one project during the baseline period [see again Tables 4.1. and 4.2.]. All but one were/are being implemented in Ecuador. The first of the two major projects, Farmer Participatory Research (FPR), began in 1995. But it really got off the ground in 1996 when LAC offered its first nationwide course on this subject for Ecuadorian NGOs and GOs. Conducted in collaboration with INIAP and the MOA, the FPR project trains farmers, their NGO supporters and GO extensionists, and other groups in on-farm, farmer-managed adaptive agricultural research. Together, they learn how to select, test, and disseminate agroecologically and economically appropriate technologies for improving farmers' harvests and income. Since the first FPR course, 8 customized workshops have been offered to such groups as: a US PVO working in Peru (TMI); various GO agricultural, social, and educational agencies and projects; an FAO forestry project; and a national forestry research institute. An additional 20 organizations have received LAC TA in this participatory methodology. A second nationwide course followed in 1999. USAID-Matching Grant Evaluation Report 64 Supported by a self-sustaining CIAT endowment and in loose collaboration with an Ecuadorian university R&D center, this project scales up CIAT's participatory research model. But LAC modified the model, adding what it saw as a missing steps in the 13-step CIAT booklets: one module on experimental design, and another on data monitoring and registration. At the same time, LAC simplified the scientist-written CIAT materials and "de-Colombianized" their Spanish to make them more comprehensible to Andeans. Further, in collaboration with another Ecuadorian university, LAC produced a distance-learning version of the materials. And beginning in 1998, the university added the FPR methodology to its 3-year training program for farmer promoters. Besides training other organizations in FPR, LAC itself implements FPR training as a means of testing and refining its materials and the methodology itself. To date LAC has successfully established and supported 20 farmer-researcher groups around the country. They have experimented with a wide range of agricultural options, such as: new breeds or varieties of guinea pigs, fava beans, potatoes, and fruit trees that are better adapted to local conditions; new crops like raspberries; traditional cattle￾raising systems; organic fertilizer use; and alternative pest control techniques. Box 4.3. provides a mini-case-study of how this project works, based on the evaluator's visit to one research site. Box 4.3. Farmer Participatory Research Members of one community co-op had noted how, over time, their main potato variety had become small, ever more subject to pests and diseases, and generally "tired and tasteless." Crop earnings suffered accordingly. In fact, the co-op was raising a variety distributed long ago by INIAP that had probably eroded genetically. Happily, however, the local MOA extensionist was in LAC's second National FPR Workshop. Mindful of the co-op's concern, at the workshop he obtained a commitment from INIAP to supply 10 new clones for on-farm testing, to see if any might meet co-op criteria for an improved market-quality potato. Seven of the co-op's 24 male members agreed to conduct the tests on behalf of the co￾op as a whole. Selected by their peers as especially "serious and responsible men," they received a praticum in on-farm research from the MOA extensionist. Next, they established carefully matched comparison plots in two microecozones for the 10 varieties, along with the old one as a control; drew up charts to hang in the co-op office for monitoring varietal yields and other characteristics as evaluated by co-op members; and started a 3-year program of testing. The experimental plots were located where the whole community could easily observe them; and the 7 farmer-researchers gave regular reports on their progress at co-op meetings. USAID-Matching Grant Evaluation Report 65 After the first year (Y1) of testing, 7 of the 10 varieties were eliminated as unsuitable. At the end of Y2, an eighth was discarded. This left two (a red and a white) that met farmers' taste tests, yield demands, hardiness criteria, and market requirements. In examining the new varieties as compared to the old one, the evaluator could see they were 2 to 4 times larger and little marred by insects and disease. Moreover, the farmer￾researchers' carefully charted yield data showed significant production increases. The research group plans to invite all co-op members and other interested parties to the Y3 harvest of the experimental plots, to see the results for themselves. After that, their next step is to multiply the new varieties for sale as seed potatoes (which fetch the highest prices). First "dibs" on purchases naturally go to co-op members. As seed stocks grow, the co-op will also sell to other community members and outsiders. The second large LAC project is the F2F [recall Chapter 3] or farmer-led extension, supported by a CARE project, LWR, and SwissAid. Interestingly, this approach builds on IIRR experiences in the 1970s with farmer-scholars in people's schools in the Philippines. The F2F's principles and its impressive training numbers have already been described in Box 3.5. Based on the evaluator's visit to one F2F site, Box 4.4. gives a flavor of the positive people-level impacts that can result. Box 4.4. Farmer-led Extension In 1998, women farmers of Aloguincho Community began discussing the possibility of an F2F effort on cuy (guinea pig) husbandry, having seen the results of other F2F efforts with their husbands. Cuy-raising is a female income-earning activity and a strategic source of meat in the Andean diet. Research has shown that production of this delicious, pre-Incaic food animal can be significantly increased with only a few simple and cheap interventions such as cleaner and airier quarters, herd divisions by sex and age (to control mating, weaning, and contagious disease), better nutrition and healthcare (including medicinal herbs), and some improved breedstock -- all but the last using only locally available materials. While Aloguinchas expressed interest in these ideas, they had little experience in organizing as a group or, for that matter, receiving attention from extensionists. LAC therefore arranged for 8 women to visit farmers in another F2F community where improved husbandry methods had already been adopted. When the visitors saw the extra-large animals there and heard of cuy litters of up to 7 (in contrast to the Aloguincho average of only 2), all doubts were removed. USAID-Matching Grant Evaluation Report 66 Back home, the women came together and, after watching a training video on husbandry methods that LAC provided, set about constructing "modern" cuy quarters like those in the other community -- except that Aloguinchas figured out ways to build them more cheaply. In Feb 1999, the women received an allotment of improved breedstock (1 male and 3 females each) to start their new herds. As of the evaluator's Nov 1999 visit, the group reported they were already getting three times their former cuy production and that the animals were larger and fattened faster under the new regimen. Their families were all looking forward to "bigger plates of food," they said, once herd sizes and composition were stabilized. Meanwhile, other Aloguinchas were observing and asking the group about the new husbandry techniques and lining up to buy the progeny at double the normal price per animal! Even more striking was the real joy of achievement on these women's faces as they proudly toured the evaluator through their cuy quarters and described their F2F learnings. LAC has drafted multi-community case studies cum training bulletins on six of the F2F technologies being extended: guinea-pig husbandry, trout aquaculture, sheepraising, greenhoused organic tomatoes, smallscale irrigation, and integrated farming. Each case study presents: details of the technology in question; farmers' experiences, innovations, and evaluations of the technology; lessons learned -- social, cultural, and economic as well as technical; and photos of farmer-facilitators and their peers in training sessions and at work on the technology. Some cases also render an accounting of all training costs, such that future F2F-ers can calculate and plan for these. Due out in early 2000, this practical, hands-on process documentation will permit wider diffusion of the F2F program and its technologies, independent of LAC. An important observation about LAC's two major field projects is the way they are functionally interlinked. Validated findings from farmer participatory research are fed into the F2F program in appropriate ecozones. In the same way that LAC has tied publications into training and project work, it has also planned its major projects to be synergistic. This makes for smaller costs and bigger impacts across LAC programming as a whole. Again, other IIRR units would be well-advised to emulate these astute feedback and feedforward mechanisms, along with LAC's strategic approach to TA as a marketing tool. USAID-Matching Grant Evaluation Report 67 4.3. Partnerships: More Bang for the Buck As signaled throughout this report, IIRR takes a collaborative -- indeed, a sincerely participatory -- tack in nearly everything it does. This is one of the Institute's greatest strengths. For IIRR, the benefits are multifold: extra outlets for IIRR publications; likewise, greater bodies of development experience that IIRR can tap for substantive inputs into all the Institute's programs; potential new trainees and "customers" for IIRR services; access to fresh field sites; added financial and inter￾institutional backing for development issues the Institute identifies as important; of course, increased visibility in the development community at large; and thus potentially wider impacts for IIRR outputs. As described throughout this report and in FG commentary, IIRR partners also reap many benefits, both individually and institutionally. Partners at all levels are almost universally enthusiastic about their association with the Institute. Even those who start from a stance of jealousy or competition [recall Box 3.2.] soon come to find IIRR a supportive mentor who is always "there for them," often gratis. This is especially true in Institute relations with NGOs and GROs. However, IIRR has made little effort systematically to track and report on the outstanding quality, scope, spread effects, and impacts of its many and rich partnerships. Thus, in addition to the data already presented, this section offers one further dataset plus one case study in support of this positive evaluation finding. At the evaluator's request, Figures 4.1. through 4.3. were generated. These maps were intended to gauge the types and numbers of IIRR partnerships between the baseline and MG periods, with growing numbers of partnerships and continuing partnerships taken as a proxy for IIRR success in this regard. It should be noted that in the absence of systematic M&E (and in LAC, even a simple MIS), Figures 4.1-4.3.'s counts are probably underreported. Still, they were the best to be had during the evaluation. USAID-Matching Grant Evaluation Report 68 Figure 4.1-A. Partnerships with GROs and NGOs in MG Focus Countries, as of 1995 (N = 19) Ecuador • Abya-Yala • Centro Medico de Orientacion y Planificacion Familiar (CEMOPLAF) • Ecuadorian Network of Regenerative Agriculture • Ecuadorian Network of NGOs (CEOP) • Fondo de Inversion Social del Estado (FISE) • Foundation for Cooperatives Development (ADRAI-FUDECOOP) • Fundacion Esquel Ethiopia • Christian Relief and Development Agency (CRDA) – National Umbrella Network • Family Guidance Association of Ethiopia (FGAE) Bangladesh • Bangladesh RR Association • Community Development Association • Development Center International • Unnayan Shahagojy Team • Village Education Resource Center Nepal • Community/Welfare and Development Society • Friends Service Council • Kind Mahendra Trust for Nature Conservation • Nepal RR Association Figure 4.1-B. Partnerships with GROs and NGOs in MG Focus Countries, as of 1999 (N = 46) Ecuador • Abya-Yala • ACORDES • Centro Investigacion de Bosques Tropicales (CIBT) • Centro Medico de Orientacion y Planificacion Familiar (CEMOPLAF) • Direccion de Educacion Intercultural Bilingue de Chimborazo (DEIBCH) • Ecociencia • Ecuadorian Network of Regenerative Agriculture (CEA) • Ecuadorian Network of NGOs (CEOP) • Fondo de Inversion Social del Estado (FISE) • Fondo Ecuatoriano Populorum Progresia (FEPP) • Foundation for Cooperatives Development (ADRAI-FUDECOOP) • Fundacion Accion y Desarrollo (FAD) • Fundacion Antisana • Fundacion Esquel • Fundacion Jatun Sacha • Fundacion Natura • IBIS • Proyecto Saraguro Yacuambi • Red Agroforestal Ecuatoriana (RAFE) • Tierra de Hombres Ethiopia  Christian Relief and Development Agency (CRDA) – National Umbrella Network  Emmanuel Development Association (EDA)  Ethiopia Rural Self Help Association (ERSHA)  Ethiopian Environment NGO  Ethiopian Evangelical Church Mecanyesus (EECMY)  Family Guidance Association of Ethiopia (FGAE)  Gurage/Ireland Development Program  HORCO  Hundee  LEM - Ethiopia  Ogaden Relief and Development Organization  Oromo Self Help (OSHO)  Ogaden Welfare Association (OWS)  Rift Valley Children and Women Development Association (RCWDA) Bangladesh • Association of Development Agencies of Bangladesh • Bangladesh RR Association • CARITAS – Bangladesh • Community Development Association • Rangpur Dinajpur Rural Services Nepal • Community Welfare and Development Society • Gulmi Arghakanchi Rural Development Project • Nepal RR Association • NGO Federation Nepal USAID-Matching Grant Evaluation Report 69 Figure 4.2-A. Partnerships with PVOs in MG Focus Countries, as of 1995 (N = 11) Ecuador  ACT  CARE  CRS  Movimiento Agroecologico Latino Americano (MAELA)  World Neighbors Ethiopia  ActionAid  Adventist Development and Relief Agency (ADRA)  Canadian Food Grains Bank (CFGB)  Norwegian Church AID (NCA) Bangladesh  CARE  PACT Figure 4.2-B. Partnerships with PVOs in MG Focus Countries, as of 1999 (N = 27) Ecuador  ACT  CARE  CRS  FAO  Lutheran World Relief (LWR)  Movimiento Agroecologico Latino Americano (MAELA)  SNV  SWISSAID  UNICEF/Ecuador  World Neighbors/Ecuador Ethiopia  ActionAid  Adventist Development and Relief Agency (ADRA)  African Development Aid Association (ADAA)  FARM Africa  Food for the Hungry  Goal Ethiopia  Irish-AID  Norwegian Church AID (NCA)  Private Agencies Collaborating Together (PACT)  Redd Barna  Save the Children  SNV  SOS – Sahel  World Vision Bangladesh  CARE  PACT Nepal  SouthAsia Partnership (SAP) Nepal  CARE  SouthAsia Partnership (SAP) USAID-Matching Grant Evaluation Report 70 Figure 4.1. looks at the growth of IIRR partnerships with NGOs and GROs. Along with Figure 4.2., it is limited to the MG focus countries identified in the evaluation SOW. This figure shows that, in those countries where IIRR activities were not blocked by legal problems beyond the Institute's control (Nepal) or by misdirection of MG funds (Bangladesh, Honduras) [next chapters], partnerships flourished. LAC nearly tripled its Ecuadorian NGO/GRO links while AFR increased its NGO/GRO partners in Ethiopia eightfold. A similar pattern emerges for partnerships with the national or regional branches of PVOs, which doubled and nearly quadrupled in Ecuador and Ethiopia, respectively [Figure 4.2.]. USAID-Matching Grant Evaluation Report 71 Figure 4.3-A. Partnerships with Research and Training Institutions Worldwide, as of 1995 (N = 31) North America  Cornell University International Institute for Food and Agricultural Development (USA)  IDRC (Canada)  Institute for Development Research (USA) Latin America  Centro Internacional de Agricultura Tropical (CIAT)  Cornell University  Facultad Latino Americana de Ciencias Sociales (FLASCO)  Ecuadorian National Agricultural Research Center (INIAP)  International Clearinghouse for Covercrop Information (CIDICCO) (Honduras)  Universidad de Loja (Ecuador)  Panamerican Agricultural School Africa  East Africa RR Association  Kenya Energy and Non￾governmental Organization  Kenyan Institute of Organic Farming Europe  ETC (Netherlands)  IDS (UK)  IIED (UK)  ILEIA (Netherlands)  ITDG (UK)  ODI (UK) Asia  Agricultural Training Institute (Phils.)  Asian Rural Life Development Foundation (Phils.)  Benguet State University (BSU)  Guangxi Education Commission (China)  ICLARM (Phils.)  International Crops Research Institute for the Semi-Arid Tropics (India)  International Federation for Women in Agriculture (India)  IRRI (Phils.)  RRAFA (Thailand)  SEARCA (Phils.)  Society for Promotion of Wastelands Africa  East Africa RR Association  Kenya Energy and Non￾governmental Organization  Kenyan Institute of Organic Farming Europe  ETC (Netherlands)  FAO (Italy)  IDS (UK)  IFAD (Italy)  IIED (UK)  ILEIA (Netherlands)  ITDG (UK) ODI (UK) North America  Cornell University International Institute for Food and Agricultural Development (USA)  IDRC (Canada)  Synergos Institute Latin America  Centro de Estudios y Accion Social (CEAS)  Centro Internacional de Agricultura Tropical (CIAT)  International Clearinghouse for Covercrop Information (CIDICCO)  Cornell University  Facultad Latino Americana de Ciencias Sociales (FLASCO)  German Foundation for International Development (DSE)  IFAD/PROCASUR (Bolivia)  IICA (Costa Rica)  Instituto Nacional Autonomo de Investigacion Agropecuaria (Ecuador)  Instituto Nacional Ecuatariano Forestal (INEFAN)  International Potato Center (CIP)  PAHO/WHO  Universidad Bolivar (Ecuador)  Universidad Catolica (Ecuador)  Universidad de Loja (Ecuador)  Panamerican Agricultural School Africa  African Medical and Research Foundation (AMREF) (Kenya)  Awasa Agriculture Research Center (Ethiopia)  Baraka Agricultural College (Kenya)  CABI-Africa Regional Centre (Kenya)  Center for Communicable Disease Control and Prevention (Kenya)  CTA (Holland)  East Africa RR Association (Kenya)  Egerton University Health Centre Project (Kenya)  Egerton University (Kenya)  International Center for Research in Agroforestry (ICRAF)  International Livestock Resources Center (ILRI)  International Support Group (ISG)  Kenyan Institute of Organic Farming  Manor House Agricultural Center (Kenya)  Participatory Rural Education Methodologies & Evaluation Service (PREMESE)  Regional Land Use Management (RELMA)  Sustainable Agriculture Center for Research & Development (SACRD)  Uganda Rural Development and Training Europe  ETC (Netherlands)  FAO (Italy)  IDS (UK)  IFAD (Italy)  IIED (UK)  ILEIA (Netherlands)  ITDG (UK) ODI (UK) Asia, continued (2)  FAO-RAPA (Thailand)  Guangxi Education Commission (China)  Guanxi Normal University (China)  Hanoi Agricultural University (Vietnam)  Haribon Foundation (Phils.)  Hue University of Agriculture and Forestry (Vietnam)  ICIMOD (Nepal)  ICLARM (Phils.)  Indian Council of Agricultural Research  Indian Institute of Management  International Crops Research Institute for the Semi-Arid Tropics (India)  International Federation for Women in Agriculture (India)  International Potato Center (Indonesia)  IRRI (Phils.)  Krishi Vigyan Kendra – KVK (India)  M.S. Swaminathan Research Foundation (India)  MANAGE (India)  Mekong River Commission Secretariat (Cambodia)  National Institute for Rural Development (India)  National Institute of Plant Protection (Vietnam)  OUTREACH (India)  Pakse Southern Agriculture College (Laos)  Participatory Research in Asia (India)  PCAMARD (Phils.)  PCARRD (Phils.)  PhilRice (Phils.)  Post-Harvest Technology Institute (Vietnam)  Research Institute for Aquaculture No.1 (Vietnam)  Research Institute for Fruit and Vegetable (Vietnam)  RRAFA (Thailand)  SEARCA (Phils.)  Siliman University-Center for Excellence in Coastal Resource Management (Phils.)  Society for Promotion of Wastelands Development (India)  South Asia RR Movement (India)  SouthEast Asia Fisheries Development Center (Phils.)  Tamil Nadu Agricultural University (India)  University of Agriculture and Development (Vietnam)  UP – College of Medicine  UP – Public Administration  UPLB (Phils.)  UPWARD (Phils.) Asia  Agriculture, Man and Ecology (India)  Asian Institute of Technology (Thailand)  Asian NGO Coalition (Philippines)  BAIF Development Research Foundation (India)  Bangladesh Agricultural Research Council  Bangladesh Agricultural Research Institute  Bangladesh Fish Research Institute  Bangladesh Rice Research Institute Asia, continued (1)  Bohol Agricultural College (Phils.)  Benguet State University (Phils.)  Bureau of Agricultural Research (Phils.)  CanTho University – Center for Rural Dev’t (Vietnam)  Cavite State University (Phils.)  Central Institute of Freshwater Aquaculture (Phils.)  Central Research Institute for Dryland Agriculture (India)  Central Soil and Water Conservation Research and Training Institute (India)  China International Center for Economic and Technical Exchanges  Chinese Academy of Sciences  College of Agriculture and Forestry (Vietnam)  Department of Fisheries – Bangladesh Figure 4.3-B. Partnerships with Research and Training Institutions Worldwide, as of 1999 (N=107) USAID-Matching Grant Evaluation Report 72 Links with international research and training institutes worldwide are also important [Figure 4.3.]. They input cutting-edge technical information and development methods and thinking into IIRR programming, and thence into its partnerships with PVOs, NGOs, and GROs. Between the baseline and MG, these higher-order partnerships also burgeoned, by nearly 350%! Box 4.5. illustrates the impetus of development thinking and action that can result when all such partners pull together under IIRR aegis -- not only NGOs, PVOs, and international institutions, but also different IIRR units and regions. Box 4.5. The Power of IIRR Partnerships Participatory monitoring and evaluation (PM&E) can be viewed as a learning process that may involve many different stakeholders: PVOs, NGOs, GROs, other kinds of development institutions, donors, and government agencies. It is also useful as a tool in empowerment, conflict negotiation, capacity building, collaboration, and the elaboration of new forms of mutual accountability and governance. Yet, while many development practitioners now acknowledge the value of PM&E, few can describe how it is really done. Until IIRR marshaled itself and a number of partners to tackle this lacuna, there had been little documentation (and fewer training opportunities) on how PM&E works in practice -- its dynamics and impacts, successes and failures. In response to PVO partner demands, beginning in 1995 AFR pioneered the design and delivery of a PM&E course for managers of PVOs and later also of NGOs and local￾government agencies. Meanwhile, since its inception LAC had been designing and promoting "participatory systematization" -- a PM&E method for PVOs/NGOs to document, evaluate, and learn from their development projects and activities. By 1996 LAC had published a how-to workbook on the method in both English and Spanish (Selener et al. 1996 a&b). Based on the visibly growing desire for information and training in PM&E generally, in 1996 HQ initiated and headed up the planning for a global PM&E conference, in cooperation with three other international institutes (IDRC, IDS, and IIED) plus a number of African and Asian PVOs/NGOs.a The conference was designed to foster consultations, partnerships, and networking among practitioners and proponents of PM&E while they shared their state-of-the-art experiences via discussions, case studies, and commissioned literature reviews. Held at the HQ campus on 24-29 Nov 1997, the event was attended by 52 NGO representatives, GO officials, academics, researchers, and donor representatives drawn from 41 organizations in 27 countries. The AFR Director also attended. For many participants, the conference represented the first opportunity systematically to share their PM&E learnings with an international gathering. USAID-Matching Grant Evaluation Report 73 This multi-partner effort really "got the ball rolling" on PM&E, with follow-on outcomes like: • production and widescale distribution of 1,500 copies of the conference proceedings (IIRR, IIED, IDS, and UPWARD 1998); • publication of the literature review in IDS' working paper series (Estrella and Gaventa 1998); • publication of selected conference case studies in an issue of PLA Notes (IIED 1998); • a summary of PM&E policy implications, released as an IDS Policy Briefing Paper (Guijt and Gaventa 1998) and subsequently translated into Spanish with funding from a Latin American evaluation network based in Costa Rica; • a related conference held in May 1999 and organized in conjunction with IDS by Latin American participants from the first conference to focus in on NGO and GRO applications of PM&E plus planning for a Latin America-wide PM&E conference; • also in 1999, preparation of a PM&E tool manual (IIRR, IIED, IDS, and UPWARD 2000); • edited by 8 conferees (including IIRR's VP for Program) and supported by IDS, IIED and IIRR, a scholarly volume that deepens analysis of conference case studies and discussions (Estrella et al. in press). Within IIRR itself, beginning in 1998, HQ added a PM&E course to its offerings while AFR enriched its PM&E training with cases from the conference. AFR also embarked upon a trilogy of training materials consisting of a handbook on M&E that includes participatory elements (Schroll 1999) plus a facilitator guide and a student workbook. And LAC reprinted its popular workbook in 1998. These IIRR efforts in PM&E illustrate how the Institute as a whole forges multi-level partnerships and leverages resources to promote self-reliant, people-oriented approaches and tools in the development community at large. By pulling together diverse funding sources as well as scholars and practitioners from every type of development organization and dozens of Southern and Northern countries, and by building on its new cross-regional strengths, in the space of only a few years IIRR has directly contributed to the increased understanding and utilization of an important participatory approach worldwide. To conclude this case study, it should be noted that IIRR garnered outside funding for the PM&E conference plus preparation and publication of the proceedings and the in-press "tools" volume, including coverage for all IIRR persontime expended. In like vein, AFR and LAC training and LAC publications in PM&E pay for themselves. In other words, partnerships are an extremely cost-effective way to achieve widespread results and impact. a PVOs/NGOs included Oxfam/Hong Kong, Sikiliza International (of Uganda), and from the Philippines, UPWARD plus the PRA Network represented by Kaisahan. USAID-Matching Grant Evaluation Report 74 4.4. Lessons and Recommendations 4.4.1. Lessons As we have seen throughout this report, everywhere IIRR operates, it has built strong and diversifying partnerships with key development, research, and training institutions whose capacities it can both strengthen and complement. In turn, partners provide many benefits to IIRR, as enunciated and illustrated above. Clearly, it pays for PVOs to partner. In field projects as in other arenas, often IIRR partnerships got their start in training [recall Box 4.2.] bolstered by various forms of follow-up, later evolving into collaborative efforts. Thus, a lesson for other PVOs is that, even when staffing and funding may be short, it is wise to build up and on organizational "buddy" links, giving strategic partners "extras" that make a PVO stand out as different from the crowd. Even IIRR could improve upon its stated goal of nurturing partners in several cost￾effective ways. A handful of ideas in this vein follow, with the hope that other PVOs might also find some of them suggestive. • Share IIRR annual reports with key alumni and partners, along with a once￾yearly, Institute-wide newsletter 6 months after annual report issuance, thus making for twice-yearly contact in print. The newsletter could consist of a new, regionally integrated version of the International Sharing (which in any case should be re-tooled to reflect IIRR's shifting focus from HQ to the regions). The newsletter could then also feature IIRR-partner collaborations, whether in training or other activities. That is, make the International Sharing a cross￾functional newsletter that serves multiple aims. • Hold an annual partner appreciation and networking day in each region, featuring displays and presentations on IIRR collaborations with alumni and their organizations. • Highlight some of partners' project work and publications on the IIRR website, and intensify and perhaps formalize what are now mostly informal collaborations in distributing and promoting one another's publications and services. • Give "buddy" organizations token courtesy copies of new publications from various IIRR units and regions. • As per FG suggestions, build a donor database and share it with partners. A second lesson -- and one that IIRR itself needs to learn better -- is that bigger impacts can be achieved by a PVO as a whole when its different units all pull together in the same direction, as per the PM&E case. Put another way, partnerships across what can all-too-easily become semi-autonomous and fractious units and regions can USAID-Matching Grant Evaluation Report 75 "pay off" in many of the same ways as those with other development organizations. They can also make for PVO efficiencies. To take a semi-hypothetical example, AFR is planning a trilogy of M&E manuals; meanwhile, LAC has issued a workbook on one aspect of M&E and HQ has published a conference proceedings and a tools manual; and all three units offer courses on the subject. That being the case, then with one region taking the lead, might not all (or most) contribute materials and experiences to produce a unitary, Institute-wide product that could do at least double-duty in training and best-practice dissemination? From LAC come a couple of pragmatic lessons: the view of TA as a marketing and advertising tool that can turn one-shot service delivery into a growing partnership (a strategy also reportedly deployed by ASI); and the cost- and programmatic￾effectiveness of interlinking the substance of field projects insofar as possible. 4.4.2. Recommendations Recommendations largely follow from the above lessons but also from FG commentary and the case studies, plus findings concerning institutional structures and administration [next chapter].  Intensify the trend to mount field projects and "social laboratories" collaboratively in all regions, and consider making a policy against unilateral field implementation, barring very exceptional funding circumstances.  Relatedly, choose partners for field collaboration very strategically, such that they contribute the necessary biological/technical/sectoral or research interventions and competencies while IIRR focuses on what it does best (and probably with the most impact): participatory methodologies and people-centered capacity-building through training, organizational development, extension, and so forth.  Analyze the relative substantive impacts and other payoffs of field projects in which IIRR's contributions center on one or the other of the two foci enunciated above (biological/etc. versus methods/capacity-building) and strategize future projects accordingly.  As per FG suggestions, more systematically include local government and local representatives of national government as partners in fieldwork.  Administer all field projects through the regional centers.  Look for ways to design field projects (and for that matter, all activities) synergistically, not only within but also across regions. USAID-Matching Grant Evaluation Report 76  Create incentives to stimulate greater partnering and sharing across IIRR units and regions, as this can contribute to more efficient, informed, and globally relevant outputs Institute-wide.  Particularly in AFR and LAC but also at HQ and with HQ support, further strengthen links with alumni and with "buddy" organizations via added but cost￾effective mechanisms [see text for some ideas].  With regard to TA, do away with the term "consulting" as inappropriate to IIRR's vision-mission-values and also as a conflating category for M&E purposes.  Continue, and perhaps codify, IIRR's unwritten policy of accepting only TA assignments that are congruent with its vision-mission-values and its primary expertises.  Also assess TA assignments from the viewpoint of marketing and advertising opportunities that can lead to larger, longer-term relationships and, ultimately, new partnerships.  Promptly establish a systematic and Institute-wide program of M&E for IIRR field projects, TA, and partnerships. USAID-Matching Grant Evaluation Report 77 5. INSTITUTE AND PROGRAM MANAGEMENT 5.1. Overview To assess changes in Institute and program management under the MG [see Annex A's SOW Item B], it is helpful first to review the many planning processes IIRR has engaged in across the past decade, along with major changes in management and institutional structure [Table 5.1.]. Table 5.1. Planning and Management Milestones 1989 Conceptual creation of AFR, ASI and LAC Regional Offices at the Philippines HQ, under the administration of the Department of International Training and Outreach. July 1990 Based on nearly a year's prior strategic planning (SP), finalization of the first-ever IIRR 5-Year Plan. July 1992 Resignation of Dr. J. Flavier (protegé of the Institute's deceased founder) after 17 years as IIRR President to become Philippines Secretary of Health. Aug. 1992 Formation of an interim management team composed of the BOT Chair as Acting President, VP for Program, and VP for Management. Nov. 1993 Installation of Mr. J. Rigby as IIRR President. Late 1993 Launching of an SP resulting in a Seven-Year Program Scenario and Goals (IIRR/HQ 1993b), embodied in a 22-point Program Scenario by Year 2000 (IIRR/HQ 1993a) and a new institutional structure [see Figure 5.2.]. June 1995 Official resignation of HQ Finance Director in order to pursue other professional interests, with IIRR encouragement. Late 1995 Some institutional downsizing. Early 1996 Elimination of Chief Financial Officer (CFO) position in NY; appointment of a new Finance Director at HQ; and transfer of virtually all financial management to HQ. Late 1996 Resignation of VP for Management; no replacement made. Late 1996 Appointment of an energetic new BOT Chair. Feb. 1997 Week-long SPa session, resulting in a 10-point Action Plan (IIRR/HQ Mar 1997) based on the SP. June 1997 Establishment of a personnel unit at HQ. Oct. 1997 Preparation of a preliminary business plan (Shutt Oct 1997). Dec. 1997 Reorganization and downsizing of IIRR [see Figure 5.3.], but also creation of a Deputy VP position. USAID-Matching Grant Evaluation Report 78 1997-1998 Intermittent overseas leave of the Deputy VP for some 9.5 months, to pursue doctoral studies. 1998 Addition of new BOT members with extensive finance, management, and international-development experience. May 1998 Completion of a Program Procedures Manual (IIRR/HQ May 1998) to reflect the 1997 SP and reorganization. June 1998 Development of a 1998 Strategic Overview and Plan (BOT 1998). Sept. 1998 Development of a Medium-term Plan Framework (Gonsalves and Killough Sep 1998). Dec. 1998 Completion of an external Development Resources Audit and Evaluation Report (JBL 1998) as part of business plan production. Dec. 1998 End-of-contract of Rigby as IIRR President. Jan. 1999 Installation of Dr. P. Kale as IIRR President and launching of a year-long SP.a Sept. 1999 Official resignation of HQ Finance Director in order to pursue other professional interests, with IIRR encouragement. a Dubbed "action planning," this 1997 event focused on only one aspect of strategic planning (institutional management). The year-long 1999 SP was a comprehensive exercise that spanned every aspect of Institute structure and functioning. For semantic simplicity, however, the text refers to both events as "strategic planning," from each of which a number of more precise action plans emanated. As Table 5.1. shows, in addition to or in lieu of its normal Institute-wide annual workplanning, IIRR conducted strategic/action planning in: 1990; 1993; 1997 (as per DIP Item 24), accompanied by a reorganization and additional planning in 1998; and all of 1999, to be followed by another reorganization shortly. The decade also saw major changes in senior program and finance managers and structures at IIRR. For example, the departure of President Flavier and the advent of a new and different management style marked a major institutional milestone. For some longtime staff, this was an emotionally charged event that triggered considerable internal factionalization well into 1999 (AIM and IIRR 1999:II-7). Nevertheless, the 1990s rang in the greatest change in IIRR's 70-some years of existence: transformation from an Asia-only organization into a global one. As preceding chapters have suggested, with IIRR's simultaneous globalization and decentralization, a much-needed Institute-wide re-vitalization appears to be underway. Most of it is directly attributable to BHR/PVC via: MG support to the 1997 and 1999 SPs; GEM support to the latter; and especially the MG seed money that made possible AFR's and LAC's real "takeoff." Conducted using the Blondin Organizational Development Model, the 1997 SP was attended by a broad cross-section of IIRR staff plus 4 trustees. It resulted in new vision, mission, and goal statements for 1997-2000 and the following "statement of IIRR's Root Cause Problem": USAID-Matching Grant Evaluation Report 79 IIRR has serious, but solvable, financial, structural systems and management deficiencies, and inadequate integration, coordination and communication between [sic] all people within IIRR, that significantly constrain IIRR from attaining its full potential (IIRR/HQ Mar 1997:1). To address this problem, the 1997 SP identified 10 "action priorities" (IIRR/HQ Mar 1997:3) closely paralleling the DIP's Purpose II activities [see again Chapter 1]. Each priority was accompanied by an outline of actions for addressing it, designation of an Action Team to implement them, and a timeline for so doing. The priorities are paraphrased below, with their corresponding DIP activity number in brackets [see again Annex B]. 1. New organizational structure [31] 2. Clarification of staff roles, responsibilities, and decision-making processes [26] 3. Human resource development [27, 34, 35] 4. A budget system [25] 5. Improved financial management and institution of a financial information system (FIS) [25] 6. Clarification of governance issues [33] 7. Resource generation [23, 28] 8. External networking and collaboration [30] -- already discussed in Chapter 4 9. Documentation and publication of IIRR experiences to share lessons learned [32] 19. Demonstration of impacts [29] The 1999 SP is charted in Table 5.2 and Figure 5.1. It was funded mainly by the Ford Foundation and the Rockefeller Brothers Fund, with some additional MG support. AIM and GEM were contracted to lend qualitative data-gathering/analysis and facilitation TA, respectively. The final week-long SP session in Nov 1999 also included a 2-hour presentation of preliminary findings by the USAID evaluator, who observed most of the session. USAID-Matching Grant Evaluation Report 80 Table 5.2. 1999's Year-long Strategic Planning Late 1998 Discussions with GEM about its assisting the SP. Jan-Feb. SP pre-planning and fundraising by the IIRR President. March Week-long "plan to plan" sessions with AIM and GEM TA to do SWOT and AI analysis, respectively, and produce a "roadmap" [Figure 5.1] for the rest of the year's SP work; Institute-wide feedback session; 2-day staff AI workshop; and more. April-May Collection of data for SP: historical/situational analysis, "environmental scanning," internal assessment, staff interviews, external stakeholder surveys, financial data, BOT and AIM/GEM feedback -- all compiled in an overview document (AIM, GEM, and IIRR 1999). June First SP session to: "unburden" staff about IIRR problems; review data collected; adopt new vision-mission-values and goal and strategy statements; and establish 9 "theme teams" to prepare reports and issue papers for next SP session in Nov. Jul-Oct Preparation and dissemination of SP progress report, including a draft strategic plan and draft strategy formulation outputs (Killough Aug 1999); theme team work. Oct. Week-long AFR pre-SP session in Ethiopia, with GEM TA and 2 trustees. Nov. Final, Institute-wide 1.5-week SP session, with AIM and GEM consultants, 5 trustees, and USAID evaluator. Dec. Distribution of Draft Strategic Plan Document (IIRR/HQ Dec 1999) incorporating various action plans; and start on drafting a 1-year operating plan and a 3-year medium-term plan. USAID-Matching Grant Evaluation Report 81 Discovery Dream Design PROCESS Initiation of Strategic Planning Process IIRR/AIM/GEM Strategic Planning Partnership March 99 Environmental and Internal Assessments March - June Roles, Functions and Process Agreed Organizational Strengths Opportunities Identified Review of Assessment Results June Possible Futures… Dreams Explored Strategic Planning Workshop June Image Vision/Mission Goals Dialoguing Validation June -November Strategic Approaches Refined Operational Planning Workshop November Action Plan Social Architecture Destiny/Delivery Image / Vision / Mission / Goals Strategic Plan 2000-2025 Medium Term Operational Plan 2000-2003 Feedback / Monitoring / Evaluation / Annual Review and Revision Development Issues Analyzed Check Values, Philosophy, Culture Stakeholders Internal and External Reality Testing O U T P U T S Social Architecture Alternatives Explored Strategies Elaborated Implementation Continuous Learning Figure 5.1. 1999 Strategic Planning (March-November) Main Planning Steps and Intended Outputs USAID-Matching Grant Evaluation Report 82 Involving all IIRR staff from drivers and cafeteria workers up to trustees, the 1999 SP re-evaluated every aspect of Institute structure, functioning, staffing, and funding. It sought: to review IIRR's past experience and strengths; to explore opportunities for the future in the changing contexts and challenges of development; and to formulate IIRR's statements of vision, mission, and long-term goals; to define a 5-year program strategy within the framework of the Institute's vision, mission and long term goals; and to prepare a 3-year medium-term plan for the years 2000-2002 (AMATech 1999:2). Roughly following 1997's 10-point Action Plan, this and the next chapter evaluate progress toward DIP objectives from the 1997 SP to the end of the 1999 SP. The chapters draw on nearly all the evaluation methods described in Chapter 1. One aspect of IIRR SPs must be underscored. As signaled in Table 5.2., they tend to rely on relatively little analysis of quantitative data on Institute activities and impacts. In fact, the only such data found in SP documents were financial (cf. AIM, GEM, and IIRR 1999; AIM and IIRR 1999). In the absence of an Institute-wide M&E system with impact as well as process indicators and someone to analyze them, even available FIS data cannot be fully appreciated. This is the management information context in which IIRR strategizing and decisioning take place. 5.2. Progress and Trends 5.2.1. Organizational Structure As noted in Table 5.1., the 1997 SP reorganized IIRR's ponderous institutional structure [compare Figures 5.2. and 5.3.]. The 5 Program Divisions were deleted and their substantive foci wisely consolidated into 3 programmatic thrusts spread across both functional and regional units [recall Table 1.1.]. These moves allowed IIRR to focus and integrate its programmatic emphases much more tightly, thereby rationalizing financial and, to some extent, human resources. USAID-Matching Grant Evaluation Report 83 President Regional Offices Program Divisions Functional Divisions Administration Africa Asia Latin America South Asia Environment, Natural Resources & Agriculture Community Health, Reproductive Health & Nutrition Rural Enterprises Development Institutional Capacity Building Training Communication Research Administrative Services Finance US Office Integrated Community-Based Rural Development Gender and indigenous knowledge are cross-cutting themes in all IIRR activities Africa Executive VP for Management Executive VP for Program Figure 5.2. Organizational Structure as of 1995 USAID-Matching Grant Evaluation Report 84 At the same time, however, TSG was created. It housed some of the former Division Chiefs and other restructured personnel. This group appears to have replaced the former Research Division. It is supposed to contain "'world class' professional[s] with strong technical... specialization[s]" plus other skills standard for international development experts. Currently, TSG has 5 professionals specializing in aquaculture, community forestry, enterprise development, nutrition, and water resources. Among many other things, TSG responsibilities include "leading the field operational research functions...by providing professional and technical inputs into IIRR's field-based activities, which are under the management of Regional Directors" [a clear potential for role and responsibility confusion here...] and "providing guidance in trainings and workshops" Institute-wide (Rigby 1997:Annex 2, unpaginated). In sum, TSG is a sort of Board of Trustees President US Office Executive Office • Personnel and Human Resources Management Unit • Public Awareness Unit Office of the Vice President-Program • Vice President-Program • Deputy Vice President-Program Education and Training Unit Technical Specialist Group • Finance Department • Logistical Support Unit Publications Unit Latin America Regional Center Africa Regional Center Asia Regional Center Figure 5.3. Organizational Structure as of 1999 USAID-Matching Grant Evaluation Report 85 research and technical consulting unit to the rest of the Institute. Its members also sell their consulting services out-of-house on IIRR's behalf. Within IIRR, TSG is perceived as and verbally dubbed "an elite." Personnel-unit data unit indicate that TSG-ers enjoy a disproportionate share of professional￾development and conference-travel opportunities. Yet as of this evaluation, only 2 of these 5 "world class professionals" were PhDs, and some only BAs. All but two are Asians, with little experience outside Asia. And none speak Spanish. To date, TSG assistance to research or field projects and training/workshops outside HQ/ASI has been minimal. The main exception is assistance to AFR by the TSG water-resources specialist (a PhD African) [recall Box 3.5.]. TSG's minimal support to AFR and LAC is hardly surprising. The travel costs to AFR or LAC of bringing consultants from the Philippines are prohibitive given that equally qualified consultants are available locally. These are individuals who, moreover, speak the local language(s); know the peoples and ecologies involved; may be more familiar with AFR or LAC programming than any TSG-er; and may have important ties to other development organizations in the region. Still other questions arise about TSG's utility. For example, what is the cost- and program-effectiveness of maintaining an in-house consulting group when qualified consultants are also plentiful in Asia and the Philippines; when IIRR takes a strategic partnership approach to most of its activities [recall Chapter 4]; and when ASI is moving into "new" countries? A further problem has been TSG's overshadowing of ASI self-management and growth [see below and AIM and IIRR 1999:II-17]. Also created in the 1997 restructuring was the position of Deputy VP for Program. A former Division Chief was promoted into it assigned myriad frontline responsibilities, e.g.: heading TSG, coordinating HQ and regional operations, overseeing budget and finance plus the finance and personnel units, ensuring attention to human resource development, seeing to installation of an MIS, conceptualizing and implementing an M&E system, and still more. Concurrently, the VP position was sharply redefined as one of mainly networking, fundraising, and general institutional liaison plus program ideation, thus removing all day-to-day managerial functions from it (Rigby 1997:Annex 2 unpaginated). Another SP outcome was acknowledgement of ASI's problematic (almost virtual) existence as a de facto appendage of HQ (Rigby 1997:6). Various solutions were proposed but apparently poorly implemented since, at the time of this evaluation, according to many knowledgeable accounts, by comparison to AFR and LAC, ASI still had limited functional autonomy vis-à-vis HQ. This is why preceding chapters were largely unable to evaluate ASI activities as distinct from other HQ units' on a programmatic, structural, or management par with AFR and LAC. This structural peculiarity reportedly came about for several reasons. USAID-Matching Grant Evaluation Report 86 For one, largescale projects contracted in Asia under the pre-regional structure were not readily transferrable managerially to what was at first a skeleton ASI Office [recall Table 1.2.]. Relatedly, desegregating financial data on these projects and then budgeting them separately under ASI aegis initially proved technically (until late 1998) and managerially infeasible because of an obsolete HQ FIS [Section 5.2.4.] compounded by inadequate finance-unit leadership and a general inattention to finance/program links at the top two levels of senior management [as per reliable reports from various financially savvy internal, external, and BOT sources, as well as the acting ASI Director.] For reasons beyond IIRR's control, the Nepal and other ASI country offices did not materialize, blocking ASI's decentralization plans. In any case, it became evident that it was economically more sensible to house ASI at HQ, where there was much surplus office space -- so much that IIRR is also (astutely) able to rent offices to a Philippine-German grantmaking foundation associated with GAA, which is a longtime IIRR partner. However, this propinquity to HQ invited ASI's continued domination by top HQ program personnel and TSG, which the former ASI director and the current acting director reportedly were unwilling or unable to combat. It bears noting that, unlike AFR and LAC, ASI has never had a PhD at its helm. Although ASI is still floundering, the 1999 SP may at last have won real, as versus rhetorical, consensus that ASI must operate in the same mode as AFR and LAC. This SP also saw a further streamlining of what staff aptly described as IIRR's still "convoluted and confusing" structure (AIM and IIRR 1999:II-5). A flatter, more modern￾style structure has been designed for 2000 [Figure 5.4.]. In the new organigram, the HQ publications and training units are bundled together along with various campus￾based logistic functions into a center of their own, retaining the name "Yen Center" in honor of the Institute's founder. If the evaluator understood aright, this center will form the nexus of non-region-specific activities, and will be expected to earn its way on the same footing as the regional centers. At the same time, the latter are assigned their proper standing as the Institute's real "center of gravity," from which all field operations will now be managed (IIRR/HQ Dec 1999). Relatedly, as per evaluator recommendations, the TSG is done away with, in part by assigning some of its personnel to HQ or the regions. USAID-Matching Grant Evaluation Report 87 Figure 5.4. Proposed Organizational Structure as of 2000 5.2.2. Staffing The 1999 SP did not determine the precise "social architecture" (i.e., staffing) for Figure's 5.4.'s organigram, although some broad outlines were sketched for major management positions. Staffing decisions were postponed until after the 1999 holiday season. But judicious staffing is critical to reaping the benefits of the new organizational structure and of truly shifting IIRR's center of gravity to the regions. Otherwise, most of the 1999 SP will have been for naught. Numbers, Distribution, Skills. "Judicious" naturally means taking into account: sheer quantity of staff and their distribution relative to managerial, program, and support needs; gender balances; staff skill mixes and qualifications (including language abilities) vis-à-vis position demands; and still other variables. In the first regard, IIRR planning documents make reference to downsizings in 1995 and 1997. But as Table 5.3. shows, between the baseline and MG periods, staff numbers in fact grew. Growth was appropriate for adding much-needed technical staff in the regions [see Annex D's Tables D-2 through D-4] but not for HQ. However, the Institute did gain some flexibility in staffing by halving the number of professional hires on regular/permanent contracts. Nor did its near-equal gender ratio suffer in the process. Also, the staff mix became more international [Table 5.3]. Board of Trustees Headquarters US Office Asia Regional Center Yen Center Latin America Regional Center Africa Regional Center USAID-Matching Grant Evaluation Report 88 Table 5.3. 1995 and 1999 Staffing 1995 1999 Staff Category Female Male Total Staff Category Female Male Total Professional Regular/Permanent Contractual Project-based Support Regular/Permanent Project-based 23 13 6 35 1 29 16 8 14 1 52 29 14 49 2 Professional Regular/Permanent Contractual Project-based Support Regular/Permanent Project-based 14 17 7 27 8 12 17 15 30 6 26 34 22 57 14 Totals 78 68 146 Totals 73 80 153 Nationalities Nationalities American (8), Argentinian (1), British (1), Canadian (1), Chinese (1), Ecuadorian (5), Ethiopian (1), Filipino (121), German (1), Indian (2), Kenyan (2), Nepali (1), Nigerian (1) 25 = Non-Filipino 121 = Filipino American (3), Argentinian (1), Australian (1), Bangladeshi (1), British (2), Chinese (1), Ecuadorian (8), Ethiopian ( 10), Filipino (111), Indian (4), Kenyan (7), Nepali (3), Nigerian (1) 42 = Non-Filipino 111 = Filipino Distribution is critical when it comes to decentralization. Program realities can be assumed to follow personnel and budget allocations. In light of this principle, figures provided by IIRR's Financial Analyst suggest the Institute has a long way yet to go in shifting its center of gravity from HQ (and NY) operations to the regions. For example, from MG monies alone, expenditures on HQ+NY personnel (salaries+fringes, exclusive of 25% indirect costs) across the MG summed to $461,000 [rounded from sums of Item 1.a. in Table 5.5.]. In contrast, the totality of MG expenditures on AFR+ASI+LAC personnel equaled only $234,000, in a grant intended for the very purpose of decentralization! Moreover, according to the Financial Analyst, the whole of the 25% indirect costs applied to MG salary supports (another $174,000) was appropriated by HQ. Further analysis to determine how HQ deployed this sum (e.g., to regional as versus HQ overhead) would have been desirable. But as noted elsewhere in this report, computerized miscegenation of funds made such analysis very difficult, if not impossible, at the time of evaluation. To verify whether the above 2:1 MG ratio (exclusive of indirect costs) might have been balanced off somehow by IIRR's match, an SP financial table was consulted for comparative data for 1999 (AIM, GEM, and IIRR 1999:Year 1999 Institute Fixed Cost Schedule, unpaginated), when IIRR's MG match should have peaked. But that table shows the same 2:1 ratio in personnel expenditures. As a rough indicator of institutional "gravity," this ratio is revealing. Were the HQ-to-regions shift proposed in the MG in fact achieved, one would expect the ratio to be reversed, or at the very least 1:1. USAID-Matching Grant Evaluation Report 89 One implication of even such a simple indicator is that HQ is probably bloated, in several senses of the word. This is a particularly distressing finding given that, beyond basic administrative functions, the evaluator could find few HQ-provided services to AFR and especially LAC to justify the present staffing ratio. Several trustees (one of whom recently visited AFR) echoed this observation of HQ bloat. As he quipped, "All you really need at HQ is 10 guys, not 100." Table 5.4. speaks for itself. Table 5.4. Staffing Distributions as of December 1999a Staff Type HQ US AFR ASI LACb Total Management/Finance Only 14 1 1 1 1.8 ≈19 Technicalc 19 --- 8 22 3.5 52.5 Support 64 1 10d 6 2.0 83 Total 97 2 19 29 7.3 ≈154 Sources: Calculated from: a list of IIRR professional staff compiled by the HQ personnel unit at evaluator request; a "Head Count Report" (IIRR/HQ Oct 1999); Table 5.3; and Annex Tables D-2 through D-4, constructed on-site by the evaluator. Note that Table 5.4.'s HQ figures for technical versus support staff are subject to some interpretation by job title. Also note that the 1-person variance between Tables 5.3. and 5.4. is due to conflicting data sources. a Staff counts exclude interns, volunteers, and secondments. b LAC astutely employs staff at various parttime levels according to staffers' family-life and other choices as well as available funding. c Regional directors are included here rather than in management/finance. Although naturally they have myriad duties in both arenas, they have not heretofore been vouchsafed top management status within IIRR; and in any case, they hold a unique responsibility for the technical content and quality of regional programming. Also included here are research, training, and field assistants. d Security concerns in eastern Africa make for relatively greater numbers of guards and caretakers. One sense in which HQ may be said to be "bloated" is in its fulltime permanent hiring, with benefits, of a wide variety of low-level logistic staff. These include, e.g., clinicians, drivers, auto mechanics, cafeteria workers, gardeners, janitors, a laundress, a carpenter, etc. -- as distinct from more substantively-oriented hirees such as office, library, and computer-support personnel. HQ should investigate whether cost-savings could be realized by privatizing many of these logistic services. Besides adding fixed costs in the form of employee benefits, the maintenance of large numbers of low-level workers on-staff adds disproportionately to the burden of Institute management and accounting (both of which are already stretched thin at present). Also, personnel, cash, and inventory control problems all become compounded [see also Section 5.2.4.]. Another source of HQ bloat is TSG. As discussed earlier, TSG is supposed to serve all regions; but in fact, it services mainly only ASI, in effect operating much like a USAID-Matching Grant Evaluation Report 90 part of that center. Those TSG services that are indeed vital to everyday ASI functioning should be folded into that unit; all others can be out-sourced as needed; and any remaining TSG personnel should be re-posted or dismissed [see again Section 5.2.1.]. Further, some HQ units may be somewhat overstaffed (or alternatively, underskilled) in terms of secretarial support. This question would bear further examination, especially in the training unit. Finally, the NY office may be "bloated" not in terms of staff numbers or cost, but rather in terms of the benefits it yields for the operational expenses it incurs. [For discussion of this point, see Section 6.1.1.] The important point here is that HQ+NY bloat may translate into regional starvation. While Annex Tables D-2 through D-4 show that the MG made possible vital formative staff growth in AFR and LAC, LAC has been cruelly staff-starved, with only 2.5 professionals besides the director. By comparison with ASI, AFR is also understaffed (or perhaps ASI is over-staffed ?). But starvation is not the worst of it. Both evaluator analysis and FG commentary [Annex B] indicate that, in personnel terms, AFR and LAC are entirely unsustainable at present. FG members note that in AFR no one but the director is of a caliber to make decisions, answer questions, negotiate contracts, and train independently. Much the same is true in LAC where, moreover, only the director can operate in English. If either of these individuals were removed for whatever reason, the regional programs would grind to a halt. There is no strong Number 2 to backstop these "one man shows" or to take charge in case of director illness or job change. Also, minus a capable Number 2, both directors are wildly overworked. In short, at present both AFR and LAC are sustainable only to the extent that their present, semi-charismatic directors remain healthy and on-the-job. Happily, the draft 1999 SP document seconds the need for strong Number 2 positions in the regions. These should be designed with a mix of management skills and duties so as to double for directors as-needed. But they should also meet critical technical needs in each center, e.g., another strong trainer/leader in AFR and a combination office manager/M&E specialist in LAC. The IIRR President also lacks a strong Number 2 competent to take over the management complexities of a global (and now also bilingual) organization during its President's necessarily heavy travel to fundraise and to visit AFR and LAC more regularly. At present, the Institute's Number 3 (the Deputy VP) takes charge during the President's absence. This adds to his already-myriad duties, which naturally suffer thereby. Clearly, there is an unbalanced and dysfunctional mix of duties assigned to HQ's Number 2 and 3 positions. Moreover, the operational link that programming must perforce share with budget (not to mention the proper management of donor monies) is not uniformly understood and respected by both incumbents. Likewise for the critical role of M&E in monitoring and testing this link. USAID-Matching Grant Evaluation Report 91 Both of these expensive positions need to be thoroughly re-examined to determine what is a reasonable, appropriate, and cost-effective skill mix for each or, indeed, whether both are warranted vis-à-vis acute needs for other management experts and literally vital Number 2 regional staff. Likely the Institute does not require both positions if the President shoulders the job of seeking major grants and donations, if the Institute is now sincere about shifting program decisioning and all field operations to the regions, and if it adds some currently missing but imperative HQ management expertise. The latter include a CFO and an M&E specialist [see Section 5.2.4. and the next chapter]. One last point on staff skills now... The evaluator was surprised by IIRR inattention to language abilities as a consideration in job qualifications and Institute functioning. This cannot help but sabotage communication and coordination within a global but decentralized Institute -- as has occurred for HQ-LAC relations [below]. Only two IIRR staff are English-Spanish bilinguals (the Deputy VP and the LAC Director). Thus HQ is dependent upon a sole on-site source for all its information from and about LAC. Moreover, no one other than the director can answer an HQ phone call at LAC, much less write a report in English! This makes a single individual automatically to blame for any of the miscommunications that inevitably occur in a complex, decentralized organization. Conversely, the dearth of Spanish skills at HQ denies the rest of LAC staff any direct communication with their own home office. Just think: What if all AFR hirees but one spoke only Swahili or Amharic, and only one HQ staffer could talk to one group or the other? Yet such a state of affairs is tolerated at LAC. It must be promptly remedied. It is also one of many compelling reasons for recruiting a LAC Number 2, who must be fully bilingual. Further, IIRR should write language requirements into job announcements at both HQ and the regions; give all staff training in and access to internet translation freeware; and identify translators who can be hired on a consulting basis as needed. Roles, Rights, Responsibilities, Decision-making. As most top HQ managers readily confess, at birth the AFR and LAC Offices "were just thrown out there [in the regions] on their own, to sink or swim." Indeed, HQ was unable to produce any document prior to the 1998 Program Procedures Manual stipulating operating mechanisms and relative roles, rights, responsibilities, and decisioning power of HQ and the regions. Only the LAC Director was able to locate such a document, a 6-page Operating Guidelines for the Regional Offices (IIRR/HQ ca. 1993). The upshot of this initial lack of role, etc. definition was that, while ASI endured parental stifling, AFR and LAC suffered from parental neglect. Thus, the latter two largely went their own way in terms of setting (or not setting) operating and quality-control policies, handling personnel, managing monies, and writing proposals. With the advent of a procedures manual in 1998 and an outstanding new Institute president in 1999, HQ began to try to regain custody of its abandoned children. But by USAID-Matching Grant Evaluation Report 92 then, workstyles and role/responsibility understandings inevitably clashed. To take one simple example, squabbles arose over who paid for whose DHL to whom due to conflicting (unwritten) directives on this issue from different HQ managers. At the start of the USAID evaluation, tensions were running so high that the President and BOT were considering taking certain legal action against one regional center. At one point during the 1999 SP, among many other options, IIRR even considered closing down AFR and LAC. For their part, these two regions were toying with the idea of secession from IIRR in view of minimal HQ support and, worse, what to-regional-eyes sometimes looked like a greedy yet fiscally inept HQ's attempt to raid hard-earned regional funds [see also Section 5.2.4.]. Virtually every IIRR manager the evaluator interviewed expressed confusion, frustration, or anger over HQ-region relations. Without going into detail, suffice it to say that with some evaluator and GEM interventions and with renewed face-to-face contact among all concerned parties, at the 1999 SP there was general agreement that the fundamental trouble lay in the continuing muddle over relative roles, rights, and responsibilities between HQ and the regions, as well as in an at-once bloated and managerially weak HQ itself. To this assessment the evaluator would add far-too￾limited communication among all units. In other words, the "root cause problem" identified at the 1997 SP was still pervasive: "serious...financial, structural systems and management deficiencies, and inadequate integration, coordination and communication" (IIRR/HQ Mar 1997:1). A gratifying outcome of the 1999 SP was a renewed and seemingly sincere resolve amongst all concerned to arrive at management systems that are "effective, transparent and accepted by and responsible to the IIRR community" (IIRR/HQ Dec 1999:12). To this end, a 9-point action plan was elaborated, to be implemented by a "global management team for collective, institutional decision-making," with the understanding that regional directors will now be given their proper place as top Institute managers and decisioners. The team is to elaborate "clear, specific parameters of decision-making authority for the different management positions" with "sanctions of non-compliance with management decisions" (ibid.). Of course, it remains to be seen whether IIRR will in fact follow through on these good intentions this time around. Thus, one further step is recommended: in order to increase intra-Institute communication, understanding, program integration, and exchange of useful ideas and strategies generally, for the foreseeable future IIRR should hold an annual management-team meeting. Whether in addition to or in lieu of the Institute's "mass" annual workplanning, the locale of this meeting should rotate among the three regions. Furthermore, the first such meeting should take place in LAC, for a variety of reasons. The president just visited AFR in late 1999 but neither she nor any AFR staff have ever viewed LAC operations. LAC clearly has been the poor cousin throughout the MG, and a management-team visit there would help drive home its many needs. Finally, HQ- USAID-Matching Grant Evaluation Report 93 LAC relations are in need of repair, and an on-site meeting with all LAC documents and personnel at hand would go far in this regard. 5.2.3. Human Resource Management and Development As noted in Table 5.1., in 1997 IIRR created an HQ Human Resources Development [hereafter, personnel] unit at HQ, with all the normal functions of such units plus oversight of the HQ health clinic (Rigby 1997:21). Prior to 1997, IIRR was long without a personnel officer; personnel policies and procedures were sorely outdated in light of globalization and decentralization; and job descriptions had not been updated for 10 years. This unit and its newly appointed director were supposed to remedy this situation. The personnel director's first act was to request 3 new personnel-unit positions. The request apparently was denied, since this individual is presently the only non￾clinical hiree in the unit [but see Box 5.1.] In late 1997, he launched a laudable effort to revise and update job descriptions. An innovative approach was taken in which small groups of 10 to 15 staff of similar ilk worked together on their 8-page Individual Task/Workload Analysis sheets (because otherwise, staff simply ignored the form). But this undertaking stalled in the face of 1999's SP and expectations of a new social architecture. As of the Jun 1999 SP session, the personnel unit was "perceived to be not meeting the needs of the staff and the growing needs of an international organization" (AIM and IIRR 1999:II-11). Problems center on: • unrealistic job descriptions and "overburdening work with little compensation"; • inadequate policies and procedures for staff appraisal, reward, and compensation; • relatedly, unclear policies for promotion; • "disgust on creating new units and positions just to accommodate people" and, in the same vein, protection of "untouchables," i.e. staff whose contributions to the organization may no longer be needed or relevant but who cannot be dislodged from their sinecures; • hiring new staff despite financial difficulties; and • lack of a well-defined staff development plan and a budget to go with it (after ibid.) The evaluator independently verified the validity of each of these complaints. For instance, with regard to "overburdening work," across the nearly 1.5 months spent at HQ or in the regions, the evaluator observed excessive night and weekend workhours that by all reports (including angry spouses') were the rule rather than the exception. Evaluation of the previous MG noted the same problem (O'Brien 1994). Moreover, one manager confided that it was standard practice to bid >100% FTEs on multiple projects. USAID-Matching Grant Evaluation Report 94 Indeed, these practices appear to be one of the main explanations for the cost￾effectiveness of many IIRR program activities. In a curious way, then, the Institute appears at once over- and under-staffed (or perhaps under-skilled). This suggests an urgent need seriously to re-assess skills-to-job fits, sinecures, and "untouchables," as well as staff distributions. As to inadequate appraisal and etc. policies, it was discovered that while HQ and AFR both have well-organized procedures for annual staff reviews, their systems differ; and LAC has no system whatsoever. A new annual appraisal system has been designed at HQ, but not yet implemented. No written promotion policies currently exist, nor an employee handbook. HQ reports it is hard at work on a Human Resources Development Manual. But the only draft documents the evaluator saw were some revised job descriptions. The evaluator also identified cases of new-unit creation (e.g., the TSG) and sinecures. The latter were transmitted privately to the President and BOT. At a larger level, recordkeeping leaves something to be desired in the personnel unit. For example, the unit experienced difficulty generating the data for Table 5.3 -- perhaps because it does not keep dossiers on AFR and LAC employees other than the regional directors. Neither does it evince much awareness of personnel procedures, staff development, and other such matters in AFR and LAC. Indeed, the unit has difficulty gathering staff development information just at HQ. Overall, the personnel director appeared to assume that AFR and LAC lay largely outside his purview, echoing the disconnect in policies and oversight between HQ and the regions found for the publications and training units. In general, data-management capabilities and professional skill levels in the personnel unit appear inadequate to the job of maintaining complete, orderly, and up-to￾date personnel data, much less proactively advising the Institute on the complexities of managing an increasingly dispersed staff, with more diverse contractual statuses than heretofore, operating under many different legal systems and in at least two working languages. Also missing is the capacity to investigate current staffing gaps and superfluities, to forecast evolving staff needs, and to alert top management accordingly. Such tasks cannot await the haphazard visits of external evaluators... In short, this unit needs thorough overhauling and upgrading in order to bring it to a point where it can support a globalizing but decentralizing organization. That said, HQ/ASI and AFR are to be commended on their wise, cost-effective, and sometimes innovative use of volunteers, interns, and secondees to make up staffing deficiencies or add special skills in the short term [Box 5.1.]. In view of its starved staffing situation, LAC would do well to emulate this strategy, assuming LAC moves from its current sardine-tin-sized offices to quarters that can accommodate extra help (and evaluators). USAID-Matching Grant Evaluation Report 95 Box 5.1. Creative Staffing: A Win-Win Proposition HQ attracts professionally-oriented development interns gratis from a wide range of organizations, including volunteer groups like the American Jewish World Service or the UK's VSO, NGOs such as BRAC, the MOAs of different Asian nations, international research and training institutes like IDS, and foreign universities (e.g., in China and the US). One example of the kinds of services they render IIRR is a Canadian volunteer's updating and modernizing of the Institute's website in 1999. Other examples are editorial and MIS assistance. Since 1997, ASI has enjoyed a unique relationship with a Manila hospital, which monthly sends several community-medicine interns to do fieldwork on the ASI/LWR health program. Also, trained local residents volunteer their time at the HQ clinic as midwives, nurses, and social workers. Further, area colleges and universities send students for OJT practica at HQ. Annually, about 60 students contribute between 200 and 720 personhours each in, e.g., computer technology, basic management, marketing, accounting, electronics, and auto repair. These interns save IIRR hundreds of thousands of pesos in local labor costs AFR has taken the HQ model further. AFR accepts recent university graduates as interns in areas where it is short-staffed and provides them not only OJT (e.g., in computer softwares) but sometimes also coursework. Some of these interns have gone on to become permanent AFR staff, as did AFR's accountant. Others have applied to MA programs. AFR has been uniquely successful with another strategy: secondments from donors. For example, for the past 3 years DED has supplied AFR with M&E specialists, trained for their AFR secondment in Germany at DED expense. The first such secondee went on to become the GAA Program Officer for Southeastern Africa in Bonn. These creative staffing arrangements are a real win-win "deal." Volunteers, interns, and students gain new practical skills, sometimes along with course credits or additional formal training, plus valuable professional contacts; afterwards, they have tangible work products and references to show prospective employers or university admissions committees. In AFR, interns and secondees also get the chance to travel to new countries; but whether in AFR or at HQ, they are exposed to people of other cultures. And everywhere, temporary staffers enjoy the opportunity to contribute to important development goals. IIRR wins in several ways. The added personpower frees up more highly skilled regular staff to do more of what only they can do. Products that might otherwise languish (e.g., the website update) get done sooner. IIRR is exposed to fresh ideas, information, and perspectives. And as AFR has found, interns may turn into valuable employees while secondees may bring in "new business" down the road. For example, AFR's first DED￾supplied M&E Specialist now manages AFR's recently-won GAA microfinance project [recall Box 4.1.]. Turning now to staff development, this has proceeded apace during the MG, albeit without any organized system or budget for planning, rules for approving, or procedures for conscientiously reporting it. Beyond activities noted later in this and the USAID-Matching Grant Evaluation Report 96 next chapter, staff development included the following, as per an HQ table compiled at evaluator request. • At HQ/ASI three professionals pursued PhD and MA studies (one MA was awarded in Dec 1999); some support staff received training in selected secretarial skills; appropriate staff took short courses in library management and various sectoral and methodological topics; and TSG-ers attended numerous technical conferences and short courses. Also, a trustee trained 39 HQ/ASI staff in proposal writing. • Highlights for AFR were: two core staffers attended the flagship HQ course; new hirees in the Ethiopia Country Office took AFR courses; selected staff studied topics such as gender, good governance, PRA, and simulation games in the Netherlands; and most Kenya and Ethiopia staff trained in desktop publishing. • Aside from a special mentoring program [below] and the proposal-writing training noted above, ASI staffers reported only three activities: a certificate course on applied ethnobotany for one individual, a PACT course on facilitating organizational capacity-building for another, and GAA training in project planning and management for a third. • In LAC, formal professional development was limited to a participatory-research and a distance-education course for one staffer each. The LAC Director obtained OJT in writeshopping [recall Box 2.6] and then likewise shared his learning with two staffers. An innovation has been one trustee's mentoring HQ professionals in career development. So far, he has worked with 5 supervisory/technical staff. It is to be hoped that this program will not be restricted to HQ staff. AFR and LAC managers could also benefit from some management mentoring by knowledgeable trustees. A final word of advice to IIRR: staff development aims to upgrade skills of already valuable employees or exceptionally promising employees. It is not a remedial program. Staff lacking the basic skill mix and outlook to perform effectively in a globalizing organization should be dismissed. 5.2.4. Financial Management As Table 5.1. hints, IIRR financial management has been rocky. But some progress was made on the 1997 SP objectives during the MG. Historically, the NY Office kept consolidated books for the Institute while another set of books was maintained at HQ. However, discrepancies between the two arose, if only as a result of coordination problems. Thus, in 1996 the CFO slot in NY was eliminated and all but "residual financial functions" were consolidated at HQ. The former HQ Finance Director -- who officially resigned -- was temporarily replaced by a consultant who later (Jan 1996) was promoted into the Director position. This individual served throughout the MG until Aug 1999, when he also officially resigned. As of Dec 1999, a replacement had not been appointed. USAID-Matching Grant Evaluation Report 97 During the first half of 1996, a VSO volunteer conducted an internal financial￾information needs assessment among HQ staff, with the assumption that regional needs would be similar. Findings were reported to the BOT who then took a hard look at HQ financial reporting. They noted a profound need to improve the Institute's financial management system. Said the Chair, "The reports we were getting were mush," and greatly delayed to boot. At the time, HQ's DOS-based ledger system was malfunctioning and no one was able to repair it. Thus in Oct 1996 a consultant was hired to rationalize accounting systems, improve workflow, and reduce the lagtime in monthly reporting. In Nov 1996, with MG support the VSO volunteer was appointed Program Monitor. Her job was to advance IIRR's financial sustainability goals, improve existing program databases, and link the latter to finance [see next chapter]. In mid-1997, a trustee helped HQ select a local accounting firm to design a whole new FIS that would meet the needs of all Institute units. But the Filipino firm selected never turned up for its contract meeting. So in Oct, another trustee advised adopting an off-the-shelf software, QuickBooks Accounting and Payroll (Version 5.0). The following month, the supplier trained HQ finance staff in the software, and they began re-configuring it to meet IIRR needs such as a set of "Bluebook" reports for the BOT, public accounts for the external auditors, donor reports, and internal accounting reports for IIRR managers. Come Feb 1998, the QuickBooks reference-numbering system was changed to reflect new activity-management responsibilities under the 1997-1998 reorganization. This entailed re-coding all 1998 FIS information in both dollar and Filipino-peso files, because the software lacked a currency-conversion function. Nevertheless, by Jul the QuickBooks system was nearly completely installed, re-coded, and tested. A signal achievement was a one-month reduction in the lagtime for Bluebook reports. To speed reporting further, QuickBooks needed to be installed in AFR and LAC. (The same will be true for ASI once it gains its independence.) So in summer 1998, the Resource Development Support Officer (promoted from Program Monitor) visited AFR, LAC, and NY to assess their capacity to adopt the new FIS. With MG support, AFR was able to promote its accounting intern to accountant [Annex Table D-3], assist her in CPA studies (now nearly concluded), and bring in a new intern. In early 1999, both these individuals received QuickBooks training locally; and in Jun 1999 the AFR accountant took further training at HQ. At the same time, HQ finance staff refined the FIS and developed a manual on it tailored to AFR's specific needs. "I love love love it," said the AFR accountant to the evaluator in describing the changeover to QuickBooks. By Oct 1999, she had trained the Ethiopia Office cashier in it. AFR financial data handling and reporting have thus improved significantly [recall Box 4.1.]. LAC was unable to adopt the FIS due to its obsolete computers and its staff's inadequate knowledge of modern computer-based accounting software. But from its inception, LAC has had an experienced senior accountant. MG support allowed LAC to USAID-Matching Grant Evaluation Report 98 bring this individual to near fulltime status and add a fulltime assistant accountant [see again Annex Table D-4]. In LAC as well as AFR, accounts have been meticulously kept and all documents are well-archived. Back at HQ, in Oct 1998 the finance unit began configuring a separate FIS file for building annual budgets. This resulted in much higher-quality financial information and budget construction for 1999. For example, for the first time, IIRR was able to distinguish such variables as core versus activity costs and gross contributions to margin. This more sophisticated budget and other financial analyses were reported in the Nov 1999 SP session by the Program Financial Analyst (promoted from the position of Resource Development Support Officer). These new capabilities will give IIRR managers more integrated FIS-MIS information with which to improve their program planning. In Aug 1999 the HQ Finance Director officially resigned to pursue other professional interests, with the full encouragement of IIRR. As of this evaluation, he had not been replaced. However, in Oct a consultant was hired to recommend policies and to streamline procedures in the finance unit, and also to reconcile and collect outstanding cash advances and receivables (a nagging problem at HQ). The Nov 1999 SP session led to a decision to reinstate the CFO position, but siting it now at HQ, since it was evident that a higher skill level than that of a finance director/comptroller was required to manage IIRR's increasingly complex and diversifying accounts [next chapter] and to provide analytic financial information. The SP also accepted the evaluator's recommendation to hire an internal auditor, which AIM advisors agreed would be a prudent fiduciary move. Looking ahead, in early 2000, QuickBooks' currency-conversion deficiencies will have to be addressed. Thereafter, HQ staff plan to help LAC duplicate the resulting FIS. In 2000 and beyond, policies, procedures, sanctions, computer connectivity and equipment, etc. will be put in place to: complete integration of all IIRR units in financial+program reporting and analysis; ensure timely data submission from all units; and more systematically and transparently share financial data and consolidated reports. Also looking ahead, the evaluator recommends upgrading staff skills in the finance unit or re-configuring the staff to give it greater analytic capacity. Evaluator data requests indicated that the unit is unable independently to formulate useful analytic questions and then query financial data accordingly, using exceedingly simple measures (e.g., percentages, ratios, means). Looking back, however, management of MG monies clearly suffered from the lack of appropriate financial systems and skills, as well as from inattention to finance-to￾program linkages at the top-most levels of HQ. According to the BOT Chair, once the former IIRR President hired a finance director, he largely washed his hands of such matters; and the VP claims no interest in or understanding of them. Although a Deputy VP was appointed specifically to ride herd on an increasingly complex finance situation, USAID-Matching Grant Evaluation Report 99 he had no formal training in this area. Moreover, he was on leave during a significant part of the MG [Table 5.1.]. In any case, he was vouchsafed little formal programmatic responsibility. Thus by default, many unwritten programmatic decisions devolved to the Finance Director by virtue of his control of the purse-strings. This default was substantiated in evaluator interviews with regional directors, the BOT, and others. A sign of shortcomings in financial skills and savvy was IIRR's failure to include a revised and detailed budget in the DIP, to parallel the detailed budget submitted with the MG proposal. (However, neither did BHR/PVC note this omission). The proposal had requested $1.8 million in USAID funding for the 3-year MG, but only $1.5 million was actually awarded (Marshall 1996). In the absence of a detailed budget for the actual award, Tables 5.5. and 5.6. are presented as a basis for evaluating how MG monies were in fact expended, as compared to various statements of expenditure intent. USAID-Matching Grant Evaluation Report 100 Table 5.5. MG Expenditures by Units/Regions (US Dollars) PARTICULARS AFR ASI LAC NYª HQ TOTAL 1. Program Elements a. Salaries and Fringes b. Travel and Per Diem c. NGO Training Costs d. Country Office Costs e. Costs of TA to Partners f. Other Direct Costs g. Monitoring and Evaluation 98,047 13,227 20,847 61,911 28,452 19,413 - 78,888 53,279 3,000 2,870 - - - 116,776 10,096 - 19,636 - 1,614 - 51,174 37,005 - 4,440 - 48,469 - 409,702 49,562 11,500 482 - 30,823 1,405 754,588 163,170 35,347 89,339 28,452 100,318 1,405 SUB-TOTAL 241,897 138,037 148,122 141,088 503,474 1,172,620 2. Procurement 1,245 5,650 257 - 23,152 30,304 3. Indirect Costs (24.79%) 59,966 34,219 36,719 34,976 120,477 286,358 TOTAL 303,109 177,907 185,098 176,064 647,103 1,489,282 Source: 1 Oct. 1996 to 30 Sept. 1999 data complied by IIRR's Financial Analyst at evaluator request. ª Does not include salaries paid through NY to AFR and LAC staff. USAID-Matching Grant Evaluation Report 101 Table 5.6. MG Direct Costs as Budgeted versus Expended by Units/Regions ($000s, including procurement but excluding indirect costs) (1) (2) (3) (4) (5) HQ 361 (25%) 300 20% 276,557 (34%) 526,626 (44%) NY ---a ---a ---a 45,513 (6%) 141,088 (12%) Subtotal HQ + NY 361 (25%) 300 20% 322,070 (40%) 667,712 (56%) AFR 362 (25%) 300 30% 105,824 (13%) 243,139 (20%) ASI 360 (25%) 299 30% 181,964 (23%) 143,682 (12%) LAC 360 (25%) 299 20% 191,540 (24%) 148,379 (12%) Total ≈ 1.4 ≈ 1.2 801,398 1,202,916 Key (1) Proposed budget as indicated in Attachment 1 to the MG Proposal (IIRR/HQ Dec 1995), showing cash and percent assignments to units/regions, exclusive of indirect costs. (2) Straightlined proposal budget (from Column 1) diminished by 17% as per reduction of $1.8 million proposal request to a $1.5 million award, exclusive of indirect costs [calculated by evaluator]. (3) "Inter-regional allocation"/"estimated percentage breakdown... independently of overhead" of MG funds as stated in Message #164 (Rigby 31 Jul 1996b). (4) "Calculation of Salaries To Be Charged to Cooperative Agreement" in memorandum (Rigby 1 Jul 1996a:B-3) to Sherry Fachet, Senior USAID Grant Specialist, with the percent breakdown by region/unit of total salary charges ($801,398) as shown in Column 4. (5) Actual charges to the MG, exclusive of indirect costs, as per Table 5.5. Source: Data compiled by the evaluator based on the sources indicated in the key. a In these calculations, NY expenses were lumped in with HQ's. As Column 1 in Table 5.6. shows, IIRR's MG proposal, submitted in 1995, intended to share MG monies essentially equally among the regions and HQ+NY operations. Column 3 represents a modification from Column 1 as per intended percentage breakdowns that the IIRR President instructed NY to report to the BHR/PVC Project Manager. But at the time of negotiations with the USAID Contract Officer, this breakdown mounted dramatically in favor of HQ+NY (Column 4). Column 4's salary breakdowns were the only figures detailed by region/unit to be submitted with the DIP, instead of a properly revised and detailed budget. They assign 40% of all MG salary expenditures to HQ+NY. This percentage ballooned to 56% toward the close of the MG, as per the 30 Sep 1999 figures of Column 5 [the latest accounts available during the evaluation]. USAID-Matching Grant Evaluation Report 102 IIRR's MG indirect-cost multiplier is 25%. According to the Institute's Financial Analyst, no indirect-cost monies flowed to the regions. So, 25% of Column 5's total $1,202,916 expended (i.e., $300,729) should be added to the $667,714 already absorbed by HQ+NY in direct costs and procurement. When this is done, these two entities' share of MG monies rockets to $968,443 -- nearly $1 million of the $1.5 million MG award. In other words, only about one-third of MG funds appear to have been directly devoted to regional operations in a grant explicitly designed "To fully shift its [IIRR's] center of gravity from its headquarters to regional and country-based support initiatives" (IIRR/HQ Dec 1995:Item A7). According to evaluator interviews with both IIRR and BHR/PVC managers and inspection of records held (or not held) by each, IIRR never reported or renegotiated this major shift in the MG's financial "center of gravity" to/with USAID, despite the shift's likely vast implications for programmatic gravity. On the other hand, BHR/PVC apparently overlooked IIRR's failure to submit a revised, detailed budget with the DIP, to reflect the difference between the $1.8 million requested in the MG proposal versus the $1.5 million actually awarded. BHR/PVC also seems to have overlooked the advisability of its doing some interim MG monitoring. Of course, one can argue around the edges of the nearly $1 million appropriated by HQ+NY. For example: • Some $50,000 is documented as having gone for valid HQ staff travel to the regions, another $12,00 for training regional students and AFR staff at HQ, and some $26,000 for a temporary Program Monitor for M&E [see below]. • Presumably, HQ units like publications and, again, training lent other kinds of assistance to the regions -- although the evaluation did not identify very much such support to AFR or LAC. • Probably a goodly portion of ASI money is "hiding out" in HQ funds, as per ASI dependency on HQ/TSG. • One would also expect some increased HQ+NY transaction costs due to decentralization start-up and then continuing demand for greater investments in global information technology, translation, staff travel, international banking, and other communication and coordination needs. • Of course, a modicum of HQ+NY manager and support-staff salaries and benefits can appropriately be assigned to the MG insofar as managers properly see to their regional coordinating responsibilities. Beyond the travel, training, and monitor costs noted above, however, financial records were such that it was not possible to tease out details of what total share of HQ+NY expenses were spent directly or indirectly in support of the regions. That said, it is distressing to learn that $146,094 in HQ salaries were budgeted for HQ M&E specialists (Rigby 1996a:B-4) who were never hired, with the brief exception of the USAID-Matching Grant Evaluation Report 103 Program Monitor mentioned above. It is not clear to what use these M&E monies were instead put. Note, however, that the evaluation found no indication of fiscal fraud. Rather, the issue is HQ grant management vis-à-vis the MG's two explicit purposes [recall Chapter 1]. The suspicion (shared by the evaluator and all regional directors) is that bloated, inefficient, or overly-expensive HQ and NY operations desperate for funds may have raided regional budgets (and corresponding overhead shares) as outlined in the MG proposal. One can only wonder: how much greater might the impressive successes of AFR and LAC have been, and how might ASI have fared, had the three regions in reality received a share of MG funds commensurate with the proposal's rhetoric of decentralization? 5.2.5. Governance "Governance" refers to the BOT. IIRR currently has 21 trustees and 2 trustees emeriti. The evaluator did not attempt to gauge BOT functioning in the same pre- and post-MG manner as for other topics. However, the foregoing sections suggest that the BOT may have been somewhat lackadaisical in past, particularly in financial oversight. For instance, the BOT never noted the lack of an internal auditor at IIRR. But many findings indicate that the BOT entered a new, more dynamic era during the MG. As the former IIRR President wrote in his statement to a 1997 BOT meeting: "1997 has...been a year of substantial engagement by the Trustees in the functioning of IIRR. ...besides four sets of meetings...and voluminous communications, we have had seven visits by New York based Trustees and advisors -- coming to...roll up your sleeves...and get into the nitty gritty of the operation. ...trying to influence those functions...that affect the way you carry out your stewardship... helping...the management staff better to carry out our responsibilities" (Rigby 1997:2). Much of this engagement can be attributed to the 1996 appointment of an energetic new Chair, as well as the 1998 addition of some savvy new trustees [Table 5.1.]. Echoing the sentiments of many IIRR staff, one top HQ manager said, "He [the new Chair] has brought a new level of enthusiasm and leadership to the Board." As noted earlier, examples of hands-on BOT involvement during the MG are assistance and advice on an FIS, training in proposal writing, and a career mentoring program. Also, two trustees travelled to AFR to look at operations there and participate in AFR's Oct 1999 SP. [None have ever gone to LAC, however.] Interestingly, the new IIRR President was a sitting BOT member when she graciously accepted the presidency. At that point, she had been a trustee for at least 5 years. Thus BOT-president relations are exceptionally cordial. USAID-Matching Grant Evaluation Report 104 A concern might be that the BOT is overstepping its governance role into management. However, both the former and current IIRR presidents unequivocably write/state that this is not the case. Evaluator interviews and observations support their claim. IIRR's current BOT deserves nothing but kudos for its devotion to duty. That said, it is recommended that the BOT clarify its duties and procedures in writing. At present, only a by-laws document exists (BOT 1991). It gives "listing-style" information on committees, meeting logistics, officers, etc., but it offers no real performance guidance to incoming trustees. The excellent manuals and videos available from the National Council for Non-Profit Boards might be helpful in elaborating a more complete and up-to-date document. In the process of updating the by-laws, the BOT needs to set term limits and attendance criteria. At present, trustees seem to serve until they lose interest or become too old or ill to travel. Some individuals have been on the BOT for so long that no one remembers when they first started! Conversely, several members who have been inactive for some time are still "on the rolls." Downsizing would be a good idea too. A BOT of 20+ is not likely to be an effective action body. It could also be very costly, except for the fact that trustees are generally expected to defray most of their own travel expenses. But again, this expectation is not clearly codified. A further recommendation is that selected trustees and the IIRR President visit LAC, especially in view of thorny HQ-LAC relations recently [Section 5.2.2.]. During the Nov 1999 SP, the BOT Chair wrote most of the evaluator's recommendations into a 6-point action plan for governance (IIRR/HQ Dec 1999:11). Paraphrased, the six points are as follow. 1. Reduce the BOT from 21 to 15 active members. 2. Assure that the BOT's collective skills include capacities to provide oversight in program and financial management and also resource generation. 3. Include representation from each of the regions on the BOT. 4. Conduct BOT and committee meetings in each of the regions. 5. Establish a flexible policy for rotation and term limits for trustees. 6. Make a realistic budget for IIRR's governance function. USAID-Matching Grant Evaluation Report 105 5.3. Lessons and Recommendations 5.3.1. Lessons It is difficult to draw many lessons for other PVOs from the specifics of one PVO's management situation. However, a few general observations follow. One is that, while SP is valuable, it is possible to have too much of a good thing. At some point, an organization must move from planning to action. Otherwise, SP becomes a vitiating rather than an invigorating process. If after weeks, months, or (in IIRR's case) years of document preparation, meetings, team-building exercises, speeches from various luminaries, internal and external evaluations, and what-have￾you, little real change results, then an organization risks demoralization. Thanks to MG support, IIRR made some concrete strides in program management between its 1997 and 1999 SPs, e.g.: streamlining its structure, strengthening its regional staffing, installing an FIS, and more [next chapter]. But the 1999 SP found many fundamental issues unchanged or even worsened because of failure to act on the previous SP. Another lesson is that it is hard to do good planning in the absence of actual performance and impact data as analyzed in the context of financial data. Perhaps this is why IIRR's previous SP stumbled in some areas, such as de facto decentralization of ASI. At least the 1999 SP had the advantage of better financial data than in past. And thanks to the USAID evaluation, it has now pulled together some general program data, both quantitative and qualitative. A pair of interlinked observations is that: PVOs cannot afford top-heavy and duplicative HQ staffing, no matter how complex and decentralized they are; but by the same token, neither can they afford to have second-rate human resources in their limited but key management positions. Examples are a provincial- rather than an international-caliber personnel director, a finance director instead of a CFO, and a BA or MA instead of a PhD as a regional director (and no one at all for M&E). Scrimping on quality in key personpower is usually a false economy. On the other hand, a lesson other PVOs can certainly learn from IIRR is the value of creative staffing options for non-key positions. Other PVOs considering major decentralization may be warned that it is easy to underestimate accompanying transaction costs. These include not only things like increased travel costs, bigger communication bills, more computer equipment and software licenses [next chapter], but also language barriers, diverging workstyles in distant locales and cultures, and the need for greater management policy and administrative machinery to keep tabs on quality and coordinate disparate units [recall Chapters 2 and 3]. USAID-Matching Grant Evaluation Report 106 The importance of good financial management to good grant and program management is obvious. Less obvious, perhaps, is the need for granting agencies to do a good job of monitoring and reinforcing this link too. Finally, the idea of trustee￾mentors might prove interesting for PVO staff, along with other ways in which IIRR's dynamic BOT deploys itself in hands-on support of the organization. 5.3.2. Recommendations Problematic IIRR management makes for numerous recommendations. Because of their number, they are grouped into subcategories below. Given that the USAID evaluation was designed to form part and parcel of the final phase of IIRR's 1999 SP, many of these recommendations have already been incorporated into the Institute's 1999 Draft Strategic Plan Document. Conversely, others arose from the SP itself. Organizational Structure and SP  As per the Institute-wide strategy outlined in the MG proposal, continue and intensify all efforts to shift IIRR's center of gravity from HQ to regional and country-based initiatives.  In particular, promptly put ASI on the same structural, managerial, and financial footing as AFR and LAC.  Expeditiously implement the many other excellent decisions taken in the 1999 SP concerning Institute restructuring, e.g., flatten management structures, disband the TSG, elevate regional directors to management-team status, downsize HQ, build M&E capacity, and so forth.  Move on now from SP to action. With a medium-term plan in place and hopefully with greater quantitative and impact analysis [next chapter], annual workplanning and regular, rotating management-team meetings should suffice for SP for the foreseeable future. Staffing  Trim the fat off any HQ units that are/may be unnecessarily overstaffed (e.g., top management, ASI), close down superfluous HQ units (i.e., TSG), re-balance HQ￾to-region technical staff ratios (e.g., by out-posting relevant HQ staff to the regions), and privatize some HQ logistic services.  Hire an ASI Director of equal status to that of AFR and LAC. USAID-Matching Grant Evaluation Report 107  Beef up AFR and especially LAC staff in order to plug serious gaps or correct stopgap measures in professional-technical-managerial expertises in the two regions.  Relatedly, in order to make AFR and LAC sustainable "staffwise," immediately put in place strong and polyvalent Number 2's to the directors there, according to their pressing (but differing) management and technical backstopping needs. Evaluate the need for an equivalent position in ASI, once a permanent ASI director is appointed. NOTE: The LAC No. 2 must have native-level English-language skills, as well as some Spanish-language capabilities.  Re-evaluate the roles and skill mixes of the HQ Number 2 and 3 positions to determine whether both positions are warranted in light of factors such as the shift of program decisioning to the regions and acute management needs in other arenas, and then take action accordingly.  Re-tool personnel, finance, publications, and other HQ functions in such a way that unit heads are constantly and fully cognizant of counterpart functions in the regions and they take appropriate responsibility for supervision, support, and regular communication with regional counterparts. Human Resource Management and Development  Bring all existing personnel files and data-handling systems up-to-date and complete regional and country files that are currently missing at HQ.  Take a hard look at whether current record-keeping, data-processing, duty assignments, authority systems, and professional skills within the personnel unit are adequate for maintaining orderly and up-to-date personnel information in future, for advising on Institute-wide personnel policies, and for supporting management in its oversight of an increasingly internationalized, multilingual, and decentralized staff.  Institute-wide, standardize policies and procedures for staff recruitment, position classification, terms of employment, and promotion.  Establish, and hold supervisors accountable for, a unitary and impartial system of Institute-wide annual performance review.  Do not program >100% of FTEs, nor expect excessive after-hours staff work. USAID-Matching Grant Evaluation Report 108  Continue to take advantage of, and where possible expand, the creative use of temporary or supplemental low- or no-cost staffing.  Stipulate language requirements/preferences in all job announcements and qualifications, and ceteris paribus give priority to candidates with relevant bilingual skills. Meanwhile, train existing staff in the use of internet translation freeware and make provision for accessing other translator services as-needed.  Set guidelines for staff professional development and conference attendance and the regular reporting thereof.  Ideally, extend the trustee mentoring program to AFR and LAC.  Make greater use of staff-to-staff OJT, especially across IIRR units and regions, as a way not only to build skills but also to increase coordination and communication.  Obviously, revise job positions, descriptions, and qualification requirements in light of all personnel recommendations plus emerging social-architecture decisions; and re-staff accordingly. Financial Management  ASAP, hire a competent CFO with strong FIS skills plus an internal auditor.  Building upon the excellent work with QuickBooks, now move on toward putting in place a sufficiently powerful but cross-regionally integrated FIS that can handle the present and future financial monitoring, management, and reporting needs of an increasingly globalized Institute with a burgeoning diversity of partners and donors.  Increase analytic competencies within the HQ finance unit generally.  Provide additional training, guidance, and support from HQ to regional- and country-level accounting staff as financial monitoring and reporting needs grow more complex with the addition of new donors, country offices, etc.  Relatedly, promptly upgrade LAC's obsolete computer and e-mail systems. USAID-Matching Grant Evaluation Report 109 Other Management  Put in place the necessary management-team skills to understand and analyze the institutional implications of FIS data as integrated with data from programmatic, personnel, and other MISs, so that management is able to decision-make wisely and to report to donors, the BOT, etc. knowledgeably and accurately.  Relatedly, hold all top managers (which includes regional directors) accountable for integrated financial+programmatic decisioning and reporting.  To this end, obtain additional, professional training for managers/directors, as needed, in the areas of finance, budget, contracts, and business planning.  Thereafter, give directors the maximum authority possible over all regional operations consistent with Institute-wide quality controls as overseen by the appropriate HQ unit heads.  Based on all the foregoing recommendations and with input from all key players, write a "charter" clearly stipulating the relative roles, rights, and responsibilities of HQ versus regional directors in: supervising different functional categories of staff; managing financial matters, with clear lines for funding flows and financial communications established; and programmatic decisioning in the context of both supervisory and financial matters.  Hold annual management-team meetings and rotate them among the regions, beginning with LAC.  Again, attend to the obsolescence and other shortcomings in office equipment, systems, organization, and space in LAC, as these pose a serious drag on management efficiency and other functions there, such as clear and frequent communication with HQ.  Establish and then enforce sanctions for all staff for non-compliance with management policies and decisions.  Do not open any new country offices until all the foregoing recommendations plus those on M&E [next chapter] have been implemented. Governance  Implement the excellent 6-point BOT action plan elaborated during the 1999 SP.  Review and, as necessary, update, expand, and clarify the 1991 BOT by-laws. USAID-Matching Grant Evaluation Report 110 6. LOOKING AHEAD: SUSTAINABILITY AND M&E In discussing progress and trends, recurrent problems, innovative practices, and emerging opportunities, this report has taken a forward- as well as backward￾looking perspective on IIRR change during the MG. But two elements in particular are key to the Institute's future: financial sustainability; and systematic, Institute-wide M&E that builds and informs the finance-to-program links upon which sound SP and management decisioning for longterm sustainability must be based. IIRR has struggled mightily with both these elements. In the sections to follow, USAID readers please note that no fundraising or public awareness efforts have been supported by USAID monies. IIRR has other sources of such support, such as its Readers' Digest Trust Fund. 6.1. Progress and Trends in Sustainability A key recommendation in the evaluation of IIRR's previous MG (O'Brien 1994) was to strengthen Institute fundraising and budget processes. Per DIP Item 28, these were major subjects in both the 1997 and 1999 SP. With the present MG award, IIRR began moving aggressively on both fronts at all structural levels and across all functional categories. 6.1.1. The Institute and the Regions Throughout the MG, IIRR has pushed to diversify its sources of support as a means of increasing chances of a sustained revenue stream. In addition to intensifying its strategy of cost-effective partnering in most activities [Section 4.3], IIRR has diversified by: • strengthening relations with IIRR's primary heritage partner, the Philippines NRRM, as well as with other NRRMs -- all as recommended in the previous MG evaluation; • accessing more donors and other funders [private contributors are little discussed in this report]; and • as underscored throughout the MG proposal, increasing income earned from HQ publications, training, and other services. Table 6.1. tracks change in both grant and earned income by numbers and types of sources between the baseline and MG periods. As the Table shows, IIRR succeeded in diversifying its supporters overall, and particularly in winning new PVO and NGO "customers" for IIRR services -- as per one of the MG's two explicit purposes. Likely this felicitous finding reflects mainly AFR and LAC dynamism in USAID-Matching Grant Evaluation Report 111 building new partnerships, including paying ones. Notably missing from Table 6.1., however, are national, subnational, and local government agencies as funding sources or even as in-kind contributors. Admittedly such agencies may be poor. But FG commentary emphasized the importance of their tangible stakeholdership for the longterm sustainability of certain kinds of IIRR initiatives. FG members felt that, as greater power and funding devolve to lower levels of government under democratization, IIRR should be able strategically to link with funded programs at these levels, at least in such a way as to stretch Institute resources by not duplicating local development programs. Table 6.1. Revenue Sources (excluding private donations) 1992-1995 1996-1999 No. of Agencies No. of Agencies Variance Grant Income UN/CGIAR System Multilateral Agencies Bilateral Agencies Foundations PVOs/NGOs Private Sector Total 3 1 4 10 11 3 32 3 0 5 10 14 5 37 0 -1 +1 0 +3 +2 +5 Earned Income UN/CGIAR System Multilateral Agencies Bilateral Agencies Foundations PVOs/NGOs Private Sector 3 0 7 1 15 3 7 3 13 0 23 4 +4 +3 +6 -1 +8 +1 Total 29 50 +21 Source: Tables prepared and categorized by IIRR's Deputy VP at evaluator request. They are displayed in Annex D's Tables D.-5. and D.-6. Figure 6.1. pie-graphs revenues for a 45-month-each baseline and MG period. The periods are structured so as to be strictly comparable according to the latest financial information available at the time of evaluation. The Figure documents progress on two aims, both of which are implicit or explicit in the MG proposal. USAID-Matching Grant Evaluation Report 112 51% 13% 36% GRANTS & CONTRIBUTIONS USAID EARNED INCOME ($ 4,027) ($ 1,500) ($ 5,698) 23% 28% 49% GRANTS & CONTRIBUTIONS USAID EARNED INCOME ($ 7,468) ($ 4,220) ($ 3,499) 1 Jan 1992 to 30 Sept 1995 previous MG began 1 July 1990 Total = $15,187 1 Jan 1992 to 30 Sept 1999 previous MG began 1 July 1996 Total = $11,225 Figure 6.1. IIRR Revenue ($000s) Source: Data compiled by the IIRR finance unit at evaluator request. 49% 23% 28% 13% 36% 51% USAID-Matching Grant Evaluation Report 113 One, implicit in IIRR's submission of an "exit" MG proposal, is decreased reliance on USAID support. As Figure 6.1. shows, IIRR dependency on USAID went from 23% ($3.5 million) to only 13% ($1.5 million) of total revenue. The other is increased reliance on earned income. The MG proposal set an ambitious target of 80% of Institute-wide and regional revenue to derive from earned income by the MG's end. The DIP revised this figure to a more realistic 50%. As Figure 6.1. shows, IIRR hit the latter target Institute-wide. Earned income grew from some $4 million (28% of total revenue) during the baseline period to more than $5.5 million (51%) toward the end of the MG. How did the regions do on this target? Figure 6.2. displays annual grant-to￾earned income and total income/expenses for AFR and LAC. [Recall from Chapter 5 that similar calculations cannot be made for ASI.] Calculating from aggregate income between Jan 1996 [IIRR operates on a calendar fiscal year] through Aug 1999 [the latest available HQ financial data at the time of evaluation], Figure 6.2. shows that AFR garnered exactly 50% of its > $1 million revenue from its own earnings. For LAC, these figures are 30.5% and ≈ $0.7 million USAID-Matching Grant Evaluation Report 114 Earned Income Grant Revenues $27 $546 $164 $93 $220 $143 $92 $132 $49 $156 - 100 200 300 400 500 600 1995 1996 1997 1998 Aug-99 $573 / 245 $257 / 253 $363 / 206 $224 / 392 $205 / 258 $126 $59 $56 $81 $38 $262 $23 $59 $84 $89 - 100 200 300 400 500 600 1995 1996 1997 1998 Aug-99 $185 / 120 $137 / 140 $300 / 140 $82 / 191 $ 1 7 3 / 8 9 Latin America Africa Source: Data compiled by the IIRR finance unit at evaluator request. a The number after the forward-slash following the grant+income total at the top of each column represents the Regional Center's total expenses for the year shown. Figure 6.2: Grant and Earned Income ($000s) USAID-Matching Grant Evaluation Report 115 In sum, IIRR as a whole made significant progress on diversifying its revenue sources and, in both relative and absolute terms, on increasing its earned income. AFR and LAC moved closer to self-sustainability. Just how close they got is impossible to say, however, in light of conflicting financial information from HQ. While Figure 6.2's annual revenue-to-expense numbers look very promising, the 1999 data there must be discounted as incomplete. For 1998, according to IIRR commentary on the draft evaluation report, Figure 6.2's data are contradicted by end-of-year external auditor reports indicating that AFR and LAC ran deficits of $22,000 and $49,000, respectively. The same source reports respective deficits of $101,000 and $13,000 for 1999. Obviously, the IIRR finance unit still has a ways to go in terms of tracking expenditure data precisely... In any case, Institute achievements in revenue diversification and earned income were overshadowed by a negative trend in total revenue, which dropped from approximately $15.2 to $11.2 million between the baseline and MG periods. Average monthly income perhaps gives a better sense of available operating capital and overall trends. As calculated from Figure 6.1., monthly averages dropped from ≈$337,500 to ≈$250,000 (-26%). Indeed, for the past several years, IIRR has run BOT-approved deficit budgets. Funds have become so tight that the IIRR President forewent some of her benefits in 1999; and downward renegotiation of 1999 salary and benefits for the AFR and LAC Directors [as well as other managers?] was mooted. During the MG, IIRR initiated many measures to promote financial sustainability. In Jan 1997, 4 of the 5 then-Divisional Chiefs attended an NGO workshop on this issue, delivered under BHR/PVC's Sustainable Development Services Project. The chiefs took advantage of this US trip to meet with the BOT on financial and other matters. Then, during the 1997 SP in Feb, HQ managers, Regional Directors, the BOT Chair, and several trustees together developed an action plan emphasizing strengthening institutional systems in support of financial sustainability goals. In Oct-Nov 1997, a business-plan proposal was written and approved as part of medium-term planning [see again Table 5.1.]. Also in Nov 1997, the HQ staffer charged with tracking resource generation took a fundraising course on Business for Social Progress held at the University of the Philippines in conjunction with the Asia Foundation and the American University School of Philanthropy. In Dec 1997, she took an IRRI proposal-writing course. With BOT approval, some of the ideas from these courses were later implemented. For example, IIRR established a library of materials on donors and partners for use in fundraising. Between Jan and May 1998, large numbers of HQ staff attended workshops on marketing and proposal-writing, respectively delivered by AIM and a trustee. In May, USAID-Matching Grant Evaluation Report 116 IIRR contracted a consultant to design a comprehensive Public Awareness Program (e.g., Siron-Freeman 1998) and produce a new annual report format to do double-duty in fundraising, plus a number of "slick" flyers and brochures advertising IIRR's vision￾mission-values, its conference and training facilities, writeshops, and so forth. Another consultant was hired to develop a medium-term financial plan as a way of moving forward on the 1997 business-plan proposal. Her recommendations bear repeating here, with notes on IIRR progress to date (Fuchs-Carsch 1998). • Develop and work within an ambitious medium-term plan. [In progress.] • Engage a senior, expatriate Director of Finance and Administration. [The evaluator seconds this recommendation; see Chapter 5.] • Prepare a resource mobilization policy and communication/outreach materials to go with it; and hire someone to oversee the plan and promote public awareness of IIRR work. [The above-referenced Public Awareness consultant was contracted; and in Jan 1999 a Public Awareness Officer was local-hired.] • Rethink the Institute's approach to restricted funding, e.g. seek fewer but larger grants from larger donors. [In progress.] • Pay staff more and charge more. Otherwise good staff are going to be lost. Most IIRR salaries are lower than those of comparator organizations and what the market will bear. [Awaiting new social architecture and further analysis.] • Control budgets in multi-partner projects, and adopt a new proposal budget format. [The latter was completed and placed in the Program Procedures Manual.] Concerning this last item, based on a review of proposals written at the time, the consultant noted many budget-construction problems: omission of key costs; no provision for inflation or currency fluctuations; under-calculation or omission of indirect costs; ignorance of differing unit costs by country; and in general, a misperception of donor funding ceilings such that proposals are underbudgeted, forcing IIRR to subsidize the resulting projects. The USAID evaluator confirmed these observations and thus endorses all the consultant's recommendations. Additional evaluator recommendations are: • add 5% to 10% of total direct costs into every proposal budget in order to support M&E [Section 6.2.] and plan for the corresponding staff level of effort (LOE) without surpassing 100% FTEs; and • with HQ leadership, for each region (1) do a simple comparator study of the prices of services akin to its own, (2) analyze actual current costs of IIRR delivery of these services in the region, and (3) survey regional partners' estimation of IIRR service quality in the context of (1); based on the results, write region-specific business plan aimed at ensuring that regions do not "price themselves out of business" but also do not undercharge for their service USAID-Matching Grant Evaluation Report 117 packages vis-à-vis real costs and relative quality. In fact, a "principal activity" enunciated in IIRR's MG communication with the USAID Grant Officer was to "prepare business plans for each office [i.e., today's Regional Centers], including cost and feasibility studies for leveraging funds" (Marshall 1996:10). This activity was postponed in view of the 1999 SP, but it now needs to move forward. In 1998, IIRR successfully negotiated a renewed agreement with the Philippines NRRM that allows the Institute to continue to use the HQ campus from 2000 to 2025, with an option to renew for an additional 25 years by mutual accord Also in 1998, the BOT contracted John Brown, Ltd. to perform an Institutional Resource Generation Audit (JBL 1998), with the goal of increasing fundraising among US corporations and individuals. When the new IIRR President took over in Jan 1999, she instituted a thorough￾going policy of complete transparency in sharing finance, budget, and fundraising information widely within the Institute. This policy has been a good staff motivator that also allows all members of the organization to do better-informed, more realistic and strategic thinking and planning about their work. As noted above, in early 1999 HQ local-hired a fulltime Public Awareness Officer to design a fundraising plan-of-attack and coordinate resource generation efforts. Working with the 1999 SP's Resource Generation and Public Awareness Team, in Mar she produced a consolidated internal assessment of IIRR's history, strengths, weaknesses, opportunities, and threats in fundraising (IIRR/HQ Mar 1999). Earlier, a computerized system to track and manage these efforts had already been designed [Figure 6.3.]. Throughout 1999, the BOT took vigorous fundraising measures, such as a New York reception, mass mailings, special seasonal appeals, and large personal contributions from trustees and IIRR Counsel members. The BOT also prepared a detailed schedule of the use of IIRR's various trust funds, with an eye to amalgamating them for more efficient investment. Meanwhile, the BOT Resource Development Committee created and discussed a 7-page tabular outline of 30 issues and ideas for increasing revenue (Mudge Oct 1999). In Nov 1999 AIM delivered a customized marketing course to a large HQ group. A "sustainability analysis" of IIRR fixed and variable costs vis-à-vis revenue was presented at the Nov 1999 SP session. This analysis demonstrated the need to reduce fixed costs and overhaul the whole cost structure of the Institute via a number of radical but positive changes. Most were noted in Chapter 5, but they bear repeating USAID-Matching Grant Evaluation Report 119 IIRR PTS Current Project Info (See PTS form) Phase I – Long Term Initiatives Collecting and maintaining target data Data Source: Directors and Executive Officers Data Gathering Tool: IIRR Contact Form Frequency: As gathered Info Flow: Source – RMO for entry in IIRR database Possible Data Use: - Strategic mass mailings eg - Marketing of training, facilities and publications - Sharing of IIRR information & publications - Sharing of index & abstract IIRR donor info t ll i t ti l ffi Phase II – Medium to short term future Using target data for strategic resource generation and management Data Source: Directors, Executive Officers Data Gathering Tools: PTS, Anticipated Donation/Fund Raising Income, Training & Facilities Booking forms Frequency: As gathered Info Flow: Source – RMO for database input Possible Data Use: - Production of reports for BOs and Ds on new proposals monthly status chan ges IIRR ORGANIZATION CONTACT INFO - TARGET BASE - (See tool IIRR Organization Contact Form for information collected and stored) IIRR Individual Contact Information (See Individual Contact F f d t il f IIRR Facilities Reservation Info. (See facilities bookin g form) IIRR Training Reservation Info. (See training IIRR PTS Proj. Proposal Info (See PTS IIRR Anticipated Donor Gifts or Event Fundraising Income Through BOT IIRR FA D Report on cashflow sources for the month IIRR Friends categorized: - Corporate and Individual, - Fundraisers and IIRR Facilities Clients: - International & local NGOs - Commercial IIRR Alumn i IIRR Projec ts and their Donors Phase IV – Pas t Maintaining historic data to enable improved resource generation strategies Data Source: Encoded information from previous phases Data Gathering Tool: Database queries Frequency: As required, regular and ad hoc Possible Data Use: - More refined Phase I uses - Statistics and distributions as required Phase III – Current and short term future Maintaining data for efficient and effective current resource management Data Sources: Directors, Executive officers and FAD Data Gathering Tool: PTS form, project documents, finance reports Frequency: On event occurrence or project commencement Possible Data Use: USAID-Matching Grant Evaluation Report 119 here in that they flow from data-based IIRR-internal analysis. As summarized by IIRR's Financial Analyst, these changes involve: • radically restructuring financial and other operating relationships among HQ, NY, and the regions; • relatedly, shifting staff from HQ to the regions; • adjusting the proportion of core to project-based staff; • rationalizing cost centers within the Institute; and • contracting certain services out-of-house. To the foregoing list, the evaluator would add one further item: • carefully weighing the costs and benefits of keeping the US Office in New York as versus moving it to the Washington DC area. The reasons for this item are multifold: higher office and labor costs in New York than in, say, suburban Maryland or Virginia; the IIRR President's observation that the New York Office location is so inconvenient as to be useless on her visits there, but other locations are too expensive; the wealth of PVOs, PVO service organizations, and PVO networks in the DC area; the greater number of and readier access to PVO￾oriented donors in the DC area -- where, indeed, some donors like USAID have standing committees on PVO assistance and give regular public briefings to PVOs; and the possibility of hiring more development-savvy staff, possibly on a money-saving parttime basis. Also presented in the Nov 1999 SP session was a comprehensive proposal for resource generation and public awareness. The resulting 8-point action plan emphasized the following, revenue-raising and cost-saving steps (IIRR/HQ Dec 1999:13-15). 1. "Establish a well-articulated resource generation and marketing strategy with defined revenue targets...roles and responsibility; accountability; policies and procedures, including budget guidelines, incentive-based revenue allocations; and [a] management information system which [sic] supports a dynamic process of planning, implementation, monitoring and evaluation, and learning." 2. "Systematically analyze institutional costs and needs in terms of: core costs (as determined by the new structure), plus management overhead requirements; program costs determined by program goals and objectives...; and specific resource needs for institutional strengthening, regional coordination and communication, etc." 3. "Establish revenue targets from diverse funding sources based on the costs above." USAID-Matching Grant Evaluation Report 120 4. Identify and analyze the needs of...broad groups of financial partners/donors and/or revenue sources..." 5. "Identify focused 'niches' between the specific interests and needs of donors/clients and IIRR's products and services..." 6. "Establish a desired mix of products and services with funding sources that ensures (1) that individual units generate enough revenues to cover their core costs; and (2) that the Institute generates sufficient revenues to cover all its costs." 7. "Develop a well-articulated and funded resource development and marketing strategy for donors..." 8. "Work towards the organization of a donor consortium to support IIRR's programs." 6.1.2. Publications, Training, Field Projects and TA Meaningful analysis of the differential contribution to Figure 6.1.'s revenue from publications, training, field projects, and TA is not possible because such desegregate FIS data go back only a couple of years. So this section focuses mainly on marketing and other strategies relevant to the financial sustainability of these activities. Publications. The HQ publication/communication unit has aggressively marketed its own and, to some extent, also regional products. Besides the general dissemination strategies noted in Chapter 2 -- most of which double as marketing tools -- marketing steps can be summarized as follows. • For mail marketing, a simple 4-page, black-and-white handout/order-form lists all current IIRR publications. • Mainly for HQ/ASI publications, the HQ unit has generated a series of marketing "fact sheets." Entitled Let's hear it from the press, What do people say about our publications? and About the workshop [sic] process, they compile book reviews and press releases about IIRR publications and writeshops, plus purchaser/participant commentary on them. • In 1997, LAC designed a sturdy yet artistic full-color, four-fold flyer and order form for its Spanish-language publications. Roughly translated as Experiences in Participatory Development: Publications that Gather Together Our Lived Experiences, this eye-catching flyer provides the same information as the HQ catalogue [next bullet] plus a précis of document contents. • In 1999 the HQ unit created a 15-page spiral-bound catalogue with color photos of the covers of all IIRR and affiliated volumes. Each photo is accompanied by a publication abstract, ordering and outlet information, and prices (given in Philippines pesos and US dollars). • Similar photo layouts and information are presented on IIRR's website, along USAID-Matching Grant Evaluation Report 121 with a handy electronic order form. Spanish-language publications are conveniently broken out into a separate category. HQ plans to institute a credit-card payment option for internet sales as soon as Philippines banks are able to support it (possibly in 2000). • Other PVO catalogues advertise and distribute selected HQ and regional publications. For example, World Neighbors' In Print includes IIRR publications in which WN has collaborated; and PACT distributes IIRR publications about local/indigenous knowledge. • The HQ unit has extended distribution beyond its own bookstore (which stocks all IIRR publications regardless of regional origin) to consignment sales in Philippines and German bookstores. • AFR has done likewise, with consignments to development and technical bookstores in Nairobi. • In tandem with HQ's new Public Awareness Office, the HQ publications and training units provide every HQ staffer who travels with a packet of outreach materials to distribute. • As noted in Chapter 2, AFR partners and alumni often distribute books on AFR's behalf. As noted earlier, it is not possible to obtain Institute-wide data on earned income from publications alone, nor clear cost-to-earnings ratios for publication preparation, production, and sales. In any case, IIRR publications have a somewhat conflicted purpose: to make money, yes; but more importantly, to get development information "out there." Most publications are generated with multi-donor/organization support and participation. Multiple stakeholdership makes for wider information dissemination but also for many gratis copies of publications to immediate collaborators. IIRR also gives away many courtesy copies -- by one estimate, at least 5% of a print run. They advertise the Institute; prime the market for more purchases of the publications thus distributed; and of course share the technical information and best practices therein. Because the latter is the ultimate goal, the Institute has also sought new, cost￾free ways (both to itself and readers) to disseminate the information embodied in its publications. Recall the internet posting idea [Box 2.7.] and the CD-ROM project described in Chapter 2. As noted there and in Section 6.2.2. below, IIRR's uncopyrighted publications further serve as templates that are freely translated, modified, and re-published by other PVOs/NGOs, and as powerful models that stimulate development organizations of all ilks to marshall and share their own learnings in similar formats. Thus, to the extent that IIRR does not lose money on its publications, they should be considered cost-effective. But it would be prudent to do a double-check USAID-Matching Grant Evaluation Report 122 analysis to make sure all LOE (and associated share of staff benefits), printing, distribution, and overhead costs are in fact being fully recovered. [The evaluator suspects they are not.] Guidelines for budgeting publication proposals should then be written, and book prices revised upwards, according to findings from this analysis. Appropo, to insure the sustainability of its regional publication program, LAC has established a sort of revolving fund by which earnings from book sales are banked to provide seed money for promoting/producing subsequent volumes on development topics LAC deems important. Other IIRR units may wish to consider such a strategy. Training. The HQ training unit devotes two of its 9 training professionals solely to marketing and materials packaging. Given eroding HQ/ASI enrollments across the MG [Chapter 3], the unit has mounted a major marketing push. For example, it has done detailed analyses of HQ/ASI course costs, content, participants, and etc. as compared to those offered by other development organizations worldwide (IIRR/ETD n.d.-a). Additionally, LOE for HQ/ASI course preparation, delivery, evaluation, and reporting has been analyzed, and then strict LOE guidelines set in an effort to rationalize this cost (Espineli 1998). Further, course advertisements have been targeted to useful new trainee recruitment channels, e.g.: member directories of PVOs, international clubs, UN agencies, and donor groups that nominate and support candidates for training organizations; and international development magazines and newsletters. In 1996, the training unit produced marketing flyers for each HQ course and circulated them widely. In 1998, the flyer information was placed on the Institute website, where it will be updated annually, with course dates announced a year in advance. The unit now also broadcasts a Training Needs Assessment by internet as both an information-gathering and marketing tool. And in 1999 a "slick" marketing booklet was printed entitled Sharing Lessons Learned: Training, Workshops, Study Programs. Like so many HQ efforts, however, little of the foregoing flurry of activity responds to regional needs. For example, only one paragraph in the "Sharing" booklet mentions courses offered by Regional Centers; and it does so only vaguely, with no indication of content, structure, approach, etc. In the same vein, recall Chapter 3's description of the International Sharing Newsletter, which services only HQ/ASI. The IIRR website is little better. For AFR and LAC courses, it bullets a half￾dozen general topics and then simply refers readers to the Centers. In the absence of HQ support, marketing materials and efforts in AFR and LAC are minimal. AFR generates flyers as courses are announced; presumably it also uses the flyers for marketing to prospective clients. The evaluator did not see any LAC marketing materials for training. Of course, it is not efficient for an institution to generate equivalent materials four times over (HQ, AFR, ASI, LAC), except for necessarily USAID-Matching Grant Evaluation Report 123 language-specific ones (i.e., Spanish in LAC). It is to be hoped that HQ has shared/will share all its market analysis findings and techniques with the regions, and that it will assist them in similar reviews. But the HQ training unit's overall lack of marketing support to its young AFR and LAC Centers is symptomatic of findings throughout this and the previous MG evaluation of HQ's "sink or swim" attitude toward the regional centers and a pervasive disconnect between the two. Specifically for training, it is also diagnostic of tensions over HQ versus regional competition for students. This tension was openly acknowledged in the Nov 1999 SP session and needs to be resolved. One logical way to go might be for HQ to give more highly specialized, lengthier, higher-order, and accredited courses, leaving all others to the regions. Another possibility is for HQ to offer a more limited but regularly-scheduled menu of lengthier courses while regions operate on a purely customized basis, referring interested parties to HQ for alternative scheduling or more intensive study. In any case, this issue must be put to rest. IIRR certainly cannot afford to duplicate courses and marketing materials needlessly. Field Projects and TA. As noted in Chapter 4, much of IIRR fieldwork and TA consists of training (and extension), especially in AFR and LAC. Non-training activities in field projects and TA were not analyzed in detail for this evaluation due to their diversity plus the lack of indicators, and thus organized data, for them. Besides TA, certain IIRR staff sell consulting services on IIRR's behalf, notably TSG, the VP, and the LAC Director. TA/consulting can be worthwhile if done in a marketing or learning spirit to "get a foot in the door" with new clients, link with a special endeavor, or bring new ideas and experiences back to the Institute. But IIRR should beware of TA that is tangential to its programs and goals; and IIRR should do away with the concept of "consulting" insofar as this is normally understood, in a sort of "Have gun will travel" context. Consulting typically deploys the Institute's most highly skilled technical and management staff, thus robbing regular activities of their oversight and input. It may also contribute to the >100% FTE practice noted in Chapter 5 and thus prejudice LOE on donor-funded projects. For example, according to numerous reports, the VP was frequently absent on consulting jobs during the MG. Further, consulting may not be cost-effective. As reported by TSG, IIRR daily rates run only $150 to $200. As the 1998 financial consultant pointed out for very small grants (Fuchs-Carsch 1998:10), these rates may not make consulting worthwhile to pursue and/or administer, especially if it prejudices normal program management. TA should be viewed with the same critical eye. USAID-Matching Grant Evaluation Report 124 One strategy worthy of note is LAC's use of flat-task-rate and product￾deliverable contracts for some TA and training. This saves both LAC and the client a lot of budget and accounting headaches while giving LAC more flexibility in pricing and in the use of any resulting earnings. Other units may wish to experiment with this arrangement, especially for very small grants and tasks and/or with resource-poor NGOs/GROs. But again, care must be taken not to undercharge on fee [recall Section 6.1.2.] or overspend on LOE. General. The Institute needs to prepare to take radical action on its sustainability situation, including possibly drawing down trust funds in order to move forward expeditiously on all the literally vital restructuring decisions taken in the Nov 1999 SP session. This may sound like a recommendation for unsustainability. But to the Institute's detriment, IIRR management made some major missteps in deploying its single biggest block of funding (the MG) as it had originally and astutely proposed, and in following through on key elements of its MG-supported 1997 SP [recall Chapter 5 and see next section]. At this point, if IIRR is not willing to "put its [own] money where its mouth is" in order to embrace modern PVO financial, personnel, and program management and M&E methods (and to fire and hire accordingly), it is doubtful that disillusioned donors will do so. The Institute may now have to "pull itself up by the bootstraps" to a position from which it can move forward effectively on future resource generation. 6.2. Progress and Trends in M&E With the many steps and plans outlined in Section 6.1. plus a streamlined organizational structure, a leaner HQ, an exceptionally dynamic BOT Chair, and an outstanding new president, IIRR seems to be taking almost every measure possible to improve its revenue outlook and thus its sustainability. The one element missing in this all-out attack is the ammunition. That is: valid, up-to-date, and suasive process and especially impact data with which to demonstrate IIRR achievements to potential supporters. IIRR's vital need for impact data was the focus of a major recommendation in the previous MG evaluation in 1994. It was also the driving rationale for the present MG's proposal to hire an M&E specialist with 1 support staffer in 1996. And it was one of the 10 action-plan points of the 1997 SP. However, as the financial consultant who advised IIRR on resource generation found in 1998: The Institute is doing a poor job of communicating its achievements, either internally or externally. Creating awareness of the excellent work of the Institute...has been allowed to lapse to a dangerously low level (Fuchs-Carsch 1998:1). USAID-Matching Grant Evaluation Report 125 The consultant further expressed "surprise to find that no staff were fluently able to provide up-to-date statistics on such elementary facets of the Institute's work as the number of staff members, number of people trained, number of countries where IIRR was working, percentage of activities undertaken in the various continents, etc." (op. cit.:7). At its outset, the MG evaluation found the same dearth of such statistics, much less impact data. Like many PVOs, IIRR has sold itself short by not establishing a results￾oriented M&E system. By this is meant an Institute-wide minimum of: SMART (simple, measurable, accurate, reliable, timely) indicators (usually mostly quantitative) of financial, program, and other Institute operations with their corresponding baselines, targets, and timelines; mechanisms (whether manual or computerized) for regularly collecting, monitoring, and reporting on these indicators; standardized methods for analyzing them from multi-user perspectives; a complementary plan of qualitative reporting and analysis, the better to interpret changes and trends in quantitative indicators; and expert staff to design and manage such a system. This is the stance from which IIRR progress and trends on M&E are evaluated below. 6.2.1. The Institute and the Regions "Institutionalization of a system for multi-level monitoring and evaluation" was cited as DIP Item 29. According to the MG proposal, "IIRR established a Planning, Monitoring, and Evaluation Office under its most recent Matching Grant" and "a full￾time evaluation specialist is being hired" (IIRR/HQ Dec 1995:E44). MG salary support in the amount of $146,094 was budgeted for this individual plus a supporting Project Monitor, both to be housed at HQ (Rigby 1 Jul 1996:B-2). No such office or staff existed at HQ at the time of this evaluation, however. Indeed, no M&E specialist was ever hired. Broadly stated, programmatic M&E capacity at the time of evaluation consisted of computerized databases of process data in: the HQ Centralized MIS (CMIS) Office, which manages a relational database of the same name; one HQ unit (training); and one region (AFR). CMIS and AFR are also able to manipulate some linked financial data [recall Chapter 5]. Other units and regions have some computerized data-listing and statistical-analysis capabilities [Section 6.2.2.]. All are expected regularly to report selected process data to CMIS. Institute progress in arriving at this point across the MG as compared to a 1995 baseline is detailed below. In brief, however, progress during the MG centered on bringing IIRR information technology (IT) into the 20th century, improving existing databases, and doing a lot of thinking, planning, workshopping, and talking about an M&E system. USAID-Matching Grant Evaluation Report 126 IIRR's baseline (1995) IT situation can be described as follows. HQ had only 2 outside telephone lines, 1 e-mail connection, and about 30 computers, mostly 386s. For M&E, a customized DOS database built under the previous MG tracked HQ and regional data on training. It also generated descriptive statistics and supported mass mail shots for training. Training data were gathered from hard-copy semestral reports, sent by fax or DHL from AFR and LAC, albeit irregularly. The database also included information on IIRR TA, partnerships, staff trips to overseas development organizations, and HQ visitors. During 1995, however, some of these files became corrupt and thus were transferred into Microsoft Office's (MSO) relational database, ACCESS. Also, 4 separate contact databases were consolidated in ACCESS. In Nov 1996, HQ hired a Resource Systems Specialist/Project Monitor to oversee all databases and to develop a new one specifically for fundraising, aimed at interlinking medium-term program planning with financial planning. The resulting database related some program activity data to IIRR partners and alumni, thus permitting analysis of repeat clients; and it generated information for promotional materials. In 1997, IIRR contracted a telecommunications consultant (the owner of a local computer firm) to improve HQ connectivity. In a series of reports (e.g., Ira 1997a&b), the consultant produced a blueprint for the Institute to enter the information age. The following advances were then made. • External telephone lines at HQ were increased from 2 to 7, and phone costs were cut by shifting to fiber-optic land lines. • HQ computers increased to 46, of which 36 were considered adequate to users' needs after repair and rationalization. The other 10 were scheduled for upgrading. • E-mail access was decentralized from a single source to 8 addresses in the HQ administration administration, thus facilitating overseas communication. • After a review of software used by other development organizations in and around Manila, MSO '97 was adopted as IIRR's standard package. • Plans were developed for an HQ LAN. • An Institute website was established on a local ISP and two staff trained in website creation. • IIRR management became better educated about the need for and uses of IT; and a Computer Advisory Group was designated. On the M&E side, all remaining data from the old DOS system were imported into the ACCESS database, and new program activity data were added, thus creating the CMIS. For each activity, data included an activity reference number, title, type, location, dates, individual and organizational partners, and objectives for each USAID-Matching Grant Evaluation Report 127 program activity. An important result was that, to some extent, program financial transactions and activity data could now be tracked together. This made for accurate reporting to donors even on activities funded by multiple donors. By Aug 1997, other (albeit spottily) available data and reporting capabilities of the CMIS included: organizational details to permit categorization of partners according to type, interests, and location; trainee characteristics; activity pipeline reports; and reminders about donor reporting deadlines. Further, activity reference numbers served as locators for hard-copy program documents, such as proposals, donor and trip reports, evaluations, and notes on donor visits. These documents were stored in the CMIS Office for use in complementary, qualitative M&E. Despite this numbering system, however, the evaluator observed that the CMIS Office had difficulty locating key M&E and related management and governance documents [such as a complete copy of the previous MG evaluation]. Thereafter in 1997, work continued to enhance CMIS capacities to accept program data, but still focusing on process data for IIRR's training program. Unfortunately, system effectiveness was hobbled by several factors: • the CMIS's general inaccessibility to managers; • the lack of clear policies and procedures for units/regions to submit data to the CMIS, plus staff laxness in this regard; • relatedly, receipt of incomplete or overly aggregated raw data from the regions; and • overall, an admittedly stopgap CMIS design and an amateur user interface, built on an emergency basis when the old DOS system finally failed completely; thus in Oct 1997, the Computer Advisory Group advised management that professional CMIS review and reprogramming were imperative. With restructuring in 1997, the Deputy VP was assigned general oversight of M&E Institute-wide [Chapter 5]. To address the foregoing and many other M&E issues, he wrote an impressive "think piece" on M&E for the 1997 SP (Killough Feb 1997), which resulted in an SP action plan with the following, salient aims (IIRR/HQ Mar 1997:14). • Assess and reform IIRR's "multi-level (institutional, program, project) M&E system," including defining qualitative and quantitative indicators, sources of information, and methods of data-gathering and analysis. • Clarify/establish M&E roles and responsibilities within units, including codification and abstraction of field-based learning for sharing. USAID-Matching Grant Evaluation Report 128 • Upgrade the M&E skills of existing staff, including bringing in new M&E skills "if necessary." • Conduct regular multi-level M&E of IIRR work from HQ and the Regional Centers, alumni, NRRMs, including indirect impact through trainees and through the sharing of IIRR learnings through multiple avenues. The Deputy VP followed through with a series of thoughtful, comprehensive, and M&E-savvy memoranda (Killough Feb, Apr, Jun, and Nov 1997) plus staff meetings to try to stimulate forward movement on an M&E system. Highlights of his thinking included the following. • An integrated, Institute-wide system was envisioned as spanning not only data on program/project activities and grants but also on personnel matters, persontime expenditures, staff travel, visitors, physical plant operations, and of course budget and finance. • Concrete suggestions for Institute-wide process indicators were outlined for all of the above. • Also addressed were: different levels of indicators; the use of tightly targeted "point" evaluation studies; the need for special M&E planning on collaborative field projects; ways to promote M&E among IIRR partners so as to garner impact data; funding for M&E; shortcomings in staff writing abilities vis-à-vis qualitative evaluation and impact assessment; and much more. Unfortunately, after all this excellent thinking and planning, little action was forthcoming. This is hardly surprising in view of several factors. No M&E specialist was ever hired. Even the Project Monitor position ceased to exist after a year. The only person with any clearly designated responsibility for an M&E system (the Deputy VP) was absent for some time in 1997-1998. And according to various reports, higher-ups at HQ pooh-poohed M&E as unimportant. As one AFR interviewee said, they are content "every year to just talk about the same M&E stuff" and then continue program work in the absence of systematic evidence as to its impact, cost￾effectiveness, staffing efficiency, etc. Only two M&E actions were taken in 1998. The Project Monitor (promoted into another position) was replaced by the CMIS administrative assistant, who was promoted to Administrator and tasked with managing the CMIS -- although she did not know ACCESS and did not receive training in it until much later. And 2.5 pages of general guidelines focusing on more complete and timely submission of M&E data were included in the Program and Procedures Manual. Staff compliance remained poor, however. As of Oct 1999, the evaluator found that even training data were incomplete and out-of-date in the CMIS. USAID-Matching Grant Evaluation Report 129 IT fared better in 1998. HQ staff outside the administration building obtained access to the internet for the first time, albeit only at a single workstation. And IIRR hired its first Computer Systems Specialist (CSS). His immediate tasks were to inventory IIRR software and hardware; rationalize distribution of the latter; and assess "humanware," i.e., staff computer skills and training requirements. This he did at HQ and ASI and, via e-mail and other-staff visits, to some extent for AFR and LAC. The CSS was further tasked with: repairing and maintaining all HQ computers; decentralizing MSO to the regions; piloting a LAN in TSG; and developing telecommunications policies and procedures, including Institute-wide guidelines for system consistency and compatibility. Further, he trained some 20 HQ staff in Windows, MSO, internet use, e-mail, LAN basics, and virus detection and cure. Also during 1998, HQ: increased computers to 54, of which 48 were deemed user￾adequate; purchased 15 MSO '97 licenses and 18 copies of a leading antivirus software; and initiated the Institute website. IT continued to advance throughout 1999, when a Canadian intern redesigned the website. Also, more AFR and LAC content was added. Since then, however, clear responsibility for maintaining and updating the website has not been assigned to any staffer, nor have any guidelines been written as to how or to whom units/regions are to prepare and flow website content. With help from interns at a local computer school [recall Box 5.1.], the CSS extended the TSG LAN HQ-wide. Now, approximately 35 HQ staff can exchange data more efficiently through standardized templates and reports, detect and clean computer viruses more quickly, share printers, and so forth. Further HQ LAN uses being explored are: HQ-wide access to databases, which currently are maintained at stand-alone workstations; internal e-mail to and from networked workstations; increased internet access and web-browsing; and internet phone and fax [as per evaluator suggestions]. Also during 1999, the CSS drafted a manual with LAN instructions, tentative IT policies and procedures (e.g., specs, purchasing), ideas for funding IT improvements, and more (CSS 1999). During 1999 the CMIS Administrator finally received ACCESS instruction, and then trained a few other staff. She also drafted a CMIS manual. Prodded by the USAID evaluation, in Oct 1999, she and the Financial Analyst (the former Project Monitor) made Herculean efforts to update the CMIS, which they achieved by the end of Dec. Otherwise, there would have been little timely quantitative data in the present report. Meanwhile, what was happening in IT and M&E in LAC and AFR? [ASI presumably followed along in the wake of HQ events.] The answer to this question for LAC is: not much. During the MG, LAC obtained a limited form of e-mail on a cheap USAID-Matching Grant Evaluation Report 130 network for NGOs. But its e-mail connection and computers are so obsolete they cannot accept attachments or a Windows platform, and thus cannot handle QuickBooks and MSO. The Center has only 1 laser printer, and 1 phone line each for calling and faxing. Like HQ/ASI, LAC has no M&E specialist. But it does keep clear and detailed computerized lists of raw activity data. Indeed, according to HQ's data￾management expert, LAC maintains the most accurate and comprehensive program activity records in all of IIRR. By comparison to LAC, AFR is well-supplied with modern computers, printers, software, phone lines, etc. During the MG, it received HQ assistance in installing and later upgrading its e-mail capabilities, obtaining up-to-date software, and schooling its accountant in QuickBooks [Chapter 5]. As noted earlier, AFR is totally up-to-speed on this FIS. Various other Head Office and Ethiopia Country Office staff were trained in computer and internet skills during the MG. When it comes to M&E, at the touch of a button AFR's in-house MIS (built in ACCESS) can generate tables, charts, graphs, etc. and descriptive statistics of trainee data by such parameters as gender, nationality, course type or content, etc., all broken down into any temporal groupings desired. [The evaluator tested these capabilities on-site.] However, AFR's MIS is not configured in the same way as HQ's. [If anything, it works more smoothly.] Significantly, since 1997 AFR has had a fulltime M&E specialist, seconded by DED [recall Box 5.1.]. Most impressive of all is AFR eagerness to do proper M&E. More than any other IIRR unit or region, AFR seems to appreciate the rationale and importance of M&E. It was also the most adept at understanding and generating the kinds of impact-oriented qualitative data the evaluator sought. [By comparison, LAC staff below the level of the Director had difficulty grasping this idea.] Indeed, AFR could have forged ahead on design of an M&E system on its own, were it not for a Catch 22. Without agreed-upon indicators Institute-wide, AFR would be setting up a parallel system that might later have to be laboriously re-configured and re-coded. Yet already, AFR is beginning to outgrow its current MIS design. IIRR progress on IT/M&E generally during the MG can be summarized as follows. Everywhere but LAC, great strides were made in IT as compared to the baseline situation. Some IIRR staff and units/regions made good-faith efforts to move forward on M&E. And despite top-management disinterest and missteps, IIRR laid the groundwork for integrated FIS+MIS review by marshalling the necessary hardware, piloting appropriate software, consolidating available data, expanding existing databases and creating new ones (especially in finance), and training staff accordingly -- with the obvious exception of LAC. These moves have set the stage for building a proper M&E system. Still, in the absence of indicators, impact data, stipulated analytic methods, and a regular reporting schedule, M&E remains very process-oriented and USAID-Matching Grant Evaluation Report 131 ad hoc [as when an evaluator shows up]. To build an M&E system, IIRR sorely requires the professional expertise of a top-flight M&E specialist. Few Institute staff seem to understand that just sticking numbers into a database and pushing them around is meaningless without defining indicators, etc. Indeed, many staff have difficulty just conceptualizing useful arrays of raw data. Interpreting what constitutes "impact" also seems to be problematic [Section 6.2.2.]. Finally, not all key M&E information can be captured numerically, as the boxed mini-case-studies in this report should suggest. But again, few staff appear to comprehend the importance of systematic qualitative data. Yet IIRR could and should have been gathering such data all along. They require no fancy hardware or software -- although these certainly help. In fact, with an M&E specialist on-board, IIRR will want to link or cross-reference substantive text files to its CMIS and regional MISs, and perhaps try out some ethnographic software. Looking ahead, the 1999 SP renewed IIRR resolve to: install "Participatory monitoring and evaluation systems (M&E)...in all operating units to track impacts and outcomes of programs"; "Develop appropriate indicators...of quality for...management systems"; "Pursue management analysis and decision-making that is based on useful, timely, accurate and up-to-date information"; "Invest in...information technology" for so doing; "Establish a coordinated M&E system...that will allow for integration across the organization. Ensure that this M&E system is integrated with and relevant to the multiple functions and information needs of the organization (i.e., program impact, lessons for sharing, donor contacts, etc.)"; and include a "Monitoring & evaluation/research" specialist among HQ professional staff (IIRR/HQ Dec 1999:9,12,17). More concretely, IIRR envisions hiring 2 inexpensive IT support staff (one for hardware, one for software) in 2000. This will free the CSS to work closely with the to￾be-hired M&E specialist. The 1999 SP also tasked the Computer Advisory Group with finding ways to enhance and further integrate the CMIS. The Group recommended a holistic assessment of IIRR information needs by unit/region in order to maximize and institutionalize benefits from IT/M&E investments under the MG. Likewise for professional redesign of the CMIS interface and reporting capabilities. These moves would permit greater immediate use of the valuable management and program data already in the CMIS, while also informing its redesign to serve as an Institute- and program-wide system. With all these pieces in place, plus a major upgrading of LAC IT equipment and skills, it is hoped that sometime in 2000 units and regions will become able to access and input data into the CMIS directly, using simple templates, e-mail, and the internet. USAID-Matching Grant Evaluation Report 132 In the operating strategy outlined above, the emphasis on coordination and integration across the Institute is vital for an M&E system. Program achievements and impacts are most impressive and powerful when all IIRR units and regions pull together [recall Box 4.5.], tallying and analyzing their successes and failures in a unified framework -- as this evaluation has endeavored to model. Whether in program as well as financial M&E, truly the whole is greater than the sum of the parts when it comes to informing organizational strategizing and resource generation. 6.2.2. Publications, Training, Field Projects and TA Publications. As noted in Chapter 2, the Institute's publication/communication program constitutes "the shining star of IIRR." But systematic evidence for how far (outreach/spread) or how brightly (impact) this star casts its light is poor. The following insights are offered in illustration of some relatively simple ways that IIRR can measure such variables, or at the very least communicate better about them to potential funding sources. Some of the same principles sketched below can also be applied to other IIRR program activities. The fact sheets described in Section 6.1.2. offer anecdotal evidence of customer satisfaction and program impact for IIRR publications. But in May 1999, the HQ unit formalized this heretofore somewhat serendipitous effort by mailing a Publication's [sic] Monitoring and Evaluation Letter to a sampling of customers, asking how IIRR publications came to the customer's attention, how they relate to his/her work, how they are used, who are their main users, how they may have helped reach "the poorest of the poor," and what is good/bad about content and format. Although mail-out response rates are expectedly low, the feedback is nevertheless valuable. As with most HQ-unit work, however, the fact sheets and letters focus on HQ/ASI. If systematically applied across all units/regions, they would be more useful, both as M&E and marketing tools. For instance, sheets could be generated in different arrays for niche marketing to donors and partners with different regional, topical, or audience foci. With varying degrees of formality, accuracy, and detail, all IIRR units/regions that produce publications make some effort to track customer types, their organizational affiliation, location, product use, and so forth plus sales/distribution. But tracking is difficult for the HQ unit, with its more far-flung outlets; procedures vary across unit/regions; and the data are not stored in the CMIS, where they could undergo aggregate analysis. Until publications indicators are standardized and databased, it is impossible to do any linked, quantitative evaluation of spread and impact effects, whether globally or regionally. Until then, it is also impossible for management to access MIS data on publication topics, audiences, etc. for strategizing about, e.g.: whether to revise or reprint existing publications; what future publications USAID-Matching Grant Evaluation Report 133 to initiate; what potential donors or partners to target for them; or how to slant fundraising materials. Obviously, sales and distribution data can indicate spread; and volume and velocity of sales can serve as rough indices of product relevance and excellence. Collecting such data by origin, purchaser, and end-use parameters makes them much more informative, however. For example, if a publication by one unit/region is bought in other regions, global relevance can be imputed to it. If tracked by customer type or end-use, sales signal general arenas of imputed impact: libraries buying IIRR publications for reference use, PVOs/NGOs purchasing them for training, academics and universities ordering them for textbooks, and so forth. Where numbers of trainees or students involved are known, impacts can be further quantified. With such data, IIRR could easily build proxy indicators of impact at different levels of institutional aggregation. Managers would then have some information about what kinds of publications best speak to what functions or audiences, making for better forward planning and marketing. For conferences and writeshop publications, regional/topical/etc. data on numbers and types of organizations that send participants and/or lend financial support would be likewise useful. Participation offers a measure of IIRR organizational capacity-building through informal training [recall Chapter 3]. As with sales, financial support provides evidence of the relevance and value that other development organizations attach to IIRR products and the participatory way in which they are generated. Such data were not readily retrievable in summary form at the time of this evaluation, but they appear to exist in hard-copy. A real oversight has been the Institute's failure to monitor the many follow-on publications produced with or without IIRR assistance. Recall the spread effects of the "runaway" volumes of Box 2.2; the CARE and SwissAid manuals that resulted from participant learning in the LAC agroecological writeshop of Box 2.6.; the Honduras cover-crop manual; or the many products following on the heels of the PM&E conference (Box 4.5). Apparently, this evaluation report was the first to document the plurality of such replicator and multiplier effects. Unfortunately, in the course of an external evaluation it is not possible to back up and collect distribution, use, and qualitative impact data that should have been being tracked all along. But with such data, IIRR can rightly lay claim to far wider regional or global impacts with regard to the distribution of development information, institutional capacity-building, or significant influence on development thinking. In like vein, HQ should follow up with the CD-ROM project to find out how many CDs have been distributed, to whom, etc., and thus how many more groups now have access to IIRR-generated development information. Materials posted on the internet USAID-Matching Grant Evaluation Report 134 should include a site counter to monitor "hits." Another idea is to access library￾science services that do citation searches. Further, regular write-up of "boxes" of qualitative process and impact information, like those collected for the present evaluation, is strongly recommended for all IIRR work [see below]. Training. As discussed in Section 6.1.1., training has enjoyed the lion's share of M&E attention within IIRR. Indeed, the HQ training unit has designated one of its professionals an Evaluation Specialist. Recently, it also won its first-ever unit project, the Sustainable Agriculture Training of Trainers or SAToT, which features a hefty M&E component as per donor requirements. Still, in all units and regions, training M&E has so far been limited mainly to: numerous before-during-after course evaluations (e.g., pre/post application of a trainee Confidence Level Test, various participatory methods); trainee "body counts" (numbers and sex); and ad hoc analyses of trainee characteristics (such as nationality, education, organizational affiliation, professional position and functions, funding source for training), and financial data for curricula re-design, marketing, and donor reporting. Strangely, though, forms requesting trainee information traditionally have not included parameters that might allow IIRR to impute at least potential spread and multiplier effects -- such as numbers of persons supervised, serviced, or to-be￾trained in turn by trainees. Even for process data, IIRR has had difficulty operationalizing or tracking some variables. An example is how to code whether trainees are new ones or repeaters, to get at totals trained from a given organization. A persistent problem for ASI is said to be under-reporting of training, especially at the community level. There is some question as to whether AFR's MIS may cause double-counting under some circumstances. And so forth. However, most such problems should be solvable by an experienced M&E specialist. More worrisome is the lack of impact assessment. As noted in Chapter 3, the Institute emphasizes trainee action-planning; but this is difficult to follow up, even with HQ/ASI Alumni Associations. Moreover, some of the impact indicators HQ has identified may be fallacious. For example, HQ/ASI tries to track trainees' management-level promotions to gauge the effectiveness of its flagship course; yet this measure may be tautological, in that only the best-and-brightest or most privileged may be sent for training in the first place. IIRR has made no effort to track the emulation or outright borrowing of its training materials by alumni and other TOTs. Yet this appears to be a common practice that, as with publications, could suggest replicator and multiplier effects of IIRR's training programs. USAID-Matching Grant Evaluation Report 135 Spurred in part by SAToT, during the MG HQ began explicitly to engage the question of how to do meaningful but cost-effective training impact assessment. For SAToT, the HQ unit was required to draft a detailed logframe and M&E plan (Arevalo and Saladores Feb 1999). They include a few intermediate impact indicators such as pre/post comparisons of TOTs' knowledge-skills-abilities and the numbers and types of participants subsequently trained by TOTs. Significantly, SAToT also calls for post￾course in-country meetings with alumni, to interview them directly [see below]. In May 1999 AFR prepared and delivered an M&E workshop for CARE/Kenya staff and their partners from CARE's Girl Child Project (IIRR/AFR Jun 1999). AFR itself hosted another workshop on the subject in Jun-Jul 1999, attended by the HQ training unit's Evaluation Specialist. An in-house summary report on these workshops was inconclusive as to recommendations for impact assessment, however. On the other hand, AFR has applied or will apply the OCAT to assess pre/post effectiveness of its NGO/GRO training in institutional capacity-building. This should give a fair measure of organizational impacts. Meanwhile, the HQ training unit has proposed holding an Expert Consultation on Evaluating the Results of Capacity Development Programs, for which it has won partial funding from IDRC, with the remainder possibly to come from ISNAR. The evaluator recommends that IIRR do a thorough-going literature review in preparation for such events. In a quick internet search, for instance, the evaluator happened upon a healthy literature on training evaluation in the business world, often organized by simple models such as: reaction, learning, behavior, and results/impacts. Again, having an experienced M&E specialist on-board should go far towards demystifying this issue. Certainly, IIRR should use more qualitative forms of training impact assessment. For this, there is little substitute for periodic evaluation, using simple but reasoned sampling frames to do participatory evaluation exercises, open-ended interviews or FGs (which allow trainees to define their own notions of "impact"), and mini-case-studies [like those of Boxes 3.6. through 3.8.] that illustrate concrete and verifiable impacts or lack thereof. Such face-to-face evaluation is particularly appropo for illiterate community-level trainees. Bolstered by thematic analysis and other techniques for quantifying qualitative data, qualitative methods are probably the best way to get at training impacts. Again, mechanisms for organizing, databasing, and analyzing them will be required, however. Finally, as suggested in Chapter 3, it would be helpful for IIRR first to put its training activities into a conceptual framework. In this way, the Institute will not continue to overlook informal and non-formal kinds of training that it provides, such as: its 1,091 IIRR writeshoppers and conferees during the MG [calculated from Table USAID-Matching Grant Evaluation Report 136 3.2.]; the local editors, artists, and wordprocessors who receive OJT in off-campus writeshops; and the Institute's many interns and volunteers [Box 5.1.]. Field Projects and TA. Doing M&E for field projects and TA can be difficult insofar as IIRR may not be the controlling project partner, and activities can be quite diverse. On the other hand, field projects are where IIRR stands the greatest chance of demonstrating tangible people-level impacts at the grassroots. And indeed, impact analysis is vital in this arena, given that the main point of IIRR involvement in field projects is to test out Institute curricula, participatory methods, development information materials, etc. before extending them more widely [recall Chapter 1]. As the Deputy VP noted in his excellent M&E memoranda, IIRR must do special planning for M&E on collaborative field projects and identify ways to promote M&E among its partners so as to garner impact data. As suggested in Chapter 4, one trick is to break out any IIRR-assisted or - stimulated training entailed in field projects and TA, and monitor and evaluate it in the same M&E framework as other training. Ditto for publications. For specific biological/technical/sectoral and gender impacts of field projects, IIRR may have to rely on its collaborators. But it could also do periodic "point" evaluations of people￾level impacts using qualitative techniques as described above for training. Indeed, once IIRR gets its own M&E house in order, conceivably it could sell M&E training and TA services and TA to its field-project partners -- and M&E practica during training could serve to generate desired data [recall Box 4.2.]. One minor point now. All IIRR programs use trip report data. For some TA, these may constitute the only output. While IIRR does have a standard trip-report format, its present formulation makes it extremely difficult to input into the CMIS￾database. This should be corrected, perhaps in the Travel Manual HQ is currently drafting. General. No matter what the activity -- finance, personnel, physical plant operations, publications, training, field projects, TA -- the important thing is for IIRR to establish a minimum dataset of both quantitative and qualitative, process and impact indicators that have been agreed upon for Institute-wide use. This will provide the data comparability necessary for getting at Institute efficiency and effectiveness around the world and down through time. Of course, donor-, region- and/or project￾specific indicators and analysis will always also be needed. In the meantime, IIRR must "back up" and gather as much qualitative impact data on recent program activities as possible by whatever means possible -- staff recall, documents and reports, FGs and interviews. This is imperative for sustainability. The correlation between declining Institute revenue and lack of impact USAID-Matching Grant Evaluation Report 137 data is no accident. In the Philippines, the evaluator spoke with some major IIRR donors and partners who felt that "IIRR is an anachronism"; it "lives in the past," "has seen its heyday," and "hit bottom." But when presented with a few mini-case-studies, FG commentary, and interview data from this evaluation (drawn from AFR, as it happens), these same individuals expressed pleased surprise at IIRR dynamism. Said one, "I had no idea IIRR was doing all this!" The message here is clear. In today's competitive PVO environment, impact data are imperative for fundraising among larger donors for larger grants. The process data that IIRR has collected are alone insufficient to "tell the story." Fortunately, even anecdotal data can be suasive if multiplied many times over and packaged appealingly. Something as simple as fleshing out and multiplying the mini￾case-studies illustrating this report could go far toward raising the dangerously low level to which communication of IIRR excellence and impact has fallen. Such a task must be undertaken urgently. With staff shifts and cutbacks likely under reorganization, considerable institutional memory could be lost. As an initial tool in this task, IIRR should try using its excellent writeshop process on itself. When it comes to qualitative impact data and also proposal preparation, problems noted throughout IIRR were a generally low level of English-language writing skills and little time for those few staff who can write well to do so. One solution to the latter problem is a policy of "protected time" for writing. For the former, solutions are less evident in an international development organization like IIRR, where few staff are native English speakers. Consulting writer-editors are a possibility, if they are sufficiently knowledgeable in development issues to draw out lessons and impacts from raw qualitative data and staff drafts. Certainly part of the solution is to hire an M&E expert with proven skills at writing for development audiences in English. In fact, all job announcements for professional hires should emphasize this skill. Further, the M&E specialist ideally should be able to read Spanish. Otherwise, he/she will be unable to access the many reports, raw data, and supporting literature necessary for astute qualitative M&E for LAC. In any case, HQ sorely needs more Spanish-language capacity on-staff. As IIRR re-staffs after restructuring, it should ask LAC to post job announcements in Spanish-language development outlets and to screen and recommend outstanding bilingual job applicants of all ilks. Finally, both for sustainability and for M&E, IIRR urgently needs to build an "institutional capability database." By this is meant up-to-date computerized records of the sorts of elementary statistics referenced throughout this report, plus other data like BOT and staff profiles (including disciplines and degrees), regional profiles, donor lists, publications and course descriptions, etc. USAID-Matching Grant Evaluation Report 138 In fact, IIRR has most elements of such a database already in place, whether in its MIS, website, "learning-sharing database," various donor/partner and contact databases, and a new master acronym list started at evaluator suggestion. It just needs to add some more "meat" in the form of, e.g.: fact sheets and mini-case￾studies; compilations of other quotable quotes, as from donor letters or external evaluations like this one or Bachmann 1999; easy-to-digest graphics [recall Figures 2.1., 4.1., and 4.2.]; and prepared, "off the shelf" concept notes and proposal outlines (of which a number already exist). All information should be entered in such a way that it can be keyword-searched, e.g.: for region, country, sector, specific topics (AIDS, gender, livestock, water harvesting, what-have-you), participatory methods, staff qualifications, donor(s), collaborators, etc. With such a database, IIRR can then tailor marketing materials to specific donors and take maximum advantage of managers' and BOT travel and meeting opportunities to fundraise. Also, this would better prepare the Institute for the next evaluator who happens by... 6.3. Lessons and Recommendations 6.3.1. Lessons AFR and especially LAC progress toward self-sustainability suggests that, despite today's shrinking development funding worldwide, there is still room for dedicated, focused, and quality-oriented PVO units to thrive, even in the face of a non￾supportive HQ (at least for a while). But as IIRR's experience also shows, if HQ does not keep a close eye on the bottom line, survival of the organization as a whole is jeopardized. Besides quality programming, survival requires at least two fundamental tasks: wisely managing both the cost and revenue sides of the sustainability equation; and generating impact data. The need for cost controls is obvious. From IIRR experience, other service￾delivery PVOs can take a lesson to beware inefficient and unsustainable bureaucratic and programmatically duplicative units' feeding off those units that are efficient and high-performing. If this situation arises, something probably is seriously amiss, signaling the need for a hard-headed review of low-performing units' staff competencies and bloat, their perhaps mistargeted activities, and maybe even their continued relevance and comparative advantage within the organization. Of course, meaningful review is impossible in a management-information vacuum. Hence the importance of M&E in more mundane arenas (finance, personnel, persontime deployment, physical plant and logistic services, etc.) as well as programmatic ones. For revenue generation, some lessons appear to be: make sure staff know how to budget proposals realistically, or hire competent staff to begin with; relatedly, charge what the market will bear; go after bigger donors and grants; and build the USAID-Matching Grant Evaluation Report 139 M&E system that is imperative for putting teeth into resource generation efforts. Programmatic M&E is particularly crucial for "intermediary" PVOs and NGOs that emphasize service delivery in the form of capacity-building through such activities as publications, training, and TA. Impacts are more diffuse by comparison with field projects, where such things as agricultural yields, hectares treated/planted, footage of hedgerows established, fish-ponds or wells dug, trees established, clinics built, clinic visits made, new businesses started, income earned, and etc. are easier to measure and count. Grassroots, people-level impacts from PVO capacity-building with client NGOs and GOs are not easy to detect and attribute. More refined, and usually extra, qualitative M&E machinery is required to demonstrate such transitive impacts. In this regard, other PVOs can learn from IIRR's foray into the modern world of IT. Although in today's complex and competitive "development business," computerized MISs and FISs have become a virtual necessity, they are not sufficient for M&E. As the now-old adage says, "Garbage in, garbage out." An integrative M&E system is required, complete with: a reasoned conceptual framework; SMART organization-wide process and impact indicators (and their baselines, targets, etc.), usually along with additional region-, project-, or donor-specific indicators; a complementary plan of qualitative evaluation; accompanying manuals and OJT for PVO staff worldwide about the regular collection and computerized or manual handling and flow of data; and top-flight analytic expertise on-staff to query and interpret all these data methodically, detect anomalies or trends with important implications for PVO administrative or programmatic functioning, define evaluation needs accordingly, keep on top of the latest literature, and do M&E re-design as circumstances change. A PVO shirks M&E and especially impact assessment only at the risk of its own demise. No amount of "slick" reports and brochures nor even well-organized and comprehensive process data can substitute for impact data. Donors and the constituents to whom they are accountable have a right to know not only whether their money was spent and on what (hopefully, as agreed), but also whether it made any difference. In other words, "Where's the beef?" Moreover, any serious PVO should be asking itself the same question all along. That is, are its programs achieving what the PVO envisions for them, and doing so in a professionally competent and fiscally responsible manner that usefully reaches a maximum of beneficiaries? In the international development arena, at least, gone are the days when PVOs could expect to be handed large grants and then go their own way, without having to report on the money's impact-ful use. PVOs that do not learn this lesson will soon be "out of business," no matter what their resource-generation endeavors. USAID-Matching Grant Evaluation Report 140 6.3.2. Recommendations Sustainability  Congratulate AFR and especially LAC on their progress toward financial sustainability.  Continue making highly cost-effective use of funds by leveraging multiple strategic partnerships for activities. Ditto for diversifying funding sources.  In the foregoing regards, pay more attention to national, subnational, and local government agencies as potential sources of support.  Keep an eagle-eye on budget building in proposals, and give all relevant staff further training in budgeting if problems continue.  Give the new President kudos for her policy of transparency in fiscal matters; and keep it up.  Seriously consider re-locating IIRR's US office from New York to the Washington DC area.  Do a forward-looking analysis of all Institute revenues and expenditures taking into account staffing structures, equipment needs, infrastructure costs, BOT expenses, institutional reorganization, decentralization transaction costs, HQ/region duplication of efforts, and so forth. Then devise staged, multi￾scenario (e.g., low-medium-high or minimum/maximum) financial/business/marketing plans for the next several years for the Institute as a whole and for each unit and region.  More specifically: review publications pricing vis-à-vis total costs; and settle on a viable distinction and distribution between HQ and regional courses so that they reinforce rather than compete with each other.  Relatedly, integrate and rationalize marketing and outreach materials and systems across HQ units and regions to avoid costly duplication and get "the biggest bang for the buck" out of investments in marketing. Basically, HQ has done little to support AFR and LAC in this regard.  Follow through on all the excellent plans for financial sustainability emerging from the 1999 SP. USAID-Matching Grant Evaluation Report 141  Take a cue from LAC's use of revolving funds, flat-task rates, and product￾deliverable contracts to experiment with self-sustaining forms of programmatic (as versus merely project) financing and more flexible contracting (and hence income-disposition) arrangements in other IIRR units/regions.  Steer clear of tangential TA that may not be worthwhile either in earnings or in skilled personpower robbed from core management and program work.  Be prepared to draw down trust funds if necessary in order to move forward expeditiously on all the restructuring decisions taken in the 1999 SP. Prompt implementation of these decisions is literally vital to putting IIRR in a position from which it can hope to work toward sustainability and, indeed, survival. Monitoring and Evaluation  Build 5% to 10% of total direct proposal costs into every proposal budget to pay for the necessary M&E.  Make greater use of the IIRR library as a secure and reliable yet accessible and transparent repository for hard-copies of evaluations, annual and project reports to donors, and other M&E, management, and governance documents.  Promptly put in place the necessary mechanisms and sanctions to ensure that staff report all stipulated M&E data to HQ in a timely and proper manner.  Relatedly, bring closure now to the longstanding efforts to build modern, interlocking and cross-regional IT systems to facilitate the timely input, flow, analysis, and use of M&E data as well as better HQ-to-region communication.  Designate appropriate and competent HQ staffers to take charge of website "care and feeding," i.e., technical design/functioning and content.  Add guestbook sign-ins and counters to all websites in order to monitor visitors and "hits" on sections where such data are useful for M&E.  Immediately mount a major push to capture and compile qualitative data on IIRR program impacts in recent years. Try using the IIRR writeshop process as an initial tool in this task.  As stipulated under the MG, hire a fulltime, top-flight M&E specialist at HQ to design, implement, and manage a true M&E system. USAID-Matching Grant Evaluation Report 142  NOTE: This hiree ideally should be able to read Spanish, and must be able to demonstrate solid skills at English-language writing for development audiences.  As restructuring proceeds, advertise new jobs in Spanish-language development outlets and ask LAC to screen and recommend outstanding bilingual candidates. At the same time, include strong English writing skills as a preference in all job announcements for professional positions. Also, consider establishing a policy of protected time when critical writing assignments arise.  Establish a comprehensive "institutional capability database" and keep it up-to￾date. Annex A Evaluation Scope of Work I. PROGRAM IDENTIFICATION PVO : International Institute for Rural Reconstruction Cooperative Agreement # : FAO-0158A-00-6050 Dates of Evaluation : October-November 1999 Country programs : Philippines, Kenya, Ecuador II. PROGRAM BACKGROUND The International Institute of Rural Reconstruction (IIRR) has a USAID Matching Grant (USAID Cooperative Agreement No. FAO-0158A-00-6050) for the three-year period September 30, 1996 through November 30, 1999. IIRR, incorporated in the US in 1960, provides training, technical, and management consultancies, and develops information resources, and carries out collaborative field research projects. Since 1980, IIRR has had 4 Matching Grants with BHR/PVC. This program was proposed as the final 3-year Matching Grant Program to complete a fundamental restructuring of IIRR s NGO capacity strengthening activities. IIRR s program is derived from its 7-year strategic plan (1994-2000). Partners Local NGOs in Ecuador, Kenya, and Philippines. Program goal and purpose (objective & outputs) Goal : The goal of this Matching Grant is to promote sustainable improvements in income, health, and food security for the poorest of the poor in selected developing countries. Purpose 1: To strengthen the capacity of IIRR's NGO partners to implement participatory, people-centered, local development programs. Purpose 2: To increase the capacity and sustainability of IIRR's regional centers a) to deliver and sustain services to NGOs; b) to undertake process documentation and impact evaluation and c) to promote inter￾institutional learning and networking. Sectoral focus: The critical development issue to be addressed has been the capacity of NGOs to achieve positive results in: a) food security at the household level b) sustainable use of natural resources c) economic empowerment of women and household income generation d) reproductive health e) maternal and child health f) environmental health. Geographical focus: IIRR's regional centers in Africa, Asia, and Latin America have played the primary role in the design and implementation of IIRR's capacity building activities in each region. The efforts focused on strengthening the capacities of NGOs in Ecuador, Ethiopia, Bangladesh, and Nepal. (Initial plans to include Honduras and Uganda have been refocused on other program activities in the regions.) Availability of data: The evaluator will refer the following documents: original proposal, detailed implementation plan (DIP) which includes the logframe, cooperative agreement with USAID, annual reports, prior evaluation, PVO M&E system data, other sources. III. PURPOSE OF THE EVALUATION BHR/PVC Purpose: This final evaluation fulfills the requirements of the USAID/BHR/PVC Matching Grant Program. BHR/PVC will use this information in its annual Results Report to the Agency as well as in the review of any follow-on proposals from IIRR. It is, therefore, important that the evaluation report present material that is quantifiable and measurable as BHR/PVC is held accountable for results from the programs it funds. PVO Purpose: IIRR Strategic Planning Initiative: In line with the objective of increasing IIRR's institutional capacity and sustainability, the final year of the Matching Grant has coincided with a major strategic planning initiative designed to focus, strengthen and expand ambitiously IIRR's programs and impact. In 1999, this strategic planning exercise has three purposes: C. to review IIRR's past experience and strengths; to explore opportunities for the future in the changing contexts and challenges of development; and to formulate IIRR's statements of vision, mission, and long-term goals; C. to define a 5-year program strategy within the framework of the Institute's vision, mission and long term goals; and C. to prepare a 3-year medium-term plan for the years 2000-2002. Two complementary teams of facilitators, Global Excellence in Management (GEM), and the Asian Institute of Management (AIM), are providing high quality technical support to facilitate the strategic planning process. Two phases of the strategic planning process have already been completed, resulting in formulation of new Vision, Mission, and Values statements. The planning process is now focusing on developing a social architecture and long-range action plan. The AIM and GEM teams will facilitate planning sessions between November 8 and 14, 1999 at IIRR's global headquarters in the Philippines. IV. EVALUATOR STATEMENT OF WORK The evaluator will assess the following program and institutional elements, providing evidence, criteria for judgment and citing data sources. The evaluator(s) will assess both headquarters and the regional offices in the following countries: Philippines, Kenya, and Ecuador. An estimate of the emphasis or level of effort for each segment of the SOW is in italicized brackets. Further, based on the lessons learned from the Matching Grant, the evaluator will provide an overview and specific recommendations that will feed into IIRR's program priorities and strategic planning process. A. Program Implementation: [EMPHASIS FOR THIS EVALUATION --- 40%] 1. Assess progress towards each major objectives. Based on the logframe/Planning Matrix or statement of program purpose from the proposal, have objectives been met? Assess achievement toward each purpose statement as set forth in the Detailed Implementation Plan (DIP). Identify constraints and unanticipated effects. Identify major successes and why these elements were successful. (BHR/PVC is interested in what worked and did not work.) Discuss changes that have occurred as a result of the grant. 2. Assess progress towards sustainability What program elements were intended to be sustained? What are the sustainability objectives? What are the indicators used to measure progress? What are the achievements to date? And what are the prospects for post-grant sustainability? Describe the existence and status of cost-recovery mechanisms and their effectiveness. What was the local-level financing or approaches used to generate resources to support project operations? 3. Assess the status of strategic partnership(s) with NGOs, community￾based organizations or local-level government. Characterize partnerships with local-level partners.role, responsibilities (decision-making power); mechanisms employed (MOU, subgrant, contract etc); and fiscal autonomy & amount of grant funds directly managed by NGOs and partners in past year. How did the program plan to assess the quality and scope of partner relations? impact of the partnership on the program? What change in capacity of local level partner was planned? What has been achieved? Effects of training or resource transfer on local partners capacity? Assess the local level partners satisfaction with the partnership has the interface and communications among PVO headquarter and partners been effective? Do partners have access to email/internet? Cite the major implementation lessons learned and recommendations. What major approaches or methods were used? And how effective were these methods and why? Was the use of these approaches/methods expanded or institutionalized? B. Program Management [EMPHASIS FOR THIS EVALUATION --- 35%] 1. Assess change in the IIRR s management capacity (structure & quality of management) as a result of grant. Strategic Approach and Program Planning What changes have occurred in the organization s capacity for critical and analytic thinking regarding program design and impact? How has the Strategic Planning process assisted IIRR in achieving this? Use information/data from the 1994-2000 Strategic Plan and the current Strategic Planning process with GEM and AIM. Provide evidence that the program has Fostered analysis and self-evaluation in country programs, or conducted quantitative or qualitative analysis to refine interventions; Conducted periodic review of performance data by project personnel and taken actions as a result of review; Institutionalized performance monitoring and impact evaluation systems into other non-PVC grant funded programs; and Acted on recommendations from mid-term evaluation, or program reviews. Are there changes in headquarters capacity to manage the planning process -- program renewal, strategy integration, project design; address over-arching program issues of replicability, scale-up, sustainability; forecast needs and management of a process strategic planning; strengthen financial planning; and manage organizational change. Monitoring and Evaluation Assess the capacity of IIRR to monitor program performance and measure impact. Has the program: developed appropriate intervention specific, capacity building and sustainability objectives & indicators; conducted baseline and final assessment; used impact and performance data to refine program implementation; developed a system (e.g., MIS) to consolidate, analyze and interpret project data; transferred monitoring and evaluation skills and capacity to local partners; and disseminated IIRR s required reports in an adequate and timely manner. Financial Management Are adequate financial monitoring systems in place to verify program revenue, operating and financial expense, other inputs and outputs? Has the program leveraged additional resources (beyond the match)? What is the cost effectiveness of the program? Human Resource Development Did IIRR assess staff training needs and strengthen the organization s professional or technical capacity and that of the local partner? Supervision Assess if there were sufficient staff with the appropriate technical and management skills to oversee program activity at both headquarters and in the field programs. Cite the management lessons learned and recommendations. Forward Plans [EMPHASIS FOR THIS EVALUATION --- 25%] Refer to page 2 of this evaluation SOW where IIRR strategic planning exercise is discussed. What are the implications from the lessons learned for future: technical direction? organizational direction? organizational financial sustainability and competency? Viability of the regional centers? EVALUATION METHODS The evaluator will review documents, conduct interviews and visit IIRR s regional offices, project sites, and NGO partners in Kenya and Ecuador, in addition to IIRR s global headquarters in the Philippines. Approach The PVOs program was developed and funded prior to the Agency's emphasis on results-oriented program designs and the development of PVC s Strategic Plan. The data from all BHR/PVC funded programs are critical to PVC's ability to report on achievements against the PVC s Strategic Plan. Therefore, it will be necessary for the evaluator to focus on results-oriented data. B. Methodology The Evaluator will: Assess the appropriateness of using the data collection approaches; Use the Agency's microenterprise (MED) indicators to assess the status of the MED intervention (if or where applicable); Document data sources; Provide, a copy (electronic or paper) of all primary data collected and analysis performed; and Use focus group meetings with local partners, if appropriate and possible. TEAM COMPOSITION AND PARTICIPATION The evaluation will be carried by out by one evaluator who will be the team leader in collaboration of IIRR s management and staff. The evaluator will have a background and technical expertise in strategic planning and organizational development. The evaluator will be contracted by AMaTECH, the support contract under contract to BHR/PVC. SCHEDULE (includes GEM schedule for field portion only) FINAL EVALUATION SCHEDULE AS OF Wednesday, October 06, 1999, 3:30pm: Washington, DC: Week of Oct 4th - document review. October 7 (telephone meeting) - Dr. McCorkle talks with Peggy Meites regarding AID evaluation needs. Philippines (1st trip to Philippines IIRR pays airfare, 6 days per diem (includes lodging & M&IE) October 8-9 - travel to Philippines. Hotel at airport for 1st night. (Airport Hilton) October 10-15 - Visit IIRR center in the Philippines. (Regional TPM) Meetings with senior management and staff, Asia regional office staff, trustees, partners. IIRR will also arrange focus group discussions with partners for Dr. McCorkle. Details to be worked out with IIRR and Dr. McCorkle. October 15 Meeting with AIM representative, Bobby Basilico, (GEM interview). Ecuador: October 16 18 En route to Ecuador from Philippines. Local travel and hotel to be booked by IIRR (not Hotel Quito) October 18 -22 -- Ecuador, visit Regional Office for Latin America. Meetings with staff, partners, site visits. (Regional TPM.) IIRR Deputy Vice President for Programs, Scott Killough, would accompany Dr. McCorkle on Ecuador portion. IIRR will also arrange focus group discussions with partners for Dr. McCorkle. Details to be worked out with IIRR and Dr. McCorkle. Washington, DC: October 23 Return to US. Some report writing possible. October 25 (date/time: Oct 25, 10:30am. Location: TBD) Interview and meeting with IIRR Vice President for Programs, Julian Gonsalves, in Washington DC. Attendees: Constance McCorkle, Julian Gonsalves. Kenya: October 26, evening travel to Kenya (arrive late October 27th). Accommodations at hotel near IIRR. October 28 Dr. McCorkle spends 1 day doing GEM NGO interviews. October 29-November 3 -- Nairobi, Kenya, visit Regional Office for Africa. Meetings with staff, partners, site visits. (Regional TPM.) IIRR will also arrange focus group discussions with partners for Dr. McCorkle. Details to be worked out with IIRR and Dr. McCorkle. November 4 Leave Kenya for Philippines Philippines (trip # 2 USAID pays airfare, per diem (includes M&IE & lodging) for both IIRR and GEM portions) November 4-5 Travel to Philippines. Local travel & hotel arranged by IIRR (not at IIRR Guest House). November 7-10 , 14-18 Visit to IIRR center in Philippines & IIRR-related activities. November 11-13 Dr. McCorkle conducts GEM NGO interviews. (SOW & interview guide provided by GEM evaluators). GEM Nov 11-13 GEM-related activities. Dr. McCorkle will use GEM evaluation team interview guide to begin to evaluate IIRR for GEM and conduct interviews of 2 other NGOs for GEM evaluation. Dr. McCorkle will coordinate with GEM evaluators when her product must be delivered (Nov 15) to them in order to be incorporated into final report. PVC will coordinate with GEM/US about which NGOs will be contacted and provide the information to Dr. McCorkle or to GEM evaluators. IIRR Nov 6-10 Visit to IIRR center in the Philippines. Meetings with Senior Staff and Regional Directors to discuss initial findings and enable IIRR to incorporate evaluation into Strategic Planning process. Nov 7-10 IIRR strategic planning. Observation days at strategic planning meeting. Dr. McCorkle will give input into strategic planning process (Nov 7 & 8) and observe (Nov 9 & 10). Other days may be used for drafting evaluation report. Meeting with Asian Institute of Management (AIM) which is collaborating with GEM to facilitate Strategic Planning. Nov 14 & 15 Evaluate IIRR with regard to GEM. Helpful if GEM representatives are still available. IIRR & GEM Nov 14-18 Continuation of IIRR work and some write-up for GEM evaluation. November 18 Travel back to US November 19-22 If necessary, report writing for GEM evaluation. Final report due by Nov 23. Nov 22-Dec 10 draft final report due to: BHR/PVC, AMaTECH, and PVO (simultaneously); debriefing, executive summary, and final revised report will follow after all parties have read report. VII. REPORTING AND DISSEMINATION REQUIREMENTS This scope of work will serve as the outline of the evaluation report. AMaTECH will provide the evaluator with a style guide for the evaluation report. The word-processing package to be used is WORD 97 and EXCEL (if necessary). Delivery Schedule: The evaluator will complete the draft report for review and comment and distribute to the following parties at the same time. AMaTECH (To fulfill contract agreement. The consultant will provide a paper and electronic copy.) a. BHR/PVC Project Officer a. IIRR staff and trustees Review/Revision Policy: The parties listed above will review the draft report in the same time period and provide their comments to the evaluator before the final evaluation debriefing where any changes to the report will be discussed and agreed upon. In addition, issues from the evaluation as well as findings and recommendations may also be discussed in the meeting. When appropriate, the evaluator will incorporate the changes and comments into the final draft of the report. All comments on the evaluation report must be sent to AMaTECH before the debriefing and will be made available to the evaluator. Annex B USAID Evaluation Report Findings from Focus Groups with IIRR Partners The Focus Group Concept The following pages summarize the findings from a series of focus groups held during Oct-Nov 1999 by an independent external evaluator contracted by the US Agency for International Development. These groups formed a key part of the evaluation approach and methodology. Using this method, a sampling of IIRR partner opinion was drawn as per the issues raised in Item IV-A-3 of the evaluator's Scope of Work: "Assess the status of strategic partnership(s) with NGOs, community-based organizations or local-level government" (SOW p. 4). Focus groups (hereafter FGs) represent a rich, robust, and well-structured methodology that today is widely used in business for marketing research (Kreuger 1994). It reveals what customers appreciate about a company's products and, conversely, what customers find unsatisfactory. It also reveals nascent or untapped customer demand. FGs consist of expertly moderated small-group discussions that center on the perceptions and experiences of knowledgeable 'customers' concerning key issues of interest to the 'company' commissioning the FG. These perceptions and experiences are elicited via carefully structured but open-ended questions. As a complement to almost any kind of R&D involving 'products' on offer and their actual or potential users -- in this case, IIRR services to CBOs, NGOs, and other PVOs -- an FG approach offers many advantages. First, FGs are easily understood by, and comfortable for, participants, who meet in a relaxed yet confidential setting that promotes candor. Second, the approach permits the expert moderator to identify and probe any unanticipated issues, thereby capturing diverse customer experiences with the product(s) under discussion. Third, FGs are a time- and cost-effective way both to add interpretive value and to put a human face on findings from other, non￾participative sources of data. Fourth, results have been shown to have high validity as well as clear practical implications for senior managers. Fifth but by no means last, FGs typically convey to customers a strong sense of "connection," communication, and empowerment vis-à￾vis the entity comissioning the FG. FG Methods and Participants One FG was held in each of IIRR's three geographic regions of operation: Africa (Kenya), Asia (Philippines), and Latin America (Ecuador). The English-language FG Interview Guide elaborated on the basis of SOW Item IV-A-3 is displayed as Attachment A to this summary. For use in Latin America, the guide was professionally translated into Spanish. Each group lasted about 3 hours (see Attachment B's illustrative schedule). Annex B USAID Evaluation Report Participants were drawn from a convenience sample of staff from organizations that had formally partnered with IIRR at some time during the period under evaluation (1996-1999). Partnerships typically took the form of: training or technical assistance received from IIRR; participation in IIRR-led "writeshops" to promote inter-institutional learning and to generate practical, development-oriented publications; or collaboration in joint field projects with IIRR. A majority of FG members consisted of community leaders and both office and field staffers from CBOs, NGOs, and the country offices of PVOs, as per the MG mandate to IIRR to strengthen such entities. Both religious and secular organizations were represented. However, also included were staff of: local and national government agencies, universities, and projects; UN-system projects and IARCS; and labor unions. FG participants can be characterized as displayed in Table 1. Findings from each group are organized into three categories: the strengths and weaknesses of IIRR work and partnership in the region, and FG recommendations to IIRR as to how to improve its performance, outreach, visibility, etc. Table 1. Focus-Group Participant Profiles PARTNER PARTICIPANTS BY REGION SEX Africa (Kenya) - Food Security Desk Officer for one branch of the National Council of Churches of Kenya. - Communications Officer, FAO-supported Forest Action Network. - Program Officer, CARE/Kenya. - Assistant Program Director, International Christian Support Fund. - Program Coordinator, Nairobi Parents of the Deaf and Blind. - CEO, Kenya Adult Education Association. - Ag Development Coordinator of one Kenyan Catholic Diocese. - Deputy Principal, Baraka Agricultural College. - Program Manager, Dupoto-E-Maa (a Masaai primary-school CBO). - East Africa Liaison Officer, Arid Lands Information Network. - Ag&NRM Projects Officer, Catholic Relief Services/Kenya. - Fieldwork Director, Compassion International/Kenya. TOTAL: 12 FG Members 1 F 1 F 1 F 1 F 1 F 1 M 1 M 1 M 1 M 1 M 1 M 1 M 7 M/5 F Annex B USAID Evaluation Report PARTNER PARTICIPANTS BY REGION SEX Asia (Philippines) - President, Tua Community Cooperative. - President, Mother's Association of Tua Community. - President, Tua Community Coordinating Body. - Representatives, IIRR's Bicol Project. - Member of the Board, Asian NGO Coalition for Agrarian Reform and Rural Development (ANGOC). - Research Associate, ICLARM. - Staffer of the Filipino NGO UPWARD, formerly Director of ILEIA/Philippine and formerly an IIRR staffer. - Policy Analyst and Campaigns Manager, PRRM. - A Provincial Environmental Management Officer, formerly Executive Director of an NGO and currently Action Officer of the Northern Negros Forest Reserve. TOTAL: 11 FG Members 1 M 1 F 1 M 1 M/1 F 1 M 1 F 1 M 2 M 1 M 8 M/3 F Latin America (Ecuador) - An extensionist from a large FAO forestry project. - A rural school teacher and community leader. - Program Officer on an environmental NGO project. - Ecuador Country Representative, Lutheran World Relief. - Projects Coordinator of an Ecuadorian economics NGO. - Micro-enterprise Director for a provincial-level bilingual program of the Ecuadorian government. - A member of a community Association for Organic Agriculture. - Technical Director of a park-protection NGO. - Farmer-leader of a community irrigation association. - Program & training officer, credit & savings officer, and financial assessor of the Belgian PVO ACT. - Ag technician/social promoter, Habitat (formerly with a CARE/Ecuador partner of IIRR). - Managers/Project Coordinators from a major Ecuadorian environmental NGO. - Representative, National Federation of Peasant, Indigenous, and Negro Organizations. - Representative, a local Union of Farmers' Organizations. TOTAL: 18 FG members 1 M 1 M 1 M 1 M 1 M 1 M 1 M 1 M 1 M 1 M/2 F 1 F 2 M/1 F 1 F 1 F 13 M/5 F Annex B USAID Evaluation Report AFRICA FG FINDINGS STRENGTHS Training IIRR/AFR does in-depth design of all training materials, and not just for its "customized" courses. It tailor-makes materials to fit its partner/client needs. Relatedly, the design of materials and curricula is done participatorially; i.e., IIRR/AFR allows partners/clients to review training plans and materials before they are finalized. Furthermore, IIRR/AFR does daily evaluations during training, whereby courses and materials are even more tightly attuned to partner/client needs, cultures, etc. IIRR/AFR's highly participatory methods in-class are a real strength. Because of them, trainees learn a great deal from each other, as well as from the Institute instructors. A major strength of IIRR/AFR training is its post-course follow-up with trainees and their institutions. Although this is done formally via questionnaires, FG members felt that the most important follow-up takes place in the form of gratis, informal post-course advice, idea￾exchange, and encouragement.  As one FG member noted in no uncertain terms, were it not for IIRR/AFR's follow-up policy, "I would not have taken the trouble to be here today."a  Others observed that this aspect of IIRR/AFR training is what, in essence, "keeps us coming back for more" fee-based Institute services.  Two FG participants noted that another, indirect mechanism for following up and maintaining contact is that IIRR/AFR visits former partner-trainees' field worksites, often as a part of a field experience for new trainees. Overall, IIRR offers higher-quality training in everything it does (e.g., fundraising, resource mobilization, capacity-building needs assessment, strategic planning, institutional policy￾setting, stakeholder analysis, proposal writing, PM&E, documentation, and technical training in sustainable ag) than the vast majority of other PVOs, NGOs, and even private-sector enterprises (such as Deloit and Touche) that service development clients in Eastern Africa. Not surprisingly therefore, FG members [nearly all of whom had been IIRR/AFR trainees] said they find themselves able to put IIRR/LAC training to immediate use in their work, with concrete and tangible outcomes both for their own organizations and for their partners. Frequently cited in this regard were trainees' abilities to: subsequently train others in their own and partner institutions in IIRR/AFR subject matters;b and to write proposals that then won funding. Annex B USAID Evaluation Report All IIRR/AFR training is "output-oriented" in the sense that at the end of training, everyone must draft a realistic action plan for his/her own projects. Also, IIRR/AFR training is appropriately "adult-oriented" in that it is so practical. Documentation/Publications IIRR/AFR is particularly competent at facilitating processes that lead to concrete products such as publications. IIRR/AFR is very good at documentation, including its ability to capture, in detail, critical learning events from the field and fieldworkers -- "not just theories." Relatedly, IIRR/AFR has the right personnel and equipment to do professional-level publications. In evidence of how much they are impressed by IIRR/AFR documentation work, several participants described how they:  advertise Institute publications freely in their own organization's newsletters;  travel with spare copies of IIRR/AFR books when they visit partners, which they voluntarily sell on behalf of the Institute; and  include Institute publications in displays at various events of their own. As with training, IIRR/AFR is good at designing or adapting materials to fit client institutions' specific needs, and doing so participatorially.  An example is IIRR/AFR's adaptation of the OCAT instrument to become the Participatory Capacity Assessment (PCA) tool that, in collaboration with the Institute, CARE/Kenya then applied to its basic-education project grantees. Partnerships/Capacity-Building/Netowrking IIRR/AFR is always willing to learn from its trainees, partners, and clients. Good accounting systems are a real strength of IIRR/AFR -- so much so that partners are now often asking the Institute to shoulder this task in some of their collaborative efforts.  One example is ALIN's asking IIRR/AFR to handle all the "money end" of production and sales for their upcoming collaborative publication of a volume on agriculture in Africa's drylands.  Also, the PVO German AgroAction independently approached IIRR/AFR, asking the Institute to take on all accounting and money-management responsibilities for an upcoming Micro-Project Funds initiative of GAA. IIRR/AFR demonstrates outstanding organizational and facilitational skills in courses, workshops, and meetings. These are such that IIRR/AFR can even overcome inter- Annex B USAID Evaluation Report organizational jealousies and suspicions where no other such institution in Eastern Africa could. The fact that IIRR/AFR is not just another national or regional entity but rather an international one is a major strength, in that it allows IIRR/AFR to bring experiences from all around the world to share with its partners in Africa. Also appreciated is "a touch of the South" that IIRR/AFR brings as a PVO based in the developing world. WEAKNESSES Training IIRR/AFR has too few trainers in general, and too few high-level trainers in particular ("There is only Isaac")c to meet current client timelines and especially the demand for IIRR/AFR's customized courses. IIRR/AFR training tends to be more expensive than that of other PVOs, NGOs, or training institutes in Eastern Africa. Even though at present the market for IIRR/AFR training is far from saturated, IIRR/AFR should nevertheless be thinking ahead to the time when it may exhaust its primary paying market for training (currently mainly PVOs like CARE or German AgroAction and their urban￾based NGO partners), to strategize about what other groups it might train or what other kinds of services it might emphasize. Documentation/Publications IIRR/AFR needs to do more to publicize the existence of some of its publications, its work in documentation, and still other kinds of TA. At present in Africa, the Institute is mainly known for its training. Partnerships/Capacity-Building/Networking The name, "International Institute for Rural Reconstruction," is confusing and misleading. It sounds like an organization that builds rural infrastructure. Relatedly, IIRR/AFR is still relatively little known in the region as a whole. IIRR/AFR needs to be in closer touch with the grassroots, the better to inform its training, presentations, and documentation with a good understanding of relevant on-the-ground situations.d Annex B USAID Evaluation Report IIRR/AFR needs to pay more attention to matching personnel with task assignments according to partner/client culture and language.  A specific example is how, during Participatory Capacity Assessments with CARE/Kenya partners, neither NGO/CBO interviewees nor CARE staff could understand the accent of an IIRR interviewer from Ethiopia. The result was that interviewees gave answers that had little relation to the questions posed, and the assessment work suffered accordingly. IIRR/AFR is too much a "one-man show," in that it is insufficiently staffed with senior personnel who are knowledgeable about the whole program and/or who can decisonmake.  As one FG member put it, "When the Regional Director is not there, you can't get an answer even to a simple question."  Or as another participant summed up, "At the moment, Isaac [the Regional Director] is IIRR/Africa." FG RECOMMENDATIONS TO IIRR/AFR Build more links to the field and fieldworkers, so as to better inform and target IIRR/AFR training, publications, and technical assistance. Add at least one additional IIRR/AFR staff member with expert training skills who is able to work independently, so that training events do not continue to depend upon Isaac's availability. Alternatively or additionally, keep a stable of top training consultants on tap so that partner/client demands can be met in a timely fashion without compromising training quality.e Build up the capacities of existing staff -- not only in their ability to train independently but also to respond to partner/client queries and to decisionmake independently. Establish a scholarship fund so as to permit more kinds of development workers, who could otherwise not afford the full fees, to take IIRR/AFR courses. Also, in order to offer less expensive courses, break larger courses down into modular components. This will allow "poorer" clients' to take advantage of IIRR/AFR's quality training by targeting their scarce financial resources to the subject matters they feel they most urgently require. Explore further and perhaps more innovative outlets, venues, and arrangements for showcasing IIRR/AFR-led publications plus those it has assisted partners to produce.  A simple, immediate suggestion in the above regard was for IIRR/AFR to mount a display of publications in its entry hall. As one FG member observed, "I saw nothing there when I came today." Annex B USAID Evaluation Report Relatedly, produce and disseminate PR materials that make clear what IIRR/AFR is, what it does and offers. In this regard, FG members offered some concrete suggestions.  They cautioned against relying exclusively on electronic outlets in Africa.  They felt a written rationale for the choice of IIRR/AFR course offerings and an explanation of how all IIRR/AFR course offerings were inter-related would be helpful. In like vein, find a non-misleading “nickname” for IIRR in Africa.f a Indeed, the simple fact of attendance at the FG is a key indicator of the strong partnerships IIRR/AFR has built in a country (Kenya) where, according to a senior development source, NGO staff expect to be paid special "sitting fees" for attending any work-related event outside their own office, including even events funded and hosted by their own organizations! b IIRR/AFR comments that this is the main reason why many people take the Institute's courses, i.e. they are responsible for training within their own organizations. c In fact, one such trainer (a Filipina PhD) has just recently joined the team. But this was not known to the FG. d This FG took place in Kenya, where IIRR/AFR at present has no collaborative field projects (although one is slated to start shortly). Thus this group was unaware of the many such projects being carried out by IIRR/AFR in Ethiopia. e IIRR/AFR informs that it in fact does so. f For example, LAC maintains the written use of the acronym IIRR; but in speaking of the Institute, IIRR is pronounced as "ir," the verb meaning "to go," and here carrying a connotation of "Let's get going/moving." Annex B USAID Evaluation Report ASIA FG FINDINGSa STRENGTHS Training The fact that much of IIRR/Asia's training has been community-based and/or community￾oriented is what allows communities to make and manage their own organizations successfully.  As representatives of one such organization explained in the FG, "Now we know how to communicate our needs to higher-ups." They then went on to describe how, thanks to a seminar they attended at the IIRR/HQ campus -- where they learned about leadership, proposal writing, and budgeting -- they were able to write a successful proposal to local government to install a potable water system in their community and, later, to form a registered credit co-op.  The same group described how, thanks to IIRR training in primary healthcare, they have gone from a single traditional birth attendant in 1982 to 8 trained healthworkers today.  Focus-Group members of a different, 13-community group described how IIRR helped them to: organize into 5 interest groups in 1984, and then in 1987 into a federation, winning legal recognition from the Bureau of Cooperative Development in 1989; and by 1992, train and establish a core group of 7 local Technical and Extension Specialists to serve all 13 communities. They attribute their evolution to IIRR training in leadership, community-building, simple bookkeeping, enterprise management, and training of trainers. IIRR/Asia staff have excellent facilitation and extension skills that result in real farmer-to￾farmer or people-to-people learning.  Focus-Group members of the 13-community initiative, noted above, described how they participated in IIRR-facilitated exchange visits to farmers of other communities where IIRR had a "model site," and thus learned to applied new ag technologies on their own farms. Documentation/Publications IIRR/Asia has a remarkable ability to bring together highly diverse groups to generate "popular" (as versus scientific) information kits that are immediately useful. In like vein, IIRR/Asia is very good at organizing multi-sectoral and multi-agency workshops. IIRR's focus on the "tooling process" -- i.e. articulating and packaging processes, tools, and techniques -- is special. Annex B USAID Evaluation Report Partnerships/Capacity-Building/Networking IIRR/Asia is particularly good at conceptualizing projects that are appropriate to the felt needs of rural communities. Relatedly, IIRR/Asia is sincerely participatory in its work with communities. As representatives of one community with whom IIRR has long worked said in the FG, "IIRR/Asia has been more than participatory with us." Other partners appreciate the fact that, in its field projects, IIRR/Asia conducts regular evaluations. Still others say they appreciate IIRR/Asia's provision of "full and complete staff support" to its field projects, i.e. the Institute does not short-staff its projects. Also valued is the interdisciplinary mix on the project teams that IIRR/Asia fields. IIRR/Asia and Institute staff generally sit on all kinds of committees -- from, e.g., the joint coordinating councils of various localized projects all the way up to the CGIAR NGO Committee -- where their inputs are welcomed. IIRR/Asia makes wise use of different partnering modes, like: co-sponsorsing workshops, courses, and research initiatives with other NGOs and also IARCs; helping distribute partner publications; and assisting partners to design participatory evaluation and other tools/approaches. WEAKNESSES Training In community-based training and related field activities, IIRR/Asia is very good at training people to become organizers, but less so at following through on training them to become community managers, i.e. at leaving behind local managerial capacity once the Institute withdraws. Documentation/Publications IIRR/Asia has done an inadequate job of documenting its own experiences and lessons learned at the community level. Annex B USAID Evaluation Report Relatedly, it has not done enough in terms of helping local people themselves to document their own developmnet and organizational experiences. Yet such documentation could have very positive advocacy effects, in that rural groups or communities could learn from each other. Partnerships/Capacity-Building/Networking IIRR/Asia needs to do more in terms of collaborating and linking with (especially local) government, both in start-up project planning and in planning for sustainability, because "When IIRR leaves, it's the local government to whom farmers must turn." Again, IIRR/Asia does not do enough to win government stakeholdership and funding, or to avoid duplication of efforts with government priorities and projects (although one FG member felt the Institute has visibly improved in this regard across the years). Relatedly, IIRR/Asia must take care not to overstep its role in terms of taking on jobs that are more properly the purview of government or other agencies. Some of the examples given included mounting irrigation schemes and distributing work animals. FG RECOMMENDATIONS TO IIRR/ASIA "Graduate" community organizers into community managers. Make a greater effort to document IIRR/Asia's own experiences in rural development, and to help communities do likewise. Feed such documentation into policy dialogue. Strengthen information sharing, outreach, institutional ability to respond to queries and walk￾ins, and demand-led networking and information dissemination. Do more field-based projects, à la "social labs," as IIRR/Asia used to. However, make sure always to do such projects collaboratively, involving all relevant existing institutions. In particular, include local government, as this is where many resources are now to be found that would help support IIRR/Asia field projects. At the same time, such linkages and the resources they bring could help free up some of the Institute's own funds for other purposes. On the other hand, in the collaborative process, take care that IIRR/Asia does not lose or submerge its own identity. Annex B USAID Evaluation Report Influence government to allocate more resources for sustainable agriculture and natural resource management within communities. Take a more proactive stance on "the issues." Come up with a summary of key ideas/positions based on IIRR's many years of work. Increase opportunities for internal institutional reflection and for Institute workers and partners to have input into the organization's directions. a One FG member had to leave before the strengths/weaknesses and recommendations exercise. A limited selection of his subsequent e￾mailed comments is included here, insofar as these echoed opinion expressed by the group at large. More idiosyncratic comments from this individual are excluded insofar as they were not validated in group discussion (the whole point of a FG). Annex B USAID Evaluation Report LATIN AMERICA FG FINDINGS STRENGTHS Training IIRR/LAC staff are clearly knowledgeable in, and have good control over, the subject matters in which they provde training, to wit: diagnosis, needs assessment, logframing and other kinds of planning, farmer-to-farmer and other extension methods, "action learning" research, and evaluation -- all in a participatory mode. Both IIRR/LAC's training in -- and its actual use and modelling of -- participatory methods of all sorts is outstanding. IIRR/LAC is extremely responsive to its partners'/clients' real training needs, both in terms of methodological and technical information for practical use in the field. The same cannot be said of many other PVOs or NGOs in the region. In like vein, IIRR/LAC often goes "way beyond the original work contracted for" by a given institution, e.g. to include additional training topics or TA as such needs become visible in the transcourse of training (or other activities). Moreover, it does this at no extra charge. Trainees have been able to put their IIRR/LAC training to work in concrete ways, such as: writing proposals; building participatory approaches into proposals and fieldwork; organizing and reporting to donors; training others of their own and partners' staff in the methods and skills learned from IIRR/LAC; extracting, analyzing, compiling, and publishing lessons learned from an institution's work.  One FG member observed that the latter type of assistance has helped her organization "not to re-invent the wheel."  Another member described how "IIRR/LAC training in logframing helped us [a large environmental institution] prioritize what had been many different random activities thrown together."  Still another avowed that his institution has now applied all its IIRR/LAC lessons institution-wide and is now training its own partners both nationally and internationally, with significant multiplier effects. Documentation/Publications IIRR/LAC publications are consistently excellent and professional. IIRR/LAC's publications are "famous," especialy its agroecology manual. Particularly appreciated is IIRR/LAC assistance in the "systematization" of the vast amount of information and experience that a given institution collects -- but does not always otherwise compile and analyze. Annex B USAID Evaluation Report Partnerships/Capacity-building/Networking IIRR/LAC takes extra trouble to put its partners in contact with other potentially helpful organizations, and with possible sources of funding, according to each of its partners' needs. Whether in its partner PVOs/NGOs, in rural associations or federations, or in CBOs, IIRR/LAC is successful at stimulating processes of self-management. In the words of one FG member, IIRR/LAC "has a way of getting organizations going." In all its interactions, IIRR/LAC is noteworthy for its egalitarian and very "human" treatment of all persons, be they technicians, extensionists, farmers, or still other kinds of groups. IIRR/LAC is never dogmatic about the methods it promotes. It always makes room for its partners, clients, trainees, etc. to add their own knowledge and insights to the effort at hand. Said one FG member, "It is always flexible." Put another way, IIRR/LAC operates in a truly collaborative mode. Or as another FG member noted, "IIRR/LAC makes people feel like they own the work they do with the Institute and the methods they learn from it." Relatedly, IIRR/LAC is very sensitive to the varying and often changing needs of each of its different partners. IIRR/LAC personnel are always available to help when one of their partners needs advice, support, or a critical eye. This is "a natural part of IIRR/LAC's work style."  As one FG member testified, and many others concurred, "I have never known IIRR/LAC staff to turn phone calls or people away." Another strength of IIRR/LAC is the interdisciplinarity of its regional team, and their evident sense of teamship. Likewise for IIRR/LAC's emphasis on valuing and rescuing local knowledge and on appreciating farmers' own skills at practical, on-farm research. IIRR/LAC has strengthened 100% of the institutions it has worked with, as per a vote taken among all organizations represented in the FG. General/Other IIRR/LAC's participatory methods are of universal applicability -- across sectors, nations, or what-have-you. This methodological core is what makes IIRR/LAC different from other PVOs/NGOs in the region, who typically specialize in a sector and/or a geographic region. Relatedly, IIRR/LAC represents "the gold standard," as it were, when it comes to participatory methods in the region. As the representative of an Ecuadorian "think tank" recounted, "When Annex B USAID Evaluation Report we needed someone to train us in participatory methods, we asked around; and 80% of the people we talked to said 'Get IIRR'." Although IIRR/LAC has very limited resources, it invests them very astutely and in a tightly targeted fashion. For its size, IIRR/LAC's reach is impressive. IIRR/LAC clearly practices what it preaches with regard to "learning by doing." WEAKNESSESa IIRR/LAC requires more field staff in order adequately to service the areas in which it is already collaborating with partner projects. With more staff, IIRR/LAC needs to expand its services beyond just the sierra, to include more PVOs, NGOs, and farmer groups working and living in the costa and selva (coast and lowland) ecozones of the region. FG RECOMMENDATIONS TO IIRR/LAC Make clearer or reiterate the relative roles and comparative advantage of IIRR/LAC and its partners, i.e. the former brings the participatory methodological and capacity-building expertise while the latter contribute the more biological/technical skills and savvy to their collaborative efforts. In adding field staff to work on IIRR/LAC collaborative field projects, make sure that one such staffer is Kichwa-speaking. Broaden IIRR/LAC's agroecological areas of work to give more attention to the coast and the lowlands. Produce a second, updated and expanded edition of the agroecological manual, incorporating new findings and field experiences from IIRR/LAC and its partners and field collaborators since the first edition (i.e., since 1995). Also produce a new volume that presents case studies and systematizes the experiences of IIRR/LAC and its partners in the realm of participatory methods and institutional capacity￾building. Be prepared to offer new services as partner needs and demands shift. Annex B USAID Evaluation Report Do more in terms of serving, systematically, as a "bridge" between IIRR/LAC partners and possible funding sources for them.  A few FG suggestions in this regard were to establish a database of donors, and to pursue joint modes of financing. In order to achieve many of the foregoing recommendations, intensify strategic collaborations (in the sense of rational divisions of labor) between IIRR/LAC and appropriate partners who can functionally yet cost-effectively allow the Institute to extend its reach and impact -- whether geographically, thematically, technically, or service-wise. a Despite repeated prodding from the evaluator, the FG could not think of any additional weaknesses (except one wag's mention of the LAC Director's hair style). They explained that this was because IIRR/LAC responds so directly to the specific, actual methodological and other capacity-building needs of its partners/clients. Annex B USAID Evaluation Report ATTACHMENT A: FG INTERVIEW GUIDEa 1. Please introduce yourself and, briefly, your organization and your place within it. 2. How long has your organization been aware of IIRR's existence here in [country] and how did the organization first learn of IIRR? Also, when (and who from) your organization first initiated contact with IIRR, whether formally or informally? 3. What has been your organization's relationship(s) to IIRR? Please indicate all that apply.  General sharing of information, networking, other informal relations.  Contracting IIRR personnel as consultants.  Receiving IIRR training or technical assistance.  Collaborating with IIRR on the joint design and/or delivery of training events, workshops or conferences, publications.  Preparing and submitting joint proposals.  Partnering with IIRR in field projects.  Other. 4. Please describe one or two instances of these interrelationships, including:  the formal mechanism employed (MOU, subgrant, contract, etc.);  relative roles, responsibilities, and decision-making power, of your organization and IIRR, including your organization's control over funds involved;  the quantity and quality of communication from IIRR to your organization (and, does your organization have access to e-mail/internet?);  what changes in the capacity* of your organization was hoped for --and then achieved or not achieved -- thanks to its relationship with IIRR; and  any efforts made to help your organization learn how to track and evaluate, and report on these or other kinds of results achieved;  any efforts made to evaluate the quality and scope of the relationship itself. (*) Capacity may include new or improved: skills among staff, staff members, activities, beneficiaries, organizational or beneficiary earnings, beneficiary quality of life, equipment, communications access, legal status, etc. 5. As a focus group, let us now chart the strengths and weaknesses of IIRR as a partner to your organizations and others like them. 6. Also as a focus group, what recommendations would you make to IIRR in order to help it improve its work with organizations like yours in future? a The professionally-translated Spanish-language version of the guide is available upon request. Annex B USAID Evaluation Report ATTACHMENT B: ILLUSTRATIVE FG SCHEDULE Date 1999 Pre-session Arrivals, coffee/tea, materials distribution. 09:00-09:05 Welcome by senior IIRR staffer, who then departs. 09:05-09:15 Opening comments, purpose of session, the FG concept, and agenda. 09:15-10:30 FG questions, as per interview guide. 10:30-11:00 BREAK 11:00-11:30 Strengths/weaknesses exercise. 11:30-11:45 Brainstorming partner recommendations. 11:45-12:00 Wrap-up, with IIRR staffer back now to close the session with thanks. ANNEX C: DIP SCHEDULE OF ACTIVITIES PROGRAM PURPOSE 1: To strengthen the capacity of IIRR’s NGO partners to implement participatory, people-centered local development programs TASK TARGET ACHIEVED VARIANC E REMARKS HEADQUARTERS 1. Conduct training courses on rural development and related critical issues • International Course on Rural Development Management 6 5 -1 Only 1 course in 1999. • International Development Communication 3 1 -2 Course discontinued due to low demand. • Training of Trainers on Coastal Resource Management 3 1 -2 None. • Systems in Community Managed Health 3 3 0 None. • Gender Analysis in Agriculture, Forestry and Natural Resources 3 1 -2 Course partnership with IRRI discontinued. • Training of Trainers on Sustainable Agriculture 3 3 0 None. • Environmental Management Seminar on Integrated Conservation and Development 3 2 -1 1999 course postponed until Year 2000; planned for Nepal venue. • International Course on Farmer-Led Extension 3 3 0 None. • International Training on Enterprise Development Approaches 2 0 -2 Course development not yet realized. • International Course on Participatory Monitoring and Evaluation 3 2 -1 1999 course postponed until Year 2000 due to course reformulation. • Watershed Training 1 1 0 None. • Household Food Security Through Home Gardening Course 3 2 -1 Course re-formulated in 1998 for higher-level course offering in 1999. • Regenerative Agriculture for Project and Program Managers 2 0 -2 Combined with Household Food Security course. • Local Government Course 0 1 +1 Course development for Bangladesh context. 2. Production of trans-regional communication materials and publications • Information kit on environmental health 110 Completed in 1999. • Information kit on coastal livelihood options 110 Completed in 1998. • Information kit on coastal resource management 110 Completed in 1998. • Other unplanned trans-regional publication 0 11 +11 Various titles. ASIA 3. Conduct national and regional training courses, workshops, seminars • Development of curriculum on Regional Training of Trainers on Training Course Design and Management (Philippines) 110 Upgraded to an International Course and offered in 1999. • Conduct Regional Training of Trainers on Training Course Design and Management (Nepal) 110 None. • Development of curriculum for a regional course on managing training programs (Philippines) 110 International Course realized in 1999. • Holding of the Regional Course on Managing Training Programs (Nepal) 110 None. • Training of Trainers on Presentation and Facilitation Skills (Nepal) 110 None. • Sustainable Rural Aquaculture for Small Scale Farmers (Thailand) 110 Conducted in 1999; held in Bangladesh rather than Thailand. • Training Workshop on Documentation of Development Experience (Bangladesh) 2 1 -1 None. • Training on NGO Capacity-Building (Bangladesh) 3 1 -2 None. • Others 4. Respond to requests for technical advisory services of selected NGOs and GOs: • Needs assessment of CIDSE training activities and evaluate current training program (Cambodia) 110 None. • Case studies on impact of CIDSE projects (Cambodia) 110 None. • Workshop on Participatory Rural Appraisal for Philippine partners of Christian Aid (Philippines) 110 None. • Program development and management seminar-workshop for CRS Southeast Asia (Philippines) 1 0 -1 CRS changed venue of workshop. • Develop capacity of the Philippines National Red Cross in integrated health development (Philippines) 1 0 -1 None. • Capacity building of Plan International in undertaking health projects (Philippines) 110 None. • Others 5. Initial profiling of RR alumni for the purpose of developing a database benchmark data (Nepal/Bangladesh) 1 0 -1 Not completed. 6. Identification and selection of NGOs to be assisted: • Consultation with NGOs to identify organizational needs 1st Quarter 1997 N/A N/A Completed on a need-basis with several NGO’s in the three regions throughout the MG period. • Determining appropriate interventions (field programs, training, consultancy, etc) 2nd Quarter 1997 N/A N/A Completed on a need-basis with several NGO’s in the three regions throughout the MG period. 7. Generate knowledge through the implementation of field projects using participatory approaches in collaboration with selected organizations: • Design and implement the Ecological Health Care System Development Project with Rural Reconstruction Nepal (Nepal) 1 Partially completed N/A Completed Activities: 1. PAR training 2. Data collection 3. Proposal development 4. Proposal marketing • IIRR-ICIMOD agricultural technology profiling, development and dissemination (Nepal) 1 0 -1 None. • Design and implement a project on integrating primary health care in rural Nepal with the Community Welfare and Development Society (Nepal) 1 0 -1 None. • Design and implement a project on establishing health systems at the district level with the Support Activities for Poor Producers of Nepal 110 None. • Design a sustainable agriculture program (Bangladesh) 1 Partially completed N/A Completed Activities: 1. Partner identification 2. Data collection 3. Proposal development 4. Proposal marketing • Seek funding and implement integrated development field project with the Community Development Association (Bangladesh) 1 Partially completed N/A Completed Activities: 1. Data collection 2. Proposal development 3. Proposal marketing • Start Phase 2 of the Appropriate Health Model Project in 4 communities in Cavite (Philippines) 110 None. • Phase-over the Children’s Program to the Santo Domingo People’s Cooperative in Albay (Philippines) 1 Still on-going N/A To be completed in year 2000. • Pursue implementation of the water and sanitation aspect of the Sustainable Area Development Project in Marinduque with the Philippine Rural Reconstruction Movement (Philippines) 1 0 -1 None. 8. Regular monitoring and participatory evaluation of on-going field projects Continuous N/A N/A Realized in select field projects in all three regions: 1. Appropriate Health Model (Philippines) 2. Children’s Program (Philippines) 3. Upland Farm Management Project (Philippines) 4. Farmer-to-Farmer Extension (Ecuador) 5. Farmer Participatory Research (Ecuador) 6. Local NGO Cluster Group Capacity￾building (Ethiopia) 7. Small-scale Water Resources Development (Ethiopia) 9. Process documentation of field programs and capacity development efforts for future sharing of information and learnings 10. Production of technical papers and/or communication materials to share learnings generated from the field program and networking • Publication of the SARR Newsletter: Development Monitor (Nepal) 10 -1 None. • Preparation of discussion/working papers on development experiences (Nepal) 10 -1 None. • Preparation for the production of an information kit on aquaculture (Bangladesh) 11 0 None. • Actual production of the information kit on aquaculture (Bangladesh) 11 0 Bengali language publication produced in 1998. • Production of an information kit on natural resource management in rainfed drylands (India) 11 0 English language publication produced in 1998. AFRICA 11. Holding of national and regional training courses/seminars/workshops for all interested GOs and NGOs in the region especially those based in the focus countries: • Regional Food Security Through Watershed Management Course (Kenya) 32 -1 Course discontinued as such, as other groups are offering similar courses for free. Aiming to re-tool the course to be more field-oriented. Concept paper exists. • Regional Training of Trainers Course on Sustainable Agriculture (Kenya) 33 0 One course conducted in Tanzania, • Regional course on Needs Assessment, Project Design and Proposal Writing (Kenya) 34 +1 None. • Farmer-to-Farmer Extension Training (Ethiopia) 11 0 None. • Regional Course on Participatory Monitoring and Evaluation (Kenya) 33 0 None. • Regional Managers’ Course (Kenya) 33 0 None. • Gender Analysis Course (Ethiopia) 34 +1 None. • Rural Development Management (Ethiopia) 14 +3 Additional course in Ethiopia completed. Also, completed course in Gambia and Uganda. • South-to-South Exchange in Dryland Agriculture: India￾Africa Experience, a Roving Workshop (Kenya) 10 -1 Course sponsorship not acquired. • Participatory Monitoring and Evaluation (Ethiopia) 11 0 Completed for Ethiopian NGO umbrella organization – CRDA. • Customized courses on PM&E 30 -3 Not realized. • TOT for sustainable Agriculture (nationally customized) 04 +4 Three in Ethiopia, one in Zambia. 12. Strategic plan development of the organizations in the cluster and based on the results, development of a detailed capacity building and mentoring plan (Ethiopia and Kenya) • Implementation of the developed capacity building and mentoring plan (Ethiopia) 15 +4 Worked with 12 Ethiopian NGO’s, of which six have completed their strategy documents under the mentorship of IIRR. 13. Development of a plan for the implementation of Six-month pilot study funded by reproductive health and population initiatives for the region Belgium (see LOWO report). Proposal developed. • Conduct trans-regional conference on reproductive health and communications at the grassroots: Africa-Asia Experience (Kenya) 11 0 None. • Production of manual of case studies from above-referenced conference. 11 0 None. 14. Development of a more focused gender program centered around women economic activities (Ethiopia) • Formation of informal networks among organizations of cluster groups interested in pursuing gender and development initiatives 11 0 None. • Conduct training courses/planning workshops on gender auditing, gender planning and development 15 +4 None. 15. Process documentation of field programs and capacity development efforts Helped CARE write case studies of their experiences in Basic Education and produced a manual. 16. Production of technical papers and/or communication materials to share learnings generated from the field program and networking • Production of a manual on Micro-Level Efforts to Stop Dissertification in Sub-Saharan Africa: Best Practices (Kenya) 1 Re-scheduled for Year 2000 or 2001 N/A Modified concept to become a proposal to do a manual of best practices in dryland farming for East and South Africa. Half of funding already obtained. • Holding of a workshop to produce a manual focusing on the best practices on sustainable agriculture in the Great Lakes region (Kenya) 11 0 None. LATIN AMERICA 17. Provision of technical advisory services to the following organizations and others upon request throughout the duration of the Matching Grant Program • Catholic Relief Services (CRS) on systematization (documentation and learning) of their credit and health development project (Ecuador) To be continuous through 1999 Work completed in 1997 CRS changed the project after 1 year of implementation. • Ecuadorian Research Institute (INIAP) on participatory production of sustainable agriculture information materials (Ecuador) 11 0 Continuous relationship. • Pan American Health Organization (PAHO) on project design and planning of water sanitation projects (Ecuador) 11 0 None. • Support in the form of technical details for publication production, individual technical review and editing of follow-on publications. 03 +3 • SWISS AID manual • INIAP material(new) • CARE-PROMUSTA manual 18. Holding of at least 10 national training courses and workshops mainly focusing on participatory tools for project identification, planning and systematization and participatory tools for the development, dissemination of sustainable agriculture technologies and sustainable technologies per se • Workshop on Systematization (Ecuador) 3 2 -1 None. • Workshop on Participatory Technology Development (Ecuador) 6 5 -1 1999 course offered in Peru, not Ecuador. • Workshop on Participatory Rural Appraisal (Ecuador) 3 3 0 1999 course offered in Paraguay, not Ecuador. • Workshop on ZOPP (Ecuador) 3 4 +1 All 4 courses offered in 1998. • Workshop on Participatory Rural Appraisal (Honduras) 1 0 -1 Negotiations with Honduran NGO were completed; but course never materialized. • Course on Participatory Research with Farmers (Ecuador) 1 1 0 None. • Course on Farmer-to-Farmer Participatory Extension (Ecuador) 1 1 0 Three additional courses in 1998. • Courses on Sustainable Agriculture Technologies (Ecuador) 10 87 +77 19 courses in 1997 26 courses in 1998 42 courses in 1999 19. Provision of follow-up activities to training alumni upon their request focusing on the practical application of skills acquired during the training Continuous 35 % of alumni request follow-up and are responded to - CIALs - Additional courses for farmer￾farmer extension communities 20. Pursue the development and implementation of new field program initiatives and continue implementation of on-going activities • Farmer-to-Farmer Participatory Research 1 Started in 1995. Still on-going. 0 29 active CIALS with ??? farmers; ??? courses with ??? participants • Farmer-to-Farmer Extension Program 1 Still on￾going. 0 150 active promoters throughout the Ecuadorian Andean region 21. Process documentation of field programs and capacity development efforts for future sharing of information and learnings 22. Production of technical papers and/or communication materials at the end of the program to share learnings generated from the field program • Information kit on bio-intensive gardening (Ecuador) 11 0 None. • Book on farmer-to-farmer extension in Spanish (Ecuador) 11 0 None. • Book on farmer-to-farmer extension in English (Ecuador) 11 0 Translation done by HQ staff. • Workbook on participatory rural appraisal in Spanish (Ecuador) 11 0 English translation done by regional office staff. • Workbook on logical framework in Spanish (Ecuador) 10 -1 In process; re-evaluating need for the publication based on literature review. • Workbook on systematization in Spanish and English (Ecuador) 22 0 Both language versions completed by regional office staff. • Information kit on cover crops/green manure (Honduras) 11 0 None. • Others 01 +1 Farmer Participatory Research Manual￾1999. PROGRAM PURPOSE 2: To increase the capacity and sustainability of IIRR’s Regional Centers (a) to deliver and sustain services to NGOs; (b) to undertake process documentation and impact evaluation; and (c) to promote inter-institutional learning and networking. TASK TARGET ACHIEVED VARIANC E REMARKS HEADQUARTERS 23. Participation in a workshop on strategies for sustainable development to identify inputs for IIRR’s resource 1 1 0 Four Director-level staff participated. generation strategy. 24. Institutional action planning to identify key issues and actions required to help IIRR attain sound financial, structural and management systems and make it an effective and efficient learning and sharing organization for rural reconstruction. 110 February 1997 “Ten-Point Action Plan” developed. 25. Design and implementation of an efficient and effective financial management information system 1 Partially completed N/A Installation and refinement still on- going. 26. Clarification of staff roles, responsibilities and decision-making processes 1 Partially completed N/A Some clarity achieved during Nov. 1999 Strategic Planning workshop. Additional details still to be developed. 27. Implementation of a human resource development system 1 Partially completed N/A Installation and refinement still on￾going. 28. Development and implementation of a comprehensive resource generation strategy that will cover all core costs associated with relevant high quality programs 1 Partially completed N/A Some improvements realized, but more work still required. 29. Institutionalization of a system for multi-level monitoring and evaluation 1 0 -1 Minimal progress realized. 30. Implementation of policy and guidelines on NGO networking and cooperation 1 0 -1 None. 31. Analysis of alternative organization structure 110 Organizational structure modified in early 1998 resulting from the Feb. 1997 Action plan. 32. Development of systems, processes and procedures for packaging and sharing lessons learned during IIRR’s field experiences 1 0 -1 None. 33. Clarification of issues of governance of IIRR 1 Partially completed N/A Significant progress realized. 34. Hiring of required staff (personnel officer, marketing & evaluation) 1 Partially completed N/A Personnel and marketing staff hired; M&E staff not hired. 35. Enhancement of staff capabilities • Training of Trainers for IIRR staff 1 1 0 Various HRD and staff development opportunities realized for IIRR training personnel (both in-house, and external courses). ASIA 36. Establishment of sub-regional office for South Asia 1 1 O Office established in 1997; but as of 1999, still not fully operational. 37. Collaborate with headquarters the development and implementation of systems, processes and procedures indicated in item # 25-32 in the region N/A N/A N/A See items #25 – 32 above. ROA Director involved in the systems development work mentioned in those items. 38. Setting-up of a development weather station for South Asia to monitor current thinking and emerging trends on critical development issues in the region 1 0 -1 None. 39. Enhancement of staff capabilities • Participation in Rural Development Management training activities 2 1 -1 None. • Attendance in a skills enhancement course on financial management & office administration 1 0 -1 None. • Participation in a training of trainers course 1 0 -1 None. 40. Hiring of required additional staff • Office Manager 1 1 0 None. • Project Officer 1 1 0 Hired, then left. • Director (Nepal Office) 1 0 -1 None. AFRICA 41. Explore the possibility of establishing learning centers in the region 110 Environmental learning center is in an Ethiopian high school. Started after an exposure visit by the principal, 29 teachers and two students. On-going. 42. Collaborate with headquarters the development and implementation of systems, processes and procedures indicated in item # 25-32 in the region. N/A N/A N/A See items #25 – 32 above. ARO Director involved in the systems development work mentioned in those items. Quick book financial system is installed and staff trained on how to use it. 43. Enhancement of staff capabilities • A training staff of the regional center participates in the International Course on Development Management conducted at the Headquarters 1 3 +2 None. 44. Hiring of required additional staff • Monitoring and evaluation officer 1 1 0 Secondment by DED. • Gender specialist 1 2 +1 None. • Program coordinator 1 1 0 Hired, but left. LATIN AMERICA 45. Collaborate with headquarters the development and implementation of systems, processes and procedures indicated in item # 25-32 in the region N/A N/A N/A See items #25 – 32 above. ROLA Director involved in the systems development work mentioned in those items. ANNEX D: Selected Raw Data Compilations D-1 IIRR Training and Related Data. D-2 AFR Regional Staff D-3 Ethiopia Country-Office Staff D-4 LAC Regional Staff D-5 Grant Income Sources D-6 Earned Incomes Sources D-7 Notes on Data Analysis Process Annex D-1: IIRR Training and Related Data 1992 - 1995 1996 - 1999 No. of Events No. of Agencies No. of Nations No. of Women No. of Men Total No. of Events % Change No. of Agencies % Change No. of Nations % Change No. of Women % Change No. of Men % Change Total % Change AFRICA Regional Courses National Courses Community Level Courses Writeshops Conferences TOTAL 9 15 12 1 0 37 114 111 3 11 0 239 7 6 3 1 0 7 39 44 31 8 0 122 146 270 399 29 0 844 185 314 430 37 0 966 18 27 0 3 1 49 100% 80% -100% 200% N/A 32% 197 352 0 60 32 641 73% 217% -100% 445% N/A 168% 20 3 0 9 10 27 186% -50% -100% 800% N/A 286% 71 131 0 36 18 256 82% 198% -100% 350% N/A 110% 274 482 0 107 16 879 88% 79% -100% 269% N/A 4% 345 613 0 143 34 1135 86% 95% -100% 286% N/A 17% ASIA/YC International Courses Regional Courses National Courses Community Level Courses Study Programs Writeshops Conferences TOTAL 14 11 47 2 45 16 9 144 227 174 344 12 129 286 231 1403 42 22 11 1 29 20 39 61 82 45 264 1 75 125 56 648 203 213 725 13 250 474 242 2120 285 258 989 14 325 599 298 2768 29 7 16 2 36 16 5 111 107% -36% -66% 0% -20% 0% -44% -23% 365 119 199 2 85 233 123 1126 61% -32% -42% -83% -34% -19% -47% -20% 38 20 7 2 21 9 31 51 -10% -9% -36% 100% -28% -55% -21% -16% 134 35 109 22 48 147 54 549 63% -22% -59% 2100% -36% 18% -4% -15% 377 115 379 29 219 439 98 1656 86% -46% -48% 123% -12% -7% -60% -22% 511 150 488 51 267 586 152 2205 79% -42% -51% 264% -18% -2% -49% -20% LATIN AMERICA Regional Courses National Courses Community Level Courses Writeshops Conferences TOTAL 1 20 0 0 1 22 8 269 0 0 51 328 8 5 0 0 1 11 2 100 0 0 15 117 54 365 0 0 51 470 56 465 0 0 66 587 2 27 87 4 0 120 100% 35% N/A N/A -100% 445% 16 207 87 55 0 365 100% -23% N/A N/A -100% 11% 6 5 1 7 0 11 -25% 0% N/A N/A -100% 0% 7 139 500 21 0 667 250% 39% N/A N/A -100% 470% 38 354 640 155 0 1187 -30% -3% N/A N/A -100% 153% 45 493 1140 176 0 1854 -20% 6% N/A N/A -100% 216% TOTAL INSTITUTE International Courses Regional Courses National Courses Community Level Courses Study Programs Writeshops Conferences GRAND TOTAL 14 21 82 14 45 17 10 203 227 296 724 15 129 297 282 1970 42 36 21 4 29 21 39 64 82 86 408 32 75 133 71 887 203 413 1360 412 250 503 293 3434 285 499 1768 444 325 636 364 4321 29 27 70 89 36 23 6 280 107% 29% -15% 536% -20% 35% -40% 38% 365 332 758 89 85 340 155 2124 61% 12% 5% 493% -34% 14% -45% 8% 38 46 15 3 21 23 34 67 -10% 28% -29% -25% -28% 10% -13% 5% 134 113 379 522 48 204 72 1472 63% 31% -7% 1531% -36% 53% 1% 66% 377 427 1215 669 219 701 114 3722 86% 3% -11% 62% -12% 39% -61% 8% 511 540 1594 1191 267 905 186 5194 79% 8% -10% 168% -18% 42% -49% 20% Annex D-2: AFR Regional Staffa 1994 1995 1996 1997 1998 1999 Regional Director I. Bekalo (M) Regional Director Same Regional Director Same Regional Director Same Regional Director Same Regional Director Same Training Coordinator N. Mwaura (F) Training Coordinator Same Training Coordinator Same Training Coordinator Same Position vacant Sr. Training Officer V. Callo (F) Driver S. Liuva (M) Driver Same Driver Same Driver Same Driver Same Driver Same Executive Secretary G. Kinyanjui (F) Admin. Officer Same Admin. Officer Same Admin. Officer Same Admin. Officer Same Accountant Intern D. Siboe (F) Accountant Same Accountant Same Accountant Same Accountant Same Secretary E. Juma (F) Secretary Same Secretary Same Secretary Same Office Assistant E. Mulinya (F) Office Assistant Same Office Assistant Same Office Assistant Same Training Officer J. Adede (M) Training Officer Same Training Officer Same Training Officer Same M&E Officer M. Schroll (M)b M&E Officer Same M&E Officer M. Schueller (M)b Accountant Intern E. Ratemo (F) Accountant Intern Same Caretaker G. Bakhwenya (M) Caretaker Same Sr. Program Coordinator J. Juma (M) Position eliminated Communications Spec. G. Obanyi (M) Research Intern C. Obote (M) a Staff are fulltime unless otherwise noted. b Staff seconded by the German development agency, DED. Annex D-3: Ethiopia Country-Program Staffa 1994 1995 1996 1997 1998 1999 Program Coordinator M. Woldemariam (F) Program Coordinator Same Program Coordinator Same Program Coordinator L. Berhanu (M) Secretary/Cashier E. Tefera (F) Secretary/Cashier Same Secretary/Cashier Same Secretary/Cashier Same Driver/Clerk W. Girma (M) Driver/Clerk Same Driver/Clerk Same Driver/Clerk Same Training Officerb S. Gerbu (M) Training Officerb Same Training Officerb Same Food Security Expertb T. Alemu (M) Food Security Expertb Same Position vacant at mid-year Gender Specialistb A. Gashew (M) Gender Specialistb Same Jr. Gender Officerb H. Bekele (F) Sr. Gender Specialistb Same Office Assistant B. Seboke (F) Office Assistant Same Two Guards (M) Two Guards (M) a Staff are fulltime unless otherwise noted. b Staff supported full- or part-time by project monies. Annex D-4: LAC Regional Staffa 1993 1994 1995 1996 1997 1998 1999 Regional Director D. Selener (M) Regional Director Same Regional Director Same Regional Director Same Regional Director Same Regional Director Same Regional Director Same Accountant M. Espinosa (F) (1 hour/day) Accountant Same (2 days/week) Accountant Same (2 days/week) Accountant Same (3 days/week) Accountant Same (3.5 days/week) Accountant Same (4 days/week) Accountant Same (4 days/week) Office Assistant W. Cisneros (M) (1 hour/day) Office Assistant Same Office Assistant Same Position vacant Project Secretaryb W. Cisneros (M) Project Secretaryb Same Project Secretaryb Same Project Coordinatorb J. Carvajal (M) Project Coordinatorb Same Project Coordinatorb Same Project Coordinatorb Same Project Coordinatorb Same Program Officer N. Endara (F) Program Officer Same Program Officer Same Program Officer Same Secretary W. Llumiquinga (F) Secretary Same Secretary Same Secretary Same Assistant Accountant M. Espinosa (F) Assistant Accountant Same Assistant Accountant Same Project Coordinatorb V. Proaño (F) (4 hours/day) Project Coordinatorb Same (4 hours/day) Project Coordinator Same (4 hours/day) a Staff are fulltime unless otherwise noted. b Staff on project-based contracts. Annex D-5: Grant Income Sources 1992 – 1995 1996 – 1999 UN/CGIAR Agencies (N = 3) (N = 3)  CIAT (Colombia)  UNDP  UNICEF  CGIAR NGO Committee (US)  CIAT (Colombia)  UNDP Multi-lateral Agencies (N = 1)  World Bank Small Grants Program Bilateral Agencies (N = 4) (N = 5)  Dutch Embassy (Philippines)  SIDA (Sweden)  Taiwan Government  USAID  DGIS (Netherlands)  Dutch Embassy (Ethiopia, Philippines)  GTZ (Germany)  Taiwan Government  USAID (US) Foundations (N = 10) (N = 10)  Asia Foundation (US)  Food for All (US)  Ford Foundation (US)  IDRC (Canada)  International Foundation (US)  John D. and Catherine MacArthur Foundation (US)  MAZON (US)  McKnight Foundation (US)  Rockefeller Brothers Fund (US)  Starr Foundation (US)  Asia Foundation (US)  Bedminster Fund (US)  Ford Foundation (US)  IDRC (Canada)  International Foundation (US)  MAZON (US)  McKnight Foundation (US)  RMAF (Philippines)  Rockefeller Brothers Fund (US)  Starr Foundation (US) PVOs and NGOs (N = 11) (N = 14)  GAA (Germany)  ICCO (Netherlands)  IIED (UK)  Irish Freedom from Hunger Campaign  LWR (US)  MISEREOR (Germany)  NOVIB (Netherlands)  SwissAid (Switzerland)  UK Freedom from Hunger Campaign  World Neighbors (US)  WWF  BILANCE (Netherlands)  CEBEMO (Netherlands)  CIIFAD – Cornell University (US)  CTA (Netherlands)  GAA (Germany)  ICCO (Netherlands)  IDS (UK)  IIED (UK)  IIZ/DW (Germany)  LWR (US)  MISEREOR (Germany)  NOVIB (Netherlands)  OXFAM-America  SwissAid (Switzerland) Private Sector (N = 3) (N = 5)  Bristol Myers Corporation (US)  Citibank (US)  FICAH (US)  Abbott Laboratories (US)  Bristol Myers Corporation (US)  Citibank (US)  Cleary, Gottlieb, Steen and Hamilton (US)  FICAH (US) Annex D-6: Earned Income Sources 1992 – 1995 1996 – 1999 UN/CGIAR Agencies (N = 3) (N = 7)  FAO  ICIMOD (Nepal)  UNICEF  CGIAR NGO Committee (US)  FAO  IFAD  IRRI (Philippines)  UNESCO  UNICEF  WHO Multi-lateral Agencies (N = 3)  ADB (Philippines)  EU  SMISLE (EU- Philippines) Bilateral Agencies (N = 7) (N = 13)  ASEAN – Canada Fund (Singapore)  British Council (India, Bangladesh)  CIDA (Canada)  DANIDA (Denmark)  DENR (Philippines)  ODA (UK)  SDC (Switzerland)  British Council (India, Bangladesh)  CICETE (China)  CIDA (Canada)  DANIDA (Denmark)  DENR (Philippines)  DFID (UK)  GTZ (Germany)  ICAR (India)  MANAGE (India)  PCARRD (Philippines)  SDC (Switzerland)  SIDA (Sweden)  SNV (Netherlands) Foundations (N = 1)  Asia Foundation PVOs and NGOs (N = 15) (N = 23)  ACT (Belgium)  Africa 2000 Network (Ghana)  Asian Soil Conservation Network  Bread for the World  CIDSE (Cambodia, Laos and Vietnam)  CRS (US)  CTA (Netherlands)  EZE (Germany)  HPI (US)  ICCO (Netherlands)  MISEREOR (Germany)  OXFAM  PACT (US)  SEEP (US)  ACHAN (Philippines)  ACT (Belgium)  ActionAid (Vietnam, Kenya, others)  AMREF (Kenya)  BEAF (Germany)  CARITAS-Bangladesh  Christian Aid (UK)  CIDSE (Cambodia, Laos and Vietnam)  CRDA (Ethiopia)  CRMP (Philippines)  CTA (Netherlands)  Haribon Foundation (Philippines)  Helvetas (Switzerland)  ILEIA (Netherlands) 1992 – 1995 1996 – 1999 • WWF • ITDG (UK) • LWR (US) • MISEREOR (Germany) • OXFAM-America (US) • PAMANA (Philippines) • PRRM (Philippines) • RECOFTC (Thailand) • SCF (US) • WWF Private Sector (N = 3) (N = 4)  Inter-Forest (Sweden)  Louis Berger (GEM-Philippines)  Orgut Consulting (Sweden)  DAI (US)  ETC (Netherlands)  Inter-Forest (Sweden)  Louis Berger (GEM-Philippines) Item D-7: Notes on Data Analysis Process The following is a free-flowing description of how the evaluator and IIRR jointly decided and worked together on generating the data for the present report, in the absence of a fully functioning M&E system at the Institute. It is offered in hopes that other PVOs might profit in some way from this experience. Before the Evaluation While the evaluation was still in the planning stage and before any contact with the PVO, the evaluator requested that the SOW include a rigorous Focus Group (FG) methodology [Annex B]. The intent was to provide for a double-check on IIRR-generated data, via indepedently gathered assessments of Institute services and achievements from the vantage of IIRR's partners and clients. This bit of methodological design was done with no foreknowledge of existing IIRR M&E data. As it turned out, IIRR had no comprehensive M&E system and had done little impact assessment. In this event, FG findings became preeminent instead of merely validating data. At the Start of the Evaluation Upon first arriving at IIRR's HQ in the Philippines, the evaluator was surprised to discover that no M&E Specialist had ever been hired, as per the MG proposal's plan and budget. Hence, as noted above, no proper M&E system had ever been built, including the necessary framework of indicators and their accompanying baselines, targets, timelines, and analysis methods. On the positive side, however, IIRR had done considerable work under the MG to install a new FIS [Chapter 5]. Thus some important financial data were available. Also, under this and its previous MG, the Institute had established some Institute-wide data tracking, reporting, and handling mechanisms, including a Central MIS (CMIS) at HQ using Microsoft's relational database, ACCESS. And for some years, HQ units and the Regional Centers (RCs) had systematically collected a limited body of process-level program data (mainly on training) for feeding into the CMIS. On the negative side, however, HQ's receipt, compilation, and computerized entry of even training program data had fallen a year behind for AFR and LAC, for a variety of reasons [see text]. Both a positive and a negative was that many other kinds of data existed, but only in scattered, hard-copy form. In view of this fragmented M&E situation, both IIRR and the evaluator were in something of a quandry as to how to satisfy the SOW's very clear requirements for quantitative evaluation analysis and impact assessement. Fortunately, however, the original designer of the CMIS -- who was also an expert in the FIS -- still worked at IIRR, albeit in a new position. In consultation with the IIRR manager responsible for oversight of M&E, the decision was taken to allow this data analyst (IIRR's only one) to take time off from her other duties to try to bring basic data up-to-date in the CMIS. Thereafter, both these individuals worked with the evaluator to compile the resulting CMIS and other data into useful arrays for evaluation analysis. The evaluation schedule was such that this plan seemed feasible. It spanned several months of intermittent tasks and travel on the evaluator's part (including commitments other than the evaluation). Also, for IIRR-internal reasons, the schedule called for an initial and then a second evaluator visit to HQ, in Oct and Nov 1999, with evaluator travel to AFR and LAC in the interim. So, there seemed to be time between visits for IIRR to do some basic data updating and compilation. Once all her overseas travel was completed, the evaluator was to draft report back in the US during Dec. During Oct, the ad hoc "team" of the evaluator, the IIRR analyst, and the M&E manager sat together to review the kinds of information actually or potentially available and -- in the absence of any M&E framework -- to begin thinking about what kinds of general indicators and baselines (but not, of course, targets or timelines) it might be possible to construct ex post facto, and what analytic methods could reasonably be applied to them. In fact, only simple descriptive statistics (e.g., raw numbers, percentages, means) were feasible under the reigning data and time constraints. The evaluator then drew up a number of "table shells" for IIRR to be filling in until her return to HQ, as missing or backlogged data were flowed to the HQ and entered in the CMIS. The schedule also made it possible to take some additional "emergency data catch￾up" measures, as it were. Initiated during the first HQ visit, these consisted of three additional methodological steps aimed at gathering supporting qualitative and impact data to complement the mainly process-level data being computerized at HQ. First, the team jointly discussed and designed a number of chronograms and other tables and graphics of both quantitative and qualitative data from the CMIS and other sources. These were to then be prepared between Oct and Nov. Their design decisions were based on: evaluator descriptions of the data-types ideally desired to respond to various SOW items; the data analyst's and M&E manager's knowledge of which such data were actually or potentially available, where and from whom; and how best to use the latter in order to speak to the former. Second and relatedly, the M&E manager and the evaluator drew up a "Data Request Sheet" consisting of a comprehensive list of other information that each RC could reasonably gather up for the evaluation (Figure D.-7.1.). This was shared with AFR and LAC immediately it was designed, so they could begin compiling the information requested. Figure D.-7.1. Data Request Sheet Third, the evaluator initiated a "crash program" of mini-case-studies. The aim was to capture at least anecdotal impact data plus other kinds of exceptional qualitative information pertinent to the PVO's and its partners' institutional growth and development during the MG. These resulted in the "boxes" the reader sees throughout the text of the report. This information was generated in the following way. As the evaluator interviewed IIRR staff and FG members -- first at HQ/ASI and later in AFR and LAC -- she identified especially "impactful" or otherwise telling "stories" about IIRR activities and outcomes during the MG, suitable for mini-case￾study presentation. Once identified, stories were assigned for write-up in a few paragraphs by the most knowledgeable or appropriate IIRR staff or, in some instances, by the evaluator [see below]. At the same time, a system for staff to submit their writing assignments to the HQ M&E manager was established. And a deadline for submissions was set, to coincide with the evaluator's return to HQ. This bit of organizational planning gave staff some lagtime in which to collect their thoughts and write their boxes while ensuring systematic follow-through, centralized collection, and timely submission of assignments from IIRR's many and far-flung units. Once collected, all these "raw data" stories were given to the evaluator for her to query, combine/compress/expand, edit, and include/exclude in the evaluation report as she saw fit. Professional staff contributing to the above effort ranged all the way from the IIRR President down to Country-Office personnel. The evaluator also contributed "boxes" of her own based on interviews she independently conducted with donors, FGs, trainees, and farmers in their fields. This allowed for impact reporting directly from the horse's mouth, so to speak. Thus, this "mini-case-study" approach captured a wide range of inputs on Institute impacts (or the possible lack thereof -- although this issue rarely arose). This approach proved especially helpful in at least two ways. For one thing, in the spirit guiding all Institute work, it made the external evaluation a more participatory exercise. Professionals from all units were invited to "toot their own horns," as it were, by contributing stories of their program successes. This approach made the evaluation less threatening for many staff while generating valuable information and insights from their collective institutional memory. For another thing, the end result was an in-house corpus of qualitative data on which IIRR can now draw for future evaluations, annual and donor reporting, public-awareness materials, and so forth. During the Evaluation Between Oct and the evaluator's return to HQ in Nov, in a planetwide flurry of e￾mails and with assistance from the CMIS Administrator, the data analyst slaved to bring together all existing quantitative data from IIRR's various units and regions and to put these plus all backlogged data into the CMIS. Ultimately, this resulted in massive outputs of raw data. The analyst and the evaluator then re-visited and refined their rough table shells, to arrive at more elegant and meaningful presentations of the quantitative data. At the same time, the HQ M&E manager pulled together the chronograms, lists, and other data tables and graphics mentioned above, in rough initial formats. After later querying, analysis, and re-arraying by the evaluator, these resulted in items like the following. Table 1.1.'s chronogram and Table 1.2.'s and 5.1's Milestones. Not reproduced in the text because of its length, a comprehensive chart of all IIRR publications under the MG, which also tallied data on date of publication, print runs, printing costs and sales value (where known), categorization as to global or region-only relevance, language of publication, general type of publication (e.g., best-practices compilation, workshop/conference proceedings, methodology manual, etc.), and mode of creation (writeshop, research study, etc.). Some of these data are depicted in Figure 2.1.'s map of publications. Table 3.1.'s list of courses. This was initially presented merely as an unorganized list, on which the evaluator then superposed a taxonomy of subject-matter categories. Then, with assistance from the M&E manager, the categorized data were re-arrayed in a Guttman-scale-like format. The visually impactful partnership maps of Tables 4.1.-A through 4.3.-B. Table 5.4.'s informative staffing structures and distributions. The various financial graphs and tables of Chapters 5 and 6. In order to gather up-to-date information for most of the above, the evaluator and the M&E manager (who travelled together to LAC) met at length on-site with RC Directors. Also in AFR -- where data management and staff computer capabilities equalled those of HQ -- AFR staff and the evaluator together experimented with various aggregations and disaggregations of training data, their visual presentation, and simple analytic tools. On the evaluator's return to HQ, the AFR exercise proved helpful in refining the rough table shells that had been designed in Oct. As the overseas portion of the evaluation drew to a close, however, it became evident that compilation all of the data the team had planned for would not be completed by the beginning of Dec, when analysis and report-writing were to begin in the US. What to do? An Evaluation Extension As this fact began to sink in, both IIRR and the evaluator worried that all the extra time and trouble they had invested would be for naught if the data-drive could not be brought to a close in time to allow for proper analysis and write-up. Yet the goal was almost within reach. To solve this problem, they took two steps. First, the evaluator explained the situation to BHR/PVC and won an extension of the timeline. Second, at the evaluator's invitation, IIRR took the decision to fly the data analyst to the US for 1 week, where the two worked side-by-side to do a final cleaning and arraying of all raw data. The analyst concentrated on organizing the CMIS's quantitative data, in consultation with the evaluator. Meanwhile, the evaluator concentrated on refining and analyzing the qualitative data of the mini-case-studies, in consultation with the analyst. After the analyst's return to IIRR, the evaluator was then able to begin final analysis and write-up of most of the quantitative information. It should be noted that the timeline was extended at no additional cost to the donor. IIRR defrayed its analyst's travel expenses while the evaluator donated her time gratis, along with office space and housing for the analyst. This joint effort took place in early Dec. Although the very last dribs and drabs of data did not reach the evaluator until the first week of Jan, this extraordinary effort did permit completion of the ambitious agenda set in Oct "to do what we could with what we had," as the analyst put it. At the Evaluation's End At the end of this big push, not only did an evaluation report result. In addition, according to IIRR team members and other staff, the following kinds of data-handling and - analysis improvements ensued. The data-drive resulted in an up-to-date consolidation at HQ of all existing training data from all IIRR units. This entailed the analyst's first working with each RC to reconcile cross-regional discrepancies in data definition. Prior to this time, regions differed in their interpretation of activity types. The team's work during the evaluation resulted in the codification of unitary definitions. These were subsequently shared with all RCs and other relevant IIRR units. Once adopted, these definitions should ensure comparability of data across regions and also greatly speed data entry. The foregoing steps permitted the first-ever comprehensive comparative analysis of training data for the Institute as a whole [see text], as versus unit-, region-, or activity-specific analysis only. IIRR took the decision to input into the CMIS all hard-copy data on writeshoppers and conferees from years antedating the CMIS' creation, going back as far as 1990. (For the present evaluation, manual tabulation of older data had to suffice.) A new dataset on numbers of beneficiaries reached by IIRR field projects during the baseline and MG periods was constructed, and the CMIS was modified to make this a standard dataset henceforth. Likewise for the names and acronyms of IIRR's multitudinous partners worldwide. It is anticipated that this dataset will be useful in ongoing updating of the CMIS and in future M&E improvements, some of which were already set in motion as of early 2000. Similarly, the evaluator's demand for a schedule of funding sources and types caused IIRR to consolidate hard-copy data, which will now be utilized in devising the resource-generation plan called for in the Institute's Year 2000 activities. Insofar as feasible, these data will also be computerized. Likewise for the partnership maps, which were compiled from hard-copy data during the evaluation. This information can now be used to improve the CMIS' capacity to support analysis of IIRR's relationships with different types of partner organizations. In sum, IIRR has ended/will end up with a refurbished CMIS with some systematic raw data going back as far as 1990, and with a number of new analytic potentialities and applications. In addition to these more quantitative and CMIS-oriented outcomes, other impacts of the joint IIRR+evaluator efforts include the following. IIRR now has on-disc a useful (albeit still quite limited) qualitative dataset in the form of mini-case-studies. This dataset has already been made available to all IIRR staff, in hopes that they will expand it. With the institution of a LAN at HQ, all the datasets resulting from the evaluation's data-drive will be more accessible in more user-friendly formats to a much wider range of HQ staff for a greater variety of purposes. IIRR also plans to make all such M&E data electronically accessible to AFR and LAC. This should translate into better-linked M&E Institute-wide. Since the evaluation was expressly tied into IIRR's ongoing and Institute-wide strategic planning exercise [Annex A], it provided immediate and concrete input into current and future plans for M&E improvements. Shortly after the conclusion of the SP and the evaluation, IIRR acted on this input by hiring an M&E consultant to begin implementing some of the M&E lessons learned during the evaluation. IIRR staff observed that "this more interactive evaluation process" was key to their moving ahead on M&E. Similarly, thanks to the participatory nature of the evaluation, a number of at least higher-level IIRR staff gained: - a new appreciation of the benefits of M&E in general and for showing off their own successes; - a better understanding of what-all is entailed in M&E -- for example, as per an AFR staff discussion with the evaluator, the need for different kinds, arrays, and analyses of data to satisfy the different M&E and reporting needs and interests of multiple donors, NGO/PVO partners, and IIRR itself; - relatedly, perhaps a greater awareness of the value and richness of qualitative data; - some concrete ideas for starting to track impacts, or at least spread and multiplier effects, in their respective programs; and - as modelled in OJT by the evaluator, ideas about how to present data in more powerful and interesting formats. Summary and Observations The story told above is a not uncommon one: a PVO that does not fully appreciate the importance of M&E [Chapter 6] lets this work slide until it finds itself confronted by a major donor's demand for "hard data" as part of final evaluation. The normal outcome in such situations is simply for the evaluator to report this state of affairs, bemoan the lack of attention to M&E, and write a rather general and impressionistic report. Somewhat extraordinarily in IIRR's case, both the evaluator and the PVO rejected this non-solution. Instead, both agreed to "work overtime," as it were, to remedy the situation as best possible, as briefly described above. The particular plan of attack taken in this case seemed feasible because of a number of considerations that might, or might not, obtain for other PVOs in similar circumstances. A key part of the grant being evaluated had focused on M&E. Two of the Institute's top three managers (one old, one new) recognized the importance of M&E. A good FIS existed. A centralized, computerized system for entry and manipulation of at least one type of program data (training) did exist; and, however imperfect, mechanisms were in place for all PVO units to flow data to the system. The designer of this system was still employed at the PVO. Management agreed to release this individual for extraordinary work with the evaluator. All PVO units had continued to collect data, albeit mainly only process-level training data. Thus, even though data had not always been flowed to and entered into the central computerized system, at least they were available "somewhere out there." The evaluation schedule was intermittent and prolonged, making it feasible for the PVO to mount a major data-collection and -centralization push, while the evaluator pursued other tasks until data became availabile for analysis. The donor graciously consented to a no-cost extension of the evaluation schedule so as to permit the PVO to conclude its drive for data, on which the evaluator then conducted her analyses. The PVO felt confident that, despite its lack of hard data in some arenas (and of impact data in most), its programs had been successful. Based on initial FG research, the evaluator shared this opinion, and thus was motivated to assist the PVO. To be candid, she was also personally impressed by the level of many PVO staffers' dedication to their mandate, and she shared the PVO's deeply held participatory philosophy. The first two and last two points above are probably critical. The PVO in question was well-motivated (albeit belatedly) to try to meet the need for evaluation data. First, M&E had been a key part of its original proposal to the donor; so the PVO was in an especially embarassing position at not having followed through on this work. Second, in 1999 a majority of top managers came to support the need for M&E; but before then, this would have been a minority. Third, because the PVO felt sure its programs had been successful, it was not afraid to evaluate them more rigorously. A less-confident PVO might well have rejected the evaluator's offer of assistance to come up with some stopgap measures. Fourth, few evaluators would have made such an offer in the first place, since it meant going well beyond the bounds of an evaluation SOW and ultimately his/her contract time. As it turned out, however, only due to the data push described here did the PVO and the evaluator learn just how successful the PVO's program work had in fact been! For example, the evaluation provided the first-ever synthetic analysis of IIRR's publication program, suggesting its many (apparent) spread and multiplier effects. And thanks to certain financial data arrays requested by the evaluator, important insights into IIRR finance+program management were gleaned, and the needs for improvement therein were more impartially documented. At a larger level, as noted above, the PVO ended up with expanded and new datasets and staff understandings for and of M&E that can be brought to bear in future evaluations and, more importantly, in ongoing strategic planning and management decisioning. The PVO is now in a much stronger position from which to assess longterm trends and achievements (or failures), the better to guide itself into the new century. ANNEX E: PERSONS CONTACTED (exclusive of FG members) IIRR Professional Staffa and Trustees US Office and Trustees Boorstein, Jane Trustee Blitz, Eric N. US Office Director Chin, Ping-sheng Yen Trustee Gerber, Mike Trustee Kelly, Jim Trustee Mudge, Alfred BOT Chair Headquarters Executive Office: Abuton, Elinor M. Management Information System Administrator Dayrit, Jonathan F. Computer Systems Staff Freeman, Rowena Siron Public Awareness Officer Gonsalves, Julian F. Vice President for Program Kale, Pratima President Killough, Scott A. Deputy Vice President for Program Montemayor, Javier C. Human Resources Director Shutt, Catherine Program Finance Analyst Education and Training Unit: Arevalo, Mae S. Training Specialist Cruz, Nenita J. Learning Resource Center Officer Espineli, Marissa B. Director Rios, Marietta A. Training Associate Specialist Finance Unit: Deocariza, M. Germelina Chief Accountant and Officer-in-Charge Rico, Arlita B. Head, Data Control and Encoding Secretario, Alden D. Head, Data Input, Analysis and Reporting Logistics Support Unit: Perez, Maria Teresa Manager (also Marketing Officer) Publications Unit: Amutan, Celso C. Publications Development Associate Caminade, Anajoy R. Publications Unit Head/Editor Montoya, Evangeline C. Publications Development Associate Ramos, Librado L. Publications Distribution Assistant Technical Specialist Unit: Coronel, Rodolfo E. Coordinator, Community-Based Enterprises Freeman, John C. Coordinator, Community Forestry Igbokwe, Kennedy N. Coordinator, Water Resources Development Ignacio, Normita G. Coordinator, Regenerative Agriculture Jordan, Ella A. Coordinator, Nutrition Nasar, S. S. Tabrez Coordinator, Smallscale Aquaculture Africa Regional Center Head Office (Kenya): Bekalo, Isaac Regional Director Callo, Virgie N. Senior Training Officer Kinyanjui, Grace Logistics Officer Obanyi, George Communication Specialist Oketch, Rachel Accounts Clerk (intern) Oluoch, John Adede Training Officer Schueller, Martin M&E Officer (seconded by DED) Siboe, Dorothy Accountant Ethiopia Country Office: Berhanu, Lealem Country Program Coordinator Asia Regional Center Head Office (Philippines): Dela Vega, Lilian R. Program Officer and Officer-In-Charge Imperial, Demetrio Former Head (retired) Maata, Phoebe V. Program Staff Mercaida, Enrique G. Program Staff Latin America Regional Center Alexander, Maria Finance Assistant Officer Carvajal, Jose Coordinator, F2F Project Cisneros, Wilson Project Secretary and Extensionist Espinosa, Martha Finance Officer Endara, Nelly Program Officer Proaño, Veronica Coordinator, Farmer Participatory Research Project Selener, Daniel Regional Director Staff of IIRR Partner PVOs/NGOs, Projects, and Donors Bastillo, Bobby AIM Faculty Member Batten, John AMREF Director General Jinnah, Shah Mobin CDA (a Bangaldeshi NGO) Director Kebede, Amare Farm Africa/Ethiopia Sr. Program Manager Liebler, Claudia GEM Project Director Luz, Mike AIM Faculty Member Menzies, Nick Ford Foundation/Kenya Program Officer Nyaga, Florence CARE/Kenya BEF Project Coordinator Siskel, Suzanne Ford Foundation/Philippines Representative Skaggs, Tim GEM Consultant Watkins, Jane GEM Consultant Willig, Bernhard Programme Coordinator, DED/Kenya BHR/PVC Herbert, Mary IIRR MG Managers Meites, Peggy M&E Specialist a A few non-technical staff are also included, since they were interviewed by the evaluator and/or provided valuable data inputs. Annex F: Documents Consulted INSTITUTIONAL AND RELATED DOCUMENTS AIM, GEM, and IIRR. Nov 1999. Strategic Planning Workshop [notebook]. IIRR, Silang, Cavite, Philippines. Unpaginated. AIM and IIRR. 1999. Positioning for Performance: An External and Internal Assessment of the International Institute for Rural Reconstruction. IIRR, Silang, Cavite, Philippines. Unpaginated but ca. 200 pp. AMATech. Oct 1999. IIRR Evaluation Scope of Work. AMATech, Rosslyn, VA, USA. 10 pp. Arevalo, Mae S. 1998. Report on the Farmer-Led Extension Study Program for Senior Officials of the Ministry of Agriculture, Government of Bangladesh May 12-24, 1998. IIRR, Silang, Cavite, Philippines. 6 pp. plus annexes. Arevalo, Mae S., and ?. Saladores. Feb 1999. Logical Framework and Monitoring and Evaluation Plan for the Sustainable Agriculture Training of Trainers Project. IIRR, Silang, Cavite, Philippines. 13 pp. Bachmann, Lorenz. 1999. Project Appraisal Report: Micro-Project Funds. PACE, no place of publication given. Unpaginated. Blitz, Eric [presumed]. Oct 1999. Discussion Draft: U.S. Office Structure and Functions. IIRR, New York, NY, USA. 14 pp. BOT. 1991. By-Laws. Ca. 25 pp. [Some pages missing]. BOT. 1998. International Institute of Rural Reconstruction: 1988 Strategic Overview and Plan - June 18, 1998. IIRR, Silang, Cavite, Philippines. 15 pp. CMIS Administrator. Oct 1999. MIS Manual (draft). IIRR, Silang, Cavite, Philippines. Unpaginated. Compton, J. Lin. n.d. Rationale and Design for a Rural Development Institute. Unpublished ms. No place or agent of publication given. 16 pp. CSS. Oct 1999. CSS Report on IIRR Computer Matters (October 15, 1999). IIRR, Silang, Cavite, Philippines. 17 pp. Espineli, Marise. Apr 1998. IIRR Inter-office Memo: Course Development and Management Procedures. IIRR, Silang, Cavite, Philippines. 6 pp. Fuchs-Carsch, Marian. 1998. Improving Resource Mobilization at IIRR: Report to Management on May 11-22 Consultancy at Silang Campus. IIRR, Silang, Cavite, Philippines. 15 pp. Gonsalves, Julian, and Scott Killough. Sep 1998. Memo: Three-year Medicum Term Plan (MTP) Process. IIRR, Silang, Cavite, Philippines. 23 pp. Gurmu, Eshetu, Melaku Eshetu, and Charles Teller. 1999. Learning Our Way Out (LOWO) Project: An Evaluation Report. The Demographic Training and Research Center, Institute of Development Research, Addis Ababa University, Addis Ababa, Ethiopia. 27 pp. Ibus, Angelina G., and Berhanu Geleto. 1996. An Assessment Report on the Organizational Capacity Building Needs of Oromo Development & Islah Association [ODIA]. IIRR, Addis Ababa, Ethiopia. Not fully paginated, but ca. 30 pp. IIRR/AFR. n.d. The Kadida Gamela Environmentally Sound Agriculture Project, Kadida Gamela, Ethiopia: A Report Presented to Food Industries Campaign Against Hunger (FICAH). IIRR, Addis Ababa, Ethiopia. No. pp. unknown. IIRR/AFR. Nov 1997. Organizational Capacity Assessment of Four Fellow Organizations Selected to Participate in the Girl Child Project of CARE-Kenya. IIRR, Nairobi, Kenya. 21 pp. IIRR/AFR. Apr 1998. CARE Kenya Girl Child Project: Results of the Planning Workshop April 6, 1998. IIRR, Nairobi, Kenya. 8 pp. IIRR/AFR. May 1998. Supplementary Grant for Translation and Dissemination of the Sustainable Agriculture Extension Manual through Cassette Tapes and Electronic Media. Proposal to the Ford Foundation/Kenya. IIRR, Nairobi, Kenya. 5 pp. IIRR/AFR. Jun 1999. Participatory Monitoring and Evaluation Workshop Report [3rd to 7th May 1999]. IIRR/AFR and PACT. 1996. Organizational Capacity Assessment of the Food Security/Rural Development Cluster NGOs [an Executive Summary]. IIRR and PACT, Addis Ababa, Ethiopia. 19 pp. IIRR/ETD. n.d.-a. How [sic] ??? IIRR Courses Compete with Other Courses? IIRR, Silang, Cavite, Philippines. 26 pp. IIRR/ETD. n.d.-b. Lessons and Insights from the Conduct of Off-shore International Training Courses. IIRR, Silang, Cavite, Philippines. 2 pp. IIRR/ETD. n.d.-c. 77 Years of Education, Training and Communication Efforts at the International Institute of Rural Reconstruction. IIRR, Silang, Cavite, Philippines. Unpaginated. IIRR/ETD. Jun 1997. Interntional Course on Systems in Community-Managed Health Fact Sheets. IIRR, Silang, Cavite, Philippines. Unpaginated. IIRR/HQ. Jul 1991. First Annual Report on Strategic Options for International Technology Transfer and Institutional Development. IIRR, Silang, Cavite, Philippines. 30 pp. plus attachments. IIRR/HQ. Jun 1992. Second Annual Report on Strategic Options for International Technology Transfer and Institutional Development. IIRR, Silang, Cavite, Philippines. 30 pp. plus attachments. 29 pp. IIRR/HQ. Ca. 1993. Operating Guidelines for the Regional Offices. IIRR, Silang, Cavite, Philippines. IIRR/HQ. 1993a. Program Scenario By Year 2000# [sic]. IIRR, Silang, Cavite, Philippines. 4 pp. IIRR/HQ. 1993b. Seven-Year Program Scenario and Goals. IIRR, Silang, Cavite, Philippines. IIRR/HQ. Feb 1993. Strategic Options for International Technology Transfer and Institutional Development [Internal] Mid-term Evaluation. IIRR, Silang, Cavite, Philippines. 22 mpp. IIRR/HQ. Jan 1994. Third Annual Report on Strategic Options for International Technology Transfer and Institutional Development. IIRR, Silang, Cavite, Philippines. 29 pp. IIRR/HQ. Jan 1995. Fourth Annual Report on Strategic Options for International Technology Transfer and Institutional Development. IIRR, Silang, Cavite, Philippines. 22 pp. plus attachments. IIRR/HQ. Dec 1995. A Transregional Program to Build NGO Capacity for Sustained Development Impact: Matching Grant Proposal to USID-BHR/PVC. IIRR, Silang, Cavite, Philippines. Unpaginated. IIRR/HQ. Spring 1997. A Transregional Program to Build NGO Capacity for Sustained Development Impact: Detailed Implementation Plan. IIRR, Silang, Cavite, Philippines. 46 pp. IIRR/HQ. Mar 1997. International Institute of Rural Reconstruction 1997 Action Plan. IIRR, New York, NY, USA. 22 pp. IIRR/HQ. 1997. 1997 Annual Report. IIRR, Silang, Cavite, Philippines. 33 pp. IIRR/HQ. Feb 1998. First Annual Report on A Transregional Program to Build NGO Capacity for Sustained Development Impact. IIRR, Silang, Cavite, Philippines. 5 pp. plus appendices. IIRR/HQ. May 1998. Program Procedures Manual. IIRR, Silang, Cavite, Philippines. Unpaginated. IIRR/HQ. Dec 1998a. 1998-1999 Program Report Accomplishments and Plans. IIRR, Silang, Cavite, Philippines. 13 pp. IIRR/HQ. Dec 1998b. Midterm Evaluation [Internal] on a Transregional Program to Build NGO Capacity for Sustained Development Impact. IIRR, Silang, Cavite, Philippines. ??? pp. plus appendices. IIRR/HQ. Dec 1998c. Second Annual Report on a Transregional Program to Build NGO Capacity for Sustained Development Impact: Executive Summary. IIRR, Silang, Cavite, Philippines. 12 pp. IIRR/HQ. 1999. 1998 Annual Report. IIRR, Silang, Cavite, Philippines. 31 pp. IIRR/HQ. Mar 1999. Resource Generation and Public Awareness Team Consolidated Internal Assessment Updated on 13 March 1999. IIRR, Silang, Cavite, Philippines. Unpaginated. IIRR/HQ. Apr 1999 [1998]. IIRR website -- Http:/www.cav.pworld.net.ph/~iirr. Ca. 100 pp. IIRR/HQ. Oct 1999. Head Count Report (as of 11 October 1999). IIRR, Silang, Cavite, Philippines. 15 pp. IIRR/HQ. Dec 1999. IIRR: Towards a Global Development Organization: Strategic Action Plan Document (Draft). IIRR, Silang, Cavite, Philippines. 19 pp. IIRR/LAC. Mar 1993. Strategy for IIRR's Program in Latin America. IIRR, Silang, Cavite, Philippines. 16 pp. plus attachments. IIRR/LAC. 1994. Summary of Activities 1994 IIRR Regional Office for Latin America. IIRR, Quito, Ecuador. 1 pp. plus attachments. IIRR/LAC. 1995. 1995 Accomplishments Regional Office for Latin America. IIRR, Quito, Ecuador. 3 pp. IIRR/LAC. Nov 1997. Summary of Activities October 1996 - November 1997, IIRR - Regional Office for Latin America. IIRR, Quito, Ecuador. 6 pp. plus annexes. IIRR/LAC. Jun 1997. Summary of Activities January - June 1997, IIRR - Regional Office for Latin America. IIRR, Quito, Ecuador. 4 pp. IIRR/LAC. Nov 1998. Summary of Activities Regional Office for Latin America Year 1998. IIRR, Quito, Ecuador. 16 pp. plus annexes. IIRR/LAC. Nov 1999. Regional Office for Latin America Strategic Planning, Silang, Cavite. IIRR, Quito, Ecuador and Silang, Cavite, Philippines. Unpaginated. IIRR/NY. 1999. U.S. Office Structure and Functions. IIRR, New York, New York, USA. 14 pp. Ira, Gregory C., Igbokwe Kennedy, John Adede, and Berhanu Berke. 1996. Watershed Management: An Approach to Food Security and Rural Development - A Field-Assessment & Training Report. PACT and IIRR, Addis Ababa, Ethiopia. 34 pp. Ira, Joy M. Jan 1997. First Progress Report: Computers, Telecommunications, and Policy Issues. IIRR, Silang, Cavite, Philippines. Ca. 40 pp. Ira, Joy M. Apr 1997. Final Report on: Establishing a Local Area Network at IIRR - Communication and Information Technology Report. IIRR, Silang, Cavite, Philippines. Ca. 40 pp. JBL. Dec 1998. International Institute for Rural Reconstruction Development Resources Audit and Evaluation Report. John Brown Limited, Inc., Peterborough, NH, USA. 19 pp. plus attachments. Killough, Scott. Feb 1997. Evaluation and Learning: Action Planning February 16-22, 1997. IIRR, Silang, Cavite, Philippines. 6 pp. Killough, Scott. Apr 1997. Institute Monitoring and Evaluation: Inter-office Memo to All Unit Directors and Regional Directors. IIRR, Silang, Cavite, Philippines. 20 pp. Killough, Scott. Jun 1997. Institute Monitoring and Evaluation System. Memorandum to the IIRR President and Vice President for Program. IIRR, Silang, Cavite, Philippines. 14 pp. Killough, Scott. Nov 1997. Recommendations for "Evaluation and Learning" at IIRR. IIRR, Silang, Cavite, Philippines. 6 pp. Killough, Scott. Aug 1999. Progress Report on Strategic Planning Exercise. IIRR, Silang, Cavite, Philippines. 5 pp. plus numerous annexes. Marshall, Rosella. 1996. Notice to IIRR President Rigby of USAID intent to grant $1,500,000 to provide support for the Transregional Program to build NGO Capacity for Sustained Development Impact. USAID Office of Procurement, Washington DC, USA. 3 pp. Morris, Patricia T. Mar 1999. Report with Recommendations for Integrating Gender in IIRR's Medium Term Strategic Plan. InterAction, Washington DC, USA. 23 pp. Mudge, Moby. Jun 1999. E-mail to Resource Development Committee, with proposed contract for an external Institutional Resource Generation Audit. New York, NY. 6 pp. Mudge, Moby. Oct 1999. IIRR - November 1 Meeting of Resource Development Committee. New York, NY. 8 pp. Navarro, C. S., and J. F. Gonsalves. 1993a. Decisions/Agreements/Outputs of the Strategic Planning Workshop Held on July 14-16, 1993. Internal Memo to All Staff. IIRR, Silang, Cavite, Philippines. O'Brien, Robert F. Jun 1994. Strategic Options for International Technology Transfer and Institutional Development (USAID Cooperative Agreement No. OTR-0158-A-00-0084-00) Final Evaluation. No place or agent of publication given. 28+ pp. plus annexes. Rigby, John. 1 Jul 1996. Memorandum re: Coopeartive Agreement to Sherry Fachet, Senior USAID Grant Specialist. IIRR, Silang, Cavite, Philippines. 5 pp. plus numerous budget annexes. Rigby, John. 31 Jul 1996. Message #164 from Silang to New York. IIRR, Silang, Cavite, Philippines. Fragmentary document. Rigby, John. 18 Dec 1997a. Memorandum on IIRR Structure. 22 pp. + 20 pp. of annexes. Rigby, John. 11 Dec 1997a. Statement of the President of IIRR, John T. Rigby. Meeting of the Board of Trustees, New York City, December 11, 1997. 5 pp. Shutt, Cathy. Oct 1997. Linkage/Integration of Program and Finance through the Design and Implementation of a Marketing and Business Plan. IIRR, Silang, Cavite, Philippines. 13 pp. Shutt, Cathy. Aug 1998. ARO Report -- Financial Management Issues. Addendum to trip report to AFR. IIRR, Silang, Cavite, Philippines. 7 pp. Shutt, Cathy. Nov 1999. Improvement in IIRR's Financial Management Systems. [Document prepared at USAID evaluator's request.] IIRR, Silang, Cavite, Philippines. 3 pp. Siron-Freeman, Rowena S. Jul 1998. Public Awareness Program Action Plan. IIRR, Silang, Cavite, Philippines. 6 pp. Velasco, Ester Catherine C., and Mulunesh Woldemariam. 1997. Beyond Rhetorics: Building Organizational Capacity on Gender and Development of the Food Security and Rural Development NGO Sector of Ethiopia. IIRR, Addis Ababa, Ethiopia. Unpaginated, but ca. 40 pp. IIRR-PRODUCED/ASSISTED SUBSTANTIVE PUBLICATIONS Comprises publications in which IIRR or some of its staff have authorship. HQ IIRR. n.d. Farmers Changing the Face of Technology: Choices and Adaptations of Technology Options. IIRR, Silang, Cavite, Philippines. 69 pp. IIRR. 1996. Paraveterinary Medicine: An Information Kit on Low-cost Health Care Practices. IIRR, Silang, Cavite, Philippines. IIRR. 1998 [1993]. Bio-intensive Approach to Small-scale Household Food Production. IIRR, Silang, Cavite, Philippines. IIRR. 1999. Environmental Health: A Sourcebook of Materials. IIRR, Silang, Cavite, Philippines. IIRR, CGIAR-NGO Committee, MISEREOR, and GTZ-BEAF. 1998. An International Workshop on Research Partnerships Between NOGs and Agricultural Research Institutions, October 5-9, 1998. IIRR, Silang, Cavite, Philippines. 44 pp. IIRR, Ford Foundation, DENR, and Universty of the Philippines-Los Baños. 1998 [1992]. Agroforestry Technology Information Kit. IIRR, Silang, Cavite, Philippines. IIRR, and GAA. 1997. Backyard Eel Culture. IIRR, Silang, Cavite, Philippines. IIRR, with HPI. 1998. [1994]. Ethnoveterinary Medicine in Asia: An Information Kit on Traditional Health Care Practices. 4 volumes. IIRR, Silang, Cavite, Philippines. IIRR, IDRC, CIDA, et al. 1998. Participatory Methods in Community-based Coastal Resource Management. IIRR, Silang, Cavite, Philippines. IIRR, IIED, IDS, and UPWARD, with Oxfam/Hong Kong, Sikiliza International, and KAISAHAN. 1998. Participatory Monitoring and Evaluation: Experiences and Lessons. Workshop Proceedings. IIRR, Silang, Cavite, Philippines. 131 pp. IIRR, IIED, IDS, and UPWARD, with Oxfam/Hong Kong, Sikiliza International, and KAISAHAN. 2000 (in press). Participatory Monitoring and Evaluation Tools. IIRR, Silang, Cavite, Philippines. Ca. 82 pp. IIRR, IIED, IDS, UPWARD, Oxfam/Hong Kong, Sikiliza International, and Kaisahan. 2000 (forthcoming). Participatory Monitoring and Evaluation Tools. IIRR, Silang, Cavite, Philippines. IIRR, PBSP, PhilGerFund, World Vision, and Ford Foundation. 1998. Organizational Performance and Change Management: Workshop Proceedings. IIRR, Silang, Cavite, Philippines. 229 pp. IIRR, SMISLE, and EU. 1998. Livelihood Options for Coastal Communities. Volume II. IIRR, Silang, Cavite, Philippines. CGIAR NGO Committee and IIRR. Oct 1999 (draft). Research Partnerships: Issues and Lessons from Collaboration of NGOs and Agricultural Research Institutions. CGIAR and IIRR, Silang, Cavite, Philippines. 105 pp. DENR. 1997. Sustainable Livelihood Options for the Philippines: An Information Kit. IIRR for DENR, Quezon City, Philippines. 3 volumes of 484, 162, and 234 pp. MYRADA, and IIRR. 1997. Resource Management in Rainfed Drylands: An Information Kit. IIRR, Silang, Cavite, Philippines. Scarborough, Vanessa, Scott Killough, Debra A. Johnson, and John Farrington (eds.). 1997. Farmer-led Extension: Concepts and Practices. Intermediate Technology Publications Ltd., London, UK. 214 pp. Africa Bekalo, Isaac. 1992 [1994]. Participatory Approach to Rural AIDS Education: A Workshop Manual. IIRR, Silang, Cavite, Philippines. 94 pp. CARE. 1999. Accessing Quality Basic Education through Fellows and Partners: CARE (Kenya-Tanzania-Atlanta) Basic Education Fellowship Project: Experiences at the Mid-Term. CARE, Nairobi, Kenya. Ca. 122. CARE/Kenya-Tanzania-Atlanta. 1999. Accessing Quality Basic Education through Fellows and Partners: Experiences at the Mid-term. CARE-BEF Project, Nairobi, Kenya. 122 pp. FGAE and IIRR. 1998. Reproductive Health and Communication at the Grassroots: Experiences from Africa and Asia. Family Guidance Association of Ethiopia, IIRR/AFR, and IIRR/ASI, Nairobi, Kenya. 282 pp. IIRR/AFR. 1998. Sustainable Agriculture Extension Manual for Eastern and Southern Africa. IIRR, Nairobi, Kenya. 241 pp. IIRR/AFR. 1999. Gender Training Handbook. IIRR, Nairobi, Kenya. Schroll, Michael. 1999. Monitoring Made Easy: A Handbook for Development Practitioners. IIRR/AFR, Nairobi, Kenya. 56 pp. ITDG/Kenya, and IIRR. 1996. Ethnoveterinary Medicine in Kenya: A Field Manual of Traditional Animal Health Care Practices. IIRR/AFR, Mairobi, Kenya. 226 pp. Asia BARRA, IIRR, et al. 1997. Small-scale Freshwater Aquaculture in Bangladesh [in Bengali]. IIRR, Silang, Cavite, Philippines. FGAE and IIRR. 1998. Reproductive Health and Communication at the Grassroots: Experiences from Africa and Asia. Family Guidance Association of Ethiopia, IIRR/ASI and IIRR/AFR, Nairobi, Kenya. 282 pp. IFWA and IIRR. 1996 (reprinted in 1999). Environmentally Sound Technologies for Women in Agriculture. IFWA, New Delhi, India, and IIRR, Silang, Cavite, Philippines. 213 pp. IIRR/ASI. 1996-1999. Village Health Worker Training Manual. IIRR, Silang, Cavite, Philippines. Latin America IIRR-AVRDC. 1997 (1000 copies reprinted in 1998). Guía Práctica para su Huerto Familiar Orgánico. IIRR and AVRDC, Quito, Ecuador. 253 pp. CIDICCO, Universidad de Cornell, IIRR, Comunica, Vecinos Mundiales, y COSECHA. 1997. Experiencias sobre Cultivos de Cobertura y Abonos Verdes. Litografía López, Tegucigalpa, Honduras. 131 pp. IIRR/LAC. 1996. Manual de Práctics Agroecológicas de los Andes Ecuatorianos. Selener, D., J. Chenier, and Raúl Zelaya, et al. 1997a. De Campesino a Campesino: Experiencias Prácticas de Extensión Rural Participativa. IIRR-MAELA, Quito, Ecuador. 148 pp. Selener, D., Jacqueline Chenier, and Raúl Zelaya, et al. with English translation and editing by Scott Killough. 1997b. Farmer-to-Farmer Extension: Lessons from the Field. IIRR, New York. 139 pp. Selener, Daniel, Nelly Endara, and José Carvajal. 1997. Guía Práctica para el Sondeo Rural Participativo. IIRR, Quito, Ecuador. 132 pp. Selener, Daniel, Nelly Endara, and José Carvajal. 1999. Participatory Rural Appraisal and Planning Workbook. IIRR, Quito, Ecuador. 146 pp. Selener, Daniel, Christopher Purdy, and Gabriela Zapata. 1998 (second printing). Documenting, Evaluating and Learning from our Development Projects. IIRR, Quito, Ecuador. 107 pp. SwissAid. Forthcoming. La Granja Biológica Campesina. SwissAid, Quito, Ecuador. PUBLICATIONS BY IIRR WRITESHOP/CONFERENCE PARTICIPANTS AND INSTITUTIONS Comprises publications: stimulated by or linked to IIRR events, TA, partnership, etc. but written and produced independently by other organizations or individuals; or independently published by IIRR senior staff. HQ Estrella, Marisol, with Jutta Blauert, Dindo Campilan, John Gaventa, Julian Gonsalves, Irene Guijit, Debra Johnson, and Roger Ricafort. In press. Learning from Change: Issues and Experiences in Participatory Monitoring and Evaluation. Intermediate Technology Publications, London, UK. Estrella, Marisol, and John Gaventa. 1998. Who Counts Reality? Participatory Monitoring and Evaluation: A Literature Review. IDS Working Paper No. 70. Institute of Development Studies, University of Sussex, Brighton, UK. 74 pp. IIED. 1998. Participatory Monitoring and Evaluation. PLA Notes No. 31. IIED, London, UK. 91 pp. King, Wendy, and Surendra Shakya. 1994. A workshop that works -- participatory training material production. Forests, Trees and People 25(Oct):13-17. Mundy, Paul, and Evelyn Mathias. 1999. Participatory workshops to produce information materials on ethnoveterinary medicine. In: Evelyn Mathias, D. V. Rangnekar, and Constance M. McCorkle, eds., with the assistance of Marina Martin. Ethnoveterinary Medicine: Alternatives for Livestock Development. Proceedings of an International Conference Held in Pune, India, on 4-6 November 1997. Volume 1: Selected Papers. BAIF Development Research Foundation, Pune, India. Pp. 78-87. Selener, Daniel. 1997. Participatory Action Research and Social Change. Global Action Publications for the Cornell Participatory Action Research Network, Quito, Ecuador. 358 pp. Africa Nothing to report. Asia Various "runaway" publications [see text Box 2.2]. Latin America CARE/Ecuador with PROMUSTA and the MOA. 1998. Experiencias en el Manejo Sostenible de los Recursos Naturales en los Andes. CARE, PROMUST, and MOA, Quito, Ecuador. 293 pp. Gallegos, P., G. Avalos, and C. Castillo. 1997. Gusano Blanco (Premnotrypes vorax) en el Ecuador: Comportamiento y Control. INIAP, Quito, Ecuador. 35 pp. Merino, F., F. López, and M. Pumisacho. 1997. Uso de Brotes: Alternativa para Incrementar la Producción de Tubérculos-Semilla de Calidad. INIAP, Quito, Ecaudor. 17 pp. OTHER REFERENCES CONSULTED Anonymous. n.d.-a Appreciative Inquiry: An Overview. Handout [?]. Unpaginated. Anonymous. n.d.-b The GEM Initiative - "Achieving Excellence through Collaboration." GEM Project overhead transparency set. 10 pp. Anonymous. Jun 1999. Ethiopia: "Learning Our Way Out" Leads Communities to Family Planning Solutions. Partners in Population and Development South-South Newsletter 7(Jun):7. Carroll, Thomas F. 1992. Intermediary NGOs: The Supporting Link in Grassroots Development. Kumarian Press, West Hartford, CN, USA. Edwards, Michael, and David Hulme (eds.). 1996. Beyond the Magic Bullet: NGO Performance and Accountability in the Post-Cold War World. Kumarian Press for Save the Children, West Hartford, CN, USA. GEM Initiative. 1996. Appreciative Inquiry: A Brief Introduction. Unpublished ms. GEM Initiative, Case Western Reserve Univeristy, Rosslyn, VA. 9 pp. Jackson, Edward T., and Yusuf Kassam (eds.). 1998. Knowledge Shared: Participatory Evaluation in Development Cooperation. Kumarian Press for IDRC, West Hartford, CN, USA. Krueger, Richard A. 1994. Focus Groups: A Practical Guide for Applied Research (second edition). Sage Publications, Thousand Oaks, CA, London, and New Delhi. Liebler, Claudia J. n.d. Getting Comfortable with Appreciate Inquiry: Questions/Answers. Global Social Innovations [reference incomplete]:30-40. Watkins, Jane Magruder, and David Cooperrider. n.d. Organizationl Inquiry Model for Global Social Change Organizations. Unpublished ms. 17 pp. 1 Annex G: Organizational Acronyms ALRED Arid Land Resource Exploitation and Development (Kenya) AA Action Aid (U.K.) AAATP Asian Alliance of Appropriate Technology Practitioners (Approtech Asia) (Philippines) ABC Agricultural Bank of China ABLH Association for Better Land Husbandry (Kenya) ABS-CBN ABS-CBN Foundation (Philippines) ABY Agricultural Bureau of Yunnan (People’s Republic of China) ACAP Annapurna Conservation Area Project (Nepal) ACCDP Amani Christian Communty Development Project (Kenya) ACFOD Asian Cultural Forum on Development (Philippines) ACHAN Asian Community Health Action Network (Philippines) ACLC Apayao Community Learning Center (Philippines) ACORDES Acompañamiento Organizacional Al Desarrollo (Ecuador) ACT Association for Technical Cooperatives (Belgium) ACWF All-China Women's Federation (People’s Republic of China) ADA Ambassadors Development Agency (Kenya) ADA Amhara Development Association (Ethiopia) ADAA African Development Aid Association (Kenya) ADAB Association of Development Agencies in Bangladesh ADB Asian Development Bank ADBN Agriculture Development Bank of Nepal ADRA Adventist Development and Relief Agency (US) ADRAI-FUDECOOP Fundación de Desarrollo Cooperativo ADSC Agriculture Development Service Centre (Nepal) AEA Ayuda En Accion (Ecuador) AERDD Agricultural Extension and Rural Development Department – University of Reading (England) AF The Amity Foundation (People’s Republic of China) AFA Agta Fishermen Association (Philippines) AFD Action For Development AFI Agro-Forestry Institute (Vietnam) AFMPCI Adams Farmers Multi-Purpose Cooperative, Inc. AFS Academy of Forestry Science (People’s Republic of China) AFSC American Friends Service Committee AGROSKILLS AgroSkills (Sri Lanka) AGTALON Agro-Technical Assistance and Livelihood Opportunities in the North (Philippines) AHI Animal Husbandry Institute (Vietnam) AIM Asian Institute of Management (Philippines) AIP Abra Irrigation Project (Philippines) AIRD Asian Institute for Rural Development (India) AIT Asian Institute of Technology (Thailand) AKFF Albert Kunstadter Family Foundation (US) AKRSP Aga Khan Rural Support Programme (India and Pakistan) ALF Abbott Laboratories Fund (US) ALIN Arid Lands Information Network (Senegal) AME Agriculture Man Ecology (India) 2 AMPD Alternative Mangyan Program for Development (Philippines) AMREF Africa Medical Research Foundation (Kenya) ANAI Asociacion De Nuevo Alquimistas, Incorporado (Costa Rica) ANGOC Asian NGO Coalition for Agrarian Reform and Rural Development (Philippines) ANSAB Asia Network For Small-Scale Agricultural Biotechnologies (Nepal) APAN Asia-Pacific Agroforestry Network (Indonesia) APBURCEP Anhui Provincial Bureau of Urban & Rural Construction & Environmental Protection (People’s Republic of China) APH Agricultural Publishing House (Vietnam) APRTCIFF Asia-Pacific Regional Research and Training Center For Integrated Fish Farming (People’s Republic of China) APSC Agricultural Projects Services Centre (Nepal) ARCS Amhara Rural Credit Scheme (Ethiopia) ARDRI Agricultural And Rural Development Research Institute (South Africa) ARI Aquaculture Research Institute (Vietnam) ARLDF Asian Rural Life Development Foundation (Vietnam) ASEAN Association of Southeast Asian Nations ASI Agricultural Science Institute (Vietnam) ATCD INC. Appropriate Technology Center for Development Inc. (Philippines) ATCRD Appropriate Technology Center for Rural Development (Philippines) ATI Agricultural Training Institute (Philippines) ATRCI Acupuncture Therapeutic and Research Center, Inc. (Philippines) AURT Anandamaragal Universal Relief Team (Lao PDR) AU Agricultural University (Vietnam) AUW Ahfad University for Women (Sudan) AVRDC Asian Vegetable Research and Development Center BAH Bureau of Animal Husbandry (People’s Republic of China) BAI Baraka Agricultural Institute (Kenya) BAIF Bharatiya Agro-Industries Foundation (India) BAPAKA Baryohanong Panagghiusa Alang sa Kauswagan (Philippines) BARRA Bangladesh Rural Reconstruction Association BASO Badasa Afya Services Organisation (Kenya) BAU Beijing Agricultural University (People’s Republic of China) BB Bannakaroli Brothers (Uganda) BBYKI Buklod Biyayang Kristiyano, Inc. (Philippines) BCDP Baglung Community Development Project (Nepal) BDH Bangui District Hospital (Philippines) BDMPCI Bulalo Damayan Multi-Purpose Cooperative Inc. (Philippines) BDP Balai Diklat Pertanian (Indonesia) BEAF Beratungsgruppe Entwicklungsorientierte Agrarforschung (Germany) BEF Basic Education Fellowship Project (Kenya) BFCFI Batuna Farmers Communities Foundation, Inc. (Philippines) BFW Bread for the World (Germany) BHWA BHW Association (Philippines) BIDF Bohol Integrated Development Foundation (Philippines) BILANCE Social Development (Netherlands) BIND Broad Initiatives for Negros Development (Philippines) BKT Balikatan sa Kaunlaran ng Tibig Multi-Purpose Cooperative Inc. (Philippines) BOA Bureau of Agriculture 3 BOAU Bogor Agricultural University (Indonesia) BRAC Bangladesh Rural Advancement Committee BRDB Bangladesh Rural Development Board BRSP Baluchistan Rural Support Programe (Pakistan) BS Bina Swadaya – Jakarta (Indonesia) BSE Backward Society for Education (Nepal) BSU Benguet State University (Philippines) BSWM Bureau of Soil and Water Management (Philippines) BTI Biology Technology Institute (Vietnam) BTPHS Bak Thai Provincial Health Service (Vietnam) BWJWG Bukhayo West Joint Women Group (Kenya) CAAS Chinese Academy Of Agricultural Sciences (People’s Republic of China) CABI Commonwealth Agricultural Bureau International (UK) CADI Center for Alternative Development Initiatives (Philippines) CAME Andean Council Management Ecological (Peru) CAPM Chinese Academy of Preventive Medicine (People’s Republic of China) CARE Cooperative American Relief Everywhere CART Center for Alternative Rural Technology (Philippines) CAS Chinese Academy of Sciences (People’s Republic of China) CASA Church's Auxiliary for Social Action (India) CATER CATER (Ecuador) CBO Community Based Organizations CCC China Community Corp (People’s Republic of China) CCHRMDC Cebu City Hilly Resources Management and Development Commission (Philippines) CCO Canadian Cooperation Office (Nepal) CD - Kisii Catholic Diocese of Kisii (Kenya) CDA Community Development Association (Bangladesh) CDHP Community Development and Health Project (Nepal) CDPDC Colegio De Postgraduados De Champingo (Mexico) CEA Coordinadora Ecuatoriana De Agroecología (Ecuador) CEAPRED Center for Environmental and Agriculture Policy Research, Extension and Development (Nepal) CEAS Centro de Estudios y Accion Social (Ecuador) CEC Center for Environmental Concerns (Philippines) CEC Central Economics Committee (Vietnam) CECCC Cultural Education Committee of Changping County (People’s Republic of China) CECI Canadian Center for International Studies and Cooperation (Nepal) CEM Council of Ethnic Minorities (Vietnam) CEMOPLAF Centro Médico de Orientación y Planificación Familiar (Ecuador) CEOP Ecuadorian Network of NGOs (Ecuador) CERD Community Extension and Research for Development Inc. (Philippines) CESA Central Ecuatoriana De Servicios Agricolas (Ecuador) CET Center for Education and Technology (Chile) CF Community Forest (Thailand) CFGB Canadian Food and Grain Bank (Canada) CFIRAN Center for International Research and Advisory Networks (Netherlands) CFMCI Comillas Farm Systems Multi-Purpose Cooperative, Inc. (Philippines) CFRL Central Food Research Laboratory (Nepal) CFSO Children and Family Support Organization (Ethiopia) 4 CGHR Center for Goad and Hare Research (Vietnam) CGIAR Consultative Group on International Agricultural Research CHAK Christian Health Association of Kenya (Kenya) CHAMPA Community Health Action Modular Participatory Approach (Lao PDR) CHESTCR Community Health Education, Services and Training in the Cordillera Region (Philippines) CHSC Canossa Health and Social Center (Philippines) CIAT Centro Internacional de Agricultural Tropical (Colombia) CIB-CAS Chengdu Institute of Chinese Academy of Sciences (People’s Republic of China) CICETE China International Center for Economic and Technical Exchanges (People’s Republic of China) CIDA Canadian International Development Agency CIDICCO Centro Internacional de Información sobre Cultivos de Cobertura (Honduras) CIDSE Cooperation Internationale Pour Le Developpement Et La Solidarité (Cambodia, Lao PDR and Vietnam) CIED Center for Indigenous Environment and Development (US) CIIFAD Cornell International Institute for Food and Agriculture (US) CIP Centro Internacional de la Papa (Peru) CISS Community Initiatives Support Services (Kenya) CKRRC Committee for Kok River Recovery and Conservation (Thailand) CLAI Camatagan Ladies Association, Inc. (Philippines) CLN Convener/Leisa Network (India) CLSU Central Luzon State University (Philippines) CLT Cornfields Land Trust (South Africa) C-MAD Community Mobilization Against Desertification (Kenya) CMDF Community Medicine Development Foundation (Philippines) CMU Central Mindanao University (Philippines) CMU Centre for Mycological Research of National University (Vietnam) CONFENIAE Confederacion Nacional De Nacionales Indigenas De La Amazona Ecuatoriana (Ecuador) CP Community Polytechnic (India) CPK Church of Province of Kenya CPKDE C.P.K. Diocese of Eldoret (Kenya) CPRIN China Plaire Reform Information Newspaper CRC Children's Rehabilitation Center (Philippines) CRCIPR Cordillera Resource Center for Indigenous People's Rights (Philippines) CRDA Christian Relief and Development Association (Ethiopia) CRMP Coastal Resource Management Program (Philippines) CRODT Centre De Rechercher Oceanographiques Dakar-Thiaroye (Senegal) CRRC Cebu Relief And Rehabilitation Center (Philippines) CRS Catholic Relief Services CRWRC Christian Reformed World Relief Committee CSC Community Service Centre (India) CSWCD College of Social Work and Community Development (Philippines) CTA Technical Center for Rural and Agricultural Cooperation (Netherlands) CTCDFI Cordillera Tribal Communities Development Foundation, Inc. (Philippines) CTT Center for Technology Transference (Vietnam) CWDS Community Welfare and Development Society (Nepal) 5 CWF Child Welfare Foundation (Vietnam) CWO Christian Women Organization (Uganda) CWP Community Water Project (Kenya) CWS Church World Service CWSK Child Welfare Society of Kenya DA Department of Agriculture (various countries) DAC Department of Agriculture and Cooperation (India) DAE Department of Agricultural Extension (Bangladesh) DAEPA Development Agriculture and Education Project for Akha (Thailand) DAEX Department of Agriculture Extension (Pakistan) DAEXRD Department of Agricultural Extension and Rural Development (South Africa) DAI Development Associates, Inc. (US) DAL Department of Agriculture and Livestock (Papua New Guinea) DANIDA Danish International Development Agency DAPH Department of Animal Production and Health (Sri Lanka) DC Dhempe College (India) DCCDI Dayap Credit Cooperative and Development, Inc. (Philippines) DCOFM Department of Community Occupational and Family Medicine (Singapore) DECS Department of Education, Culture and Sports (Philippines) DED German Development Services (Germany) DEE Department of Extension Education (Sweden) DEIBCH Dirección de Educación Intercultural Bilingüe de Chimborazo (Ecuador) DENR Department of Environment and Natural Resources (Philippines) DEQP Department of Environment Quality Promotion (Thailand) DF Department of Fisheries (Bangladesh) DFC Desarrollo Forestal Campesino (Ecuador) DFID Department for International Development (UK) DF Department of Forestry (various countries) DF Department of Fisheries (Malaysia) DGIS Directoraat General International, Ministry of Foreign Affairs (Netherlands) DH Department of Horticulture (Nepal) DIA Dutch Interchurch Aid (Netherlands) DIGEBOS Direccion General De Bosques (Guatemala) DILG Department of Interior and Local Government (Philippines) DLFRDP Dong Land Forest Recovery and Development Project (Thailand) DLSU De La Salle University (Philippines) DMK Diocese of Mount Kilimanjaro (Tanzania) DOA Department of Agriculture (India) DOF Department of Forestry (Lao PDR) DOH Department of Health (various countries) DPHCA Department of Primary Health Care Administration (People’s Republic of China) DPPC Disaster Prevention and Preparedness Commission (Ethiopia) DPSM Department of Preventive and Social Medicine (Philippines) DSCCUI Diego Silang Cooperative Credit Union, Inc. (Philippines) DSCWM Department of Soil Conservation and Watershed Management (Nepal) DSE Deutsche Stiflung für Internationale Entwicklung (Germany) DSH Department of Southern Highlands (Papua New Guinea) 6 DSTC Dudley Senanayke Technical College (Sri Lanka) DOT Department of Tourism (Philippines) EAHNM East African Herbarium National Museums of Kenya EAP Escuela Agricola Panamericana (Honduras) EARRA East Africa Rural Reconstruction Association (Kenya) ECZMT Economic Cooperative of Zhang Mo Township (People’s Republic of China) EDA Emmanuel Development Association (Ethiopia) EDASOP Economic Development Association for Anti-Poverty of Linqu County (Peru) EECMY Ethiopian Evangelical Church Mekane Yesus ELCI Environment Liaison Center International EOC Ethiopian Orthodox Church ERAD Environmental Rehabilitation and Agriculture Development Department ERDB Ecosystems Research and Development Bureau (Philippines) ERO Ethiopian Relief Organization ERSHA Ethiopian Rural Self Help Association ESADP Edo State Agricultural Development Programme (Nigeria) ESF Ecoles Sans Frontieres (Lao PDR) ESQUEL Fundacion Esquel (Ecuador) ESRDF Ethiopian Social Rehabilitation and Development Fund ESRI Educational Science Research Institute (People’s Republic of China) ESRIG Education Science Research Institute of Guangxi (People’s Republic of China) ESWACI Earth Station Writers and Artists Collective, Inc. (Philippines) ETC Foundation for Ecology, Technology and Culture (Netherlands) EU European Union EUCORR European Committee for Rural Reconstruction (Belgium) EZE Evangelische Zentralstele Für Entwicklungchilfe (Germany) FAD Fundación Acción y Desarrollo (Ecuador) FAN Forest Action Network FANS Forum for Alternative Network System (India) FAO Food and Agriculture Organization of the United Nations FAO-RAPA FAO Regional Office for Asia and the Pacific (Thailand) FAS Faculty of Applied Science (Thailand) FC Fugong County (People’s Republic of China) FCV Forestry College of Vietnam FDRP Fish Disease Research Program (Vietnam) FD-THAILAN Forest Department (Thailand) FED Forum for Environment and Development (Nepal) FEED Foundation for Educational Evolution, Inc. (Philippines) FEF Four Eyes Foundation (India) FEPP Fondo Ecuatoriano Populorum Progresio (Ecuador) FFF Federation of Free Farmers (Philippines) FFG Farmers' Friends Group (Indonesia) FFP Forestry Foreign Aid Projects (Vietnam) FFRC Freshwater Fisheries Research Center (People’s Republic of China) FF Ford Foundation (U.S.A) FGAE Family Guidance Association of Ethiopia FHI Food for the Hungry International FICAH Food Industry Crusade Against Hunger 7 FIPI Forestry Inventory Planning Institute (Vietnam) FISE Fondo de Inversión Social del Estado (Ecuador) FLACSO Facultad Latinoamericana de Ciencias Sociales (Ecuador) FLDAU Foreign Languages Department of Anhui University (People’s Republic of China) FLE Family Life Education (Uganda) FMSA French Medical and Sanitary Aid (Nepal) FNCRDC Forest and Nature Conservation Research and Development Center (Indonesia) FPAP Family Planning Association of Pakistan FPAU Family Planning Association of Uganda FPBURCEP Fuyang Profectural Bureau of Urban - Rural Const. & Environmental Protection (People’s Republic of China) FRC Forum for Rural Concerns (Philippines) FRDC Forest Research and Development Centre (Indonesia) FSIV Forest Science Institute of Vietnam FSPTT FSP/TONGA TRUST (Tonga) FTRRM Foundation for Thailand Rural Reconstruction Movement (Thailand) FUDECOOP Foundation for Cooperatives Development (Ecuador) FUNCAR Fundacion Carvajal (Colombia) FUNDAEC Foundation for Teaching and Appliance of Service (Colombia) FUNDAGRO Fundacion Agroecologia (Ecuador) F2F Farmer-to-Farmer Project (Ecuador) GAA German Agro Action (Germany) GARDP Gulmi Arghakanchi Rural Development Project (Nepal) GATE German Appropriate Technology Exchange GB Gram Bhasti (India) GBPIHED G.B. Pant Institute of Himalayan Environment and Development (India) GCDA Genesis Community Development Assistance (Kenya) GCP Goa College of Pharmacy (India) GCZ-IPM Consultative Group of Zamoranos in Integrated Pest Management (Honduras) GDF Guiuan Development Foundation, Inc. (Philippines) GDP Gorkha Development Project (Nepal) GDS German Development Service GEM Global Excellence in Management Project (US) GFLTC Guangxi Foreign Language Training Center (People’s Republic of China) GHRRM Ghana Rural Reconstruction Movement (Ghana) GOI Government of India GI Genetics Institute (Vietnam) GIER Guangxi Institute for Educational Research (People’s Republic of China) GIFRID German-Israel Fund for Research and International Development (Israel) GMC Government of Mizhi County (People’s Republic of China) GMPC Guinsang-An Multi-Purpose Cooperative (Philippines) GPEC Guangdong Provincial Education Commission (People’s Republic of China) GPSAC Gratia-Plena Social Action Center (Philippines) GRASS Grama Sewa Samithi (India) GSLTD Godrej Soaps Ltd. (India) GSI Gender Sensitive Initiatives GTCM Garba Tulla Catholic Mission (Kenya) 8 GTZ Gesellschafttfür Technische Zusammenarbeit (Germany) GVIW Government Vocational Institute for Women (Pakistan) GVS Grama Vikasa Samastha (India) GVTI Government Vocational Teachers Institute (Pakistan) HADP Hill Areas Development Foundation (Thailand) HAU Hanoi Agricultural University (Vietnam) HCC Harargene Catholic Church HEC Hengxian Education Committee (People’s Republic of China) HFH Holy Family Hospital (Philippines) HFI Herb Foundation, Inc. (Philippines) HG Hakmuangnan Group (Thailand) HH Hengxian Hospital (People’s Republic of China) HHB Hengxian Health Bureau (People’s Republic of China) HHC Homa Hills Centre (Kenya) HIE Hunan Institute of Education (People’s Republic of China) HIRDPP Highland Integrated Rural Development Pilot Project (Lao PDR) HI Holt International (Vietnam) HKI Helen Keller International (U.S.A.) HMBEP Huainan Municipal Bureau of Environmental Protection (People’s Republic of China) HMC Hanoi Medical College (Vietnam) HORCO Hope for Rural Children and Orphans (Ethiopia) HPG Hengxian People's Government (People’s Republic of China) HPI Heifer Project International (various countries) HPRR Programa Hondureño De Reconstruccion Rural (Honduras) HRIRDP Hill Region Integrated Rural Development Project (Myanmar) HS Handicraft School (Myanmar) HSA Health Services Academy (Pakistan) HSRDP Home Stead Resource Development Program (Sri Lanka) HUAF Hue University of Agriculture and Forestry (Vietnam) IAAS Institute of Agriculture and Animal Sciences-Tribhuvan University (Nepal) IARC International Agricultural Research Center IBDR Instituto Biodinamico de Desenvolvimento Rural (Brazil) ICAR Indian Council of Agricultural Research (India) ICCO Interchurch Organization for Development Cooperation (Netherlands) ICDH Iligan City District Hospital (Philippines) ICIMOD International Centre for Integrated Mountain Development (Nepal) ICLARM International Center for Living Aquatic Resources Management (Philippines) ICRAF International Center for Research in Agroforestry (Kenya) ICRISAT International Crops Research Institute for the Semi-Arid Tropics (India) ICS International Christelijksteunfonds IDMA Instituto De Desarrollo Y Medio Ambiente (Peru) IDR Institute for Development Research (US) IDRC International Development Research Centre (Canada) IDS Institute for Development Studies (UK) IDU Indeco De Unte. Pt. (Indonesia) IE Institute of Ecology (Indonesia) IEM International Environmental Management (Thailand) IFAD International Fund for Agricultural Development IFOAM International Federation of Organic Agriculture Movements (Germany) 9 IFP Institute of Forestry-Tribhuvan University (Nepal) IFPRI International Food Policy Research Institute (U.S.A) IFSP Integrated Food Security Programme IFWA International Federation for Women in Agriculture (India) IG-CAS Institute of Geography-Chinese Academy Of Sciences (People’s Republic of China) IICA Instituto Interamericano De Cooperación (Costa Rica) IIDS Institute for Integrated Development Studies (Nepal) IIED International Institute for Environment and Development (UK) IIM Indian Institute of Management (India) IIRR International Institute of Rural Reconstruction IITA International Institute of Tropical Agriculture (Nigeria) IIYW Indian Institute of Youth Welfare (India) IIZ-DW German Adult Education Association (Germany) ILEIA Information Centre for Low-External Input And Sustainable Agriculture (Netherlands) ILRI International Livestock Research Institute (Kenya, Ethiopia) IMPECT Inter-Mountain People Education and Culture in Thailand INEFAN Instituto National Ecuatoriano Forestal (Ecuador) INIAP Instituto Nacional de Investigaciones Agropecuarias (Ecuador) INRULED International Research and Training Center for Rural Education (People’s Republic of China) INSAN Institute for Sustainable Agriculture (Nepal) IPB Institute Pertanian Bogor (Indonesia) IPC Institute of Philippine Culture (Philippines) IPHC Institute of Primary Health Care (Philippines) IPMF Integrated Pest Management Forum IRDF Integrated Rural Development Foundation (Philippines) IRDP Integrated Rural Development Programme IRDP Integrated Rural Development Project (Nepal) IRDS Integrated Rural Development Society (India) IRED Ired-Development Innovation and Networks (Sri Lanka) IRRI International Rice Research Institute-Laguna (Philippines) IRRM India Rural Reconstruction Movement ISARD Institute for Studies on Agriculture & Rural Development (India) ISEC Institute for Social and Economic Change (India) ISFI Institute for Small Farms and Industries (Philippines) ISG International Support Group (Kenya) ISI Indian Social Institute (India) ISNAR International Service for National Agricultural Research (Netherlands) ISSAS Institute of Soil Science, Academia Sinica (People’s Republic of China) ISU Iowa State University (US) ITDG Intermediate Technology Development Group (UK) ITM Institute of Traditional Medicine (China) IVS International Voluntary Services (US) IYD Institute for Youth and Development (India) JBL John Brown, Ltd. (US) JC Jiangcheng County (People’s Republic of China) JCCCI Janiuay Calvario Community Center, Inc. (Philippines) JDC Jheskang Development Committee (Nepal) JDS Jyoti Daya Sangha (Nepal) 10 JEB Jinhai Education Bureau (People’s Republic of China) JICA Japan International Cooperation Agency JIVC Japan International Volunteer Center (Cambodia, Lao PDR and Vietnam) JMKHK Jeevan Mitra Kutumbab Hivridhi Kandra (India) JS Jyotidaya Sangha (Nepal) JSFI Josefa Segovia Foundation Incorporation (Philippines) JTS Jerri Technical School (Nepal) KARI Kenya Agricultural Research Institute KAS Know Aids Society (Kenya) KBFCA Kamanga Bangus Fry Catchers Association (Philippines) KC Kiteredde Construction (Uganda) KCCI Kaunlaran Credit Cooperative, Inc. (Philippines) KCDF Kenya Community Development Foundation KCDP Kailali Community Development Project (Nepal) KCI Kiteredde Construction Institute (Uganda) KEAEO Kenya Energy and Environment Organization (Kenya) KENGO Kenya Energy and Non-Governmental Organization (Kenya) KEPP Kenya Entrepreneurship Promotion Program (Kenya) KES Kalasipan Elementary School (Philippines) KG Kaybee's Group (Nepal) KHDP Kosi Hills Development Programme (Nepal) KIB Kunming Institute of Biology (People’s Republic of China) KIBREP Kribhco Indo British Rainfed Farming Project (India) KIE Kunming Institute of Ecology (People’s Republic of China) KIM Kenya Institute of Management KIOF Kenya Institute of Organic Farming (Kenya) KKU Khon Kaen University (Thailand) KMPCI Kasaka Multi-Purpose Cooperative, Inc. (Philippines) KMTNC King Mahendra Trust for Nature Conservation (Nepal) KPYWAPP Kwan Pa Yao Watershed Area Protection Project (Thailand) KRSRDP Kware Rabom Siyat Rural Development Project (Thailand) KS Kuduamin-Samin (Philippines) KTKJ Kapisanan Tungo sa Kaunlaran ng Javalera (Philippines) KU Kasetsart University (Thailand) KUFI Kapwa Upliftment Foundation, Inc. (Philippines) KVDS Kenya Volunteer Development Service (Kenya) KVK Krishi Vigyan Kendra (Farm Science Centre) (India) KWDI Korea Women's Development Institute (South Korea) LCAEC Lao Cai Agricultural Extension Center (Vietnam) LCDE Leyte Center for Development, Inc. (Philippines) LEAD Leiden Ethnosystems and Development Programs and Social (Netherlands) LEM The Environment and Development Society of Ethiopia LHP Lihuk-Health Program (Philippines) LIT Leyte Institute of Technology (Philippines) LJSS Lanka Jathika Sarvodaya Sharamadana Sangamaya (Sri Lanka) LOWO Learning Our Way Out Project (Ethiopia) LPP Lakas Pamayanan ng Pinatubo (Philippines) LRAPI Leyte Rural Advancement Programmes, Inc. (Philippines) LRARC Lumle Regional Agricultural Research Centre (Nepal) LSP Land Stewardship Project (US) 11 LWF Lutheran World Federation (US) LWR Lutheran World Relief (US) MAAIF Ministry of Agriculture, Animal Industry and Fisheries (Uganda) MAELA Movimiento Agroecológico De América Latina Y El Caribe (Peru) MAA Ministry of Atolls Administration (Maldives) MA Ministry of Agriculture (Bangladesh) MAFF Ministry of Agriculture, Forestry and Fisheries (Cambodia) MAFI Ministry of Agriculture and Food Industry (Vietnam) MAF Ministry of Agriculture and Forestry (Laos) MAG Medical Action Group (Philippines) MANAGE National Institute of Agricultural Extension Management MANUSH Manush Project (India) MAPI Medical Ambassadors Philippines, Inc. (Philippines) MASAI Management Advancement Systems Association, Inc. (Philippines) MASL Multi-Agro Services, Ltd. (Uganda) MATEC Municipal Agro-Technical Extension Center (People’s Republic of China) MAZON A Jewish Response To Hunger (US) MBMCI Mutya Bayanihan Multi-Purpose Cooperative, Inc. (Philippines) MBRLC Mindanao Baptist Rural Life Center (Philippines) MC Marina Clinic (Philippines) MCC Mennonite Central Committee (US) MCKBCI Mt. Carmel Kilusang Bayan for Credit, Inc. (Philippines) MCRR Movimiento Colombiano De Reconstruccion Rural (Colombia) MCYG Mamuyemba Complex Youth Group (Kenya) MDPAC Maderas Del Pueblo A.C. (Mexico) MDSFDF Matara District Small Farmers Development Foundation (Sri Lanka) ME Ministry of Education (Islamic Republic of Iran) MEYC Mother Earth Young Club (Philippines) MFE-EPD Ministry of Forests and Environment-Extension and Publicity Division (Nepal) MFTEC Ministry of Foreign Trade and Economic Cooperation (People’s Republic of China) MFVDP Malakand Fruit and Vegetable Development Project (Pakistan) MF Ministry of Forestry (Vietnam) MGRR Movimiento Guatemalteco De Reconstruccion Rural MH Ministry of Health (Uganda) MI Map Internacional (Ecuador) MINSOC Management Institute for Social Change (Malaysia) MISEREOR Bischofliches Hilfswerk Misereor E.V. MIT Mara Institute of Technology (Malaysia) MMBEP Maanshan Municipal Bureau of Environmental Protection (People’s Republic of China) MMSU Mariano Marcos State University (Philippines) MNRD Ministry of Natural Resources and Development (Ethiopia) MOA Ministry of Agriculture MOE Ministry of Education (Ethiopia) MOF Ministry of Forestry (China) MOH Ministry of Health (various countries) MPA Ministry of Political Affairs (South Korea) MPAI Milagrosa Parents Association, Inc. (Philippines) MPLD Ministry of Panchayat and Local Development (Nepal) 12 MPWADP Mae Ped Watershed Area Development Project (Thailand) MRC Malaria Research Centre (India) MRD Ministry of Rural Development (Cambodia) MSF Mercedes School of Fisheries (Philippines) MSI Marine Science Institute (Philippines) MSU Michigan State University (US) MSU Mindanao State University (Philippines) MU Mangalore University (India) MUCARD Muslim-Christian Agency for Rural Development (Philippines) MFI Mag-uugmad Foundation, Inc. (Philippines) MYRADA Mysore Resettlement and Development Agency (India) NAF Nepal Agroforestry Foundation (Nepal) NARC Nepal Agricultural Research Councilgo (Nepal) NARDC Northern Agriculture Rural Development Center (Thailand) NAV Nordic Assistance to Vietnam (Vietnam) NAWA Nilgiris Adivasi Welfare Association (India) NCA Norwegian Church Aid NCCK National Council of Churches of Kenya NCYG Namuyemba Complex Youth Group (Kenya) NDF National Development Foundation (Sri Lanka) NDMU Notre Dame of Marbel University (Philippines) NECOS Nepal Community Support Group (Nepal) NEDA Netherlands Development Assistance NEED TRUST Non-Formal Education and Development Trust (India) NEPB Nantong Enviromental Protection Bureau (People’s Republic of China) NERRA Nepal Rural Reconstruction Association (Nepal) NEWAH Nepal Water Health NF The Network Foundation (Philippines) NFESC Non-Formal Education Service Centre (Nepal) NFI Norfil Foundation, Inc. (Philippines) NFSS Nairobi Family Support Service (Kenya) NFW Natural Farming Work (Zimbabwe) NGO Non-Government Organization NGOPAI NGOs for Protected Areas, Inc. (Philippines) NHAPL National Herbarium and Plant Laboratory (Nepal) NI Nutrition Institute (Vietnam) NIO National Institute of Oceanography (India) NIPP National Institute of Physical Planning (Mozambique) NISER The Nigerian Institute of Social and Economic Research (Nigeria) NIVP National Institute of Vaccine Production (Lao PDR) NLF National Labor Federation (Philippines) NMC Nanning Medical College (People’s Republic of China) NMRP Nepal Resource Management Project (Nepal) NNDCWO Nepal National Depressed Community Welfare Organization NNDSWO Nepal National Dalit Social Welfare Organization NNSWA Nepal National Social Welfare Association (Nepal) NOVIB Netherlands Organization for International Development Cooperation NPC National Planning Commission (Nepal) NPG Nairobi Parents of the Deaf-Blind Self-Help Group (Kenya) NRCS Nepal Red Cross Society (Nepal) NRDF National Rural Development Forum (South Africa) 13 NRI Natural Resources Institute (England) NRRM National Rural Reconstruction Movement (various countries) NSCB National Statistical Coordination Board (Philippines) NSM Navjeevan Seva Mandal (India) OBA Old Boys Association K.C.I. (Uganda) ODA Oromo Development Organization (Ethiopia) ODI Overseas Development Institute (UK) ODP Organization for the Development of People (India) OFFERS Organic Farming Field Experimental and Resource Station (Philippines) OFS Opp Fire Station (India) OPSI Office of the Permanent Secretary for Industry (Thailand) ORAP Organization of Rural Associations for Progress (Zimbabwe) ORDA Organization for Rehabilitation and Development in Amhara (Ethiopia) ORSTOM Institute Francais de Recherche Scientifique pour le Dévelopment en Coopération (France) OSHO Oromo Self-Help Organization (Ethiopia) OWDF One World Development (Kenya) OWS Ogaden Welfare Society (Ethiopia) OXFAM Oxford Committee for Famine Relief (UK) PAC Pacific Adventist College (Papua New Guinea) PACAS Pag-Asa Cooperative and Allied Services (Philippines) PACCOM People's Aid Coordination Committee (Vietnam) PACT Private Agencies Collaborating Together (US) PADEK Partnership for Development in Kampuchea (Cambodia) PAFISFA Patag Farmers Integrated Social Forestry Association (Philippines) PAHO Panamerican Health Organization PAKAC Pakhribas Agricultural Centre (Nepal) PALMERA People's Action and Liberation Movement in East Rammad (India) PAMANA Pagpapaunlad ng Mamamayan sa Nayon (Philippines) PAN Pesticide Action Network (various countries) PASA People's Association for Social Action (India) PBSP Philippine Business for Social Progress (Philippines) PC Peaceful Society (India) PCAMARD Philippine Council for Aquatic and Marine Research and Development PCARRD Philippine Council for Agriculture, Forestry and Natural Resources Research and Development (Philippines) PCART Palawan Center for Appropriate Rural Technology, Inc. (Philippines) PCCI Panginay Credit Cooperative, Inc. (Philippines) PCCL People Center for Christian Living (U.S.A) PCDA Population and Community Development Association (Thailand) PCEA Presbryterian Church of East Africa (Kenya) PCEC Pubei County Education Committee (People’s Republic of China) PDAP Philippine Development Assistance Program (Philippines) PDI Project Development Institute for Asia (Philippines) PDL Pa Dec Lao (Lao PDR) PDP Palpa Development Programme (Nepal) PDPEO Provincial Development Planning And Evaluation Office (Philippines) PDRC Philippine-China Development Resource Center (Philippines) PELUM Participatory Ecological Land Use Management Network (Africa) PFA Progressive Farmer Association (Thailand) PFEC Philippine Federation for Environmental Concern (Philippines) 14 PFP Partnership for Productivity (Kenya) PG Provincial Government (Philippines) PHCFWC PHC Federated Women's Club (Philippines) PHCP Primary Health Care Project (Nepal) PHILDHRRA Philippine Partnership for Development of Human Resources and Rural Areas PHILGERFUND Philippine-German Development Foundation (Philippines) PHIRCIKSD Philippine Resource Center For Indigenous Knowledge and Sustainable Development (Philippines) PHO Provincial Health Office (Philippines) PHTI Post-Harvest Technology Institute (Vietnam) PI Plan International (US) PI Provide International (Kenya) PKC Padma Kanya Campus (Nepal) PLDHC Phu Luong District Health Center (Vietnam) PMM Paitan Mangyan Mission (Philippines) PPDO Provincial Planning and Development Office (Philippines) PPI Philippine Peasant Institute (Philippines) PRDA Puttalam Regional Development Association (Sri Lanka) PREMESE Participatory Rural Education Methodologies and Evaluation Service PROCASUR Programa Regional De Capacitación En Desarrollo Rural (Ecuador) PROGRESS People's Research Organization for Grassroot Environmental Scientific (India) PROMUSTA Programa De Manejo Y Uso Sostenible De Tierras (Ecuador) PROSHIKA PROSHIKA Manobik Unnayan Kendra (Bangladesh) PRRM Philippine Rural Reconstruction Movement (Philippines) PSWDO Provincial Social Welfare And Development Office (Philippines) PTDMCI Pulong Talahiban Damayan Multi-Purpose Cooperative Inc. (Philippines) PUC People's University of China (People’s Republic of China) PWSDO Provincial Social Welfare and Development Office (Philippines) PWWCF Philippine Wetland and Wildlife Conservation Foundation, Inc. (Philippines) QLCDRC Quirino Livelihood Concept and Development Resource Center (Philippines) QQMPCI Quezon-Quirino Multi-Purpose Cooperative, Inc. (Philippines) QS-LAOS Quaker Service (Laos and Vietnam) QTHDFA Quang Tri Holland Development and Friendship Association (Vietnam) RAFE Red Agroforestal Ecuatoriana (Ecuador) RAPA Regional Office for Asia and the Pacific of FAO RAPC Roughton and Partners Consultants (Nepal) RB Redd Barna (Norway) RBNP Royal Bardia National Park (Nepal) RCC Rural Credit Cooperative (People’s Republic of China) RCFF Rural Cooperation Fund Foundation (People’s Republic of China) RCWDA Rift Valley Children and Women Development Association (Ethiopia) RDRS Rangpur Dinajpur Rural Services (Bangladesh) RDS Rural Development Society (India) RDSP Rural Development Through Self Help Promotion (Nepal) RDTS Rural Development and Training Society (India) RECFOG Rural Education Comprehensive Reform Office of Guangxi (People’s Republic of China) 15 RECOFTC Regional Community Forestry Training Center (Thailand) REDA Rural Economic Development Association (Nepal) RELMA Regional Land Use Management Unit (Kenya) REST Relief Society of Tigray (Ethiopia) RHU Regional Health Office (Philippines) RIA Research Institute for Aquaculture No.1 (Vietnam) RISD Rwanda Initiative for Sustainable Development RISE Rural Institute for Social Education (India) RIVFT Research Institute for Vegetables and Fruit Trees (Vietnam) RIVS Research Institute for Veterinary Science (Indonesia) RKA Ramakrishna Mission Ashrama (India) RMAF Ramon Magsaysay Award Foundation (Philippines) RMDC Resource Management and Development Center (Thailand) RONAST Royal Nepal Academy of Science and Technology (Nepal) RRAFA Rural Reconstruction Alumni and Friends Association (Thailand) RRN Rural Reconstruction Nepal (Nepal) RRS Regional Research Station (India) RSCDP Rubber Smallholding Community Development Project (Thailand) RSDC Rural Self-Reliance Development Center (Nepal) RST Relief Society of Tigray (Ethiopia) RWN Rwanda Women Network SA Samuhik Abhiyan (Nepal) SAC Social Action Center (Philippines) SACDEP Sustainable Agriculture Community Development Programme (Uganda) SACRED Sustainable Agriculture Center for Research and Development (Kenya) SADOPECO Santo Domingo People's Cooperative (Philippines) SAI State Agricultural Institute (Myanmar) SA Sahabat Alam (Malaysia) SANE Social Action Network Exchange (India) SAP South Asia Partnership (Bangladesh, India, Nepal and Pakistan) SARRA South Asia Rural Reconstruction Association (India) SBBLPMC Samahang Buklod Bukang Liwayway-Primary Multi-Purpose Cooperative (Philippines) SBSF Service Bureau for Small Fishermen (Philippines) SCA Senior Citizen Association (Philippines) SCF Save the Children Foundation SCFI South Cotabato Foundation, Inc. (Philippines) SCF Save the Children Fund (UK) SDAC Seventh-Day Adventist Church (Ethiopia) SDC Swiss Development Cooperation (Switzerland) SDH Siaton District Hospital (Philippines) SDRC Social Development Research Center (Philippines) SEAFDEC SouthEast Asia Fisheries Development Center (Philippines) SEARCA Southeast Asia Regional Center for Graduate Study and Research in Agriculture (Philippines) SEARICE Southeast Asia Regional Institute for Community Education (Philippines) SEASAN Southeast Asian Sustainable Agriculture Network (Philippines) SED State Education Department (Malaysia) SEDA Selam Environment Development Association (Ethiopia) SEDC State Education Commission (People’s Republic of China) SEEDS Sarvodaya Economic Enterprises Development Services (Sri Lanka) 16 SEEP The Small Enterprise Education and Promotion Network (US) SF Siyath Foundation (Sri Lanka) SFC Southwest Forestry College (People’s Republic of China) SHDI Self Help Development International SHIPP Self-Help Initiative Promotion Project (Nepal) SHRDPI Servus Human Rural Development Programme, Inc. (Philippines) SIBAT Sibol ng Agham at Teknolohiya (Philippines) SICA Solomon Islands Christian Association (Solomon Islands) SIDA Swedish International Development Agency SIDER Si-Fu-Pin Institute for Development and Economic Research (People’s Republic of China) SIKAP Samahan para sa Ikauunlad ng Pamayanan Primary Multi-Purpose Coop., Inc. (Philippines) SINAGMCI Samahan sa Ikauunlad ng Adhikain ng Grupo Multi Purpose Cooperative Inc. (Philippines) SIT Surabaya Institute for Technology (Indonesia) SLM Save Life Mission (Uganda) SLNFFHCB Sri Lanka National Freedom from Hunger Campaign (Sri Lanka) SMAP Southern Mindanao Agricultural Program (Philippines) SMC Shanghai Medical College (People’s Republic of China) SMFHP St. Mark's Family Helper Project (India) SMISLE Small Islands Agricultural Support Services Programme (Philippines) SMPPMCI Samahan ng Maggugulay ng Pook Paliparan Multi-Purpose Coopeative Inc. (Philippines) SMS Secondary Medical School (Vietnam) SM Seva Mandir (India) SNNPR Southern Nation and Nationalities People Region SNV Netherlands Development Organization SOPHRE Society for Protection of Human Rights & Rural Env't (Nepal) SPBMCI Samahan ng Pagpapaunlad ng Buenavista Multi-Purpose Cooperative Inc. (Philippines) SPD Society for Partnership in Development (Nepal) SPEECH Society of People's Education and Economic Change (India) SPM Simao Project Management (People’s Republic of China) SPPMCI Sandigan ng Pagkakaisa ng Paligawan Multi-Purpose Cooperative Inc. (Philippines) SPWD Society for Promotion of Wastelands Development (India) SRDA Social Rural Development Association (India) SRSOC Self-Reliant Society Service Center (Nepal) SRTRI Swamy Ramananda Teerth Research Institute (India) SSNSK Shree Shantimaya Nari Sewa Kendra (Nepal) STS Senior Technical School (Indonesia) TBP The Bank for the Poor (Vietnam) TC Technical College (Sri Lanka) TCCI Talipusngo Consumers Cooperative, Inc. (Philippines) TDC Tambuyog Development Center (Philippines) TDMCI Traktora Damayan Multi-Purpose Cooperative (Philippines) TFA Tribal Filipino Apostolate (Philippines) TGHDP Thai-German Highland Development Program (Thailand) TGSSL The Gami Seva Sevana Ltd. (Rural Service Center) (Sri Lanka) TGTVC Tehran Girls Technical and Vocational College (Islamic Republic of Iran) 17 THAIGERFUND Thai-German Development Foundation (Thailand) TMCEPH Tianjin Municipal Commission on Education and Public Health (People’s Republic of China) TMI The Mountain Institute (US) TRDO Teso Rural Development Organization (Uganda) TREE Technology for Rural and Ecological Enrichment (Thailand) TRTC Tuburan Research and Technology Center (Philippines) TTTI-CRD Technical Teachers' Training Institute-Centre for Rural Development (India) TWF Township Women's Federation (People’s Republic of China) UA University of the Andes (Venezuela) UA-F University of Agriculture – Faisalabad (Pakistan) UAF University of Agriculture and Forestry (Vietnam) UAS University of Agriculture Science (India) UC University of Cantho (US) UCRCIA Universidad De Costa Rica Centro De Investigaciones Agronomicas (Costa Rica) UDLE Urban Development Through Local Efforts (Nepal) UEAB University of Eastern Africa Barator UH University of Hanoi (Vietnam) UI University of Ilorin (Nigeria) UIHSFI Urban Integrated Health Services Foundation, Inc. (Philippines) UMN United Mission to Nepal UN United Nations UNAG Union Nacional De Agricultores Y Ganaderos (Nicaragua) UNCW Uganda National Council of Women (Uganda) UNDP United Nations Development Programme UNESCO United Nations Educational, Scientific and Cultural Organization UNHCR United Nations High Commission for Refugees UNICEF United Nations Children's Fund UP University of the Philippines UPK Uganda Polytechnic Kyambogo (Uganda) UPLB University of the Philippines – Los Baños UPOU University of the Philippines Open University UPWARD User's Perspective with Agricultural Research and Development (Philippines) USAID United States Agency for International Development (US) USC University of San Carlos (Philippines) USJ University of Sri Jayewardenepura (Sri Lanka) USK Undugu Society of Kenya USM University Sains Malaysia UST Unnayan Shahojogy Team (Bangladesh) UST University of Science and Technology (Ghana) UST University of Science and Technology (China) UVPP Urban Vegetable Promotion Project (Tanzania) UW University of Wisconsin (US) VBA Vietnam Bank for Agriculture (Vietnam) VBKVK Vidya Bhavan Krishi Vigyan Kandra (India) VERC Village Education Resource Center (Bangladesh) VFCCL Vietnam Forest Consulting Co. Ltd. (Vietnam) VFRI Vietnam Forestry Research Institute (Vietnam) 18 VFSSI Vietnam Forestry Scientific Society (Vietnam) VH Vaidya Hospital (India) VHAI Voluntary Health Association of India (India) VISCA Visayas State College of Agriculture (Philippines) VM Vecinos Mundiales (Ecuador) VODP Village Organization Development Project (Thailand) VPS Volunteer Programme of Scientists (Vietnam) VSFCP Vietnam-Sweden Forestry Cooperation Programme (Vietnam) VSO Voluntary Service Overseas (UK) WB World Bank WBJWG West Bukhayo Joint Women Group (Kenya) WC World Concern WCDO Women and Child Development Organization (India) WEP Women for Environment Protection (Nepal) WFA Water Farming Aid WHO World Health Organization WI Winrock International (US) WII Wildlife Institute of India (India) WMI Woodlands Mountain Institute (Nepal) WN World Neighbors (US) WOWSK Widows and Orphans Welfare Society of Kenya (Kenya) WR World Relief WVI World Vision International (US) WWA Women Welfare Association (Nepal) WWF World Wide Fund for Nature XAES Xavier Agricultural Extension Services (Philippines) XU Xavier University (Philippines) YAD Yunnan Agriculture Department (People’s Republic of China) YASS Yunnan Academy of Social Sciences (People’s Republic of China) YCO Youth Charitable Organization (India) YGMF Yayasan Geo Meno Flores (Indonesia) YIG Yunnan Institute of Geography (People’s Republic of China) YIS Yayasan Indonesia Sejahtera (Indonesia) YMCA Young Men's Christian Association (Philippines) YMT Yayasan Mitra Tani (Indonesia) YMWS Young Men's Welfare Society (India) YPEDPA Yunnan Provincial Economic Development of Poor Area (People’s Republic of China) YT Yayasan Tananua (Indonesia) YUM Yunnan Uplands Management (People’s Republic of China) YVSS Yunbiao Vocational Secondary School (People’s Republic of China) YYRA Yan Yangchu Research Association (People’s Republic of China) YYRAH Yan Yangehu Research Association Hunan (People’s Republic of China) ZAU Zhejiang Agricultural University (People’s Republic of China) ZOA ZOA - Refugee Care Netherlands (Lao PDR and Thailand) -------------------------------- Pls. note the ff. acronyms: CEBEMO – former organizational name for BILANCE (acrdg. to Grace of Kenya Office) IBIS – it is not an acronym, but a name (acrdg. to Daniel S.)