ACDI/VOCA MID-TERM REVIEW Title II Institutional Capacity Building (ICB) Grant Agreement Number: AFP-A-00-03-00009-00 December 22, 2006 Submitted by: ACDI/VOCA 50F Street NW, Suite 1100 Washington, DC 20001 Evaluation Coordinators: Contacts: Della E. McMillan, External Consultant Avram E. Guroff, Sr. Vice-President Piero Gonzalez, ICB Coordinator Piero Gonzalez, ICB Coordinator Tel: (202) 879 0235 ACDI/VOCA ICB Mid-Term. December 22, 2006 ii Executive Summary ACDI/VOCA ICB Grant Objectives: The goal of ACDI/VOCA’s Title II Institutional Capacity Building (ICB) grant (FY04-FY08) is to enhance ACDI/VOCA’s capacity: “to promote food security among at-risk and vulnerable populations.” The grant’s activities focus on achievement of the following four strategic objectives (SO) and one cross￾cutting objective. • SO1: Capacity to address food insecurity through all possible uses of Title II resources improved. • SO2: Technical excellence in food security programming enhanced through broadened partnerships to build institutional capacity. • SO3: Ability to design food security proposals enhanced. • SO4: Evidence base that supports programmatic approaches that improve impact. • Cross-cutting objective: Providing effective support for the management of current food security programs. To achieve these objectives, ACDI/VOCA developed an ICB strategy that focused on support for new tools development, external and internal training programs, and field￾level assessment using the new tools. These activities were expected to produce seven cutting-edge tools that could be shared with other Title II Cooperating Sponsors (CS). The total budget for the ICB program is $1,400,000 from USAID with a commitment from ACDI/VOCA for a cost share of $187,000 (12 percent of the USAID award). ICB Grant’s Achievements at Mid-Term: This ICB grant’s mid-term review (November and December 2006) concluded that ACDI/VOCA has made significant progress toward achievement of the grant’s strategic objectives and intermediate results (IRs). While a small number of the original targets that were identified in the project M&E Plan need to be revised downward, a similar number of the targets need to be revised upward in areas where the project has overachieved. Significant progress has been made on development of five of the seven major “tools” that were identified in the project proposal (Table A), as well as numerous other training modules (Annex II). Several of the major tools are likely to have a major impact on CS capacity in some of the key areas identified by the recent USAID/FFP Strategic Plan for 2006-2010. The grant has also supported a wide variety of internal and external trainings that have helped train ACDI/VOCA’s first tier management (i.e., headquarters project coordinators and assistants) and some field staff (through the annual food programming annual retreat). The same training programs have provided a venue for discussing new tools and for developing the DVD and PowerPoint training tools that ACDI/VOCA needs for HQ and field (distance learning) training and retraining. ACDI/VOCA ICB Mid-Term. December 22, 2006 iii Table A. Major Tools being Developed under the ACDI/VOCA ICB, FY04-FY06 Name of Tool and ICB SO Summary Description Date Draft Produced Pilot Test/Assessment using the Tool Expected Finalization and Distribution Monetization and Best Practice Tool (SO1) Tool to track developmental impact of monetization and identify best practices according to local conditions Draft anticipated January 2007 Being developed with input from Cape Verde, Rwanda, and Uganda FY08 Emergency Market Intervention Tool (SO1) Tool that can facilitate design of commodity sales in areas with extremely high staple food prices (due to emergencies) to protect remaining purchasing power of poor consumer and to increase availability of food Draft anticipated 2007 3 field assessments (DRC, Burundi, Mauritania) FY08 Agricultural Rehabilitation Tool (SO1) Guidebook on lessons learned from projects that promoted revitalization and rehabilitation of agricultural production and marketing in areas recovering from disasters or that are in transition 1st Draft FY06 2nd draft anticipated FY07 Planned for Cape Verde FY07 FY08 OAT (Organizational Assessment Tool) (SO3) New tool for assessing local partners baseline organizational capacity for implementing food security interventions and for identifying and defining areas where organizations will need capacity assistance 1st draft FY04 2nd draft anticipated FY07 Rwanda FY04 Cape Verde FY07 FY07 Food Security Needs Assessment Toolkit (SO3) Needs assessment manual of better practices of food security assessment methodologies* Draft FY06 Finalized FY06 New designs in FY06 and FY08 FY07 Long Term Impact Assessment￾Guidelines (SO4) Tool identifies information that projects can collect and analyze to assess past performance and establish an evidence base that will inform future activities Draft FY05 FY05 Cape Verde FY06 Uganda Planned for Rwanda FY07 FY07 Computer M&E System (SO4) Data base to store and analyze quantitative data collected by Title II project M&E systems Draft FY06 Project coordinators are pilot testing the system by entering FY06 project data FY07 *Includes identification of potential beneficiaries, ration design, identification of complementary activities, methodologies for monitoring and evaluation program success. Source: Annex II and mid-term interviews. ACDI/VOCA ICB Mid-Term. December 22, 2006 iv The evaluation team found several incidences where capacity building initiated by the ICB grant has had a quantitative and qualitative impact on design and execution of ACDI/VOCA’s field programs. This impact was not always easy to track with the original ICB monitoring and impact indicators. Factors that Contributed to and Detracted from the Success of the Grant: Key factors that contributed to the successful implementation of the ICB were the extensive experience of the Title II program’s senior field and HQ staff and the agency’s commitment to hosting (with non-ICB funds) an annual food programming meeting that is attended by expatriate and national staff from all of the Title II programs. The chief factors that weakened the ability of the project to execute and report on its deliverables were the limited technical support to the ICB Coordinator for routine M&E and heavy reliance on volunteers for tasks which require long-term follow-through. Recommendations: The mid-term evaluation team identified the following five areas where the ICB could strengthen its impact on the food programming capacity of ACDI/VOCA and the other Title II PVO Cooperating Sponsors over the next two years. • Tools: Set priorities for finalization of specific tools and solicit participation of highly qualified ACDI/VOCA field and HQ staff in the final review and revision of the tools. • Training and Distance Learning: Develop better systems for tracking HQ and field-level internal and external training in areas targeted by the ICB IRs, including field-level training using the training modules developed under the ICB. • Collaboration: Supplement ACDI/VOCA’s excellent record for inter-CS collaboration on general administration and policy with more focused networking of mid-level HQ administrative staff (e.g., project coordinators and ICB coordinator) in three key areas of Title II program design and execution: HIV/AIDS, nutrition, and M&E. • M&E: Strengthen the capacity for strategic planning of field and HQ staff by reformulating five of the ICB’s 14 IRs to better reflect field realities, adopting a standard Performance Indicator Tracking Table (PITT) format for the annual report to USAID, reformulating some of the ICB monitoring and impact indicators (and methods for data collection), and better documenting the various methods used to collect data that are aggregated in the new ICB-developed computerized M&E database. • Documentation: Develop better systems for informing field and HQ staff about internal (ACDI/VOCA) and external (e.g., FANTA and USAID/FFP) resources that they can consult to respond to specific questions. • ICB Management: Strengthen the project’s management systems by assigning clear staff responsibility for finalizing tools, reassigning volunteers to tasks that require shorter-term commitments, soliciting more input into design and feedback on execution of ICB activities, and adjust the project accordingly. ACDI/VOCA ICB Mid-Term. December 22, 2006 v Table of Contents Executive Summary.......................................................................................................... ii List of Tables and Figures.............................................................................................. vii List of Acronyms and Abbreviations ........................................................................... viii 1.0. Introduction and Background ..............................................................................1 1.1. Context of the ACDI/VOCA ICB Grant......................................................1 1.2. ACDI/VOCA’s Title II Institutional Capacity Building (ICB) Grant .........4 1.3 Mid-Term Evaluation Methodology............................................................5 2.0. Strategic Objective 1 (SO1): Capacity to Address Food Insecurity through all Possible Uses of Title II Resources ..................................................7 2.1. Strategy ........................................................................................................7 2.2. Progress toward Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators ........................................8 2.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data ............................................................................9 3.0. Strategic Objective 2 (SO2): Technical Excellence in Food Security Programming Enhanced through Broadened Partnerships to Build Capacity ................................................................................................................12 3.1. Strategy ......................................................................................................12 3.2. Progress toward the Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators ......................................12 3.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data ..........................................................................14 4.0. Strategic Objective 3 (SO3): Ability to Design Food Security Proposals ...... 17 4.1. Strategy ......................................................................................................17 4.2. Progress toward the Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators ......................................18 4.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data ..........................................................................20 5.0. Strategic Objective 4 (SO4): Evidence Base that Supports Programmatic Approaches that Improve Impact .........................................................................22 5.1. Strategy ......................................................................................................22 5.2. Progress toward the Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators ......................................22 5.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data ..........................................................................22 6.0. Management and Finance ......................................................................................24 6.1. Formal Management Structures and Systems............................................24 6.2. Evidence of Early or Potential Impact.......................................................27 7.0. Conclusions and Recommendations......................................................................28 7.1. Summary: Progress towards Achieving Intermediate Results/Indicators and Targets and Recommendations for Refining Targets..........................28 ACDI/VOCA ICB Mid-Term. December 22, 2006 vi 7.2. Strengths, Weaknesses, Opportunities and Threats ...................................28 7.3. Cross-Cutting Recommendations for Enhancing Grantee Performance ...30 Annex I. Proposed Revision of the ACDI/VOCA ICB Mid-Term Performance Indicator Tracking Table (PITT) based on Recommendations from the ICB Mid￾Term Evaluation ................................................................................................................1 Annex II. ACDI/VOCA Institutional Capacity Building (ICB) Grant Project Bibliography and References Consulted during the Mid-Term Evaluation ................8 Annex III. ICB Facilitated Trainings and Tools..........................................................17 Annex III.A. Tools Produced under the ACDI/VOCA ICB Grant as Reported in the Revised ACDI/VOCA ICB Performance Indicator Tracking Table (PITT) (FY04-FY06) .........................................................................................................17 Annex III.B. Internal and External ICB-Facilitated Trainings under the ACDI/VOCA ICB Grant as Reported in the Performance Indicator Tracking Table (PITT) (FY04-FY06) ...................................................................................20 Annex III.C. ACDI/VOCA ICB Monetization 101 Training Pre and Post-Test...27 Annex IV. Additional Tables .........................................................................................31 Annex IV.A. Food Security Programming Experience of Key Senior ACDI/VOCA Staff in the Title II Field Programs and Headquarters Office ........31 Annex IV.B. Project Level Training in ACDI/VOCA’s Currently Funded Title II Projects in Cape Verde, Uganda and Rwanda in the Key Areas Targeted by the ICB Grant (FY06 Only).........................................................................................35 Annex V. Evaluation Methodology and Logistics........................................................40 Annex V.A. List of People Interviewed................................................................40 Annex V.B. ACDI/VOCA ICB Mid-Term Evaluation Schedule.........................41 Annex V.C. Mid-Term Surveys............................................................................42 Annex V.D. Scope of Work for ACDI/VOCA’s Institutional Capacity Building (ICB) Program FY 2004 – FY 2008 ......................................................................52 ACDI/VOCA ICB Mid-Term. December 22, 2006 vii List of Tables and Figures Table A. Major Tools being Developed under the ACDI/VOCA ICB, FY04-FY06 Table 1.1. Previous and Contemporary Title II Capacity Building Grants to ACDI/VOCA Table 1.2. Current ACDI/VOCA Title II and Non-Title II Food Programs (in terms of funds for programming and for pass/through/sub-contracts) Table 1.3. Original ACDI/VOCA ICB Strategic Objectives and Intermediate Results Table 2.1. Pre and Post-Test Values (as % of total possible value) for Individuals Attending the ICB-Sponsored Monetization 101 Workshop in FY06 Table 6.1. Internal Trainings Facilitated by the ICB in Connection with ACDI/VOCA Annual Retreats, FY04-06 Table 6.2. Number of Title II Project Staff who Attended the ACDI/VOCA FFD Meeting, FY05-06 Table 7.1. ACDI/VOCA ICB Strategic Objectives and Intermediate Results and Mid-Term Recommendations for Reformulating IR Statements Figure 1.1. New Food Programs Proposals Awarded to the ACDI/VOCA FFD Division by Category, FY91-06 (by year of award) Figure 2.2. Areas to Explore for Monetization Development Impact Indicators ACDI/VOCA ICB Mid-Term. December 22, 2006 viii List of Acronyms and Abbreviations AEA American Evaluation Association APVOFM Association for Private Voluntary Organization Financial Managers ARC American Red Cross DAP Development Assistance Proposal DRC Democratic Republic of Congo CARE Cooperative for Assistance and Relief Everywhere, Inc. CFA Coalition for Food Aid CS Cooperating Sponsor CSR2 Cooperating Sponsor Resource Request CWG Commodity Working Group EIG Evaluation Interest Group FACG Food Aid Consultative Group FANTA Food Aid and Nutrition Technical Assistance Project FAM Food Aid Management Consortium FFD Food for Development (as of October 2006 now called Food Security and Specialty Crops Portfolio) GDA Global Development Alliance GIWG General Issues Working Group ISA Institutional Support Assistance Grant ICB Institutional Capacity Building (grant) ICRW International Center for Research Women IR Intermediate Results LOL Land-O-Lakes MT Metric Tons MYAP Multi-Year Assistance Program OASIS Organização das Associações de Agricultores, Avicultores e Pécuários da Ilha de Santiago (Umbrella association on Santiago Island) OMB Office of Management and Budget PLWHA People Living with HIV/AIDS PVO Private Voluntary Organization SO Strategic Objective SOW Scope of Work TANGO Technical Assistance to NGOs (Consulting Firm) USAID United States Agency for International Development USAPEEC USA Poultry and Egg Export Council WFP World Food Programme ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006 1 1.0. Introduction and Background 1.1. Context of the ACDI/VOCA ICB Grant The ACDI/VOCA Title II Institutional Capacity Building Grant (FY04-08) is the fourth in a series of Title II-funded capacity building grants from USAID. This grant builds on a history of capacity building under the three previous Title II capacity building grants— the Enhancement Grant (EG) (FY88-92), the Institutional Support Grant (ISG) (FY93- 97), and the Institutional Support Assistance (ISA) grant (ISA) (FY98-03) (Table 1.1). The capacity built under these grants was critical in helping ACDI/VOCA create and maintain its ability to execute a steadily expanding portfolio of Title II food programming (Figure 1.1). The same grants helped create and strengthen the new Food for Development (FFD) division–which is in the process of changing its name to the Food Security and Specialty Crops portfolio (Table 1.1)1 —which is dedicated to development and backstopping of the agency’s food security programs. Since 1991, ACDI/VOCA has seen a steady increase in the amount of Food for Development division funding (Figure 1.1), resulting from an increase in the total number of grants. ACDI/VOCA quickly emerged as a national leader in the field of monetization, reflected in an increased number of “pass monetization programs for other Title II and USDA programs. In “pass through” monetization programs ACDI/VOCA is paid a fee for overseeing the monetization process, but not for executing the program. By FY03—the year the ISA grant ended—ACDI/VOCA had increased the portfolio of the Food for Development division overseeing their Title II programs to include a sizable proportion of economic development programs as well as Title II and non-Title II monetization programs (Figure 1.1). Specifically, ACDI/VOCA had (ACDI/VOCA 2004: 1): through” Monetization Commodity vegetable oil arrives at ACDI/VOCA, Rwanda. Photo by: Thomas Gibb • Become responsible for all monetization activities in Uganda and Rwanda—a total of seven Title II programs—and was directly responsible for commodity management and the food distribution component under its own program, which was implemented by partners; and 1 Since the new name is not completely official, this report will refer to the division by its old name, the Food for Development division. ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006 2 • More than doubled its commodity management (from approximately 62,500 MT (metric tons) in 1998 to 133,350 MT in 2002) and increased nearly sevenfold the amount of umbrella monetization on behalf of other PVOs (from approximately 3,000 MT in 1998 to 20,570 MT in 2002). The current portfolio includes three Title II projects (in Cape Verde, Rwanda, and Uganda) and six non-Title II food programs (Table 1.2). Table 1.1. Previous and Contemporary Title II Capacity Building Grants to ACDI/VOCA Institutional Grants USAID Institutional Priorities Activities and Global Institutional Impact on ACDI/VOCA Enhancement Grant (EG) (FY88-92) Commodity management training and support Financed a fledgling Food for Development division at ACDI/HQ that was responsible for backstopping the Uganda vegetable oil monetization program and for developing additional PL480 monetization programs elsewhere Institutional Support Grant (ISG) (FY93-97) -Assistance to NGO CS in implementing 1995 Food Aid and Food Security Policy Paper with its requirement of M&E and development programming -Second generation consolidation of “lessons learned” in monetization by training and inter-CS networking on monetization Conversion of the FFD division created under the EG into an entity dedicated to the development and backstopping of Title II programs Institutional Support Assistance (ISA) grant (FY98-03) Strengthening and institutionalizing (at the CS-level) initiatives developed under these earlier grants by: (a) M&E training that reinforced HQ oversight, (b) reinforcing CS partnering arrangements, (c) capacity building of HQ staff to better support field activities, and (d) providing a framework for improving human resources through workshops and training courses -Financed staff, travel, and per diem of FFD personnel and consultant time needed to expand ACDI/VOCA’s food programming -Development of internal manuals for monetization and a food programming M&E toolkit Institutional Capacity Building (ICB) grant (FY04-08) Consolidation of internal capacity and development of new cutting-edge tools that can be shared with other Title II CS As of Mid-Term: -Draft tools developed and pilot tested -Internal and External Training -Distance learning model developed- -Technical Backstopping/Assessments Title II Field Programs Source: ACDI/VOCA. 2001. ISA Mid-term Review. Pp. 6-7 and ACDI/VOCA. 2004. ICB Proposal. ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006. 3 $- $10,000,000.00 $20,000,000.00 $30,000,000.00 $40,000,000.00 $50,000,000.00 $60,000,000.00 $70,000,000.00 $80,000,000.00 $90,000,000.00 $100,000,000.00 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 U.S. Dollars General Devt Program Funding (WCF, Partners for Devt, GDA) Income Generated through Monitization Services for Other Organizations USDA Food for Progress, 416(b), and GFE Title II ISG/ISA/ICB Title II Umbrella Monetization Title II Comp. Mission Contracts Title II Figure 1.1. New Food Programs Awarded to the ACDI/VOCA FFD Division by Category, FY91-06 (by year of award) Methodology: Data reflects the approximate dollar figure of contract agreements by year. This figure does not reflect the actual expenditure by category of project by year. Source: Olutayo Akingbe, FFD/ACDI/VOCA, December 6, 2006 from financial records. ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006. 4 Table 1.2. Current ACDI/VOCA Title II and Non-Title II Food Programs (in terms of funds for programming and for pass/through/sub-contracts) Program Title Total Awarded to ACDI/VOCA Pass-Through and Sub-Contract Funding Total Award % Title II Cape Verde Title II $4,282,359 $9,488,108 $13,770,467 Rwanda Title II $9,816,201 $7,408,625 $17,224,916 Uganda Title II* $13,862,305* $15,298,748* $29,161,053* ICB (Title II) $1,318,001 $81,999 $1,400,000 Subtotal Title II Programs $61,556,436 62% Ecuador Food for Progress (CCC) $2,914,805 $2,149,981 $5,064,786 GDA Vietnam -- -- -- USDA Vietnam $3,410,304 $2,621,657 $6,031,961 USDA Philippines $3,160,284 0 $3,160,284 West Bank/Gaza 416(b) $2,441,551 $2,558,449 $5,000,000 Indonesia (USDA) $16,000,000 $18,284,976 Sub-Total Other Programs $37,542,007 38% Total $99,098,443 100% *These figures do not represent commodity value for distribution. Source: P. Gonzalez, ICB Coordinator, November 2006. 1.2. ACDI/VOCA’s Title II Institutional Capacity Building (ICB) Grant The goal of the ACDI/VOCA ICB was (ACDI/VOCA 2004): “to enhance its institutional capacity to promote food security among at-risk and vulnerable populations.” The activities planned to achieve this goal were conceptualized under four numbered strategic objectives (SOs) and a fifth (un-numbered) cross-cutting objective focused on improved management. Each strategic objective was broken down into measurable intermediate results (IRs) (Table 1.3). Specifically, the grant was designed to do the following. • Tools Development: Develop and (when appropriate) revise and/or adapt existing tools that ACDI/VOCA (as well as many other Title II Cooperating Sponsors) needs to achieve the ICB SOs and IRs (Box 1.1). • Training: Support the direct (e.g., trainee) and indirect (e.g., travel) costs associated with field and HQ staff attending internal and external trainings needed to achieve the ICB SOs and IRs. • Assessments: Support some costs of pilot testing new tools (through field assessments and training programs) to refine and adapt them to field conditions. • Documentation and Distance Learning: o Distance learning: Create a new model for “distance learning” based on production of user-friendly DVDs that ACDI/VOCA and the other Title II NGO Cooperating Sponsors could use for training; and o Documentation: Disseminate the results of the pilot testing of new tools and “best practice” in the field (ACDI/VOCA 2004: 8) to ACDI/VOCA’s Title II programs, as well as (when appropriate) other Title II NGO Cooperating Sponsors. ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006. 5 • Program Management: Strengthen the efficiency and impact of ACDI/VOCA’s Title II programs and monetization efforts. The total budget for the ICB program includes $1,400,000 from USAID, as well as a commitment from ACDI/VOCA of $187,000 (12 percent of the grant total) in matching funds, which is largely from “in kind” costs of volunteer labor. The direct beneficiaries of the ICB project were and continue to be ACDI/VOCA’s Title II development programs and monetization programs and other Title II NGO Cooperating Sponsors. However, the positive impacts of these capacity building efforts will also be felt in ACDI/VOCA’s other food programs (e.g., USDA). Table 1.3. Original ACDI/VOCA ICB Strategic Objectives and Intermediate Results Original SO and IR Statements in Project Proposal SO1: Capacity to address food insecurity through all possible uses of Title II resources improved IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved IR 1.3: Institutional capacity in commodity management enhanced Un-numbered cross-cutting IR: Documentation SO2: Technical excellence in food security programming enhanced through broadened partnership to build institutional capacity IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced IR 2.2: Collaboration with FAM and its member PVOs to build technical excellence through information sharing and exchange IR 2.3: Collaboration with academic and research institutions and international organizations strengthened SO3: Ability to design food security proposal enhanced IR 3.1: Capacity to identify the root causes of food insecurity enhanced IR 3.2: Nutrition impact of Title II programs enhanced IR 3.3: Integration of innovative HIV/AIDS programs strengthened Unnumbered cross-cutting: Distance learning SO4: Evidence base that support programmatic that improve impact IR 4.1: Documentation and integration of past performance successes improved IR 4.2: Institutional of Title II program data and analysis expanded Program Management 1.3. Mid-Term Evaluation Methodology The general intent of a Title II mid-term evaluation is to assess the grant’s progress in order to make recommendations for how to improve its impact and results. The specific objectives of the ACDI/VOCA mid-term evaluation were to: • Determine the accomplishments and evidence of impact of the ICB grant based on the proposal plan; • Analyze the principal factors that have contributed to/seem likely to contribute to the utility and viability of accomplishments/proposed outputs under the current program; and • Provide recommendations for improving grantee performance, including modifications to the proposed work plan and monitoring and evaluation system. ACDI/VOCA ICB Mid-Term. Introduction and Background. December 22, 2006. 6 The issues of project management, institutional strengthening, training, and examples of best practice were also examined.2 The mid-term evaluation used a variety of methods to determine the status of grant activities and accomplishments. These included: • A detailed review of official project M&E data, including the project Performance Indicator Tracking Table (PITT) and associated data tables; • Distribution of a questionnaire to the Title II field programs to gather some of the missing data on the ICB impact indicators (Annex V.C.1); • Document review (project deliverables [e.g., tools, curricula] and administrative reports); • Interviews with members of Food for Development division of ACDI/VOCA HQ; • Email and phone interviews with senior Title II program staff overseas; and • Focus group discussions of initial findings and recommendations with the Food for Development division’s HQ staff. To facilitate comparability between interviews with the Title II field program staff and the HQ-level project coordinators who backstop these programs, the external consultant used a structured questionnaire (Annex V.C.2). A total of 16 persons were interviewed at least once each through these various methods between November 1 and December 12, 2006 (Annex V.A and V.B). Della E. McMillan (external consultant) served as team leader. Piero Gonzalez (ICB coordinator) served as the internal evaluation manager and co-team leader. The remainder of the report is divided into sections two through seven. • Sections two-five describe: o Progress toward the execution of deliverables and achievement of targets based on the official monitoring indicators; and o Any evidence of early (e.g., mid-term) or potential impact based on the official impact indicators and other qualitative information from the mid￾term surveys and interviews. • Section six reviews the project management and financial systems. • Section seven examines some of the principal factors that contributed to or detracted from the successful execution of the grant and groups the main recommendations of the mid-term into six cross-cutting areas. The project PITT and other data needed to support the knowledge claims in the report are included in Annexes I-IV. Annex V includes the SOW, as well as a list of people interviewed and the questionnaires used to update the ICB mid-term impact indicators and to interview field and HQ personnel. 2 ACDI/VOCA. 2004. ICB Proposal. ACDI/VOCA ICB Mid-Term. Strategic Objective 1. December 22, 2006. 7 2.0. Strategic Objective 1 (SO1): Capacity to Address Food Insecurity through all Possible Uses of Title II Resources 2.1. Strategy Since 1989 ACDI/VOCA has been using Title II resources to implement food security programs and has achieved solid institutional capacity for commodity management while excelling at different types of monetization. When the ICB was design, however, there was an identified need for ACDI/VOCA to build on this established strength by: • Developing better systems for tracking the developmental impact of monetization programs; and • Incorporating the concept of developmental relief into its new and existing programs by: o Pilot testing state-of-the-art monetization models including market intervention methodologies that can reduce the impact of higher prices on food insecurity during shocks and natural disasters; and o Building the agency’s capacity to reduce community and household-level risk through targeted direct distribution programs as well as their traditional interventions in agricultural production and economic growth. ACDI/VOCA’s IRs and activities under SO1 were as follows (Table 1.3). • IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased. Pilot test new methodologies (i.e., indicators) for tracking developmental impacts of monetization. • IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved. Create new innovative food programming tools to help ACDI/VOCA programs and other Cooperating Sponsors better understand some of the ways that food insecurity and vulnerability can be reduced through: o Market interventions to stabilize prices in areas with extremely high staple food prices to protect remaining purchasing power of poor consumers and increase the availability of food, as well as o Agricultural rehabilitation interventions that focus on the agricultural marketing chain and “the importance of including training for farmers in crop selection, marketing, as well as identifying and addressi obstacles to marketing” (ACDI/VOCA 2004: 3), in addition to the traditional focus of most Title II programs on restorat ng ion of Farmers’ training by ACDIVOCA coordinator, Rwanda. Photo credit: ACID/VOCA. ACDI/VOCA ICB Mid-Term. Strategic Objective 1. December 22, 2006. 8 basic agricultural production at the farm-level through distribution of seeds, tools, and fertilizer. tor • IR 1.3: Institutional capacity in commodity management enhanced. Build and maintain ACDI/VOCA’s core capacity for basic commodity management including monetization and direct distribution through external and internal trainings in key areas, field visits to examples of “best practice” in direct distribution, and improved documentation of best practices. • Documentation. A fourth, un-numbered cross-cutting IR focused on improved documentation. 2.2. Progress toward Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators The SO1 monitoring indicators were designed to assess the creation of new indicators that could help ACDI/VOCA and the other Title II Cooperating Sponsors track the developmental impact of Title II monetization (IR 1.1), the number of emergency market assessments conducted using new and updated tools (IR 1.2), and the number of external and internal trainings for basic commodity and monetization management (IR 1.3). The original PITT did not include an indicator for tracking the un-numbered cross-cutting IR for documentation. Tools and Assessments: Staff turnover for some of the key first tier staff positions at the end of FY04 delayed execution of two of the three tools that were anticipated under this SO (the Monetization Indicator and Best Practice Toolkit and the Emergency Market Intervention Tool) (Annex I). With assistance from the ICB, an ACDI/VOCA volunteer did complete a draft Agricultural Rehabilitation Tool in FY06 that will be further refined by ACDI/VOCA HQ and field staff in the remaining years of the ICB. In addition, the ICB supported some of the costs of: • Revising the ACDI/VOCA Monetization Training Module (comprising of a Monetization Workbook, Pre and Post-Test Form, and PowerPoint training materials), which was developed under the ICB for HQ and field staff; and • Using the new tool to train 10 HQ￾level staff. During the second half of the grant, Thomas Gibb, a former director of FFD at HQ and soon-to-be the commodity operations manager in the Uganda Title II program, has been charged with initial development of the Monetization Indica Umbrella Monetization Program, Rwanda. The new commodity management indicators being developed under IR 1.1 should help ACDI/VOCA and other Title II Cooperating Sponsors better track the impact of monetization on development. Photo credit: ACDI/VOCA. ACDI/VOCA ICB Mid-Term. Strategic Objective 1. December 22, 2006. 9 and Best Practice Toolkit. This toolkit will examine “best practices” from various ACDI/VOCA and non-ACDI/VOCA monetization efforts and identify indicators that programs can use to track development impacts at critical points in the monetization process (Figure 2.2). A draft tool will be completed by early 2007 and will potentially be discussed at the next ACDI/VOCA FFD meeting. ICB funds have been used to support three Bellmon analyses that feed into emergency market assessments (in DRC, Burundi, and Mauritania) in connection with development of the emergency market intervention tool (see trip reports under A.2. IR 1.2: Miller 2006a, 2006b; Murphy 2004a, 2004b; Ose and Miller 2006). Training: The grant has been used to facilitate staff participation in a wide variety of internal and external trainings (Annex III.B). To date, however, these trainings have focused on basic commodity management and monetization (IR 1.1 and 1.3). Under IR 1.2 (Annex III.B), the ICB facilitated one workshop session focused on building staff capacity in direct distribution (ration size), one panel discussion that included a presentation on the new USAID/FFP development-relief paradigm, and one “on the job” three week training of a HQ staff member in connection with an emergency market assessment exercise in Mauritania (Ose and Miller 2006). ACDI/VOCA is planning to use the training materials that were updated and revised in conjunction with the FY06 internal training on monetization (which include a Monetization Work Book, PowerPoint training materials, and pre and post tests) to orient new HQ and field staff. 2.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data The original PITT anticipated tracking the field-level impact of the new tools and training being supported under SO1 through three impact indicators: • Impact Indicator 1.1: # of programs utilizing new tools/models/best practices; • Impact Indicator 1.2: # of food security proposals and reports integrating new commodity management indicators; and • Impact Indicator 1.3: # of staff showing 25% improvement in capacity. Tools: Since neither the Monetization Indicator and Best Practice Toolkit (scheduled for release in FY07) nor the Agricultural Rehabilitation Tool (draft completed in FY06) have been “rolled out” to the field, it is not surprising that no staff reported utilizing the new tools/models or best practices being tracked by Impact Indicator 1.1 (Annex I). Although ACDI/VOCA proposed to utilize the emergency market intervention methodology in its Mauritania proposal, the proposal was rejected. Training: Based on recommendations from the ISA mid-term and final evaluations, ACDI/VOCA expected to conduct pre and post-tests on each of its internal trainings as a basis for measuring Impact Indicator 1.3 (Annex I). Unfortunately, this type of pre and post-test was conducted at only one of the trainings and only in the third year (FY06) (see ACDI/VOCA ICB Mid-Term. Strategic Objective 1. December 22, 2006. 10 pre and post-test text in Annex IV.C). Only two staff members showed the 25 percent increase in capacity that the original indicator was designed to track. The chief reason was that the baseline knowledge of staff was already high before the training (80% of people taking the pre-test scored higher than 50% --Table 2.1). Rather than being a sign of failure, these high baseline figures are evidence of success of ACDI/VOCA’s focus on training staff. Specifically, the pre-test scores measure the aggregate impact of all the successful ICB-sponsored commodity and monetization trainings that had been used to train new3 HQ food programming staff during the two years that preceded the FY06 training exercise (see Annex III, Table A) that was evaluated in FY06.4 Therefore, the evaluation team recommends that the indicator be changed to: “% of FFD staff participating in internal trainings scoring more than 65% on basic commodity management/assessment tests.” The fact that there was a 30 percent increase in the number of staff who scored higher than 65 percent of the possible values on the post-test (Table 2.1) is important since this monetization pre-and post exercise (utilized by T. Gibb) is an example of best practice that needs to be continued for SO1 and replicated as a model for pre-and post training in key areas under the other SOs. Documentation: One direct impact of ACDI/VOCA’s extensive investment in internal and external trainings and tool development for monetization and commodity management has been a dramatic increase in the number of documents and examples of best practice. In anticipation of this, ACDI/VOCA created a cross-cutting (un-numbered) IR under SO1 titled “Documentation.” To date, the ICB has not yet put in place any in￾house system for presenting the different documents to internal or external audiences, such as a project bibliography. 3 Out of 10 participants in the workshop, 67% have been with ACDI/VOCA for 1.5 years or less; 22% for 2-2.5 years; and only 1 (11%) for 7.5 years. In contrast to the senior field staff supervising the ACDI/VOCA field programs, the mid-level HQ staff overseeing the programs tends to have less development and food programming experience (See Annex IV, Table A). 4 These HQ level trainings under IR1.3 include: a one day training on commodity management in FY04 (Trainer: Joe Gerstle), a three day training on food aid management at the American Red Cross office in FY06 (Trainer: Joe Gerstle) as well as a presentation on Reg 216 and OMB Circular A-122 and Cost Sharing in FY05 (Annex III.B and Annex II, Section A.3). ACDI/VOCA ICB Mid-Term. Strategic Objective 1. December 22, 2006. 11 Figure 2.2. Areas to Explore for Monetization Development Impact Indicators Commodity Flow Stakeholders Potential Indicators Comments Auction Notice Wsale/Retailer/Consum. Awareness of health benefits? Vit A fortified, less trans fats than alt. What do we say in ads? Ð Auction Training Small/big potential buyers Business training - financial analysis, competing with the bigger buyers Small buyers compete w/bigger buyers, level of partic. of women? Ð Bid / Submitting Deposit Bidders / banks Formal bidding system / utilizing new bank service - cashiers checks Making bidders accountable with non￾refundable deposit Ð Opening Bids Buyers Transparent bidding / information about competitors With public bid openings, each bidder knows exactly what competition has bid Ð Charting Bids/Selecting Winners Bidders Transparent bidding / information about competitors W/transparent pricing - potential buyer negotiating power Improved? Do they get a better price because of it? Ð Contracts with Winners Winning bidders Formal sales contracts Do buyers use formal sales contracts for other commercial purchases? Ð Pricing Consumers Pricing stability throughout the year Are auction vegoil prices more stable throughout the year than the substitutes? Ð Paying Balance Winning bidders/banks Paying balance directly to A/V bank account, Is this a common practice for commercial transactions? Ð Paying Balance Winning bidders/banks Buyer bank accounts Do more buyers have bank accounts as a result of partic. in auctions? Ð Pick-up Winning bidders/transport Contracting transportation Anything different for winning bidders vs. commercial transactions? Ð Wholesale Sales Wholesalers/Retailers Better supply/reliability, lower transaction costs (more flexible purchasing) With regular vegoil auctions, do buyers get better availability and lower transaction costs? Ð Retail Sales Retailer/Consumer Better supply/reliability? Better quality, healthier product How do retailers benefit from auction oil vs. other oil purchases? Ð Consumer Consumer Better quality/healthier product Do consumers benefit too? Ð Consumer Consumer Availability Availability of USA vegoil Source: Personal communication Thomas Gibb. December 11, 2006. Methodology: Figure represents the draft conceptual model being used by ACDI/VOCA in identifying and pilot testing (with existing data) monetization indicators. Table 2.1. Pre and Post-Test Values (as % of total possible value) for Individuals Attending the ICB-Sponsored Monetization 101 Workshop in FY06 Pre-Test Post-Test Suggested Categories n Percentage of test-takers n Percentage of test-takers Category I: score <=50% 2 20% 1 12.5% Category II: 50%65% 2 20% 4 50% Total 10 100% 8* 100% *Two people did not take the post-test. Source: ICB project coordinator. ACDI/VOCA ICB Mid-Term. Strategic Objective 2. December 22, 2006. 12 3.0. Strategic Objective 2 (SO2): Technical Excellence in Food Security Programming Enhanced through Broadened Partnerships to Build Capacity 3.1. Strategy ACDI/VOCA’s ICB proposal emphasized “collaboration” and “partnership” as important elements for achieving various intermediate results. Three types of partnership were envisioned in the following three IRs. • IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced. This IR was to be satisfied through development of an Organizational Assessment Tool (OAT) “to properly diagnose local partners’ baseline capacity for carrying out food security interventions and also to identify and define areas for which the organization will need capacity-building assistance” (ACDI/VOCA 2004). • IR 2.2: Collaboration with member PVOs to build technical excellence through information sharing and exchange. Activities under this IR included continued support for participation of various key staff in the FFD division in FAM working groups, FAM workshops, and other inter-CS coordinating committees and bodies. • IR 2.3: Collaboration with academic and research institutions and international organizations strengthened. This was to be done by working with the International Center for Research on Women (ICRW), as well as other relevant international institutions of excellence, to improve ACDI/VOCA’s techniques for incorporating gender into food security interventions. 3.2. Progress toward the Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators The monitoring indicators in the project tracking table for SO2 were designed to track the execution of gender tools (for IR 2.3), the number of meetings/workshops of Title II food security partners attended (for IR 2.2), and the number of tools and/or best practices shared with Title II food security partners (across ICB program components) (for IR 2.2). The original PITT did not include any indicators to track the execution of activities relevant to strengthening local partners’ food security interventions (IR 2.1). Tools, Assessments, and Training on Local Capacity Building and Gender: Organizational Assessment Tool (OAT): One major achievement of the ACDI/VOCA grant was the speed with which it developed a draft of the Organizational Assessment Tool in the first year of the grant. Based on the initial pilot assessments of the tool in Rwanda (in FY04) and Cape Verde (in FY07), a revised tool is to be prepared in FY07. ACDI/VOCA ICB Mid-Term. Strategic Objective 2. December 22, 2006. 13 Gender Training Tools: A second major achievement of the grant under SO2 was the speed with which ACDI/VOCA negotiated a partnership with ICRW that produced (Annex II, Annexes III.A and III.B): • In FY04: o A two-day HQ-level gender analysis training (August 11- 12, 2004) facilitated by ICRW on gender training terminology and basic concepts of gender analysis, o A compilation of resource materials for participants, and o A DVD training tool based on the August 2004 training, “How to use Gender Analysis to Strengthen Project Design and M&E” that was developed for future gender analysis training opportunities; Women sorting food rations, Uganda. Photo credit: ACDI/VOCA • In FY05: o Collaboration with ICRW that produced the curriculum for conducting a gender analysis exercise in food programs, o A second HQ-level training (May 14, 2005) that was facilitated by ACDI/VOCA staff, o Three field-level trainings using the new gender curriculum in West Bank/Gaza, Ecuador, and Indonesia that were facilitated by HQ-staff members during supervision missions, and o One plenary session at the FY05 annual FFD meeting (Annex III.D). Tools, Training, and Information Exchange with Other Title II Cooperating Sponsors (CS): Under the ISG and ISA grants, ACDI/VOCA was an active participant in the Title II-funded Food Aid Management consortium (FAM). Many of the ACDI/VOCA staff members benefited from the FAM-orchestrated trainings and participated actively in the FAM working groups. ACDI/VOCA anticipated that the majority of its activities under IR 2.2 would focus on these various FAM working groups (ACDI/VOCA 2004: 10). Once FAM ceased to operate at the end of FY04, ACDI/VOCA shifted the emphasis of its activities under IR 2.2 to participation in various other inter-CS networking groups: including the Food Aid Consultative Group (FACG), the FACG Commodity Working Group (CWG), and the Coalition for Food Aid (CFA) (Annex III. A). Several senior staff members have played important leadership roles in these groups: • Suzanne Berkey served as PVO co-chair of the FACG and at times hosted CWG meetings at ACDI/VOCA; ACDI/VOCA ICB Mid-Term. Strategic Objective 2. December 22, 2006. 14 • Buz Guroff and Thomas Gibb participated in the working group that focused on re-writing Reg. 11 in FY04 and FY05; • T.J. Ryan participated on the consortium working group that assessed the advantages and disadvantages of Cooperating Sponsors working together on Title II programs, as well as the FFP Strategy Working Group in FY04; • ACDI/VOCA, in conjunction with Catholic Relief Services, facilitated the PVO response to USAID/FFP’s FY06 draft guidelines; and • ACDI/VOCA was a member of the FACG General Issues Working Group (GIWG) in FY06. Even more staff members have participated in a wide variety of FANTA, USAID/FFP, ICRW, CFA, and FACG meetings, as well as the annual Kansas City Export Food Aid Conferences. The ICB facilitates this participation by enabling staff to charge a percentage of their time to the grant and by supporting travel costs (when relevant). 3.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data Unlike the other ICB SOs, which usually have one impact indicator for each major IR or group of IRs, SO2 has only one impact indicator in the original PITT, which pertains to activities under IR 2.3: • Impact Indicator 2.1: # of new proposals and/or existing programs that incorporate new or improved gender integration tools. Given the fact that partnership was seen as the mechanism for achieving three specific IRs, it is useful to discuss the impacts observed separately for each IR, as well as for the only impact indicator under this SO (Impact Indicator 2.1). A recent field test of the Organizational Assistance Tool (OAT) on the community association “Association Joao Garido” in Cape Verde, in November 2006. Photo credit: ACDI/VOCA. IR 2.1: Since the Organizational Assessment Tool (OAT) is being revised, it is not realistic to expect it to be used in field programs until late FY07 and FY08. During the most recent pilot test in Cape Verde (November 2006), a HQ staff member (Suzanne Berkey) tested the methodology on one community association (Joao Garrido) and one umbrella association (OASIS). Both groups understood that ACDI/VOCA was testing a methodology that would complement some of the existing capacity assessment analyses that the ACDI/VOCA ICB Mid-Term. Strategic Objective 2. December 22, 2006. 15 ACDI/VOCA Cape Verde office had already covered with them. Based on the pilot te ACDI/VOCA developed a list of recommendations for revising the format of and language of the OAT. The team also made recommendations for reducing redundancy between questions and for streamlining the interviews in situations (such as was the for the pilot test) where a certain amount of pre-existing data (on financial issues, for example) already exists. st, case oduce final draft OAT in FY07. taff ose with Commodity orking Group and the FACG) during weekly staff meetings. ors—are ting One f the . orum to operate, it could be another important inter-CS network for capacity uilding. ing” cts e way they interpret project data” and even the supervision questions that they ask. dinators o assist st half of the grant have been in three non-Title II countries (Ecuador, Indonesia, West 5 Various members of the FFD HQ and field teams will pr a IR 2.2: One strength of the ACDI/VOCA administrative model—which many of the s underscored during interviews—is the senior administrators’ commitment to sharing information that they receive from inter-CS meetings (such as th W What are lacking—not just for FAM, but for all the Title II Cooperating Spons more structured mechanisms for exchanging information on Title II technical programming and M&E, such as those that FAM facilitated for the Title II Coopera Sponsors between 1989 and 2004. This is especially important to the professional development for first tier ACDI/VOCA personnel (such as project coordinators) with good technical training, but that lack in-depth field experience in Title II projects. promising new initiative that ACDI/VOCA might wish to encourage their staff to participate in is the Evaluation Interest Group (EIG)—which was created by some o former members of the FAM M&E working group under the aegis of Interaction. Several staff participated in the recent Food Security Assessment Workshop (September 19-21, 2006) that was organized by the new PVO Collaborative Forum on Food Security ACDI/VOCA was also a PVO co-sponsor of the workshop. If the Collaborative F continues b IR 2.3: ACDI/VOCA envisioned—and has created an excellent model for—“cascad training in gender. Unfortunately, there was no pre or post-test given for any of the gender trainings. Several staff concurred (in a group interview) that the training “affe th Project coordinators are the principal mechanism through which gender “training of trainers” training is translated into field-level training. Specifically, project coor are expected to use the gender modules to train project staff during their annual supervisory trips, which are typically programmed to last two to three weeks. T with gender training, the ICB gender point person (who is also the ICB project coordinator) offers a one-on-one training/coaching session so that the project coordinator can be comfortable using the written and DVD training modules. To date, however, the only field-level use of the ICRW-ACDI/VOCA gender curricula during the fir 5 S. Berkey. Internal Notes: OAT Findings. December 2006. ACDI/VOCA ICB Mid-Term. Strategic Objective 2. December 22, 2006. 16 Bank/Gaza). One Title II project (Uganda) hired a local Ugandan trainer to design a separate training course. The deputy director of the Title II Uganda project reported that: This course has been very useful in helping the local grantees to understand the relationships between the household, agriculture, and the cultural assumptions about gender (personal communication, Sandra Blanchard, December 2006). Both Title II project coordinators recommended that future scopes of work for their supervisory travel include a clear set of project orchestrated priorities about what types of training (on gender or other topics) would be most relevant for that field program. It would also be very useful if the HQ and field-level training programs would identify which training programs were conducted using these and other types of ICB-generated training materials (including the gender curricula and the two DVD training sessions). ACDI/VOCA ICB Mid-Term. Strategic Objective 3. December 22, 2006. 17 4.0. Strategic Objective 3 (SO3): Ability to Design Food Security Proposals 4.1. Strategy The ISA final evaluation identified several areas where ACDI/VOCA felt it needed to build its internal capacity for proposal design and execution. Specifically, the activities under SO1 were designed to address the following (ACDI/VOCA 2004). • IR 3.1: Capacity to identify the root causes of food insecurity enhanced. o Build ACDI/VOCA’s capacity to use needs-assessments to identify the root causes of food insecurity; and o More fully integrate monitoring and evaluation into the design process so that objectives developed for new programs will drive monitoring and evaluation from the beginning. • IR 3.2: Nutrition impact of Title II programs enhanced. Improve ACDI/VOCA’s capacity to incorporate and measure nutrition components of its food security programs. • IR 3.3: Integration of innovative HIV/AIDS programs strengthened. Build ACDI/VOCA’s capacity to address HIV/AIDS within the context of its food security programming. • Distance Learning. A fourth un-numbered, cross-cutting IR focused on the model for turning the live trainings being developed under the different ICB IRs into training session DVDs that could be used to train new and existing staff. Given the constraints on staff time, ACDI/VOCA contracted with TANGO (Technical Assistance to NGOs) International to produce the Food Security Needs Assessment Toolkit, which was finalized in FY06. The grant anticipated paying 15 to 20 percent of a HQ-based staff member’s time beginning in year three for HIV/AIDS-related activities. It was anticipated that this person would: • Identify and participate in training on agriculture and nutrition and linkages to HIV/AIDS programming to enable her/him to provide high quality technical support to ACDI/VOCA’s efforts to enhance the impact of programs aimed at PLWHA (people living with HIV/AIDS); Primary PLWHA beneficiary at home, Ggaba Kampala, Uganda. Photo credit: ACDI/VOCA. ACDI/VOCA ICB Mid-Term. Strategic Objective 3. December 22, 2006. 18 • Serve as a trainer for ACDI/VOCA staff and, in some cases, implementing partners in the field; • Participate in development and review of new proposals and existing programs for incorporating HIV/AIDS components; and • Serve as a liaison between ACDI/VOCA and other PVOs and attend FAM activities on HIV/AIDS. Under IR 3.2, the ICB proposal anticipated the need: • To train “both its field and headquarters staff…..to upgrade their skills….about nutrition and how they can incorporate nutrition into their programs”; and • As approaches and best practices are identified….[to share these]…with PVO partners and [ensure that these are] institutionalized within the organization through brown bags, trainings in the field during supervisory visits or using distance-learning techniques” (ACDI/VOCA 2004: 13). 4.2. Progress toward Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators The monitoring indicators in the project tracking table for SO3 were designed to track the number of tools and/or best practice documents and trainings developed to support the achievement of all three IRs (IR 3.1-3.3), the number of assessments conducted using the new needs assessment tool (IR 3.3), and the number of field and HQ-level sessions that were organized using the new DVD training materials (cross-cutting IR on distance learning). The original PITT did not include any monitoring indicators that focused specifically on the execution of activities related to nutrition or HIV/AIDS.6 Tools: The original PITT anticipated development of five tools and/or best practice documents from activities under SO3 (Annex I of this report; ACDI/VOCA 2004 [M&E Plan] PITT). To date, only one tool—the Food Security Needs Assessment Toolkit—has been developed (under IR 3.1). These other four tools were presumably written documents that the project expected to emerge from use of the needs assessment tool in project designs and the project’s activities on HIV/AIDS and nutrition that were scheduled to start in FY06. Training and Technical Backstopping: IR 3.1: Achievements under this IR include, participation of several staff in the three-day food security assessment workshop in FY04 sponsored by the PVO Collaborative Forum on Food Security, a two-day internal training on value chain analysis in FY05, and one M&E presentation at the FFD meeting in FY05 (Annex III.B). 6 One impact indicator focused on HIV/AIDS. ACDI/VOCA ICB Mid-Term. Strategic Objective 3. December 22, 2006. 19 IR 3.2: To date, there have been no internal or external trainings on nutrition (Annex III.B) and no focused technical backstopping. Staff are interested in the topic, however, and have taken it upon themselves to attend information sessions, such as a recent presentation on preventative and recuperative care in Haiti. IR 3.3: There have been two general presentations on HIV/AIDS at the annual FFD meeting (Annex III.).7 There has not been any internal or external training of HQ staff on HIV/AIDS and only limited technical backstopping from the HQ staff person who was tasked with being the FFD division’s HIV/AIDS point person in FY06.8 The ICB did, however, pay for one of the Title II Uganda program staff members to attend a regional workshop on HIV/AIDS in Zambia (see Annex IV.B). Distance Learning: One innovative component of the ACDI/VOCA ICB was its commitment to developing edited DVDs to be used in ICB-related training sessions. The project anticipated producing at least eight training session DVDs over the life of the ICB (ACDI/VOCA 2004) and conducting 22 training sessions.9 Although the DVD training modules were cross-cutting (i.e., they touched on themes related to each of the IRs/SOs) they were originally conceived of and tracked as part of SO3. Better background materials and training to promote innovative HIV/AIDS programs was a major focus of the ACDI/VOCA ICB grant. Photo credit: ACDI/VOCA. The project has been very successful in developing and editing these training modules. Specifically, four of the eight DVDs have already been completed (one on value chain analysis, one on Reg. 216 Environmental Assessment, and two on gender). In addition, several PowerPoint modules have also been developed that accompany the monetization workbook (developed under SO1) and one has been developed on “How to Conduct a Bellmon Analysis” that accompanies an internal tool on conducting a Bellmon Analysis (See Annex III, Section A.4). 7 One breakout session, “HIV/AIDS Current Programs and Strategy” (Trainer: Sandra Kugonsa-Isingoma) at the FY04 meeting and one plenary session on HIV/AIDS (Presentation by Judy Canahuati, USAID/FFP) in FY06 (See Annex III.B). 8 The two Title II field programs that have incorporated HIV/AIDS activities stated that their staff’s in￾country contacts have been their principal source of information on HIV/AIDS programming options and examples of best practice. 9 ACDI/VOCA. 2004. ICB M&E Plan. PITT. ACDI/VOCA ICB Mid-Term. Strategic Objective 3. December 22, 2006. 20 4.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data The original PITT included three impact indicators: • Impact Indicator 3.1: # of new or existing program interventions that integrate HIV/AIDS activities; • Impact Indicator 3.2: # of new or existing programs that incorporate improved M&E tools or methodologies; and • Impact Indicator 3.3: # of new proposals that reflect analysis of long-term impact assessments. Even without the development of more focused training and technical backstopping, two of the three Title II projects are involved in HIV/AIDS activities (IR 3.3) (Impact Indicator 3.1, Annex I).10 In general, however, field staff reported that the impetus for these innovations came from the field, rather than the ACDI/VOCA HQ office (despite the fact that the HQ office hosts the cross-cutting HIV/AIDS team and the FFD HIV/AIDS point person position that was partially funded by the ICB starting in FY06). IR 3.3: The principal constraints that confront the ICB in achieving the goals of IR 3.3 for strengthening integration of innovative HIV/AIDS programs in ACDI/VOCA’s Title II programs include: • Conflicting obligations that have reduced the amount of time they can devote to HIV/AIDS activities related to Title II programs from the 15 to 20 percent anticipated by the grant; and • The difficulty of identifying appropriate training sessions (i.e., trainings that address the issue of HIV/AIDS within the context of food programming) and networking opportunities once the FAM11 project ended in FY04. IR 3.2: A similar set of constraints have hindered achievement of IR 3.3’s objective of enhancing the nutritional impact of ACDI/VOCA’s Title II programs. In contrast to the HIV/AIDS activities, the grant did not include any support for the HQ-level person charged with providing leadership on nutrition. As a result, the nutrition point person was given the responsibility of backstopping this component of the ICB, but no resources in terms of billable time. This problem was compounded by the difficulty of identifying a relevant training program for the HQ staff member charged with execution of these activities once FAM was no longer able to serve as a clearing house for identification of suitable CS and FAM-sponsored training programs on Title II nutritional assessment techniques. 10 Rwanda has increased its activities. Although the number of PLWHA receiving rations decreased in the new Uganda Title II program, ACDI/VOCA is trying to integrate more PLWHA into their livelihood activities. 11 FAM was identified as a major source of training and networking with other PVOs concerning HIV/AIDS in the original proposal. ACDI/VOCA ICB Mid-Term. Strategic Objective 3. December 22, 2006. 21 Distance learning: To date, one HQ-level group training and a handful of one-on-one staff orientation sessions have used the DVD training modules. To encourage greater use during the second half of the grant, the ICB is focusing on: • Equipping the HQ-based project coordinators with background training and tools needed to train field staff during their supervisory travel; • Helping to ensure that field-level training (using the modules) is included in the written scopes of work for project coordinators’ supervision missions; and • Distributing the DVD and PowerPoint training modules and manuscripts to field programs upon request. These activities would be strengthened by the preparation of a short description of the DVD and PowerPoint training modules and how they could be used to train staff on particular issues. It is also necessary to have better systems for tracking HQ and field￾level use of the modules similar to the “feeder tables” developed in Annexes III.B and IV.B. The chief constraint to increasing HQ field support for HIV/AIDS activities in Title II programs is the same constraint that is impeding technical support for nutrition (IR 3.2) and M&E (IR 3.1): none of the point persons for M&E, nutrition, or HIV/AIDS ever received any of the specialized trainings and documentation they expected to have through FAM. Furthermore, the fact that the ACDI/VOCA ICB training and technical backstopping for M&E was lumped with the needs assessment tool development activities in a single IR made it difficult to identify and track these activities in planning sessions and reports.12 12 Although the introduction to SO3 in the proposal mentioned four IRs (improved capacity for needs assessment, M&E, nutrition and HIV/AIDS), SO3 lumped M&E activities with IR 3.1, but only mentioned needs assessment in the IR statement (“Capacity to identify the root causes of food insecurity enhanced”). The recommended revision of the IR statement during the mid-term is designed to take into account the original intent of the grant, which was to consider the needs assessment tool as part of a more general improved capacity in M&E. The proposed revision of the IR statement is: “IR 3.1: Capacity to identify the root causes of food insecurity and to integrate monitoring and evaluation into project design and execution enhanced.” ACDI/VOCA ICB Mid-Term. Strategic Objective 4. December 22, 2006 22 5.0. Strategic Objective 4 (SO4): Evidence Base that Supports Programmatic Approaches that Improve Impact 5.1. Strategy One major important sub-IR of USAID Office of Food For Peace’s 2006-2010 Strategic Plan is “Evidence base for more effective policy and program approaches improved” (sub-IR 1.3) (USAID 2005: 46). The activities under SO4 were expected to contribute to this objective through the following. • IR 4.1: Documentation and integration of past performance success improved. Creating a tool that would help project managers and design teams better assess the long-term impact of food aid programs. • IR 4.2: Institutionalization of Title II program data and analysis expanded. Creating an analytical system whereby Title II M&E data on individual projects can be aggregated and analyzed. 5.2. Progress toward Execution of Deliverables and Achievement of Targets Based on Official Monitoring Indicators The monitoring indicators in the project tracking table for SO4 were designed to track the number of assessments conducted using the new needs assessment tool (IR 4.1) and creation of the computerized database “knowledge management tool” (IR 4.2). ACDI/VOCA completed a draft “Long-Term Impact Assessment Guidelines” in FY05 (Monitoring Indicator 1.1). The tool has been used for assessments in two countries (Cape Verde and Uganda) and an assessment is planned for Rwanda in FY07. ACDI/VOCA is behind in its planned activities for IR 4.2, but is moving forward with a highly innovative data entry and analytical system for Title II project M&E data. 5.3. Evidence of Actual or Potential Impact based on Official Impact Indicators and Other Data The original PITT included one impact indicator: • Impact Indicator 4.1: # of new proposals that reflect analysis of long-term impact assessments. Given the difficulty that many volunteers experienced in collecting the types of historic information on the projects that they needed to analyze long-term impacts, ACDI/VOCA is recommending that the Rwanda assessment (scheduled for FY07) be carried out, but with a less ambitious SOW than the other two assessments (in Cape Verde and Uganda). Although ACDI/VOCA anticipates sharing a final version of the tool with the other Title II Cooperating Sponsors and its own field programs, it is not recommending any additional assessments or scaling up of the tool within ACDI/VOCA’s other food security ACDI/VOCA ICB Mid-Term. Strategic Objective 4. December 22, 2006 23 programs. For this reason, ACDI/VOCA proposes eliminating Impact Indicator 4.1 after the mid-term. The new M&E computerized database is in the early stages of being pilot-tested. Over the next few months, each of the Title II project coordinators will enter the M&E data on the projects they supervise; one Title II project’s data has already been entered. The intent is that the computerized system will eventually be a web-based model that HQ and field staff can use to enter and access information. Once this data is entered, project coordinators should be able to work with in-house or consultant experts to analyze cross￾cutting trends and determine which indicators (in addition to those required by USAID/FFP) could be adopted as “core” indicators (i.e., indicators that are included in the PITT of every ACDI/VOCA Title II program). Given the critical importance of this process, the mid-term recommends adding an impact indicator that will track the extent to which the Title II programs analyze this information (new Impact Indicator 4.1: # of comparative analyses of trends in Title II programs based on data entered into the database). For the full impact of this M&E “knowledge management tool” to be realized, it needs to be strengthened by greater harmonization of the indicators themselves and methodologies for measuring indicators in Title II project PITTs. This harmonization process will also need to take into consideration FFP’s recent (August 2006) recommendations for core indicators, as well as the new State Department proposals for generic indicators. ADCI/VOCA ICB Mid-Term. Management and Finance. December 22, 2006. 24 6.0. Management and Finance 6.1. Formal Management Structures and Systems Administrative Structures: Since 1988 ACDI/VOCA’s food security programs have been coordinated by the Food for Development (FFD) division. When the FFD division started, its programs were almost exclusively financed by Title II. It now backstops USDA programs, as well as more conventional development and Title II programs (Figure 1.1 in Section 1.0). The mandate continues to expand beyond food aid programming, which is why the division’s name will eventually be changed to the Food Security and Specialty Crops portfolio. The FFD division Vice President has over eight years experience in Title II food programming (Annex IV.A). The FFD division is supervised at the highest level by the Senior Vice President, who has over 40 years of experience in food programming for USDA and Title II (Annex IV.A). Reflecting the broader growth in the number of ACDI/VOCA food security programs, the size of the portfolio has increased from four full time staff members at the mid-term point of the previous ISA grant to eight full time staff members (excluding the Senior Vice President) at mid-term of the current ICB. Despite HQ turnover at the end of the first year of the ICB grant, these positions were filled quickly and there has been almost no staff turnover in the division since the start of FY05. This is an enviable record given the fact that many Title II Cooperating Sponsors experience between 50 to 100 percent turnover in mid-level food programming positions every two years. Volunteers: One feature of ACDI/VOCA’s ICB—which distinguishes it from other Cooperating Sponsors’ Title II ICB grants—is its extensive use of ACDI/VOCA volunteers. The ICB grant work plan was designed to use 23 volunteers for various tasks associated with tools development, assessments, and training. The “in kind” contribution of the volunteers’ labor is the basis for all ACDI/VOCA cost-share on the ICB. The ICB project coordinator is responsible for recruitment, orientation, fielding, and supervision of volunteers in conjunction with ACDI/VOCA’s recruiting team. Management Communication: HQ backstopping of Title II programs (including the ICB) is centralized under three project coordinators. Each field program is led by a chief of party (COP) and deputy chief of party. Some FFD HQ project coordinators—including the ICB project coordinator—backstop both non-Title II and Title II programs. One major role of the ICB grant is to “buy the time” for portfolio staff to: (a) train in key areas that will improve their management of Title II programs, as well as (b) develop new tools and conduct assessments. Weekly meetings are held with all FFD HQ staff to coordinate between projects. In addition, coordination and information exchange has been facilitated through ACDI/VOCA’s commitment to organizing an annual food programming meeting. ADCI/VOCA ICB Mid-Term. Management and Finance. December 22, 2006. 25 Monthly division updates are posted to an internal ACDI/VOCA bulletin board to inform other divisions of current activities. Over 50 percent of the internal trainings counted in the PITT were held in conjunction with one of the three annual meetings that have occurred since the grant started (Tables 6.1 and 6.2). Staff travel expenses for these retreats are paid by individual grants and/or overhead. In most cases the only cost charged to the ICB was for consultants to conduct specific trainings. With rare exception (for which the FFD specific issues were addressed in only one day of a week-long meeting) these food programming retreats are usually one week long. They provide one of the best opportunities for ACDI/VOCA to orient staff to new tools and issues. The minimum attendance at the meetings (by Title II programs) was typically two to three expatriate staff and one to two national staff (Table 6.2).13 At the other end of the continuum was the Uganda program, which sent 15 staff members to the Rwanda FFD retreat in FY04 (11 national staff and four expatriates) and eight staff members (six national and two expatriates) to the Leesburg, Virginia retreat in FY05. This solid tradition of organizing an annual food programming meeting is clearly an example of best practice for building capacity of HQ and field personnel simultaneously. Especially important, it presents an ideal venue for training staff in the use of the tools and for getting feedback on field experiences with the tools once they are introduced. Financial Management Systems: The ICB project coordinator is the person primarily responsible for budget planning and oversight. Financial oversight is facilitated by ACDI/VOCA’s strong internal systems and is supplemented by (Annexes II. C.1 and II.C.2): • An electronic financial tracking system, which enables the project coordinator to track expenses by category for each SO and line item more or less instantaneously; • An internal Quarterly Assessment conducted by the president’s office staff; and • An electronic Mid-Term Internal Project Review, in which a project is subjected to an intensive internal team review. 13 Title II participation was lower at the FY06 retreat, which focused on ACDI/VOCA’s forthcoming reorganization with only one day focusing on food programming. ADCI/VOCA ICB Mid-Term. Management and Finance. December 22, 2006. 26 Table 6.1. Internal Trainings Facilitated by the ICB in Connection with ACDI/ VOCA Annual FFD Meetings, FY04-06 S01 S02 S03 S04 S05 Type of Training IR1.1 IR1.2 IR1.3 IR1.4 IR2.1 IR2.2 IR2.3 IR3.1 IR3.2 IR3.3 IR4.1 IR4.2 MGT FY04 in Rwanda Hygiene and Nutrition BO HIV/AIDS: Current Programs and Strategy (Isingoma) BO Plenary reports on HIV/AIDS and Monetization P P Monetization and Private Sector Capacity Building BO Direct Distribution Strategic Planning Session BO ICB (Steele) P Local Organization Assessment Tool (Pons) T (1/4 Day) FY05 in Leesburg Code of Federal Regulations and Grants Mgt. T (2.5 day) ICB Overview P Monetization Panel BO M&E (Seroussi) BO Direct Distribution (Cekan) BO Gender Awareness and Analysis (Leonhart and MacQuarrie) P HIV/AIDS Plenary (Canhuati) P ICB (Piero) BO Env.Compliance/Reg 216 B0 Developmental relief paradigm (Landis, Croushorn) P A-122 Training T (1.0 day) FY06 at ACDI/VOCA HQ FFD Contracts & Grants Training BO How to Conduct a Bellmon or a Market Analysis BO Source: Mid-term Evaluation from ACDI/VOCA FFD Unit Records Codes: BO=Break-out sessions for smaller sub-groups; several operate simultaneously (usually 1.5 hours in length); P=Presentation to entire group (for the entire group); T=Focused Training Session. Numbers (¼, ½, 1.0: 2.5) indicate the number of days training lasted. ACDI/VOCA ICB Mid-Term. Management and Finance. December 22, 2006. 27 Table 6.2. Number of Title II Project Staff who Attended the Annual ACDI/VOCA FFD Meeting, FY05-06 Cape Verde (#) Rwanda (#) Uganda Annual Meeting (#) Expat Nat Expat Nat Expat Nat FY04 Rwanda (5 days on food security ) 2 2 2 7 4 11 FY05 Leesburg (5 days on food security) 2 2 1 2 2 6 FY06 A/V HQ (1 day on food security only) 2 1 2 1 3 1 Expat=Expatriate staff; Nat=National staff; #=number. 6.2. Evidence of Early or Potential Impact Execution and Impact of Title II-Funded ICB Grant: One major strength of ACDI/VOCA’s ICB management structure and systems is that it enables project coordinators to have quick feedback on project management and the costs associated with specific project deliverables. This emphasis on comparing deliverables against the budget seems to have helped the ICB stay focused and on-task despite massive turnover in many of the key mid-level positions at the end of the first year. The chief weakness is that both internal audits focused almost entirely on the project’s financial status with very little reference to the project’s official monitoring or impact indicators or Intermediate Results (IRs). Another weakness is that there are few established channels for getting structured feedback from the field about the field’s priorities for the ICB IR’s and/or their reaction to specific ICB tools and trainings or for assessing field needs. This is not surprising since many of the tools are still in draft form or were only finalized only in FY06. Execution and Impact of Title II Field Programs: The management structure has permitted ACDI/VOCA’s HQ to provide essential project support to existing Title II programs. While there is a great deal of qualitative evidence that this technical support helped increase Title II program efficiency and impact, there is currently no system that allows the field to assess the impact of supervisory visits. ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 28 7.0. Conclusions and Recommendations 7.1. Summary: Progress toward Achieving Intermediate Results/Indicators and Targets and Recommendations for Refining Targets ACDI/VOCA has made clear progress towards achievement of all four its original SOs and eight of the nine original numbered IRs and seems likely to achieve the vast majority of its original LOA targets. The team is recommending, however, that (Annex I): • Four of the original targets be decreased slightly in large part because ACDI/VOCA is likely to be involved in fewer Title II countries than was originally expected in FY04 (Impact Indicator 1.1, Impact Indicator 1.2, Impact Indicator 2.1, and Monitoring Indicator 3.3); • Two of the original LOA targets be revised upward due to the project overachieving its original goals (Monitoring Indicator 1.4 and Monitoring Indicator 3.2); • One of the original impact indicators be eliminated because it tracks a tool (the long term impact assessment tool) that the project is no longer recommending (Impact Indicator 4.1); • One of the indicators be moved to another location (from SO3 to SO1) due to the creation of a new IR under SO1 that consolidates the two un-numbered cross￾cutting IRs (documentation and distance learning) into an official IR that can be better tracked (Impact Indicator 1.6); • Three new indicators be added to the PITT to track critical activities related to the IRs and SOs (Monitoring Indicator 1.6, Monitoring Indicator 2.3, and new Impact Indicator 4.2); and • That the “program management” indicator (that was proposed in the FY04 ICB M&E Plan) be eliminated since it is difficult to calculate and can give a false impression that the project has difficulties in timely Title II reporting when this is not the case. 7.2. Strengths, Weaknesses, Opportunities, and Threats Strengths: Key factors that contributed to the successful achievement of the projects objectives are: • The highly skilled HQ and field staff who have a combined experience of over 100 person-years in Title II food security programming (Annex IV, Table A); • The project’s excellent management, accounting, and planning systems; • The annual ACDI/VOCA food programming meetings that the chief of party, deputy chief of party, expatriate staff, and several national staff attend; and • The fact that many of the ICB tools build on pre-existing ACDI/VOCA tools (such as the partner pre-assessment form and Monetization Best Practices Manual). ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 29 Weaknesses: Factors that detracted from efficient execution and completion of some of the ICB activities during the first half of the grant were: • High rates of HQ staff turnover at the end of the first year; • The initial plan to use volunteers to develop many of the tools, which made it difficult to ensure the quality control and oversee the revisions necessary to finalize tools for internal and external audiences; • Use of a non-standard format for the PITT in the FY04 and FY05 annual reports to USAID, which made it difficult to compare achievements with targets; • Insufficient M&E backstopping of the grant by staff with experience in Title II M&E requirements for indicator quality assurance and standard reporting formats; • Lack of any standard system (linked to the project M&E system) for tracking field and HQ level-staff capacity in key areas being targeted by the grant; • The FAM project’s demise, which: o Required the project to gear up internal trainings in many core areas (M&E, nutrition, local capacity building, HIV/AIDS, and food security) that the FAM working groups traditionally sponsored; and o Increased the difficulty of sharing ICB-developed tools with other CS partners; • The fact that neither the project proposal nor the M&E plan developed a list of impact indicators or a standard PITT with which to assess HQ or field-level capacity being strengthened by ICB activities (except the pre and post-test for the monetization training); and • Poor systems for tracking HQ and field-level training in key areas targeted by the ICB developed tools, training curricula, and DVDs. Opportunities: Plans are underway to continue pilot testing and reviewing the major tools in field programs; to use the tools in actual field assessments associated with the design of new Title II, USDA, and Food for Development programs; and to accelerate staff training in key areas, which the tools will reinforce during FY07 and FY08. Factors that will contribute to the successful execution of this opportunity include the following. • Posting of material on the ACDI/VOCA website: The likelihood that some of the documents will be shared both internally and with the other Title II Cooperating Sponsors is enhanced by the fact that ACDI/VOCA has an established webmaster and a FFD website.. • Patterns of collaboration: The likelihood that the tools will be relevant to other Title II CS’s and will take into account examples of best practices from the other CS’s will be enhance by the HQ FFD division staff’s current project networking with the other Title II Cooperating Sponsors through: o Field visits (such as the one scheduled under SO1 to examples of best practice in direct distribution) and HQ-level collaboration of the three Title II project coordinators with colleagues in other Title II CSs; o Participation of the M&E point person and (when appropriate) the ICB coordinator in the Interaction EIG working group; and ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 30 o Participation of the ICB coordinator in the new emerging Consultative and Coordinating body among NGOs engaged in food security programs (funded by various USAID Title II Cooperating Sponsors and FANTA). • Experienced staff: ACDI/VOCA’s long involvement in food programming and patterns of moving staff from field to HQ and reverse means that the organization has ample internal resources in all areas (except M&E) for ensuring quality and accuracy of the final tools posted for internal and external users. • Training: In the course of the mid-term evaluation, it became clear that most programs have developed project-specific systems of tracking training at different levels that are harmonized (to various degrees) with the USAID TrainNet system or other pre-existing systems. These systems provide a mechanism for tracking what types of training are filtering down from the training of trainers training sponsored by the ICB. Threats: The chief risks associated with the grant are as follows. • ACDI/VOCA’s Title II programs vary in the degree to which they “need” and have the “flexibility within their MYAPs” to benefit from and feed into the ICB IRs. In order to avoid problems stemming from these differences, HQ leadership needs to continue to listen carefully to field needs in developing its training programs and tracking systems. Systems are currently in place to have this kind of field-level feedback. They need to remain active or the Title II programs could become hostile to the ICB. • If the project doesn’t consider new innovative ways to use volunteers that are more appropriate to the amount of time and level of commitment that they can offer to the project, then the rate of development of “tools” will lag behind and be incomplete. • Given the achievements to date for specific IRs, many of the tools will remain in draft form, filed on a shelf, and never used if the project doesn’t establish a set of priorities for which tools are to be finalized, • If the ICB doesn’t develop a simple system of annotated bibliographies that describe the tools and where they can be found, staff may not use the tools or training modules. • If the project doesn’t develop a better system for finalizing and posting the tools, other Title II Cooperating Sponsors will never benefit from the ICB investment in these tools or ACDI/VOCA’s demonstrated expertise in areas where it is developing cutting-edge tools. 7.3. Cross-Cutting Recommendations for Enhancing Grantee Performance Recommendation #1--Tools: Strengthen current systems for finalizing, organizing, training, and sharing tools. 1.a. Priorities: Set priorities for selection of tools to be finalize. Lower priority tools might include the emergency market intervention tool and others that the project might ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 31 wish to defer or eliminate in order to better concentrate on finishing higher priority tools that are more urgently needed. 1.b. Review process: Identify the most relevant field staff for initial feedback on the tools in order to enhance “ownership” and “relevance.” 1.c. Supervision: Task specific HQ and field staff—who are qualified experts in the areas being covered—with responsibility for participating in final production and revision of the tools and give them credit for the exercise on their annual work plan. Recommendation #2—Training and Distance Learning: Develop better systems for tracking HQ and field-level internal and external training in the areas targeted by the ICB IRs. 2.a. Track training: Build on the existing ACDI/VOCA HQ and field systems for tracking internal and external trainings for Title II field programs and the portfolio HQ office to develop better systems for tracking how many people have been trained in specific areas related to the ICB IRs and SOs. 2.b. Access to training documents: Strengthen ACDI/VOCA Food for Development systems for collecting and listing (in bibliographies) materials distributed during external ICB-sponsored training workshops in key areas being strengthened under the ICB (e.g., commodity management, M&E, and nutrition). 2.c. Practical teaching training of trainers with modules: Staff recommended offering a series of training of trainers workshops in conjunction with the annual ACDI/VOCA FFD meeting in which they could get “hands on” experience in teaching with the DVD modules and tools. 2.d. Link planning for the annual food security meeting to some of the training needs identified by the ICB mid-term: A high percentage of the presentations and break-out sessions during the FY04, FY05, and FY06 annual retreats focused on monetization, commodity management, and project management (Table 2.1, Section 2.0). Future planning needs to consider some of the areas related to other ICB IRs. It is important to recognize, however, that the annual meeting must respond to a wide range of identified needs and not just those of the ICB. Recommendation #3—Collaboration: Supplement ACDI/VOCA’s existing model for inter-CS collaboration with more focused networking of mid-level HQ food programming staff in three key areas (M&E, HIV/AIDS, and nutrition). 3.a. Networking: Support participation by the FFD point persons for HIV/AIDS, M&E, and nutrition in workshops and informal networking (e.g., exchange visits) with the point persons in these areas in other Title II Cooperating Sponsors. 3.b. HQ field visits to examples of best practice for direct distribution: Consider rescheduling the field visit to other Cooperating Sponsors’ examples of best practice in direct distribution that was scheduled for the final year of the grant (FY08) under IR 1.3 to FY07 in order to better backstop the design of new and existing projects that include direct distribution to people living with HIV/AIDS. ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 32 Recommendation #4--Monitoring and Evaluation: Strengthen capacity of HQ and field offices to understand ICB by reformulating some IR statements and indicators and adopting more standardized indicator methodologies (Table 7.1). 4.a. IR statements: Make the minor modifications proposed by the mid-term evaluation (indicated by brackets on the PITT—Annex I) to the project IRs in order to clarify and provide better focus on actual activities and create a new cross-cutting IR (1.4) that can be used to track the project’s DVD development and documentation activities. 4.b. PITT: Use a standard PITT form in all reports. 4.c. Monitor mid-term follow-up: Develop a structured system for monitoring follow-up on the recommendations from the ICB mid-term in a table format similar to that requested in the CSR2 guidance. 4.d Computerized database management: Supplement ongoing efforts to enter existing data into a centralized database with a thorough review of methods and guidance field programs use to collect data for indicators in a standardized manner (i.e., to enhance the potential for comparative analyses). 4.e. Staff capacity index: Pilot test a staff capacity index during the FY07 meeting that can be used to track staff needs and capacities, as well as improvements in capacities being targeted by the grant (as indicated in the original proposal). 4.f. Core indicators: Facilitate discussion (during the next annual meeting) of which cross-cutting/generic indicators (both USAID-endorsed and others) that ACDI/VOCA may wish to introduce into all its existing and new programs (taking into account FANTA’s August 2006 recommendations for “core” Title II indicators as well as the new USAID generic indicators). Recommendation #5—Documentation: Develop better systems for informing field and HQ staff about internal and external resources they can consult to respond to specific questions. 5.a. Create ICB bibliography: Develop and update regularly a project bibliography that lists ICB tools, curricula, and training DVDs and PowerPoint presentations, as well as where “hard” and “soft” (e.g., electronic) copies of these materials can be located. 5.b. Create “cheat sheets”: Create one page “cheat sheets” that cross reference common questions to materials listed in the bibliography described in recommendation 5.a. 5.c. CD download of key tools for internal distribution (to ACDI/VOCA HQ and field offices): Develop a CD download of the most critical tools, curricula, and “cheat sheets” that cross-references materials for local project staff and post the bibliography for field staff. 5.d. Post tools, bibliographies, and cheat sheets on the internal ACDI/VOCA website: Create a place on the existing FFD ACDI/VOCA webpage for posting the electronic files for download of ICB training modules and tools, bibliographies, and cheat sheets for internal (e.g., within ACDI/VOCA) distribution. 5.e. Once finalized, post electronic versions of tools on external website: Work with the ACDI/VOCA webmaster to post finished tools on the FFD webpage of the ACDI/VOCA ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 33 website and publicize this posting to CS partners. This page might include short two to three page summaries of each tool, the tool, and instructions for use. If it is difficult to develop a way of tracking who requested a particular document (as was done on the FAM website) the project might consider using the ICB coordinator as a “clearing house” for tool requests as is currently done at most of the other Title II Cooperating Sponsors. 5.f. Archive training materials acquired during external trainings: Develop a better system for archiving training manuals and reports that staff acquire during external trainings. Recommendation #6--Management and Financial Systems: Strengthen the project’s management systems by assigning clear staff responsibility for finalizing tools, soliciting more input into the design and feedback on the execution of ICB activities, and adjust the project budget accordingly. 6.a. Staff assignments: Task HQ and/or field staff with relevant technical training with the responsibility for final editing of the tools and ensure that their participation in this exercise is recognized in the text and on their annual evaluation. 6.b. Volunteer involvement in grant activities: Consider reassigning volunteers away from more long-term tasks, such as tools development, toward tasks such as training and assessment. 6.c. Budget: Adjust budget to reflect new priorities and use of consultants for certain activities to which volunteers are less well suited. 6.d. Cost-share: Pursue options for small capacity building grants from private donors (such as the Gates Foundation new initiative for African Agriculture) as a way of reducing pressure on using volunteers to satisfy the project cost-share. 6.e. Training in Title II M&E and reporting: Help accelerate the speed with which field staff can prepare useful reports by offering a one-day training session on Title II M&E and program reporting in conjunction with each of the annual food programming meetings. ACDI/VOCA ICB Mid-Term. Conclusions and Recommendations. December 22, 2006. 34 Table 7.1. ACDI/VOCA ICB Strategic Objectives and Intermediate Results and Mid-Term Recommendations for Reformulating IR Statements Original IR Statements in Project Proposal Mid-term Recommendations for IR Statements SO1: Capacity to address food insecurity through all possible uses of Title II resources improved IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased Same IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved Same IR 1.3: Institutional capacity in commodity management enhanced IR 1.3: Institutional capacity in basic commodity management guidelines and principles enhanced Un-numbered cross-cutting IR: Documentation IR 1.4: Institutional capacity for distance learning and documenting examples of best practice increased SO2: Technical excellence in food security programming enhanced through broadened partnership to build institutional capacity IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced. Same IR 2.2: Collaboration with FAM and its member PVOs to build technical excellence through information sharing and exchange. IR 2.2: Collaboration with other Title II Cooperating Sponsors to build technical excellence through information sharing and exchange. IR 2.3: Collaboration with academic and research institutions and international organizations strengthened. IR2.3: Technical excellence in gender addressed through broadened partnerships to build institutional capacity SO3: Ability to design food security proposal enhanced IR 3.1: Capacity to identify the root causes of food insecurity enhanced. IR 3.1: Capacity to identify the root causes of food insecurity and to integrate monitoring and evaluation into project design and execution enhanced.” IR 3.2: Nutrition impact of Title II programs enhanced Same IR 3.3: Integration of innovative HIV/AIDS programs strengthened. Same Unnumbered cross-cutting: Distance Learning Merged with documentation into IR 1.4 above SO4: Evidence base that support programmatic that improve impact IR 4.1: Documentation and integration of past performance successes improved Same IR 4.2: Institutional of Title II program data and analysis expanded Same Improved Management Improved Management Annex I. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 1 Proposed Revision of the ACDI/VOCA ICB Mid-Term Performance Indicator Tracking Table (PITT) based on Recommendations from the ICB Mid-Term Evaluation ([]=Mid-Term Recommendations for revising PITT based on indicator text from M&E Plan) Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved SO1: Capacity to address food insecurity through all possible uses of Title II resources improved Impact Indicator 1.1. # of [new and existing Title II] programs utilizing14 new [commodity] tools/modules/b est practices [developed under SO1] 0 2 0 0% 6 (rec. revising to 4)15 Impact Indicator 1.2. # of [Title II] food security proposals and reports integrating new commodity management indictors 0 1 n/a not yet completed n/a 1 (revise annual target) 4 (revise annual target) 5 Impact Indicator 1.3. [% of staff participating in internal trainings16 scoring 65% or higher on basic commodity management 0 60% 50% 100% (data from 1 training only) 60% -- 14 Defined as active use of at least one of the major new SO1 tools introduced under ICB [e.g. revision of the monetization manual in FY04 does not count). 15 The original target for this indicator in the proposal was three. It was revised upward in the M&E Plan when A/V anticipated adding more new projects. 16 Only trainings that included pre and post-training. Only one of the internal trainings included a pre and post training examination between FY04 and FY06. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 2 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved capacity/assess ment test ]17 Monitoring Indicator 1.1. # of [commodity mgt.] tools developed and/or updated18 0 1 119 100% 1 0.5 (half Ag. Rehab. Tool completed) 50% 3 2 draft + 0.5 (second half Ag. Rehab. Tool (completed) 83% 1 TBD TBD 0 5 TBD Monitoring Indicator 1.2. # of assessments conducted 0 1 1 (DRC) 100% 1 0 0% 3 2 (Mauritania and Burundi20 ) 66% 1 TBD TBD 1 TBD TBD 6 (rec. revising to 5) TBD Monitoring Indicator 1.3. # of commodity management indicators developed or modified 0 1 n/a (not yet dev.) n/a (not yet dev.) 1 n/a (not yet dev.) n/a (not yet dev.) 1 n/a (not yet dev.) n/a (not yet dev.) 4 (rec. revising target) -- -- 0 -- -- 4 -- Monitoring Indicator 1.4. # of internal and external trainings/worksh ops21 [on commodity management and monetization] supported by the ICB 0 1 5 500% 2 4 400% 3 3 100% 3 -- -- 3 -- -- 12 -- Monitoring Indicator 1.5. [NEW MID- TERM] # [A/V ICB bibliographies n/a -- -- -- n/a -- -- n/a -- -- 4 -- -- 4 -- -- 4 -- 17 Originally: # of staff [rec. adding: attending internal commodity training workshops] showing [adding or maintaining] a 25% improvement in capacity. 18 See Annex III. Table A. 19 Update and revision of Monetization Best Practices Manual 20 In collaboration with CRS, Land O’Lakes, and World Vision. 21 See Annex III, Table B. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 3 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved prepared and/or updated] 22 Monitoring Indicator 1.6. [MOVED FROM SO3]] # of training sessions on DVD [and PowerPoint] shared23 DVD/PP HQ24 DVD/PP Field & HQ Sessions 0 2 6 2 0 100% 0% 3 6 3 1 curricula training used gender 100% 0% n/a 4 3 PP 0 0% TBD 4 TBD 4 8DVD and PowerPoint 22 HQ& Field -- SO2: Technical excellence in food security programming enhanced through broadened partnerships to build institutional capacity Impact Indicator 2.1. # of new food security proposals and/or existing programs that incorporate new or improved gender integration tools [into programming reporting or new proposals/ extension]25 0 0 0 0 0 3 0 0% 0 -- -- 4 -- -- 7 (Rec Revision to 4) -- Monitoring Indicator 2.1. # of gender integration tools developed or improved26 0 3 2 66% 1 2 200% 0 -- -- 0 -- -- 0 -- -- 4 -- 22 Recommend adding at mid-term to facilitate achieving cross-cutting IR documentation. 23 Annex III, Table B. 24 Mid-term recommendation is to track DVD/PowerPoint training modules produced as well as training sessions. 25 Proposed rephrasing of indicator that was suggested at mid-term7 26 See Annex III, Table A and Annex II, Section IR. 2.3. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 4 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved Monitoring Indicator 2.2. # of trainings on gender integration and food security programs 27 0 2 2 100% 2 2 HQ [& 2 USDA] 100% 0 0 -- 0 -- -- 0 -- -- 4 -- Monitoring Indicator 2.3. [NEW MID- TERM] # organizational assessment tools developed 0 n/a 1 tools 1 HQ training 2 field tests 100% N/A -- -- N/A -- -- 1 1 field test -- N/A -- -- 1 Tool -- Monitoring Indicator 2.4. # of meetings/ workshops of [Title II] food security partners attended 0 9 13 144% 9 15 166% 9 15 166% 9 -- -- 9 -- -- 45 -- Monitoring Indicator 2.5. # of tools and/or best practices shared with [Title II CS] food security partners (across ICB program components) 2 1 128 100% 1 0 0% 0 informal exchange only 0% 0 -- -- 3 (minimum) -- -- 3 -- SO3: Ability to design food security proposals enhanced Impact Indicator 3.1. # of new or existing program interventions that integrate HIV/AIDS activities 0 1 2 200% - - - 2 - - 3 -- Impact Indicator 3.2. # 0 1 n/a (tool developed n/a 2 -- -- 3 -- 27 See Annex III, Table B. 28 Risk and liabilities sheet from the commodity management sheet shared with Land O’Lakes. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 5 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved of new or existing programs that incorporate new M&E methodology tools or methodologies developed under S03 (e.g., needs assessment tool) 29 in FY06) Monitoring Indicator 3.1. # of tools and/or best practice documents developed or updated [in key areas targeted by S03]30 0 1 0 0% 1 0 0% 1 1 100% 1 -- -- 1 -- -- 5 -- Monitoring Indicator 3.2. # of internal and external trainings/worksh ops attended [that support S03 31 -0 0 1 100% (target not set for year) 0 2 200% (target not set for year) 3 2 66% 2 -- -- 2 -- -- 7 (revise up to 9)32 -- Monitoring Indicator 3.3. # of assessments conducted [using needs assessment tools] 0 -- -- -- 1 n/a (not yet developed) -- 1 n/a (completed in this year) 2 (rec. revising to 0) -- -- 2 -- -- 3 (rec. revising to 2) -- SO4: Evidence base that supports programmatic approaches that improve impact [e.g., long-term impact assessment tool, data management system] Impact Indicator 4.1. [MID-TERM 0 0 -- -- 2 n/a -- 0 Field tests in CV and UG 1 -- -- 0 -- -- n/a recommend eliminating -- 29 For example, tools and methodologies being developed by the ICB. 30 For example, program design tools, program development process, nutrition and HIV/AIDS program integration documents (see M&E plan, data collection table, page 2). 31 Annex III, Table B 32 Recommendation to revise target upward is to take into account the need for more training on HIV/AIDS and Nutrition. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 6 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved RECOMMEND ELIMINATING THIS INDICATOR] # of new proposals that reflect analysis of long-term impact assessments suggest that assessment is not that useful indicator New Impact Indicator 4.1. [NEW MID- TERM] # of comparative analyses of trends in Title II programs based on data entered into the data base 33 0 n/a -- -- n/a -- -- n/a -- -- 1 -- -- 1 -- -- 2 -- Monitoring Indicator 4.1. # of knowledge management tools developed or revised 0 1 0 0% 0 1 100% 0 -- -- 0 -- -- 0 -- -- 1 -- Monitoring Indicator 4.2. # of long-term impact assessments [of new or existing Title II programs] conducted 0 0 -- -- 2 0 0% 0 1 UG 1 CV 100% 1RW34 + -- -- 0 -- -- 3 -- Program Management [MID-TERM RECOMMENDS ELIMINATING THIS INDICAOR]# 0 7 4 57% 6 5 71% 7 reports submitte d 7 on time 100% TBD - -- TBD -- -- TBD -- 33 Included in annual reports or best practice bulletins. Mid-term recommendation for impact indicator. 34 This volunteer led activity will include developing an extensive bibliography of the old project documents. ACDI/VOCA Mid-Term. Annex I: PITT. December 22, 2006. 7 Indicator Base- line FY04 Target FY04 Achieved FY04 % Achieved vs Target FY 05 Target FY05 Achieved FY05 % Achieved vs Target FY06 Target FY06 Achieved FY06 % Achieved vs Target FY07 Target F07 Achieved FY07 % Achieved vs Target FY08 Target FY08 Achieved FY08 % Achieved vs Target LOA Target LOA Achieved of reports submitted on time to DCHA/FFP Acronyms: PP=PowerPoint; HQ=Headquarters; TBD=To be Determined; FY=Fiscal Year. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 8 Annex II. ACDI/VOCA Institutional Capacity Building (ICB) Grant Project Bibliography and References Consulted during the Mid-Term Evaluation (By Strategic Objective and Intermediate Result35 for Completed Materials and Those Projected for Completion by the end of FY07) Piero Gonzalez, ICB Coordinator The ACDI/VOCA Title II Institutional Capacity Building Grant (ICB) (FY04-FY08) was designed to achieve three strategic objectives (SOs) and 11 cross-cutting Intermediate Results (IRs). This bibliography is divided into three sections. • Section I lists the tools, training modules, and assessments that have been drafted or finalized as part of the ICB. For tools that have been through successive drafts, each draft is listed in order to ensure a correct reference. • Section II lists the routine administrative and financial reports of the ICB project—both internal and external—that were reviewed as part of the mid-term evaluation. • Section III describes the official proposals and administrative reports (e.g., annual reports, proposals, and annual CSR2) for the three major ACDI/VOCA Title II projects whose design and execution overlapped with the capacity building efforts supported by the ICB grant.36 Tools and training modules that the project anticipates completing in FY07 are indicated with an asterisk (*). I. ICB TOOLS, TRAINING MATERIALS (EXISTING AND PROJECTED), AND ASSESSMENTS A. SO1: Capacity to address food insecurity through all possible uses of Title II resources improved A.1. IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased ACDI/VOCA. 2003. Version 1. 2004. Version 2. Monetization Best Practices Manual. Washington, DC: FFD Unit, ACDI/VOCA. December 1, 2003. Miller, Lisa. 2006. What You Need to Know and Ask to Make a Bellmon Determination and Conduct a Market Analysis. Trip report, ICB Capacity Building Exercise in 35 Indicator statements include mid-term recommendations for revision (see Table 7.1). 36 Although this literature is not a product of the ICB it is related since many of the ICB indicators were designed to build capacity and/or strengthen the impact of new and existing Title II funded food security programs. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 9 connection with the ACDI/VOCA Mauritania Market Assessment. Washington, DC: ACDI/VOCA. November 20, 2006. Miller, Lisa. 2006. Guide to Opening Letters of Credit (Draft Version). Trip Report, ICB Capacity Building Exercise in connection with the ACDI/VOCA Mauritania Market Assessment. Washington, DC: ACDI/VOCA. Gibb, Thomas. 2006. Monetization Training Modules. Washington, DC: ACDI/VOCA. (Includes: Pre and Post-test Form, Monetization Work Book; and PowerPoint Training materials for training.) A.2. IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved A.2.1. Market intervention capacity (in areas with extremely high staple food prices) A.2.1.1. Tools and training * Gibbs, Thomas and Barry Elkin. (in press). Monetization Best Practices/Models/Market Access/Indicator Tracking Tool (title of final tool might be adjusted). -First Draft: Anticipated January 2007 (to discuss and revise in annual meeting) -Revised Draft (for sharing with other cooperating sponsors): Anticipated September 2007. Cekan, Jindra. 2005. Ration Design Workshop. ACDI/VOCA FFD Annual Retreat. June 15, 2005 (1/4 day) *ACDI/VOCA (A/V Volunteer Activity). (in press). Monetization field Assessments: Rwanda and Uganda. Washington, DC. *ACDI/VOCA. (in press)). Direct Distribution Training PowerPoint Module (planned for after the FY07 annual FFD A/V retreat using an experienced volunteer): -One at HQ (1 day; approximately 10 people) -One at the retreat (1/4 day; approximately 40 people). A.2.1.2. Assessments Murphy, Emmet. 2004. Emergency Market Assessment: Eastern DRC. Washington, DC: ACDI/VOCA. Murphy, Emmet. 2005. Emergency Market Assessment: Burundi. Washington, DC: ACDI/VOCA * Symbol indicates materials that will be completed in FY07. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 10 Ose, Iven and Lisa Miller. 2006. ICB Institutional Capacity Building Exercise for Monetization: Bellmon Analysis Mauritania. Washington, DC: ACDI/VOCA. (Version: May 15, 2006). A.2.2. Agricultural rehabilitation capacity Sabwa, Nicholas. 2006 First Draft. 2007 Anticipated Final Draft. Agricultural Rehabilitation Tool. Washington, DC: ACDI/VOCA. *ACDI/VOCA. (in press). Agricultural Rehabilitation Workshop PowerPoint Presentation. (To be developed in conjunction with the FY07 training program planned for Cape Verde). Washington, DC. A.3. IR 1.3: Institutional capacity in basic commodity management guidelines and principles enhanced Gerstle, Joe. 2004. Commodity Management Training. Washington, DC: ACDI/VOCA. September 7, 2004. Gerstle, Joe. 2006. Food Aid Management Training. Washington, DC: American Red Cross, January 31 – February 2, 2006 (3 days -3 HQ staff attended). APVOFM. 2005. USAID Rules and Regulations Training. Nairobi, Kenya. April 25-27, 2005 (1 local staff member participated with ICB support).37 A.4. IR 1.4: Institutional capacity for distance learning and documenting examples of best practice increased Kula, Olaf and Banu Akin. 2006. Training Module on Value Chain Analysis. (Edited by: Peter Leninger. (2 hour tool with videotape chapters and accompanying PowerPoint slides). May 12-13, 2006. Washington, DC: ACDI/VOCA. Meyer, Maggie. 2006. Training Module on Regulation 216 Environmental Impact Assessment (videotape). Washington, DC: ACDI/VOCA. Johnson-Welch, Charlotte Leonhardt, and Tom Leonhardt. 2004. How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation (2 edited videotapes). Washingon, DC: ACDI/VOCA. 37 Several national staff have participated in commodity training with support from their projects (see Annex III, Table III.A. of this report). This indicates training that was supported under the ICB. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 11 OMB (Office of Management and the Budget) 2005. Circular A-122, Cost Principles, and Cost Share Training (videotape). (August 24, 2005). Washington, DC: ACDI/VOCA. Miller, Lisa and Bernie Runnebaum. 2006. How to conduct a Bellmon Analysis? Washington: ACDI/VOCA. (Draft PowerPoint presentation not yet edited based on training conducted at October 2006 annual retreat.) Gibb, Thomas. 2006. Monetization Training Modules (PowerPoint training materials). Washington, DC: ACDI/VOCA. B. SO2: Technical excellence in food security programming enhanced through broadened partnership to build institutional capacity B.1. IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced Pons, Ed. 2004. Organizational Assessment Tool. Version 1.0. Washington, DC: ACDI/VOCA. (Field tested in Rwanda.) Keatts, Adam. 2006. Organization Assessment Tool. Version 2.0. Washington, DC: ACDI/VOCA. (To be field tested in Cape Verde in FY07). B.2. IR 2.2. Collaboration with other Title II Cooperating Sponsors to build technical excellence through information sharing and exchange Cederstrom, Thoric, David Evans, and Keith Wright. 2006. Assessment of Future PVO Willingness to Collaborate on Food Security Programming. (SOW and report). Washington, DC: Counterpart International and Food for the Hungry International. Collaborative Forum on Food Security and/or Food Aid Issues. 2006. Food Security Assessment Workshop. September 19-21, 2006. Program and Abstracts.38 B.3. IR 2.3: Technical excellence in gender addressed through broadened partnerships to build institutional capacity ICRW. 2004. Review of ACDI/VOCA M&E System. Washington, DC: ICRW (International Center for Research on Women) for ACDI VOCA. (Internal report not for circulation.) Johnson-Welch, Charlotte and Tom Leonhardt. 2004. Training Workshop: How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation. A Training Workshop for: ACDI/VOCA Food for Development Staff. August 11-12, 2004. 38 Same as preceding footnote. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 12 Washington, DC: ARC. (Documentation that came out of the training is a deliverable.) (Training was taped.) ICRW and ACDI/VOCA. 2005. Gender Analysis Tool. Washington, DC: ACDI/VOCA. ICRW and ACDI/VOCA. 2005. Curriculum for Conducting a Gender Analysis Exercise in the Field. Washington, DC: ACDI/VOCA. C. SO3: Ability to design food security proposals enhanced C.1. IR 3.1: Capacity to identify the root causes of food insecurity and to integrate monitoring and evaluation into project design and execution enhanced C.1.1. Tools ACDI/VOCA. 2006. Food Security Needs Assessment Toolkit. Prepared for ACDI/VOCA by Tango International, Inc., Tucson, Arizona. (January 2006.) ACDI/VOCA. 2006. Conceptual Approach and Background Information for Food Security Assessment. Prepared for ACDI/VOCA by Tango International, Inc., Tucson, Arizona. APVOFM (Association for Private Voluntary Organization Financial Managers). 2005. Training for Trainers Workshop for the Purpose of Building the Facilitation Skills of Finance Staff to Deliver Relevant and Effective Financial Management Training to NGO Staff. Washington, DC: APVOFM. ( July 18-21, 2006). C.1.2. Assessments HQ level capacity planned for FY06, but not enough lead time on priority countries. Planned in connection with 2007 designs—Mozambique and Haiti. C.2. IR 3.2: Nutrition impact of Title II programs enhanced ACDI/VOCA. N.D. Health and Nutrition Resources. Binder #1(informal internal resource document). C.3. IR 3.3: Integration of innovative HIV/AIDS programs strengthened ACDI/VOCA. N.D. HIV/AIDS and Food Security Resources. Binders #1(informal internal resource document). ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 13 World Vision International, FANTA, and the World Food Programme. 2006. Food Assistance Programming in the Context of HIV and AIDS. Handbook Review Workshop. December 11-12, 2006. Washington, DC. Draft agenda. (Program and program documents on file; ACDI/VOCA staff participated and use materials for technical backstopping.) Bwire, Fred. 2006. Trip report: 2006 Africa Forum on HIV/AIDS and Food Security in Lusaka, Zambia, May 8-12, 2006. Kampala: ACDI/VOCA. D. SO4: Evidence base that supports programmatic approaches that improve impact (e.g., long-term impact assessment tool, data management system) D.1. IR 4.1: Documentation and integration of past performance successes improved Edings, David Dalziel, and Elizabeth Dalziel. 2005. Long-Term Impact Assessment— Guidelines. Washington, DC: ACDI/VOCA. Draft (July 5, 2005). Hahn-Grelling, Susan. 2005. Trip Report. Long-term Impact Assessment: Cape Verde. Washington, DC: ACDI/VOCA. (September 2005). *Russell, John. 2007. Trip Report. Long-term Impact Assessment: Uganda. Washington, DC: ACDI/VOCA. D.2. IR 4.2: Institutionalization of Title II program data and analysis expanded ACDI/VOCA. 2006. Draft of computerized data management system. Washington, DC. (First Draft being pilot tested.). Washington, DC: ACDI/VOCA. E. Improved Management Gonzalez, Piero. 2006. The ACDI/VOCA ICB Grant—what it can and could mean to you! Washington, DC: ACDI/VOCA. Berkey, Suzanne. 2005. Presentation: The ACDI/VOCA ICB Grant. Kenya Retreat for Africa/Middle East field staff. Washington, DC: ACDI/VOCA. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 14 II. PROPOSALS AND ROUTINE ADMINISTRATIVE REPORTS A. Proposal and contracts ACDI/VOCA. 2003. ACDI ICB Proposal. Washington, DC: ACDI/VOCA. B. Monitoring and evaluation ACDI/VOCA. 2003. ICB Monitoring and Evaluation Plan. Washington, DC: ACDI/VOCA. December 31, 2003. C. Internal assessments and audits C.1. Internal quarterly assessment (electronic) FY04: four per year FY05: four per year FY06: four per year C.2. Internal mid-term review (electronic) Polidoro, Bill. 2006. Memo to Note Users concerning Internal Project Reviews. Washington, DC: ACDI/VOCA. Phippard, Jim, Robin Caveney, and Piero Gonzalez. 2006. Internal Mid-term Project Review: ACDI/VOCA Title II ICB Grant. Washington, DC: ACDI/VOCA. Electronic report (notes). October 12, 2006. C.3. ACDI/VOCA Food for Development Annual Meeting ACDI/VOCA. 2004. ACDI/VOCA Food for Development. Annual Meeting. Kigali, Rwanda (including special sessions with trainers whose participation was financed by ICB). June 21-25, 2004. ACDI/VOCA. 2005. ACDI/VOCA Food for Development. Annual Meeting. Leesburg, Virginia. June 13-18, 2005. ACDI/VOCA. 2006. ACDI/VOCA Food for Development Day. Annual Meeting. Washington, DC. October 2, 2006. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 15 C.4. Annual performance reports and implementation plans C.4.1. Performance reports ACDI/VOCA. 2004. FY2004 Annual Performance Report. Washington, DC. ACDI/VOCA. 2005. FY2004 Annual Performance Report. Washington, DC. C.4.2. Implementation plans ACDI/VOCA. 2003. FY2004 Implement Plan. Washington, DC. ACDI/VOCA. 2004. FY2005 Implementation Plan. Washington, DC. ACDI/VOCA. 2005. FY2006 Implementation Plan. Washington, DC. C.5. Mid-term and final evaluations (ISA) (FY99-FY03) ACDI/VOCA. 2001. Mid-term Review: PL480 Title II ISA Program. Washington, DC. June. ACDI/VOCA. 2002. Final Evaluation: PL480 Title II ISA Program. Washington, DC. December 31, 2002. ACDI/VOCA ICB Mid-Term. Annex II: Bibliography. December 22, 2006 16 III. OTHER TITLE II FOOD SECURITY PROGRAMS A. Cape Verde ACDI/VOCA. 2000. Cape Verde-ACDI/VOCA, FY01-FY06 DAP Proposal. Washington, DC. ACDI/VOCA. 2003. Mid-Term Evaluation: Cape Verde PL480 Title II Food Security Program. Washington, DC. Langworthy, Mark, Antonio Sabino, and Jao Cardosa. 2005. Final Evaluation of the PL480 Title II Monetization Program. Washington, DC: ACDI/VOCA. December 2005. B. Rwanda ACDI/VOCA. n.d. ACDI/VOCA Development Assistance Proposal—Rwanda. Gaudreau, Martha M. 2003. Mid-term Evaluation: Rwanda PL480 Title II Program. Washington, DC: ACDI/VOCA. Swanson, Richard. 2004. Final Evaluation: ACDI/VOCA Rwanda PL 480, Title II Monetization Development Program (2000-2005). Washington, DC: ACDI/VOCA. Swanson, Richard. 2006. Title II Development Activity Program (DAP): Revised ACDI/VOCA-Rwanda M&E Plan. Washington, DC: ACDI/VOCA. C. Uganda ACDI/VOCA. 2001. Uganda-ACDI/VOCA, FY02-FY06 DAP. Washington, DC. Riley, Barry and Carl Harris. 2004. ACDI/VOCA. Uganda. 2002-2006 Title II DAP. Mid-term Evaluation. Washington, DC: ACDI/VOCA. 9/20/04 draft. Pierce, Robert Gardner, and Alison Gardner. 2006. ACDI/VOCA, Uganda: FY2002-2006 Title II DAP Final Evaluation Report. Washington, DC: ACDI/VOCA. April 2006. Annex III. ICB Facilitated Trainings and Tools ACDI/VOCA ICB Mid-Term. Annex III.A. December 22, 2006 17 Annex III.A. Tools Produced under the ACDI/VOCA ICB Grant as Reported in the Revised ACDI/VOCA ICB Performance Indicator Tracking Table (PITT) (FY04-FY06)39 (* indicates materials that are scheduled for completion in FY07) Tools Produced or Anticipated Monitoring Indicator 1.1. #of commodity management tools developed and/or updated Monitoring Indicator 2.1. # of gender integration tools developed or improved Monitoring Indicator 2.3. # of local capacity tools developed, trainings organized and assessments conducted Monitoring Indicator 3.2. #of tools and/or best practice documents developed or updated in key areas targeted under SO3 Monitoring Indicator 4.1. # of knowledge management tools developed or revised S01: Capacity to address food insecurity through all possible uses of Title II resources improved Monetization Training Modules (T. Gibb) (FY06) which includes: a) Pre and post test form b) Monetization work book c) PowerPoint training materials 1 Emergency Market Intervention Tools. (FY06) (being developed in conjunction with emergency market intervention assessments) a) What You Need to Know and Ask to Make a Bellmon Determination and Conduct a Market Analysis (internal document) b) Guide to Opening Letters of Credit (internal document) Not reported as an achievement because reports are inputs into the tool which is not yet produced *Monetization Best Practices/Models/Market Scheduled for completion in FY07 39 Indicator statements include mid-term recommendations for revision (see Table 7.1). ACDI/VOCA ICB Mid-Term. Annex III.A. December 22, 2006 18 Tools Produced or Anticipated Monitoring Indicator 1.1. #of commodity management tools developed and/or updated Monitoring Indicator 2.1. # of gender integration tools developed or improved Monitoring Indicator 2.3. # of local capacity tools developed, trainings organized and assessments conducted Monitoring Indicator 3.2. #of tools and/or best practice documents developed or updated in key areas targeted under SO3 Monitoring Indicator 4.1. # of knowledge management tools developed or revised Access/Indicator Tracking Tool (T. Gibb) (draft report anticipated January 07) *Agricultural Rehabilitation Tool (first draft N. Sabwa; under revision in FY07) 1 Training Module on Value Chain Analysis (2 hr tool w/video chapters and accompanying PowerPoint slides) 1 Training module on Reg. 216 (tape) 1 Gender Modules (2) 2 OMB Circular A-122 (tape) 1 Monetization PowerPoint Training Modules (accompany the monetization workbook) 2 How to Conduct a Bellmon Analysis? (draft PowerPoint presented at retreat) 1 SO2: Technical excellence in food security programming enhanced through broadened partnership to build institutional capacity *Organizational Assessment Tool. Version 1. (Pons 2004) 1 *Organizational Assessment Tool. Version 2. (Keatts 2006) 1 Training Workshop Proceedings: How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation (Johnson-Welch and Leonhardt 2004) 1 ACDI/VOCA ICB Mid-Term. Annex III.A. December 22, 2006 19 Tools Produced or Anticipated Monitoring Indicator 1.1. #of commodity management tools developed and/or updated Monitoring Indicator 2.1. # of gender integration tools developed or improved Monitoring Indicator 2.3. # of local capacity tools developed, trainings organized and assessments conducted Monitoring Indicator 3.2. #of tools and/or best practice documents developed or updated in key areas targeted under SO3 Monitoring Indicator 4.1. # of knowledge management tools developed or revised Gender Analysis Tool (ICRW and ACDI/VOCA 2005) 1 Curriculum for Conducting a Gender Analysis Exercise in the Field (ICRW and ACDI/VOCA 2005) 1 SO3: Ability to design food security proposals enhanced *Food Security Needs Assessment Toolkit (TANGO 2006) 1 HQ Health and Nutrition Resources Binder #1. Internal document (not official internal or external tool) HQ HIV/AIDS and Food Security Resources. Binder #2. Internal Document (not official internal or external tool) SO4: Evidence base that supports programmatic approaches that improve impact *Long-Term Impact Assessment: Guidelines Completed in FY05 but not counted under Monitoring Indicator 4.1 *Computerized Data Management System (for Title II programs) 1 Annex III.B. Internal and External ICB-Facilitated Trainings under the ACDI/VOCA ICB Grant as Reported in the Performance Indicator Tracking Table (PITT) (FY04-FY06) (* indicates that figure is an estimate) (Ret: Retreat; P=Presentation; BO=Break-out session at Annual ACDI/VOCA Retreat)40 ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 20 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E SO1: Capacity to address food insecurity through all possible uses of Title II resources improved IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased FY04 Ret Monetization Training (Trainer: Gibbs) I 1 FY06 Monetization 101 (Trainer: Gibbs) I 1 FY04 Ret Monetization and Private Sector Capacity Building (BO) I 1 FY06 Ret How to Conduct a Bellmon or Market Analysis (BO) I 1 IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved FY04 Ret Direct Distribution Strategic Planning Session (P) I 1 FY05 Ret Donor Panel: Presentation on New Dev-Relief Paradigm (P) I 1 FY06 On the job training in an emergency market assessment in Mauritania I 1 40 Indicator statements include mid-term recommendations for revision (see Table 7.1). ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 21 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E (Lisa Miller) IR 1.3: Institutional capacity in commodity management enhanced FY04 Commodity Mgt Training (FY04) (Trainer: Gerstle) I 1 FY04 Ret Ration Design (Trainer: Cekan) I 1 FY05 APVOFM. USAID Rules and Regulations Training E 1 FY05Ret Reg. 216 Environmental Rules and Regulations Training I 1 FY05 Ret OMB Circular A￾122 and Cost Sharing Training I 1 FY06 Food Aid Management Training (T. Gibb) I 1 IR 1.4: 41 Institutional capacity for distance learning and documenting examples of best practice increased (See Annex II) n/a Value Chain Module (DVD) n/a 1 n/a Reg.216 (Environmental Module) (DVD) n/a 1 n/a Gender Modules n/a 1 41 Numbers should refer to the number of staff who have been trained using the existing DVD modules. ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 22 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E (2) (DVD) n/a OMB Circular A￾122: Cost Principles and Cost Share Training (DVD) n/a 1 n/a Monetization modules (accompany monetization work book) (PowerPoint) n/a 2 n/a How to Conduct a Bellmon Analysis (Trainers: Miller and Runnebaum) (PowerPoint) n/a 1 SO2: Technical excellence in food security programming enhanced through broadened partnerships to build institutional capacity IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced FY04 RET Organizational Assessment Training (Trainer: E. Pons) 1 1 IR 2.2: Collaboration with other Title II Cooperating Sponsors to build technical excellence through information sharing and exchanges FY04 only FAM ACDI/VOCA participated actively in FY04 before FAM dissolved FY04- FY06 FAM: M&E/EIG Working Group meeting ACDI/VOCA participated in FY04 but did not ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 23 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E participate in EIG working group in FY06 FY04- FY06 FACG: Food Aid Consultative Group (FACG) Commodity Working Group (CWG) Disaggregated data from FY04 and FY05 not available 4 (FY06 only) FY04- FY06 FACG: General Issues Working Group (GIWG) Disaggregated data from FY04-05 not available 7 (FY06 only) FY04- FY06 CFA (Coalition for Food Aid): general meetings Disaggregated data from FY04 and FY05 not available 1 (FY06 only) FY04- FY06 CFA (Coalition for Food AID): Working Group Rewriting Reg. 11 Disaggregated data from FY04 and FY05 not available 0 (FY06 only) FY04- FY06 Title II Cooperation Sponsors (CS) (Group meetings) Disaggregated data from FY04 and FY05 not available 2 (FY06 only) FY06 only Collaborative Forum on Food Security and/or Food Aid Issues (organizational meetings only— trainin g meetin g in 1 organizational meeting (06 only) ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 24 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E FY06 counts under needs assessment) IR 2.3: Technical excellence in gender addressed through broadened partnerships to build institutional capacity FY04 Gender Training on M&E ICRW (April 04-1 day) I 1 FY04 How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation (August 04) I 1 FY05 Gender Analysis and Awareness Training of Trainers Session (Trainers: Dunne and Gonzalez) (May 05) I 1 FY05 Plenary session at the 05 meeting (1/2 day) (Trainer: Leonhart and MacQuartie) I 1 SO3: Ability to design food security proposals enhanced IR 3.1: Capacity to identify the root causes of food insecurity and integrate monitoring and evaluation into project design and execution enhanced FY04 Measuring Food Insecurity Workshop (April 15-16,2004) E 1 FY05 Value Chain I 1 ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 25 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E Analysis Training (May 12-13, 2005) FY05 Ret M&E (Trainer: M. Seroussi) I 1 FY06 Food Security Assessment Workshop (Sept 19-21, 2006) E 1 IR 3.2: Nutrition impact of Title II programs enhanced No trainings or retreat presentations IR 3.3: Integration of innovative HIV/AIDS programs strengthened FY04 HIV/AIDS Current Programs and Strategy (for food security programming) I 1 FY05 HIV/AIDS Plenary (Speaker: Canhuati) I 1 FY06 Lusaka Conference on the dual epidemic of HIV/AIDS and Food Insecurity (May 2006 (See Annex IV.B) 1 (national staff member from Title II Uganda) SO4: Evidence base that supports programmatic approaches that improve impact IR 4.1: Documentation and integration of past performance successes improved No trainings or retreat ACDI/VOCA ICB Mid-Term. Annex III.B. December 22, 2006 26 Monitoring Indicator 3.2. SO/ IR/ FY Training Theme Internal (I) External (E) Monitoring Indicator 1.4. Internal and External Trainings on Comm. Mgt & Monetization Monitoring Indicator 1.6. DVD and PowerPoint Tools Monitoring Indicator 2.2. Gender Integration Tools Monitoring Indicator 2.3. Local Capacity Building Tools, Trainings, Assess Monitoring Indicator 2.4. Meetings/Workshops with Title II Food Security Partners (FASIG, FANTA, Consultative Group, etc.) HIV/ AIDS Nutrition New Needs Assessment Tool, M&E presentations IR 4.2: Institutionalization of Title II program data and analysis expanded Training in data entry and analysis using new data base (formal and/or informal one on one) Improved Management FY04 Ret ICB (P) I FY04 Ret ICB (P) I FY05 Ret Training in code and Federal regulations and grants management training (2.5 days) I FY06 Ret FFD Contracts and Grants I ACDI/VOCA ICB Mid-Term. Annex III.C. December 22, 2006 27 Annex III.C. ACDI/VOCA ICB Monetization 101 Training Pre and Post-Test 1. What is monetization? a. The sale of US commodities to the USG by US farmers b. The donation of US commodities to developing countries c. The generation of funds for use in developing countries through the sale of US commodities d. The shipment of US commodities to developing countries 2. What is one reason behind a PVOs choice to monetize instead of obtaining cash for programs? a. Monetizing infuses needed food in local economies b. Monetizing is simpler than obtaining cash programs c. US farmers lobby PVOs to choose the monetization route d. None of the above 3. What USG agency manages monetization programs? a. USDA b. Commerce c. USAID d. Both a and c 4. Which food aid program enables PVOs to access only surplus US commodities? a. Food for Progress b. Global Food for Education c. 416(b) d. Title II USDA Programs and USAID programs can exist within the same country. 5. Which program allows PVOs to have a separate cash account to pay for US-based administrative expenses? a. Title II b. 416(b) c. Food for Progress d. Both a and b ACDI/VOCA ICB Mid-Term. Annex III.C. December 22, 2006 28 6. Which of the following regulations apply to ALL food aid programs? a. OMB A-122 b. 22 CFR 226 c. 7 CFR 1499 d. 7 CFR 3019 7. What does a Bellmon Analysis NOT cover? a. The quality of in-country storage facilities b. An analysis of the risk for discouraging local production c. The local partnering options d. All of the above 8. What are the usual marketing requirements? a. Rules on how to promote US food commodities in-country b. Guidelines on how PVOs should market their development programs c. A determination of the ability for the market to handle the importation of a commodity beyond what is being sold commercially d. None of the above 9. What issues would NOT be considered in a market assessment? a. The level of local production of a commodity b. The level of importation of a commodity c. The local diet d. The storage facilities at the port 10. What is a call forward? a. A document that tells the host government about a food aid program b. A communication from the USG to the PVO that has won a program c. A request for commodity to be shipped out under a PVO’s program d. A movement of commodity from ship to shore 11. USDA requires a formal call forward process. a. True b. False 12. Under development programs, once a commodity is requested for delivery it is generally in the host country within two weeks. ACDI/VOCA ICB Mid-Term. Annex III.C. December 22, 2006 29 a. True b. False 13. Freight forwarders help the PVO with the following (choose one): a. Selecting the commodity b. Designing the development program c. Negotiating with the donor d. Arranging for shipment of the commodity 14. What is a bill of lading? a. An invoice from the seller to the buyer of the commodity b. A document that shows how much commodity has been loaded onto the vessel at the US port c. A document that shows how much commodity was off-loaded from the vessel at the port in the host country d. A sales agreement 15. Commodity losses are a problem with which most PVOs must deal. a. True b. False 16. Which of the following is not considered to be a negotiated sale? a. A closed bid small lot sale b. An outcry auction c. None of the above d. All of the above 17. What is an umbrella monetization? a. A monetization that occurs in the rainy season b. A monetization that is managed for multiple organizations by a lead organization c. A sealed bid sale d. An outcry auction 18. What is the key difference between a negotiated sale and an outcry auction? a. The size of the lot that is sold to the buyer b. The location of the sale c. The country in which the sale occurs d. The entity selling the commodity ACDI/VOCA ICB Mid-Term. Annex III.C. December 22, 2006 30 19. The USG often encourages the sale of US commodities in one country with the proceeds being used in another. a. True b. False 20. Monetization can be used as a development tool. a. True b. False Annex IV. Additional Tables ACDI/VOCA ICB Mid-Term. Annex IV.A. December 22, 2006. 31 Annex IV.A. Food Security Programming Experience of Key Senior ACDI/VOCA Staff in the Title II Field Programs and Headquarters Office HQ vs. Field Office Years with ACDI/VOCA Years Experience with Title II Programming Years Experience with other types of food programming (non Title II) Years experience with Development (including time with ACDI/VOCA) Past or projected role on the ICB HQ FFD Office Avram “Buzz” Guroff, Senior Vice President of Porfolio 9 9 9 40 + Oversight and coordination with USAID/FFP and other Title II Cooperating Sponsors Piero Gonzalez, ICB Coordinator 2 2 5 5 ICB Coordinator; HIV and Gender Point Person; HQ backstop for West Bank/Gaza Suzanne Berkey, Director, FFD Division 8.5 8.5 (including 3 years in Cape Verde) 8.5 8.5 years (including 3 years in Cape Verde) Technical oversight and supervision Olutayo Akingbe, Project Assistant 1 1 1 1 Works closely with USDA and Food for Progress cocoa programs in Ecuador, Vietnam, Philippines (no direct involvement ICB other than trainings) Elizabeth Dalziel, Project Assistant 3.5 1.5 3.5 3.5 Senior project coordinator cocoa programs Vietnam, Ecuador, Philippines (no direct involvement ICB other than trainings) Lisa Miller, Project 1.5 0.5 1.5 4 Backstops Philippines ACDI/VOCA ICB Mid-Term. Annex IV.A. December 22, 2006. 32 HQ vs. Field Office Years with ACDI/VOCA Years Experience with Title II Programming Years Experience with other types of food programming (non Title II) Years experience with Development (including time with ACDI/VOCA) Past or projected role on the ICB Coordinator and Rwanda; participated in ICB emergency market assessment in Mauritania Cape Verde FIELD TEAM Chief of Party/Program Manager: John (Bic) Riley 2.5 5.25 13.25 18.4 Deputy Chief of Party/M&E Specialist : Michel (Michka) Seroussi 2 4 10 15 SWC Program Manager, David Brito 1.75 7.75 10 18 Administrative/Commodity Monetization Assistant, Helena Pina 14 14 7 21 HQ Backup Team Elizabeth Adams, Project Coordinator 1.33 1.33 0.66 (USDA) 3.33 Training on and off site and some supervision; Title II M&E Point Person Ashley Mullinax, Project Assistant 1.33 1.33 1.33 (USDA) 1.33 (all at ACDI/VOCA) Training on and off site and some supervision Rwanda Field Office COP/Country Representative: Paul DeLucco 6 6 0 25 None Deputy Chief of Party/Monetization Manger: Barry Elkin 6 8 0 13 Assistant Monetization 4.5 4.5 0 4.5 None ACDI/VOCA ICB Mid-Term. Annex IV.A. December 22, 2006. 33 HQ vs. Field Office Years with ACDI/VOCA Years Experience with Title II Programming Years Experience with other types of food programming (non Title II) Years experience with Development (including time with ACDI/VOCA) Past or projected role on the ICB Manager: Chantal Umuliysa HQ Backup Team Katie Ceretani, Project Coordinator 2.5 2.5 2.5 2.5 Training on and off site and some supervision; Title II Nutrition Point Person Ashleigh Mullinax, Project Assistant See above Training on and off site Uganda Field Team COP: Bernie Runnebaum 15+ 15+ 0 25+ Deputy/COP/M&E Officer: Sandy Blanchard 12 (On and Off) 12 0 20 Monetization and Program Officer: Kim Kippels 2 2 0 2 Senior Finance Manager: Boniface Lekdyang Ogwang 9 9 0 9 Senor GDU Compliance Officer: Robert Komakech 9 9 0 9 Monetization and sales manager: George Bamugye 15 15 0 30 HQ Backup Team Katie Cerretani, Project Coordinator See above Ashleigh Mullinax, Project Assistant See above Other Thomas Gibb, Consultant who is rejoining ACDI/VOCA in Uganda 8 8 8 (Including field experience in the ACDI/VOCA HQ office and on TBD Consultant to the FFD unit (possibly returning ACDI/VOCA line employee 12 /1) ACDI/VOCA ICB Mid-Term. Annex IV.A. December 22, 2006. 34 HQ vs. Field Office Years with ACDI/VOCA Years Experience with Title II Programming Years Experience with other types of food programming (non Title II) Years experience with Development (including time with ACDI/VOCA) Past or projected role on the ICB ACDI/VOCA field projects in Rwanda and Cape Verde) Source: P. Gonzalez, December 2006 based on interviews for the ICB Mid-term Evaluation. Annex IV.B. Project Level Training in ACDI/VOCA’s Currently Funded Title II Projects in Cape Verde, Uganda, and Rwanda in the Key Areas Targeted by the ICB Grant (FY06 Only) (Note: IR statements include mid-term recommendations for revision, see Table 7.1 Section 7.0 of this report) ACDI/VOCA ICB Mid-Term. Annex IV.B. December 22, 2006. 35 Funding Cape Verde Uganda Rwanda Training Theme, Date, and Duration by SO and IR42 Non- ICB43 ICB Facilitated or Partially Facilitated National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff SO1: Capacity to address food insecurity through all possible uses of Title II resources improved IR 1.1: Ability to integrate and measure the food security impact of monetization programs increased No recorded field level trainings IR 1.2: Institutional capacity to implement developmental relief programs with Title II resource improved Environmental Assessment and Environmentally sound design (3/06) X 1 1 Drip Irrigation Training X 6 Fruits and products processing X 1 Value chain analysis/cooperative principals X 5 1 Workshop (drip irrigation and horticulture) X 1 IR 1.3: Institutional capacity in basic commodity management guidelines and principles enhanced USDA-USAID Export Food Aid Conference (3/06) X 1 Financial rules and re gulations (Nairobi) X 2 42 Indicator statements include mid-term recommendations for revision (see Table 7.1). 43 Funded through a Title II project or other source. ACDI/VOCA ICB Mid-Term. Annex IV.B. December 22, 2006. 36 Funding Cape Verde Uganda Rwanda Training Theme, Date, and Duration by SO and IR42 Non- ICB43 ICB Facilitated or Partially Facilitated National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff 3/06 Food Security Monitoring in Uganda (FAO)—one year monthly meetings X 1 monthly 1 monthly Annual Crop Outlook and Pricing (Nairobi Kenya) 11/05 X 1 Food Aid Regulations (Lusaka Zambia) 6/06 X 1 USAID/NGO Business Writing Course (10/06) (Kampala Uganda) X 9 1 Food Aid Manger’s course (Dar es Salaam) (7/06) X 1 1 Finance training: Regulation and accounting X 1 1 Training for the quarterly web- interface commodity reporting (QWICR) Tool, April 19-21, 2006 1 1 Commodity management team visit to port of 1 ACDI/VOCA ICB Mid-Term. Annex IV.B. December 22, 2006. 37 Funding Cape Verde Uganda Rwanda Training Theme, Date, and Duration by SO and IR42 Non- ICB43 ICB Facilitated or Partially Facilitated National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff Mombasa (May 31- June 3, 2006) FFP Food Aid Manager Course (Dakar Senegal) X 2 Monetization Training (sponsored by CRS), August 11, 2006 1 IR 1.4: (MID-TERM NEW) Institutional capacity for distance learning and documenting examples of best practice increased No recorded field level trainings SO2: Technical Excellence in food security programming increased through broadened partnerships to build institutional capacity IR2.1: Capacity to strengthen local partners food security interventions enhanced No recorded field level trainings IR2.2: Collaboration with other Title II Cooperating Sponsors to build technical excellence through information sharing and exchange Cooperating Sponsor Meeting USAID/Kigali (Oct- Dec 2005); (Jan, March, April, May, June, July, August , September 2006) not training 1-2 attended meeting (not training) 1 attended meeting (not training) (9/21/06) Rwanda Food Security Crisis Meeting (Jan 2006) 2 attended meeting (not training) IR 2.3: Technical excellence in gender addressed through broadened partnership to build institutional capacity Creating Gender X 2 ACDI/VOCA ICB Mid-Term. Annex IV.B. December 22, 2006. 38 Funding Cape Verde Uganda Rwanda Training Theme, Date, and Duration by SO and IR42 Non- ICB43 ICB Facilitated or Partially Facilitated National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff awareness in Uganda’s agricultural households (2/06) (USAID/ UG) SO3: Ability to design food security proposals Increased IR 3.1: Capacity to identify the root causes of food in security and to integrate monitoring and evaluation into project design and execution enhanced Environmental Assessment (Queen Elizabeth Park, Uganda) (March 2006) 1 1 IR 3.2: Nutrition impact of Title II programs enhanced No recorded field- level trainings IR 3.3: Integration of innovative HIV/AIDS programs strengthened Lusaka Conference on the dual epidemic of HIV/AIDS and Food Insecurity (5/06) (Zambia)44 X 1 SO4: Evidence base that supports programming approaches that improve impact (M&E) IR 4.1: Documentation and integration to past performance successes improved Field level trainings in connection with field assessments IR 4.2: Institutionalization of Title II program data and analysis expanded Food Security Monitoring in Uganda (Kampala) X 1 1 44 ICB funded. ACDI/VOCA ICB Mid-Term. Annex IV.B. December 22, 2006. 39 Funding Cape Verde Uganda Rwanda Training Theme, Date, and Duration by SO and IR42 Non- ICB43 ICB Facilitated or Partially Facilitated National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff National ACDI/VOCA and Partner Staff Expatriate ACDI/VOCA Staff 10/05-9/06 (monthly meetings) Improved Management USAID NGO Business Writing Course Kampala X 8 1 Finance Training (May 2006) X 1 USAID Rules and Regulations (Nairobi) (3/06) X 2 Strategic Planning X 10 Biannual retreat meeting for Africa and Middle east (Kenya) for ACDI/VOCA X 1 Annual Food for Development Retreat (Washington, DC/ACDI/VOCA) X 1 2 Intensive language training (Portuguese) X 1 Source: Project Records submitted to P. Gonzalez, ICB Coordinator, for the ICB Mid-term Evaluation. December 2006. ACDI/VOCA ICB Mid-Term. Annex V.A. December 22, 2006. 40 Annex V. Evaluation Methodology and Logistics Annex V.A. List of People Interviewed ACDI/VOCA HQ Office Buzz Guroff, Senior Vice President of the Food for Development Division Suzanne Berkey, Director, Food for Development Division Piero Gonzalez, ICB Project Coordinator Elizabeth Adams, Project Coordinator Katie Cerratani, Project Coordinator Lisa Miller, Project Coordinator Olutayo Akingbe, Project Assistant Elizabeth Dalziel, Project Assistant Ashleigh Mullinax, Project Assistant ACDI/VOCA Title II Field Programs Cape Verde John (Bick Riley), Chief of Party/Program Manager Michele (Michka) Seroussi, Deputy Chief of Party/M&E Specialist Rwanda Paul DeLucco, Chief of Party/Country Representative Barry Elkin, Deputy Chief of Party/Monetization Manager Uganda Bernie Runnebaum, Chief of Party Sandy Blanchard, D/COP/Monitoring and Evaluation Officer Other Thomas Gibb, Consultant ACDI/VOCA ICB Mid-Term. Annex V. Table B. December 22, 2006. 41 Annex V.B. ACDI/VOCA ICB Mid-Term Evaluation Schedule Dates Activity October 25, 2006 Contract signed October 30- November 13, 2006 Pre-evaluation planning (Preparation of ICB Document Briefing Book and Logistics) November 14- November 20, 2006 Initial debriefing and discussion of initial recommendations and observations with the core members of the Food for Development Division November 27- December 8, 2006 Additional interviews (by telephone and email) with field and HQ staff December 11, 2006- December 15, 2006 Preparation of draft final report December 18- December 22, 2006 HQ staff review of draft final and final revision ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006. 42 Annex V.C. Mid-Term Surveys Annex V.C.1. Field Level Interview Concerning Impact Indicators To: Cape Verde Team: Bick Riley and Michka Seroussi Rwanda Team: Paul DeLucco and Barry Elkin Uganda Team: Bernie Runnebaum and Sandy Blanchard From: Della McMillan (ICB Mid-Term Evaluation Team) Date: December 4, 2006 Re: Impact Indicators 1. Impact Indicator 1.1. # of Title II programs utilizing new ICB tools/modules/best practices for commodity management and monetization ICB facilitated Tools being Developed or Revised under the ICB Do you know about this tool? If so, how did you learn about it Have you used this tool or module in any capacity in your current program or the design of a new program or extension of an existing program? How could ACDI/VOCA increase the impact of this tool in your current program or any new program you are associated with? SO1: Capacity to address food insecurity through all possible uses of Title II resources improved Monetization Best Practices Manual (developed under ISA grant and revised and updated under ICB) (FY04) Monetization Training Modules (T. Gibb) (FY06): d) Pre and post test form e) Monetization work book; f) PowerPoint training materials Emergency Market Intervention Tools (FY06) (being developed in conjunction with emergency market intervention assessments): a) What you Need to Know and Ask to Make a Bellmon Determination and Conduct a Market Analysis b) Guide to Opening Letters of Credit *Monetization Best Practices/Models/Market Access/Indicator Tracking Tool (T. Gibb) (draft report anticipated January 07) *Agricultural Rehabilitation Tool (first draft N. Sabwa; revision in FY07) ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006. 43 2. Impact Indicator 2.1: # of new food security proposals and/or existing programs that incorporate new or improved gender integration tools. ICB facilitated Tools being Developed or Revised under the ICB Do you know about this tool? If so, how did you learn about it Have you used this tool or module in any capacity in your current program or the design of a new program or extension of an existing program? Have you used any of these tools to train partners or your own staff? How could ACDI/VOCA increase the impact of this tool in your current program or any new program you are associated with? IR2.3: Technical excellence in gender addressed through broadened partnerships to build institutional capacity Johnson-Welch and Leonhardt. 2004. Training Workshop Proceedings: How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation ICRW and ACDI/VOCA. 2005. Gender Analysis Tool. ICRW and ACDI/VOCA. 2005. Curriculum for Conducting a Gender Analysis Exercise in the Field 2 Gender Modules (DVD) 3. Impact Indicator 3.1: # of new or existing program interventions that integrate HIV/AIDS activities. a. ORIGINAL DESIGN: Did the original design of the current program you are executing or ending include a series of activities to address HIV/AIDS and food security? b. OCTOBER 1, 2003: Did the current program you are executing include interventions that integrate HIV/AIDS activities as of October 1, 2003? (activities that were either planned or added) c. DECEMBER 1, 2006: Does the current program include interventions that integrate HIV/AIDS activities? ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006. 44 d. What were the most important sources of technical assistance to your current program (and any new program you have designed since October 1, 2003) for the HIV/AIDS activities in your existing or new Title II program? e. What types of technical support would you like to receive from ACDI/VOCA for the HIV/AIDS activities in your Title II programs? 4. Impact Indicator 4.1: # of new proposals that reflect analysis of long-term impact assessments. a. Have any of the new Title II proposals and/or project extensions in your country program used the long-term impact assessment tool that was developed under the ICB? b. Has the tool been used to gather information that you have used in annual reports to USAID/FFP? 5. Impact Indicator 4.2: # of comparative analyses of trends in Title II programs based on data collected using the new ICB long-term impact assessment tool. a. Have you seen drafts of the new computer data entry and analysis tool developed under the ICB? b. Has the data on your project been entered into this program yet? ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006. 45 c. What types of comparative analyses—between years for the same Title II program and between A/V programs in different countries—do you see this tool contributing to? Persons completing survey (Please complete): ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006. 46 Annex V.C.2. Country Team HQ and Field Level Interviews Country (Circle/highlight correct response) Cape Verde, Rwanda, Uganda Location (circle): HQ/Field Persons participating in the exercise (list names): ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ The ACDI/VOCA ICB Grant is designed to help you and the other Title II Cooperating Sponsors. Specifically the grant is designed: • To build the capacity of ACDI/VOCA to design and execute Title II food programming in five critical areas or strategic objectives (SO); and • To facilitate ACDI/VOCA’s development of tools that can help build core capacity in these areas that can eventually be shared with the other Title II NGO cooperating sponsors. The mid-term evaluation provides with us an opportunity to assess where we stand with regard to achieving these SOs and IRs. The same mid-term helps us reflect on what types of trainings and tool revisions and updates the grant need to emphasize during the next two years. Part A.1. Tools Assessment Form: Please refer to the codes in the footnote to assess your team’s awareness of and use of each of the new tools that are under development. Many of the tools have only recently been released so don’t be surprised if you aren’t familiar with them. Please mark your assessments for the Title II programs in column 2 and for other types of food programming your team backstops in column 3. Part A.2. Priorities for Finalizing the Tools listed in Part A: Could you please rank your priorities for completion of the tools or groups of tools (for example commodity management tools or M&E tools) listed in part A for each member of your team. Some team members may rank them differently. Space is added so individual team members can rank them differently. Part B. Training Assessment Form: Please indicate how many people on the current team (either HQ or Field based) attended these trainings. Part C. SWOT: Team perceptions about the strengths, opportunities and risks associated with the grant achieving its Major IRs and SOs. Part A.1. ACDI/VOCA ICB Tools Assessment (FY04-FY06) Country (Circle or highlight correct response) Cape Verde, Rwanda, Uganda Location (circle): HQ/Field ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006 47 Are you actively using the tool/module/best practice report (Rank 0-5)45 Tool/ICB products New/ existing Title II program Other new/existing food programming Recommendations for future revision/roll out of the tools and enhancing impact S01: Capacity to address food insecurity through all possible uses of Title II resources improved Monetization Best Practices Manual (developed under ISA grant and revised and updated under ICB) (FY04) Monetization Training Modules (T. Gibb) (FY06): g) Pre and post test form h) Monetization work book; i) PowerPoint training materials Emergency Market Intervention Tools (FY06) (being developed in conjunction with emergency market intervention assessments): a) What you Need to Know and Ask to Make a Bellmon Determination and Conduct a Market Analysis b) Guide to Opening Letters of Credit *Monetization Best Practices/Models/Market Access/Indicator Tracking Tool (T. Gibb) (draft report anticipated January 07) *Agricultural Rehabilitation Tool (first draft N. Sabwa; under revision in FY07) *Completed Training Tapes/DVD Training module on Value Chain Analysis (2 hr tool w/video chapters and accompanying PowerPoint slides) Training module on Reg 216 (tape) Gender Modules (2) 45 0 aren’t’ familiar with the tool/service/training and never use it. 1. Have hard of the tool/service/training but never use it/attended it. 2. Have used it/participated in the training. 3. Have used tool/service/training in program design, implementation or evaluation. 4. Have used tool/service/training in program design, implementation or evaluation and it has significantly improved program results. 5. Have used tool/service/training in program design, implementation, or evaluation; it has significantly improved program results; and I/we feel that this tool should be shared more widely (with field programs and/or other Title II Cooperating Sponsors). Are you actively using the tool/module/best practice report (Rank ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006 48 0-5)45 Tool/ICB products New/ existing Title II program Other new/existing food programming Recommendations for future revision/roll out of the tools and enhancing impact OMB Circular A-122 (tape) Monetization Power Point Training Modules (that accompany the monetization workbook) How to conduct a Bellmon Analysis? (draft PowerPoint presented at retreat) S02: Technical excellence in food security programming enhanced through broadened partnership to build institutional capacity *Pons 2004. Organizational Assessment Tool. Version 1. *Keatts 2006. Organizational Assessment Tool. Version 2. Johnson-Welch and Leonhardt. 2004. Training Workshop Proceedings: How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation ICRW and ACDI/VOCA. 2005. Gender Analysis Tool. ICRW and ACDI/VOCA. 2005. Curriculum for Conducting a Gender Analysis Exercise in the Field S03: Ability to design food security proposals enhanced *Food Security Needs Assessment Toolkit (TANGO, 2006) HQ Health and Nutrition Resources. Binder #1. HQ HIV/AIDS and Food Security Resources. Binder #2. S04: Evidence base that supports programmatic approaches that improve impact *Long-Term Impact Assessment: Guidelines *Computerized Data Management System (for Title II programs) ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006 49 Part A.2. Priority Needs for ICB Facilitated Technical Assistance (Please rank your team’s priorities top five priorities for finalizing tools and groups of tools over the next two years) Priority Tools and groups of tools HQ Teams that backstop Title II Field Title II teams Part B: ACDI/VOCA ICB Training Assessment Tool (FY04-FY06). (Ret: Retreat; P=Presentation; B0: Break-out) Country (Circle or highlight correct response) Cape Verde, Rwanda, Uganda Location (circle): HQ/Field FY Created Topic (please add others not listed) Participation? (how many people NOW on the country team attended) SO1: Capacity to address food insecurity through all possible uses of Title II resources improved IR1.1: Ability to integrate and measure the food security impact of monetization programs increased FY04 Monetization Training (Gibbs) FY04 Ret Monetization and private sector capacity building (B0) FY06 Monetization 101 (Gibbs) FY06 Ret How to Conduct a Bellmon or Market Analysis (B0) IR 1.2: Institutional capacity to implement developmental relief programs with Title II resources improved FY04 Ret Direct Distribution Strategic Planning session (P) FY05 Ret Donor Panel: Presentation on new dev-relief paradigm (P) IR1.3: Institutional capacity in commodity management enhanced FY04 Commodity Mgt Training (04) (Gerstle) FY04 Ret Ration Design (Cekan) FY05 APVOFM. USAID Rules and Regulations Training ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006 50 FY Created Topic (please add others not listed) Participation? (how many people NOW on the country team attended) FY05 Ret Reg. 216 Environmental rules and regulations training (Myer) FY05 Ret OMB Circular A-122 and Cost Sharing Training (Esposito) (1.0 day) FY06 Food Aid Management Training. (Gerstle) IR 1.4: 46Institutional capacity for developing and documenting examples of best practice and Title II project documents increased47 n/a Value Chain Module n/a Reg 216 (Environmental Module) n/a Gender Modules (2) n/a OMB Circular A-122 (Cost Principles) and Cost Share training n/a (2 PowerPoint) Monetization modules (that accompany monetization work book) n/a (1 PowerPoint) How to Conduct a Bellmon Analysis? SO2: Technical excellence in food security programming enhanced through broadened partnerships to build institutional capacity IR 2.1: Capacity to strengthen local partners’ food security interventions enhanced 04 Ret Organizational assessment training (Ed Pons) ¼ day IR 2.2: Collaboration with member PVOs to build technical excellence through information sharing and exchanges (note if you participated in specific groups) n/a IR 2.3: Technical excellence in addressing gender addressed through broadened partnerships to build institutional capacity FY04 Gender Training on M&E ICRW (April 04-1 day) FY04 How to Use Gender Analysis to Strengthen Project Design, Monitoring and Evaluation (August 04) FY05 Gender analysis and awareness training of training session (Dunne and Gonzalez) (May 05) FY05 Plenary session at the 05 meeting (1/2 day) (Leonhardt and MacQuarrie) SO3: Ability to design food security proposals enhanced IR 3.1: Capacity to identify the root causes of food insecurity enhanced FY05 Value chain analysis training (May 12-13) FY05 Ret M&E Seroussi IR 3.2. Nutrition impact of Title II programs enhanced 46 Numbers should refer to the number of staff who has been trained using the existing DVD modules. 47 PLEASE INDICATE HOW MANY PEOPLE ON YOUR TEAM HAVE SEEN OR USED THE MODULES SEPARATELY FROM THE TRAININGS IN WHICH THEY WERE DEVELOPED. ACDI/VOCA ICB Mid-Term. Annex V.C. December 22, 2006 51 FY Created Topic (please add others not listed) Participation? (how many people NOW on the country team attended) IR 3.3: Integration of innovative HIV/AIDS programs strengthened FY05 HIV/AIDS Plenary (Canahuati, USAID FFP) (P) FY04 HIV/AIDS/Monetization (BO & P) SO4: Evidence base that supports programmatic approaches that improve impact IR 4.1: Documentation and integration of past performance successes improved Training in long-term impact assessment (field level) IR 4.2: Institutionalization of Title II program data and analysis expanded Training in data entry and analysis using new data base (informal one on one) Program Management FY04 Ret ICB ( P) FY05 Ret ICB (P) FY05 Ret Training in code o Federal regulations and grants, management training (2.5 days) FY06 Re FFD Contracts & Grants Part C. Global Analysis of Strengths, Weaknesses, Opportunities and Threats (Risks) Associated with the Grant Achieving its Major SO’s and IR’s STRENGTHS WEAKNESSES OPPORTUNITIES RISKS/THREATS ACDI/VOCA ICB Mid-Term. Annex V.D. December 22, 2006 52 Annex V.D. SCOPE OF WORK for ACDI/VOCA’s Institutional Capacity Building (ICB) Program FY 2004 - FY 2008 Background of ICB program ACDI/VOCA was awarded $1,400,000 under the Institutional Capacity Building Grant (ICB) by the Office of Food for Peace to enhance ACDI/VOCA’s institutional capacity to promote food security among at-risk and vulnerable groups. The agreement is a five-year effort and is designed to provide funding support to Title II project objectives. The objectives of the ACDI/VOCA’s ICB program are: • SO1 Resource Management: Develop ACDI/VOCA’s capacity to address food insecurity through all possible uses of Title II resources. • SO2 Collaboration: Expand technical excellence in food security programming through broadened partnerships to build institutional capacity. • SO3 Design, Monitoring, Evaluation and Reporting: Enhance ability to design food security proposals. • SO4 Evidence Base: Increase evidence base that supports programmatic approaches. • SO5 Program Management: Support the management of ACDI/VOCA’s Title II programs. Purpose of Review The broad purpose of the mid-term review is to assess progress in achieving planned results and use that information to predict if targets will be met by the end of the activity. The review will take place during Year Three of the agreement in order to influence the FY07 and FY08 activities. Information gathered during the mid-term review, including qualitative and quantitative data, should be used: 1) to assess progress toward achieving targets and 2) to examine what changes might be necessary in terms of activities or resource allocations to achieve future targets. Project management, sustainability, institutional strengthening, training, and monitoring and evaluation components should be examined. Specific tasks are to: 1) Determine the progress toward achieving intermediate results/indicators and targets; ACDI/VOCA ICB Mid-Term. Annex V.D. December 22, 2006 53 2) Review the appropriateness of the activity with respect to the problem analysis in the ICB and any subsequent amendments; 3) Identify constraints and difficulties as well as successes; 4) Refine targets if needed; 5) Make recommendations to improve the performance, or, as appropriate, suggest modifications or discontinuance of activities; 6) Determine if the activity is being implemented in a timely and cost-effective way; 7) Determine the best practices of ACDI/VOCA and how the ICB contributed to them; and 8) Assess the effects of these best practices. Methods The evaluation will consider routine data from ACDI/VOCA’s ICB Monitoring and Evaluation (M&E) system as well as copies of all ICB deliverables. During the pre￾planning process, ACDI/VOCA will update its Project Performance Indicator Tracking Table (PITT). Although the evaluation will take the PITT as its point of departure in assessing progress, it will also consider other types of qualitative and quantitative data on project results and impact. A variety of methods may be used to assess results and impacts including: surveys; interviews with ACDI/VOCA staff, other cooperating sponsor staff, and/or Food for Peace (FFP) staff; document review; and/or case studies. The evaluator should present the chosen method to ACDI/VOCA for approval before the commencement of data collection. The presentation of qualitative anecdotes that highlight inter-CS collaboration, interpersonal dimensions within the organization and mentoring relationships are also allowable and useful. The Project Performance Indicator Tracking Table (PITT) must be presented. It should be updated to show progress of impact and monitoring indicators. Expected Report Outputs The final report should include at least the following components: 1) Rationale for the ICB; 2) Existing and projected beneficiary population of the ICB; 3) State of institutional capacity of ACDI/VOCA staff participating in ICB; 4) Updated PITT showing the status of all monitoring indicators (planned vs. achieved) and status of all impact indicators as appropriate (planned vs. achieved); 5) Realistic appraisal (based on indicator review) of likelihood of achieving all targets; ACDI/VOCA ICB Mid-Term. Annex V.D. December 22, 2006 54 6) Suggested changes to M&E plan with respect to targets and indicators; 7) Concrete recommendations for modifications to agreement’s objectives or overall performance; 8) Examples of best practices developed with ICB funding; 9) Overview of research methodology and methods used for evaluation; 10) Data collection and analysis procedures; and 11) Results from data collection process. Evaluation Team External Evaluator An outside contractor is required for this exercise. The contractor will not be required to be approved by Food for Peace. It is important the contractor evaluate in light of new FFP strategy. The desired individual should have a background in monitoring and evaluation and familiarity with Title II programs. The evaluator will work with both HQ and Field staff to develop the report. Evaluation Manager The ACDI/VOCA Food for Development Director will serve as the evaluation manager and co-team leader of the evaluation. His duties will include: • Completion of the documentation briefing book for the consultant; • Updating the current PITT with existing M&E data as part of the pre￾evaluation planning process; • Preparation of the basic background information (and tables) on ICB supported training, staff hiring, and finance; • Working with the evaluator on the identification of what types of complementary information might be needed from field programs; • Identifying the individuals from outside ACDI/VOCA that should be interviewed; and • Facilitating the in-house interviews as well as an in house review to go over the principle conclusions of the evaluation. Timetable Approval of SOW by FFP: Oct. 31, 2005 Contract Evaluator: Oct. 23, 2006 Pre-evaluation Preparation Oct. 23 – 29, 2006 Documentation and methodology: Methodology presented to ACDI/VOCA and start of data collection: Oct. 30, 2006 DC Interviews - core and field teams Nov. 5 -10, 2006 ACDI/VOCA ICB Mid-Term. Annex V.D. December 22, 2006 55 First Draft due to ACDI/VOCA: Dec. 1, 2006 Review and Discussion: Dec. 1, 2006 - Dec. 13, 2006 Final Draft due to ACDI/VOCA: Dec 14, 2006 Background Materials to be Developed as part of Pre-Evaluation Planning Document Briefing Book • ACDI/VOCA Institutional Capacity Building (FY 04-08) agreement including budget • ICB Work plan for FY 04, 05 and 06 • ICB Annual Performance Reports for FY 04 and 05 • ICB M&E Plan (developed during FY 04 and amended in 05) • Food for Development division Organizational Chart • Food for Development HQ and Field Office Contacts and suggested contacts from outside the organization (email addresses and telephone numbers) • List of Title II Food Security programs: - Currently funded projects approved but not yet funded and countries where ACDI/VOCA anticipates developing projects over the next three years including budget amount and technical areas covered; - Other food security and agricultural projects (by country) that may have benefited from any ICB materials including estimated dollar amount and donor; - Other food-assisted programs (USDA, WFP) by country including estimated dollar amount and donor. ICB Activity Briefing Book (to be produced by the evaluation manager and external evaluation as part of the pre-evaluation process) • Project chronology table (listing key events) • Staffing patterns and turnover tables (including list of all individuals that have held key positions and their activities) • Training Tables • Partner coordination and executive board meetings tables • Financial systems and accounting tables • Indicator update (PITT updated as well as indicator methodology update) • Activity matrix for each of the major countries where ACDI/VOCA has Title II or non Title II but relevant funding; • Technical sector/component updates (for each SO); • World map (indicating Title II countries); • Photos illustrating ICB related activities (including captions). • Complete project bibliography (including basic reports and deliverables) following standard format (by technical area and for the project itself) ACDI/VOCA ICB Mid-Term. Annex V.D. December 22, 2006 56 Suggested Table of Contents Executive Summary Table of Contents List of Acronyms 1. Introduction: ICB Background and Evaluation Methodology 2. SO1 2.1. Progress toward achieving results 2.2. Issues: Strengths, Weaknesses, Opportunities, Risks 2.3. Recommendations 3. SO2 (same as SO1) 4. SO3 (same as SO1) 5. SO4 (same as SO1) 6. SO5 (same as SO1) 7. Management and Financial Systems 8. Priority Recommendations for Increasing Impact Annex I: PITT (Mid-term) Annex II: List of people interviewed Annex III: Activity Tables Annex IV: Management and Finance Tables