Evaluation of Capacity Building in Two Faith-Based HIV/AIDS Projects in Ethiopia Final Report December 2008 This document was submitted by The Mitchell Group, Inc. 1816 11TH Street, NW Washington, DC 20001, USA Telephone: (202) 745-1919 Facsimile: (202) 234-1697 to The United States Agency for International Development/Ethiopia under USAID Contract 663-C-00-08-00409-00 Disclaimer: The authors’ views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government. Evaluation of Capacity Building in Two Faith-Based HIV/AIDS Projects in Ethiopia This is an evaluation report carried out by team members: Carolyn Barnes, PhD Alemayehu Mekonnen, MD, MPH Yasmin Yusuf, MA Consultants to THE MITCHELL GROUP September 2008 i Table of Contents EXECUTIVE SUMMARY ...............................................................................................................................III I. INTRODUCTION........................................................................................................................................... 1 A. PURPOSE AND SCOPE OF THE EVALUATION.................................................................................................. 1 B. BACKGROUND TO THE FBO HIV/AIDS PROGRAMS..................................................................................... 1 C. EVALUATION METHODOLOGY ...................................................................................................................... 2 II. IOCC/DICAC PROJECT: HIV/AIDS RESPONSE MECHANISM PROJECT.................................... 2 A. OVERVIEW OF THE PROJECT......................................................................................................................... 2 B. ORGANIZATIONAL CAPACITY....................................................................................................................... 3 2. HUMAN RESOURCE MANAGEMENT AND ADMINISTRATION .......................................................................... 5 C. FINANCIAL MANAGEMENT AND SUSTAINABILITY ........................................................................................ 8 D. TECHNICAL CAPACITY AND GROWTH......................................................................................................... 13 E. COORDINATION AND RELATIONSHIP WITH OTHER ORGANIZATIONS .......................................................... 16 F. CONCLUSIONS............................................................................................................................................. 17 G. MAJOR ACHIEVEMENTS ............................................................................................................................. 18 H. RECOMMENDATIONS .................................................................................................................................. 19 III. PACT LED M-ARCH PROJECT ........................................................................................................... 21 A. OVERVIEW OF THE PROJECT...................................................................................................................... 21 B. ORGANIZATIONAL CAPACITY .................................................................................................................... 23 C. FINANCIAL MANAGEMENT AND SUSTAINABILITY ...................................................................................... 27 D. TECHNICAL CAPACITY AND GROWTH ........................................................................................................ 33 E. COORDINATION AND RELATIONSHIPS WITH OTHER ORGANIZATIONS ........................................................ 37 F. CONCLUSIONS............................................................................................................................................. 38 G. MAJOR ACHIEVEMENTS ............................................................................................................................. 39 H. RECOMMENDATIONS .................................................................................................................................. 40 III. GENERAL CONCLUSIONS, LESSONS LEARNED AND RECOMMENDATIONS FOR FUTURE PROJECTS ........................................................................................................................................................ 40 A. CONCLUSIONS ............................................................................................................................................ 40 B. LESSONS LEARNED..................................................................................................................................... 42 C. RECOMMENDATIONS TO USAID FOR FUTURE PROJECTS ........................................................................... 42 ANNEX A. LIST OF PERSONS MET ........................................................................................................... 45 ANNEX B PARTICIPANTS IN EVALUATORS’ DEBRIEFING SESSIONS ........................................ 47 ANNEX C FIELD TRIP SCHEDULE........................................................................................................... 48 ANNEX D. NUMBER AND TYPES OF FGD PARTICIPANTS PER INTERVIEW SITE...................... 49 ANNEX E. EOC-DICAC ORGANIGRAMS ................................................................................................. 50 ANNEX F. ORGANIZATIONAL STRUCTURE OF EMDA...................................................................... 52 ANNEX G. ROHI WEDA ORGANIZATIONAL STRUCTURE................................................................ 53 ANNEX H ILLUSTRATIVE RESULTS FRAMEWORK FOR IOCC-DICAC PROJECT.................... 55 SELECT BIBLIOGRAPHY............................................................................................................................. 57 ii Abbreviations AB Abstinence and/or Be Faithful prevention IEC/BCC ART Anti-retroviral treatment BCC Behavior Change Communication DICAC Development Inter-Church Aid Commission, of the EOC EIASC Ethiopian Islamic Affairs Supreme Council EMDA Ethiopian Muslim Development Agency EOC Ethiopian Orthodox Church ETB Ethiopian birr (currency) FBO Faith Based Organizations FDGs Focus Group discussions HAPCO HIV/AIDS Prevention and Control Office HBC Home Based Care HQ Headquarters IEC Information, Education and Communication IGS Income Generation Scheme IGA Income Generating Activities IOCC International Orthodox Christian Charities M-ARCH Muslims Agencies Recharging Capacity against HIV/ADS OVC Orphans and Vulnerable Children PAC Project Advisory Committee PEPFAR President’s Emergency Plan for AIDS Relief PLWHA People Living with HIV/AIDS PMTCT prevention of mother to child transmission RW Rohi Weddu, The Pastoral women Development Organization TOT Training of Trainers USAID/E United States Agency for International Development/Ethiopia USG United States Government VCT Voluntary Counseling and Testing YPP Youth Peer Promoters (DICAC) iii Executive Summary Purpose and Scope. This independent external evaluation centers on capacity building in two faith-based HIV/AIDS projects; the HIV/AIDS Response Mechanism project, and the Muslims Agencies Recharging Capacity against HIF/AIDS project. Led by the International Orthodox Christian Charities (IOCC), the HIV/AIDS Response Mechanism project works exclusively with the Development Inter-Church AID Commission (DICAC) of the Ethiopian Orthodox Church (EOC). The Muslims Agencies Recharging Capacity against HIV/AIDS (M-ARCH) project, led by Pact Ethiopia, centers on the Ethiopian Muslim Development Agency (EMDA), the development wing of the Ethiopian Islamic Affairs Supreme Council (EIASC), and Rohi Weddu (RW), Pastoral Women Development Organization. For the IOCC/DICAC project this report is a mid-term evaluation, whereas it is an end of project evaluation for M-ARCH. Evaluation Methodology. Conducted over a four week period, the evaluation consisted of document review, meetings, and interviews with lead organizations, their sub-grantees, and with USAID. In select parts of the country, interviews were held with staff and focal persons. Focus group discussions were held with community volunteers and religious leaders involved in project implementation. Overview of the Projects. The first phase (January 2004 – January 2007) of the IOCC/DICAC $4.6 million project centered on building capacity of DICAC, and HIV prevention, care and support in 20 dioceses. A $7.8 million project cost extension to July 2011 added 10 additional dioceses. In 2004, prior to the M-ARCH Project, Pact received a one year USAID grant of $400,000 to assist EMDA in building its capacity. In 2005, USAID extended this capacity building project for three years, with a total estimated cost of $1.15 million. The extension allowed for Pact to partner with additional Muslim faith-based organizations. The M-ARCH Project centers exclusively on HIV/AIDS prevention and capacity building. Key Conclusions . Below are key conclusions based on findings across the two projects. The body of the report presents findings, conclusions, and achievements for each project.  Both Faith based organization (FBO) projects have been involved in strengthening the financial management capacity of their partners. The results have been positive within the confines of the organizational structures and procedures of DICAC and EMDA.  Involvement of program-trained religious leaders, together with the activities of community volunteers, has led to greater awareness about HIV/AIDS, a willingness to openly discuss it, promotion of voluntary counseling and testing services, and greater community support for people living with HIV/AIDS (PLWHA), orphans and vulnerable children. iv  Both the IOCC and Pact projects provided training and mentoring related to their project monitoring and reporting systems. Both systems, however, contain critical deficiencies that affect the quality of data reported to USAID on some of the indicators associated with the U.S. President’s Emergency Plan for AIDS Relief.  Based on the evaluators’ field findings, it is apparent that both projects have increased awareness and knowledge about how HIV/AIDS is spread and ways to prevent infection among a large number of people, which is changing social norms, a key factor in behavioral change.  Networking across organizations involved in HIV/AIDS related activities tends to be informal at the district level. At regional and national levels there are formal NGO and faith-based networks in which the local partners participate. Key Recommendations. This evaluation report contains recommendations for future USAID/Ethiopia HIV/AIDS projects, as well as recommendations specific to each of the projects evaluated. Recommendations specific to each project include the following:  Prior to completion of the Pact project, EMDA and its EIASC should undertake a facilitated internal participatory analysis to identify key limitations to EMDA’s income generation capacity and thereafter take measures to address these.  EMDA and RW should address deficiencies related to financial procedures.  DICAC should strengthen its field program by increasing the remuneration of the district Focal Person, ensuring greater alignment of Project Advisory Committees with current guidance on composition and responsibilities, and providing regular support to branch offices to increase financial management capacity.  DICAC’s centralized system of fund management is a key obstacle hampering its efficiency in financial reporting and this issue should be addressed immediately.  IOCC/DICAC should strengthen project implementation through a targeted strategy for stigma reduction, and provide a trainer’s guide to training course participants.  IOCC/DICAC should revise and computerize its monitoring and results reporting system. Key Lessons Learned. The evaluation’s key lessons learned include:  Building organizational capacity within headquarters and at the field level can occur despite difficulties within the organization that are beyond the manageable interests of Lead Partners. However, the organizational capacity building efforts can be more cost-effective and have a deeper effect if the organization is not hindered by difficulties at HQs and with its higher level decision-making bodies.  Networking at grassroots level across programs and organizations brings greater efficiency in addressing the needs of OVC and PLWHA and in filling in program/project gaps. 1 I. INTRODUCTION A. Purpose and Scope of the Evaluation This independent external evaluation was conducted at the request of the USAID/Ethiopia (USAID/E) Health, AIDS, Population and Nutrition Office. The evaluation centered on capacity building in two faith-based HIV/AIDS projects. The HIV/AIDS Response Mechanism project, with International Orthodox Christian Charities (IOCC)1 as the lead partner, works exclusively with the Development Inter-Church Aid Commission (DICAC) of the Ethiopian Orthodox Church. Pact Ethiopia is the lead partner on the Muslims Agencies Recharging Capacity against HIV/AIDS (M-ARCH) that centers on the Ethiopian Muslim Development Agency (EMDA), which is the development wing of the Ethiopian Islamic Affairs Supreme Council, and Rohi Weddu, Pastoral Women Development Organization. The evaluation of the IOCC/DICAC project is a mid-term evaluation, whereas it is an end of project evaluation for the Pact led project. The evaluation addresses the following capacity building domains:  The Organizational Governance, Strategy and Structure;  Human Resource Management and Administration;  Project and Program Management, Monitoring and Evaluation;  Financial Management and Sustainability;  Technical Capacity and Growth; and  Coordination and Relationships with Other Organizations. B. Background to the FBO HIV/AIDS Programs In Ethiopia, the national adult HIV prevalence rate is 2.1 percent, while the rate in urban populations is more than three times higher (7.7%).2 Urban women are three times more at risk for HIV than urban men. About 59 percent of the HIV-positive population is female and approximately 61 percent of all HIV-positive individuals reside in urban areas. The projected growth in the total size of the HIV positive population in Ethiopia means that efforts to strengthen counseling and prevention services will be of paramount importance to a successful national HIV/AIDS program. Christianity and Islam are the main religions and account for 97 percent of the population.3 Religious leaders play an important role in shaping the opinions, attitudes and behaviors of the followers of their faiths. Since 1987, faith-based organizations (FBO) have turned their attention to the problem of HIV/AIDS. 1 IOCC is the official humanitarian aid and development agency of Standing Conference of Canonical Orthodox Bishops in the Americas, which works in cooperation with Orthodox Churches worldwide. 2 Federal HIV/AIDS Prevention and Control Office April 5, 2007. 3 Ethiopian Demographic and Health Survey 2005. 2 C. Evaluation Methodology The evaluation was carried out by a three-person team; one member focused exclusively on financial management and sustainability. The evaluation was conducted over a four week period, including preparation for debrief sessions and drafting the report.4 The evaluation methodology consisted of a document review, as well as meetings and interviews with managers and staff of the lead organizations, partner organizations, and key persons within USAID (Annex A). In addition, in select parts of the country, interviews were held with key staff and focal persons, and focus group discussions (FGDs) were held (Annexes B and C). The sites were selected by the evaluators from a list of field sites proposed by Pact and DICAC. The visits were facilitated by Pact, IOCC and DICAC staff who were provided a protocol for selecting and inviting FGD participants. Eighteen FGDs were held with a total of 124 participants (Annex D).5 Each began with a protocol explaining the purpose and intended use of the information, the confidentiality of individual responses, and the voluntary nature of participation. The FGDs covered the activities carried out, training and materials received, priority training needs, interaction with the local program coordinator/focal person, relationship and linkages with other HIV/AIDS related organizations, and major achievements.6 Four debriefs were held: 1)USAID health office and other key staff; 2) the USAID Mission Director and Deputy Director; 3) Pact, Rohi Wedda, and EMDA; and 4) IOCC and DICAC (Annex C contains a list of participants.) Debriefs allowed the team to provide a summary of evaluation activities, hear issues and concerns, and receive feedback. II. IOCC/DICAC Project: HIV/AIDS Response Mechanism Project A. Overview of the Project 1. Objectives The first phase of the project (January 2004 - January 2007) centered on building capacity and mobilization of existing Orthodox networks in Ethiopia to reinforce and support HIV prevention, care and support. The $4.6 million project covered 20 dioceses and a total of 100 church districts. As a result of achievements made, USAID extended the project to July 2011. The four-year, $7.8 million cost extension was granted to a) expand the geographic scope of the project to 10 additional dioceses; and b) strengthen HIV prevention activities, palliative care and orphan and vulnerable children (OVC) services. The stated “objectives” for the extension are similar to the first phase of the 4 The team gratefully acknowledges the preparatory work carried out by Karen Towers (Interim Evaluation Coordinator) and Brad Corner (CTO) of USAID/Ethiopia, which enabled the team to read background documents and develop a strategy for the selection of field site visits prior to beginning the interviews and finalizing field schedule. 5 The plan to have only one type of participant per FGD was thwarted in a few instances. 6 The topics were modified for the FGDs with a district PAC committee and a FGD with RW field supervisors. 3 project, with the addition of developing measurable indicators to assess program impacts. Gender and reduction of stigma and discrimination are cross-cutting themes. The objectives include:  Increasing the capacity of the EOC and its non-religious departments to respond to the national HIV/AIDS epidemic;  Contributing measurably to behavioral change through leadership in abstinence (A) and be faithful (B) messages;  Increasing EOC-DICAC’s ability to provide faith based care for OVCs;  Improving EOC’s ability to provide comprehensive palliative care support for HIV infected persons through evidence and result based interventions;  Expanding the Clergy Training Program in order to respond to the needs of the epidemic; and  Developing a series of measurable or semi-measurable indicators on the other objectives to make assessments of program impact. 2. Targets The targets for the first phase of the project were 5,000 OVC and 5,000 people living with HIV/AIDS (PLWHA) served in year three. The target for Abstinence and/or Be Faithful prevention information, education communication/behavior change communication (IEC/BCC) was stated in terms of the percentage of people in targeted districts reached with prevention messages. The targets for each year of the project extension are:  380,000 reached through AB prevention rallies;  6,600 reached on AB prevention by para-counselors;  85,000 reached on AB prevention through Sunday schools;  8,000 served with Palliative Care/home based care;  9,000 received palliative care counseling;  800 palliative care clients received income generation support;  2,000 OVC received basic services; and  800 OVC provided with income generation support. B. Organizational Capacity 1. Governance, Strategy and Structure DICAC, which is the development wing of the EOC, was registered as an independent NGO in 1972 and governed by Archbishops, until approximately seven years ago when a Board of Directors was created. The Board has 11 members and currently includes of two religious leaders and nine experts from outside the church structure, including two women. The Board Chairman position is to be held by the General Manager of the Patriarchate, who is elected by the Holy Synod, composed of Archbishops and other key religious leaders. Other Board members are proposed by the Patriarch, after soliciting 4 nominations. Membership criteria are expertise and being well-known in the Ethiopian community. Thereafter, the individuals proposed by the Patriarch have to be approved by the Holy Synod. (See organigram in Annex E) The Board’s responsibilities include guidance, approval of major changes, and oversight of projects. It meets quarterly and more often as needed. Major changes or revisions approved by the Board must be submitted to the Holy Synod for final approval. In 2005 EOC-DICAC launched a five year strategic plan, which included its vision, mission, values and principles. The EOC-DICAC Mission is stated as: To assist the disadvantaged communities in Ethiopia to attain self-reliance by tackling the root causes of poverty: drought, conflict, gender inequality and HIV/AIDS pandemics by promoting sustainable development programs and community empowerment. The strategic plan highlights and addresses critical issues that were identified in a series of analyses. In regard to EOC-DICAC structure, systems and policies, the analysis of strengths, weakness, opportunities and threats identified a critical leadership and governance issue within EOC-DICAC: little delegation of authority, with the system being too hierarchical. The evaluation team was told that reorganization of DICAC was under review including reorganization of some departments. Any changes must be approved by the Board and thereafter final approval rests with the Holy Synod. Having identified clarification and strengthening of the linkages between EOC and DICAC as another issue, the Plan stated development of a guideline defining the linkage as the highest priority, with capacity building of the clergy and Sunday school youth the second priority. The third issue was making theology curriculum development oriented. The first phase of the IOCC-DICAC project addressed integration of HIV/AIDS into the theology curriculum and both project phases address the second priority in terms of HIV/AIDS. Field Structure. Normally DICAC programs operate through a Regional Development Officer without project advisory committees. However, the structure for the USAID funded HIV/AIDS program is based on branch offices at the diocese level and offices at the church ‘district’ level (which do not always coincide with the government district boundaries, hereafter referred to as district level). (See Figure 1). The DICAC branch offices have two paid staff members: a Branch Officer and an Assistant Branch Officer. The latter has accounting responsibilities. The Branch Officer serves as Focal Person for the local district. Under each Branch, the project is implemented in five districts; each having a Focal Person. The Focal Person is a volunteer paid an incentive allowance of 800 birr a month (approximately USD $83), which includes funds for transportation to rural areas to hold meetings and monitor community volunteers. The USAID-funded HIV/AIDS program has a three-tiered system of Project Advisory Committees (PAC). The Archbishop chairs the diocese PAC, the head of the clergy at the district level heads its committee, and especially in parishes with Hope Centers (those providing OVC and PLWHA services) there is a similar committee. The new 5 guide for these committees states that the committees are to have key stakeholders, such as the government HIV/AIDS Prevention and Control Office and the health office, at least one woman, and community leaders, and is to perform a monitoring and oversight function. However, the district PAC with whom the evaluation team met was composed only of religious leaders and two male church members who are not government employees. DICAC field visit reports also identify gaps in the composition of the PACs. There is evidence that some heads of PAC committees have a strong influence on program decisions at their level. Figure 1. DICAC Project Organization at the Field Level DICAC Branch Office Diocese PAC District District PAC Focal Person Parish PAC (when have OVC and PLWHA activities) Youth Peer Promoters Counselors In some districts Para councilors In some districts HBC In some districts Clergy 2. Human Resource Management and Administration The EOC-DICAC has a personnel manual approved by the Board in 1992/3, which is being revised, and a human resource development plan. Promotion and recruitment procedures were approved in 2005, and a disciplinary measures procedure is in draft form. To fill new vacancies, the Commission advertises and sets up a committee, and committee minutes are recorded. The Commissioner and Archbishop, recently assigned to DICAC by the Holy Synod, are involved in the approval process. Prior to advertising outside of DICAC, a job vacancy is announced internally and only advertised publicly if a suitable candidate is not identified from within DICAC. The current salary scale, set in 2003, has resulted in a relatively high turnover of staff. Recently, consultants were hired to investigate salary scales in bi-laterals and FBOs and to propose three options. The challenge to DICAC is to have salary scales that enable them to meet financial obligations to staff and officers not covered by projects (external funding sources), while being able to attract and keep qualified and competent staff on projects. 6 3. Project/Program Planning, Management and Monitoring and Evaluation Program Planning. Under the IOCC-DICAC HIV/AIDS program, planning centers on activities within the branches to meet project objectives and targets. IOCC, working in tandem with the DICAC HIV/AIDS Department, develops the Project’s annual action plan and budget, and allocates targets and budgets among the participating branches. Then, the branches develop their implementation plans. For this current planning period, however, the branches will be asked to submit proposals, with targets and budgets. Thereafter, IOCC and the Department will allocate budgets based on proposals. Management. IOCC together with the HIV/AIDS Department manage the project. This entails day to day contact at headquarters. The management function includes strengthening the organizational skills and management in the branch offices. The branch office is responsible for disseminating health learning materials; a pre-rally and post-rally assessment, holding a peer educator review meeting on a quarterly bias, monitoring the number of referrals for voluntary counseling and testing (VCT), maintaining source documentation on the OVC and persons living with HIV/AIDS (PLWHA) registered in the program, having a signed memorandum of understanding for each and every OVC and PLWHA income generating activity, maintaining a roster of para counselors and their assigned areas, and maintaining records on trained clergy. Also, it aggregates the quarterly reports from its districts and submits the reports to headquarters. The branch office is responsible for making referrals to and from health and other service delivery sites, rather than this function being delegated to the district level. Supervisory/monitoring and evaluation visits conducted by IOCC and DICAC, using a check-list with items to be covered, provide immediate feedback on key gaps and weaknesses. DICAC also conducts field visits. The number of supervisory visits to a particular branch varies based on the strengths/weaknesses found previously. The supervisory visits tend to be centered on the branch office, although many also involve visits to the district level. A number of recurrent gaps have been noted in reports from 2007, such as the need to strengthen the branch documentation system, and district level follow-up and supervision of community volunteers. Monitoring and reporting. IOCC and DICAC introduced a number of monitoring and evaluation (M&E) tools, such as a training of trainers (TOT) reporting form, trained clergy reporting form, form for reporting on rallies (including follow-up discussions with groups of participants), peer referral form, and OVC and PLWHA registration forms. The Palliative care/Home based care (HBC) providers have a weekly register of activities form for recording information by name of PLWHA. It also has a monthly summary activity format on services provided and number receiving those services. The monthly activity reporting form of youth educators records activities by date, and includes type of activity, theme/topic, and total number of persons attending disaggregated by male/female. 7 Activity reports by program-trained volunteers and clergy are collected by the respective Focal Person, who keeps a copy, and submits the original to the branch office. However, trained clergy do not always submit monthly reports. The other community volunteers are much better at meeting reporting requirements. For those in rural areas outside of the district office area, the Focal Person normally travels to those areas to meet with the volunteer and collect his/her report. Youth/peer educator reports are structured in such a way as to include double-counting of persons reached. The monthly activity forms submitted by others are also susceptible to double counting. The structure of the forms also affects the ability to report on and verify the number of primary direct services received by an OVC. Currently the ‘package’ of services for OVC consists of educational support, health, psychosocial counseling and, for a limited number, economic support. Guardians/parents receive funds for school supplies, uniforms, and health services. The psychosocial counseling is provided to the guardian/parent and not the child. The economic support, that is, funds for income generating activities, is given in the name of the child to the adult and the child often participates to some extent in the activity. All of these services are counted to report on the number of OVC who directly receive three or more types of services (a reporting requirement under the U.S. Government’s (USG) Presidential Emergency Plan for AIDS Relief (PEPFAR). A review of some of the quarterly and semi-annual reports submitted to USAID on the OVC indicators suggest the tendency to record the same number for the three indicators: total number of OVC served, number of OVC who received primary direct support, and number of OVC who received supplemental direct support. This clearly shows a lack of understanding about how to calculate supplemental direct support and primary direct support. Also, in the semi-annual reports the number receiving specific types of core services does not correspond to the number recorded as having three or more services. The latter may be the result of counting spiritual counseling as a separate service, which is not in accord with PEPFAR guidance. Their Cooperative Agreements with USAID/Ethiopia require that the IOCC and Pact projects submit quarterly and annual reports in a set template, with information reported separately by program area: AB, basic palliative care, and OVC. Under each program area, data on achievements is reported on key PEPFAR indicators against targets for the period, rather than against an annual target. It allows for reporting on types of services received by PLWHA, such as ‘social’, but does not specifically ask about home-based care for the bedridden. The templates also allow IOCC and Pact to report on non-PEPFAR indicators, accomplishments, constraints, challenges, and major activities in the coming quarter. While linkages between health facilities and community-based programs are an important part of USAID’s HIV/AIDS strategy, there is no special indicator that captures achievements of the program in persons trained to promote voluntary counseling and 8 testing (VCT) and prevention of mother to child transmission (PMTCT) services, nor on number of persons referred. Also, the project reports submitted to USAID do not capture achievements related to number of persons trained in HIV/AIDS stigma and discrimination reduction, and number trained in community mobilization (two PEPFAR indicators). C. Financial Management and Sustainability 1. Financial Operation Manual, Supervision and Support DICAC Headquarters (HQ) has a financial operation manual, which it developed some years ago. This manual is being used by the HQ finance team and program branch offices. HQ has revised its financial policies and procedures and has submitted these to the Commissioner for approval. In addition, HQ has submitted for approval an internal audit charter and audit manual. As a direct result of the USG-funded HIV/AIDS program, finance personnel both at branch and HQ levels have been trained in Peachtree Accounting. However, there is a gap in orienting new staff on the organization’s financial procedures and system. This has been a challenge especially at the branch office levels due to the relatively high turnover of Assistant Branch Officers who are responsible for finance. The HQ finance department provides limited support by way of supervision and mentoring to branch offices. There has been support in selected branches from time to time, but it lacks regularity due to the shortage of persons at the HQ level and their workload. In recent times the support from IOCC related to financial matters has been focused mainly on cross-checking reports from branch offices. Currently IOCC is in the process of organizing a team composed of finance experts from IOCC and DICAC HQ who would visit the field offices and provide one-on-one mentoring and skill transfer to field finance staff, as needed. In the early years of the project, IOCC had an expatriate finance manager who was posted in the Addis Office for about two years. During that time, IOCC provided capacity building support, including setting up a chart of accounts for the project and launching the use of Peachtree software. Since this person left, there has been a gap in strengthening capacity in financial management. 2. Budgeting and Fund Flow Up to now, planning and budgeting for the IOCC-DICAC project has been done mainly at the HQ level with limited involvement of the field staff. As mentioned previously, for the forthcoming annual plan period, branches will be asked to submit plans and budgets and the allocation of funds will be informed by these. 9 The program budget is normally sufficient for planned activities at the district level. However, there are delays in releasing funds to the branch levels. At the start of the fiscal year, IOCC provides a two-month advance to DICAC HQ. This money is distributed to branches on a monthly basis. At the end of each month, each branch is to submit its financial report including copies of all supporting documents for disbursements made during the previous month, as well as a cash request for a new month to the DICAC HQ (Table 1). Once the DICAC HQ receives these reports from all of its 30 branch offices, the project accountants7 review the documents and make sure each report is complete. In cases where there are missing documents or unclear or improper expenditures, they are expected to discuss this with the respective assistant branch coordinators. Once they have a complete set of documents, the Accountants at the HQ level post the accounting entries into the Peachtree accounting software for all of the 30 branches. Then the HQ project accounting team produces an expenditure report by project line item and completes the cash request form for one month8 . This report is sent to IOCC Addis, which adds its request for IOCC Addis’s project support funds and then sends the compiled request to its HQ in Baltimore, USA. The fund flow cycle between IOCC and DICAC is summarized in Table 1. Table 1 Fund Flow cycle between IOCC and DICAC Timeframe Activity Month 1 (beginning of fiscal year) IOCC sends advance budget for 2 months of implementatio n to DICAC HQ DICAC HQ sends one month advance to its branches Branches consume fund transferred Branches start preparing financial report for month 1 and cash request for Month 2 Month 2 Financial report and cash request reaches DICAC HQ on the 3rd to 5th day of the month HQ accountants post accounting entries for all branches, prepare financial report and cash request form DICAC HQ release advance to its branches based on their cash request Financial report/ cash request reaches IOCC Addis Office on the 20th day IOCC Addis Ababa Office requests fund release from its HQ in Baltimore Month 3 If there is no missing document, the fund will 7 DICAC HQ currently has 3 Accountants (2 Accountants and 1 team leader) who are working full time on the IOCC HIV/AIDS project.. One of the accountants was recruited only in July 2008. 8 As the one month cash request and disbursement was causing difficulties in implementation, IOCC Ethiopia Office recently introduced a new system whereby DICAC HQ will request cash advance for two months in order to give some flexibility and avoid delay in implementation. 10 be released by IOCC Addis Ababa Office by the 10th day The fund flow channels and deadlines stated above assume that all reports received are complete and there is no back and forth communication. However, in practice there are problems within DICAC that cause delay in the release of funds. Some of the major causes are:  Failure of submission of complete and timely reports by some branch offices;  The centralized system of financial management;  Difficulty to effect payment at the HQ level due to the use of two check signatories (approving payment) who are outside of the organization (i.e., Archbishops);  Frequency of cash request and transfer – one month is a short time to release funds, use them, report on use, and plan and request cash for new month Neither branch offices nor DIDAC HQ analyze expenditures against budget to assess variances in a systematic manner. However, over the last two months (i.e., June and July), the branch offices have started to calculate cumulative expenditure in order to follow their expenditures over time. The finance unit at HQ does not have a copy of the implementation plan prepared by each branch office. There is only the compiled annual work plan for all branches and at the last page there is a summary sheet that provides information on the type of activity that each branch office is expected to engage in. The budget utilization trend (Table 2), an indicator of an effective financial management system, is good for DICAC. In 2005, both the HIV/AIDS project and the whole organization over spent by a small amount. In general, over spending indicates weakness in planning. However, in DICAC’s case the over spent proportion is minimal and this has been adjusted through the revision of the budget in the middle of the year from multi-year donor funded projects. This annual summary of budget expenditure, however, masks utilization breakdown by month and by branch office due to challenges related to fund release to branches. Table 2. Budget Utilization Rate, DICAC Budget utilization rate HIV/AIDS Program (USAID￾funded) (%) Organization wide (%) 2005 102 105 2006 99 78 2007 88 85 2008* Not Available Not Available Average 87 88 * Evaluation occurred prior to the end of the fiscal year. 11 3. Cash, Check Signatories, and Asset Management System Cash transactions in DICAC are kept to a minimum as per the financial manual. Hence, no more than ETB 500 (approximately USD $52) is kept as petty cash with the cashier of the Diocese at the branch office level. The petty cash is appropriately recorded and when the money is depleted, the cashier submits an expenditure report and supporting documents and gets it reimbursed. The petty cash is kept in a safe box. At HQ level, petty cash is set at ETB 1,000 and this is kept with a cashier. All branch offices have a current bank account opened in the organization’s name. Signatories on the check are the Coordinator of the Branch, and Manager of the Diocese and, in some branches, a third signatory (i.e. Head of the Church at the branch level). In this way the Diocese Offices at the branch level are aware of the financial disbursements at the branch DICAC office level. In addition, the Manager of the Diocese approves monthly financial reports for the DICAC branch office. However, given the workload on the Diocese/Church officials, this sometimes causes delay in issuing payment at that level. Similarly at HQ level, checks used to be signed by the Commissioner of DICAC, one Archbishop and the CEO of the Holy Synod. However, because this practice was causing delays, the signatories have been changed to the Commissioner of DICAC, and two Archbishops. Nevertheless, DICAC still faces challenges due to delays in check signing. There is a very small amount of inventory of office supplies kept in the branch offices. Fixed assets both in branch offices and HQ are not properly registered and labeled. Also when new staff arrive, fixed asset transfer is not documented. In accordance with the financial manual, for procurements under ETB 500 the branch offices follow a direct purchase approach. Procurements between ETB 500 and ETB 50,000, require three bids to be reviewed by the procurement committee. However, branch offices handle very small procurements primarily through direct purchase. At the HQ level, procurements are handled by a separate department. 4. Reporting and Auditing As discussed previously, branch offices send their monthly financial reports to their HQ. Assistant Coordinators in branch offices prepare monthly expenditure summary reports, including a general ledger report on details of financial expenditures made each with a code indicating the program area, branch location, type of expenditure and source of finance with reference to the voucher number. In addition, supporting documents are attached for all disbursements. The original copy of the report is maintained at the branch office, one copy is sent to DICAC HQ, and a second copy kept at the respective Diocese office. Reporting made by branch offices is reported against the USAID/IOCC project ‘components.’ 12 HQ compiles reports from all branch offices by project component and sends them to the IOCC Ethiopia Office along with a cash request for one month, and as per the new procedure, now for two months. The monthly report is also sent to DICAC HIV/AIDS Department. Budget preparation and reporting follows the project components at all levels. IOCC is responsible for reporting to USAID. Quarterly financial reporting is prepared by IOCC HQ in Baltimore and after approval by IOCC Country Office, HQ submits reports to the USAID/Washington Office. As discussed previously, DICAC an faces challenges on submitting timely and complete finance reports organization wide. The USG funded project has its own external audit report while DICAC conducts annual external audits for its programs. The audit report for the IOCC project is sent to DICAC HIV/AIDS Department, the IOCC and the DPPA. 5. Financial Sustainability The project branch offices rely on funds from USAID, and with one exception, the evaluators found no attempt at the branch level to raise funds locally. At the district level the program has raised in-kind contributions from church followers, such as food support to OVC/PLWHA by linking the needy with local business people who own hotels, and gathering clothes and other materials. DICAC has a strong financial base as shown in Table 3. The income of DICAC has been increasing significantly over time from ETB 27.5 million in 2005 to ETB 58 million (approximately USD $6 million) in 2008. In addition the financial records show that DICAC has been working more on diversifying its sources of income both from local and international sources. Currently DICAC works with 30 to 40 donors who provide multi￾year support for programs focused on HIV/AIDS, refugees and returnees, integrated rural development and water. As a result, the share of USG support in the organization’s annual income has declined markedly over time; from 45% in 2005 to 17% in 2008. Parallel with this, DICAC also undertakes income generating activities like rental of water rigs, cars, tents, and providing a garage service. The revenue from local income generating activities averages around 16% of total income of DICAC, which is significant. Table 3. Financial Sustainability, DICAC (Ethiopian birr) Year USAID Funding (A) DICAC community contribution (B) Total USAID HIV/AIDS Project (C=A+B) Total DICAC annual income (D) Share of USAID funding % (E = C/D) Share of local income generation % (F) 2005 2,293,167 3,881,460 16,174,627 27,503,683 45 13 2006 5,861,403 5,570,017 11,431,420 33,175,899 18 23 2007* 2,186,160 48,000,000 4 15 2008 10,149,576 58,000,000 17 14 Average 21 16 * This was mainly a no-cost extension year and hence the small budget 13 6. Income Generation Scheme As part of the IOCC project, DICAC has an income generation scheme (IGS) for select OVC and PLWHA. After trying other approaches, the current approach centers on IGA members organized into groups, each with 10 members, with a bank account and statute. Each group member is to have his/her own bank book and is required to deposit a minimum of ETB 10 - 30 in compulsory savings, but can also deposit an additional/voluntary amount of this level. The para-counselors follow up with these beneficiaries. In some branches where para￾counselors are tied up with their other tasks, support may be provided by home-based care providers who, however, are not trained to provide support on financial matters. The program reports that there is also follow up by the governmental microfinance agency but this does not seem to be corroborated with evidence in the field sites visited by the evaluators. D. Technical Capacity and Growth 1. Strategies and Achievements Training of Trainers (TOT). Five and seven day TOT training courses on HIV/AIDS topics, including counseling, were conducted for clergy and youth peer promoters recruited from each branch based on a quota of five clergy and five youth promoters per branch. They are responsible for training other clergy and youth peer promoters within their branch. Selection criteria for the youth peer promoters who attended the TOT included one to two years’ stability in the local community. Normally, five trainers from DICAC provide the TOT training course. The training methodology used for both groups was largely based on lectures. The TOT for clergy was based on a trainer’s guide/participant manual in Amharic “Telikote Samrawi” adopted by DICAC in 2006. Training forms completed by trainers indicate that neither the clergy nor peer promoters of the TOT course were evaluated based on a pre- and post-training assessment tool. Each clergy who participated in the TOT course is expected to train 100 clergy at the district or parish level, whereas as each TOT youth peer promoter (YPP) is supposed to train at least 15 youth volunteers at community level. Other Training. Existing training reports provide insight into two other types of training for HIV counselors and Home Based Care (HBC) providers. Each group was trained for 21 days. Trainees for both types of training had completed at least the 10th grade and there was a gender balance among participants. Three youth counselors and ten home-based care providers respectively per branch were recruited for the training. Seventeen dioceses have home-based care providers, totalling 170 providers. Thirty branches have trained counselors, averaging three per branch. Consultant trainers selected by competitive bidding conducted these trainings based on national guidelines with a pre and post training assessment tool. 14 Refresher courses were found to have little attention. This becomes an issue especially when there is a significant turnover of volunteers. Mentoring. IOCC and DICAC conduct a number of monitoring field visits to branches and districts involved in the program. DICAC conducts planned mentoring visits twice per year to branch offices. Field visit reports reveal that the visits have resulted in ensuring that trained clergy have a copy of the Clergy Manual and IEC/BCC materials prepared in the main local languages. Mentoring has involved providing possible solutions to issues and had led to peer educators establishing Anti-AIDS clubs. A controversial issue centers on the prevention message that clergy should provide to discordant couples and this issue was referred to church scholars for a unified response. Short-term training/Workshops. IOCC facilitated a number of short-term trainings abroad and locally. Two DICAC HQ staff each attended 21 days of training in monitoring and evaluation which was held in another African country. Participation in local training courses included the M&E officer participating in M&E training organized by HAPCO, and M&E training for a Care and Support HQ officer for 21 days, organized by the Ethiopian Red Cross society. Access to HIV/AIDS technical resources. Under this USG supported project, IOCC facilitated and supported the provision of computers with internet access for all branches . Although provision of computer and web access are believed to be the source of HIV technical information, the applicability does not appear to be wide spread. For example, in some branches the team visited, the Coordinators didn’t use the web for more than Email correspondence. The Ethiopian Government’s AIDS resource center web site www.etharc.org was not widely known. 2. Major Findings from Focus Group Discussions Clergy. Two FGDs were held with a total of 18 participants. All reported that they had taken the community-level three-day training for clergy for the first time in July 2008. Except for a few participants, all had received reference material/manual during the training. Several topics were covered in the training: the areas most frequently mentioned in which they were not comfortable on their knowledge were: counseling, modes of and misconceptions on HIV transmission, the different stages of HIV, and bringing of behavioral change through AB messages. Clergy made the following suggestions for improvement of future trainings:  Duration of training should be at least 5-7 days;  Trainers should have enough knowledge about HIV to respond to all questions raised by Clergy and preferably be from the Church;  Daily subsistence allowance should be increased, especially for Clergy coming from far places (daily subsistence allowance was reported to be 25 birr or about USD $3). 15 The FGD participants mentioned a number of activities during which they convey prevention messages. They all deliver prevention messages on a regular basis at daily and weekly church services and monthly home-based gatherings (locally called Mahiber or Tsiwa). In addition, they convey abstinence (for unmarried individuals) and ‘be faithful’ (for married people) messages during house-to-house visits, Sunday school service, large Church ceremonies, and during the third day of mourning and/or funerals. All clergy reported that they encourage people to go for VCT, and about half of them followed up on the individuals until they went for testing. Very few knew the test results. In regard to reduction of stigma and discrimination, the clergy mentioned approaches such as delivering spiritual counseling, advising family members not to stigmatize and seek revenge, and encouraging the PLWHA to go for holy water even though they are taking anti-retroviral treatment. Two thirds of the discussants felt that the magnitude of stigma and discrimination was low, while the others said that it was still high in their community. Youth Peer Promoters. Two FGDs were held with a total of 18 participants. They had been trained in 2006/2007 for 5 consecutive days and most felt that the knowledge gained was fairly adequate. Enough materials were received during the training, although they would like a loud speaker to make public announcements encouraging youth to come for group education. None of the discussants in “Shebe” area had submitted their monthly activities reports for several months. This was because of the absence of reporting forms and lack of guidance and interaction since their district has been without a Focal Person for about five months and the Branch Coordinator is not nearby. Home-Based Care Providers. Two FGDs were held with seventeen HBC providers. In one FGD, participants were all single, age ranged from 20 to 26 years and all had completed at least secondary education. The majority was trained in 2006 for 17 days by DICAC and a few were trained for 7-10 days in 2005/2006 by Organization for Social Service AIDS. HBC providers gave an exhaustive list of training topics covered. HBC providers suggested that future training include additional time for practical sessions in a health facility. Also, they would like to learn more about how to care for a dead body, self protection from infection, and giving proper counseling to PLWHA. HBC providers in the FGD received teaching and other materials, during and after training. They would like to receive supportive commodities such as bicycles to make home visits, rain gear and uniforms. In one area, HBC providers had a total sum of sixty-eight PLWHA clients out of which thirty-seven were taking ART. They identified activities they were carrying out during their three times per week home visits, which included:  Palliative care for bed ridden PLWHAs (washing, massaging, cleaning…etc); 16  Training older children in the family to give care to PLWHAs;  Conveying messages to the families not to share razors with PLWHAs;  Giving adherence counseling and fetching drugs for bed ridden; and  Providing nutritional advice and encouraging PLWHAs to go for CD4 counts three times monthly for PLHIV not currently on ART. In comparison, the HBC providers in the other area only listed activities specifically related to the care and support of their bedridden clients. They said that most of their clients were living alone or the family members did not want to care for the person. At times, the HBC providers have been able to secure housing from the local administration for PLWHA who have been kicked out of their rural homes. They felt that a more coherent strategy should be developed to involve clergy and other volunteers to reduce stigma and discrimination, especially because it results in PLWHA not being cared for by their family members. Three of the providers are HIV+ and told about their own experiences with family members. In both areas, the HBC providers reported meeting weekly with their Branch Coordinator. Participants in one area reported semi-annual review meetings with the DICAC coordinator and group experience sharing discussions among themselves twice a month. These providers believed that the interaction greatly helps them work together and network with other organizations for support to needy PLWHA. Counselors and Para–counselors. One FGD was held with two counselors and three para-counselors. The counselors work out of an office in their church. Individuals come seeking counseling on a range of topics, including those related to HIV/AIDS. The counselors attended a three week counseling course. Para-counselors make home visits to PLWHA and OVC and oversee IGA activities. E. Coordination and Relationship with Other Organizations IOCC has assisted DICAC in establishing relationships and coordination mechanisms with other organizations working on HIV/AIDS. The collaboration with John Hopkins University TSEHAI project led to a jointly conducted field visit and collection of important qualitative data on holy water users at different sites. The results led to the Patriarch supporting the taking of both holy water and ART at the same time. The Coordination with Population Services International (PSI) yielded provision of water agar to treat unclean water and bed nets for PLWHA through training of HBC providers in a specific program site. Pact-Ethiopia has been a good collaborator and was contracted to conduct a follow-up Organizational Capacity Assessment of DICAC. Also, the evaluators found collaboration between IOCC and Regional Health Bureaus and HAPCO in the areas of Prevention of HIV. Though both organizations have their own objectives and constraints to address the high demand, the World Food Program and Ethiopia Red Cross Society in collaboration with DICAC used to provide needy PLWHA 17 with food. Collaboration at the grass roots level also was identified in interviews with Focal Persons and FGDs with HBC providers. F. Conclusions  A large, established wing of the Ethiopian Orthodox Church, DICAC’s governance structure and procedures are hierarchical. This results in delays in funding flows and necessary revisions to strengthen its organizational, financial management, and technical capacity. o The current practice of check signatories including religious leaders causes delays in the flow of funds. o The DICAC low salary levels/scales that have not been adjusted since 2003, have led to loss of people in key positions associated with the Project, such as accountants and finance officers.  Program management is not strong at the point where services and IEC/BCC occur, that is the district level and below. The program has only one volunteer, entitled Focal Person, who is provided a relatively low monthly incentive allowance that includes funds for transportation.  Evidence suggests that some of the PACs do not involve any key government stakeholders and women. In addition, there is evidence to suggest that the chairmen in some PACs have a strong voice in deciding project matters in their communities beyond the stated role of PACs to monitor and oversee project implementation.  IOCC has institutionalized a good system for structured supervision/monitoring and evaluation of organizational and technical aspects of branch and district offices within the HIV/AIDS Department.  The monitoring and reporting system involves several forms for reporting on activities and all is done manually. The system and requirements for reporting indicate several weaknesses related to PEPFAR indicators. o Some forms used by community volunteers lend themselves to double counting. They also require time consuming efforts to extract the information into the appropriate indicator categories. o No master lists exist of those trained as trainers nor is there a list of who these trainers have trained. This lends itself to double counting when the same people receive additional training. 18  The frequency of cash requests and transfers, which is on a monthly basis, is delaying funds transfers, given the highly centralized fund flow and financial management system within DICAC.  DICAC faces challenges on submitting timely financial reports to IOCC, which is responsible for submitting the reports to USAID. The delay is related to the practice of reviewing and posting of financial transactions into the accounting software by a small number of HQ staff after the reports from all of its 30 branches have been received.  DICAC, organization wide as well as the USG-supported HIV/AIDS program, has good financial utilization rates.  There is a gap evidenced at both the HQ and branch offices levels in asset management – both a lack of an updated proper registry of fixed assets and. available assets that are not properly labeled. Also the offices lack a system for asset transfer between incoming and outgoing staff.  Training of community level educators through a TOT approach is encouraging and promising in that re-training for attrition and refresher courses are easily manageable. However, some of the TOT clergy are not always effective trainers. DICAC lacks clearly defined criteria for selection of those trained as trainers, and individual assessments done by trainers giving TOT do not appear to influence who trains clergy at the community level.  In regard to training of clergy, absence of a trainers guide (as distinct from a manual with technical information) with a syllabus on training has negatively affected the quality of the training given by the TOT clergy.  It appears that refresher training to build capacity among community level volunteers has taken second place to scaling up.  The evaluators found evidence of links with other organizations, and promising approaches were mentioned on coordination and relationships between DICAC structures and other organizations working on HIV/AIDS. The efforts have been mostly need-oriented and asymmetrical, rather than regular and systematic. G. Major Achievements  Training of clergy has increased their involvement in spreading IEC messages about how AIDS is spread and promoting Abstinence and Be Faithful. The Patriarch’s support in promoting the use of both holy water and ART at the same time has been important, and clergy are involved in encouraging this practice.  Structured supervisory/M&E visits by IOCC and DICAC have led to instilling good practices within DICAC’s HIV/AIDS Department. 19  DICAC has benefited greatly from the financial management training provided by IOCC during the first part of the project, which set the chart of accounts and provided training in Peachtree accounting. As a result of this support, DICAC currently faces no difficulty in reporting to USAID based on the project components. H. Recommendations Organizational  DICAC needs to strengthen its program at the district level in the following ways during the forthcoming annual action plan period: o Either pay Focal Persons a monthly wage plus transportation allowance or increase the person’s incentive allowance. o Revisit the Branch Officer’s responsibilities to determine which of them should be assigned to the Focal Persons. o Give immediate attention to assignment of replacement Focal Persons in districts or create a link between the community level educators and the Focal Person in a near by district  EOC-DICAC should review the composition and role of the PACs, and the actual functioning of these committees, with a view toward ensuring that the PACs play an advisory and oversight role and have greater representation of key stakeholders (including women), with technical expertise, within the broader community.  During the forthcoming annual action plan period, IOCC in collaboration with DICAC’s HIV/AIDS Department, should submit to the USAID CTO a project results framework, which includes PEPFAR and select non-PEPFAR indicators. The purpose would be two-fold: 1) teach DICAC how to develop results frameworks and 2) be better able to report results and expenditures against the framework. (See Annex H for an illustrative framework.)  IOCC, DICAC and the USAID CTO ought to revisit the performance categories and targets set for prevention activities in the Cooperative Agreement. They should consider achievements already made in awareness building in the initial 20 branches and determine if more emphasis should be given to BCC.  DICAC’s HIV/AIDS Department should revise and computerize its monitoring and reporting system for the program, to enable it to track individuals trained and record types of training each individual receives (this would apply to staff and volunteers) during a plan period (and life of project). Moreover, the system should center on lessening the possibility of duplicative counting for BCC on prevention, and types of services for OVC. This would entail revising monthly reporting forms to avoid double counting of individuals reached with AB prevention BCC, accurate recording of direct 20 services provided to OVC, and types of services provided to PLWHA and their families. o Develop and possibly install the monitoring and reporting system in select branches this coming annual planning period, with a full roll out for the next plan year. o Obtain the services of a consultant experienced in monitoring and reporting on PEPFAR-funded community based HIV/AIDS programs to provide technical advice on development of the system. o Review current required data collection to streamline it based on use of the information. o Consider development of a formula to discount double counting of individuals reached by both IEC and BCC activities during a plan year. Financial  Particularly because of staff turnover, IOCC should provide greater support to DICAC in the area of financial management through ongoing mentoring and training of financial staff at HQ and supporting HQ to strengthen financial management capacity in branch offices. DICAC needs to strengthen capacity within its finance team to provide regular support to the branch offices. In addition DICAC should develop a strategy for ongoing capacity building of staff (as needed) and smooth transfer of organizational memory given the relatively high staff turnover at all levels.  The centralized system of funds management is the key obstacle that is hampering DICAC’s efficiency. This issue needs to be addressed immediately. More authority needs to be delegated to the branch office. DICAC’s new plan to introduce decentralized accounting entry in better off branches is a good initiative that should be implemented soon.  IOCC and DICAC should strengthen their asset management system at HQ and branch and district levels. Checking the implementation of proper asset management system should be included in visits of financial management supervisory/monitoring teams to branch offices.  Efforts by branch offices to generate income from local sources should be encouraged by all parties. In addition, a system should be established for the proper recording and reporting of utilization of funds generated locally in order to reduce possible resource mishandling. Technical  A strategy for stigma reduction should be developed by DICAC, involving clergy, counselors and palliative care/home base care volunteers. One approach could be to establish better linkages with VCT centers for referrals to facilitate follow-up of those recently diagnosed as positive to provide psycho-social counseling to the 21 PLWHA as well as family members and to educate family members about HIV to reduce stigma and discrimination.  Provide refresher courses and additional training to community volunteers, especially related to technical topics, including how to provide services and the difference between BCC and IEC, to ensure a minimum capacity. IOCC should help ensure that the volunteers have appropriate BCC materials.  Based on objective pre- and post-training assessment, TOT trainees that would further act as trainers or co-trainers for community level educators should be identified and a master list of trainers should be maintained by DICAC.  Those attending TOT courses should receive a trainers’ guide containing an evaluation tool, the curriculum, objectives of each lesson, ways to stimulate discussion, and allotted time for each session.  The benefit of scaling up versus conducting refresher trainings should be viewed from the perspective of a basic minimum level of capacity among community level volunteers, including clergy.  By March 2009, the HIV/AIDS Department of DICAC should develop a clear strategy and plan for care and support services to PLWHA including those not needing HBC. In the process, it should review the responsibilities between para counselors, counselors and Palliative care/HBC volunteers. Also stronger and clearer criteria and standards should be set for selection of PLWHA to receive IGA funds.  Attention should be given to training of para counselors and counselors to directly provide psychosocial counseling and support to the OVC, rather than to rely on parents or family members to take care of the OVC’s psychosocial needs. III. Pact Led M-ARCH Project A. Overview of the Project 1. Project Objectives Prior to the M-ARCH Project, Pact-Ethiopia’s proposal for the “The ‘Capacity Building Intervention for The Ethiopian Muslims’ Development Association/EMDA” received a one year USAID grant of $400,000 in 2004. Building on a prior EMDA HIV/AIDS program initiative, the purpose was to assist EMDA in: a) building strong organizational capacity; b) developing management and technical skills of staff; c) developing a standardized project proposal on HIV/AIDS and soliciting funds from donors; and d) assisting EMDA in developing skills to reach other Muslim organizations for greater participation in the Muslim community in HIV/AIDS prevention, care and treatment activities. 22 While progress was made during the 2004 one-year project, it was recognized that capacity enhancement required more time. Consequently, in September 2005, USAID extended the capacity building project, based on the proposal submitted by Pact. The project extension expanded the program description for the cooperative agreement and extended the completion date by three years, with a total estimated cost of $1.15 million. The M-ARCH program extension enabled Pact Ethiopia to continue capacity building and program administration related to the Ethiopian Muslim Development Association and to expand HIV activities from eight to 12 geographic areas, and to support a limited number of additional Muslim faith-based organizations. The project has four stated components:  Technical Assistance by Pact to train and mentor partner staff ;  Activity Grants to the partner organizations;  Personnel Grants to cover salaries of key personnel in partner organizations; and  Commodity Grants to procure important items to enable the partners to carry out their responsibilities. 2. Intended Results and Targets Pact Ethiopia was to provide capacity building support to sub-grantees to build HIV/AIDS prevention programs that reach the anticipated number of persons shown in Table 4. Pact Ethiopia also strongly encouraged its sub-grantees to incorporate three cross-cutting themes: gender, stigma, and community participation into their program planning, implementation, monitoring and evaluation. The extended project is expected to reach the targets presented in Table 4. Table 4: Pact Project Extension Component Targets FY 2006 FY 2007 FY 2008 Prevention Component Youth peer educators trained in AB 500 750 1,000 Youth reached with AB programs 10,000 15,000 20,000 Community educators trained in AB 500 1,000 1,500 Individuals reached with community AB programs 10,000 20,000 30,000 Number of people reached with mass media messages 2,000,000 2,000,000 2,000,000 Capacity Building Component Number trained in financial management 25 25 / Number trained in proposal development 25 25 / Number trained in program management training 25 25 / Number trained in monitoring, evaluation, and reporting 25 25 / Number of program monitoring visits 4 4 4 23 Number of data quality assessments 2 2 2 B. Organizational Capacity 1. EMDA Organizational Governance, Strategy and Structure. Established as an indigenous NGO in 2000 and legally registered as an NGO in late 2001, EMDA is the development wing of the Ethiopian Islamic Affairs Supreme Council (EIASC). The General Assembly of the EIASC selects the Board members and the Board has oversight and review responsibilities. Board related documents state that it is to be composed of nine members: five executive committee members of EIASC and four from outside of the council. As noted in the Pact-funded Organizational Capacity Re-Assessment Report of EMDA, dated March 2008, “roles and responsibilities of EMDA and EIASC are not clearly delineated.” At the time of this evaluation, the Board has not been functioning for at least eight months because of a variety of factors. Two members of the Relief and Social Development Department of the EIASC currently exercise a major role over the day to day work of EMDA. The current operational structure is presented in Annex F. EMDA received training from Pact on strategic planning. With Pact assistance, EMDA developed its second five year strategic plan, which has yet to be approved by EIASC. The plan has five priority areas: food security, health and HIV/AIDS, education (formal and informal), water, and institutional and organizational capacity building. The proposed mission statement, below, reflects a revision of EMDA’s previous statement. Assisting the broad population and poor people of Ethiopia in general and Muslim community in particular though designing and implementing participatory projects and programs. The proposed strategies to address critical issues in planning and implementing programs are: authentic community participation, effective and meaningful partnership with stakeholders, and ensuring gender and age sensitivity with particular emphasis on the most vulnerable sections of society: women and children. Currently, in addition to the Acting EMDA Director, EMDA includes nine persons, three of whom are in support positions. Project personnel grants have helped to maintain this cadre. Also, the staffing grants have helped to build a structure below the national level. Field level. EMDA has four regional coordinators. The one whom the evaluation team interviewed also serves as a Field Coordinator in the district in which he is based. Field Coordinators, based at the district level, lead and manage program implementation in the entire district, and are responsible for financial management and reporting. 24 Figure 2. EMDA Organization: Jimma Zone EMDA Regional Coordinator Jimma (also Field Coordinator for Jimma Town) Islamic Council Jimma EMDA District Field Coordinator(s) Project Advisory Committee Youth Peer Educators (Male+ Female) Women to Women Peer educators Community Level Educators (imams, Idirs) District Level Islamic Council Branch Imam’s At the district level a strong structure has been built that has resulted in positive project results. A voluntary Program Advisory Committee, composed of religious leaders and government officers (representatives from the district health office, HAPCO, youth and sports, and women’s affairs) provides advice, linkages to other organizations and oversight. The district level Islamic Council Office provides supportive functions, such as use of facilities, and financial oversight. Despite governance problems at the higher level of EMDA’s structure, Pact has helped build the organizational capacity of EMDA though a series of trainings and a mentoring process. The training topics included effective advocacy skills, administrative policies and procedures, sustainability planning and resource mobilization, good governance and leadership. The trainers were Pact Ethiopia staff and consultant specialists. Human Resource Management and Administration. EMDA uses the EIASC administrative manual and procedures and has no set staff performance appraisal practice or human resource development plan. Some staff do not have written job descriptions. In HQ, the professional staff performs dual functions as needed. Despite these deficiencies in their system, evidence suggests that EMDA has a transparent system for field staff recruitment and selection. For example, a new Field Supervisor with a degree in Management told the evaluators that he had learned of the position through a job notice posted in the place he was living, outside of the Jimma zone where he is now working. He described a transparent and sound interview process. The other Field Supervisors met by the team seemed to have been well￾chosen based on experience, credentials and commitment. In regard to access to logistical and material support for managers and staff, under the project, EMDA HQ secured computers for HQ, but none exist at the district or regional 25 levels. HQ has only one vehicle at its disposal, obtained under a prior project, which is in poor condition resulting in high maintenance costs. Regional Coordinators and Field Coordinators associated with the program receive a transport allowance, but the amount is insufficient given their supervisory and monitoring functions and taking into account the recent rise in transport costs. Program/Project Planning. Training and mentoring was provided by Pact to EMDA headquarters on participatory project design and project planning as well as gender integration. As noted above, under the initial project Pact assisted EMDA to develop a project proposal request but it was unsuccessful in securing additional funds. EMDA HQ staff has gained on-the-job experience in project/program planning, within the framework of the USAID approved annual action plan submitted by Pact. After the action plan has been approved, EMDA develops a more detailed plan and budget that includes how funds and project targets are allocated across districts. Thereafter, Field Coordinators develop annual implementation plans. However, difficulties occur because often the budgets have been inadequate related to the targets and activities specified. The training received by community level volunteers includes promotion of VCT, reduction of stigma and discrimination and community mobilization and attention to gender considerations. These topics are woven into training on prevention. Program Management. The Project Coordinator in EMDA HQ conducts monitoring visits to the field program sites. In addition, EMDA holds regular review meetings with its Field Coordinators, which allows them to share experiences, achievements and challenges with HQ staff and receive feedback. Held after the regularly scheduled Pact project meetings, the EMDA meetings were held quarterly but then were changed to be held semi-annually due to budget considerations. Monitoring, Evaluation and Reporting. Under the Pact sub-agreement with EMDA, EMDA is responsible for project reporting. Pact trained 25 staff from headquarters and field in monitoring, evaluation and reporting, in line with Pact’s Building Monitoring, Evaluation and Reporting Systems for HIV/AIDS Programs (2005) Thereafter Pact provided a User Guide for M-ARCH Project monitoring, evaluation and reporting formats. The guidance follows the format required for the quarterly reports to USAID/Ethiopia, as specified in the Cooperative Agreement. The ‘daily’ activity register form, completed on a monthly basis by community level volunteers includes each volunteer reporting on his/her activities. The columns on the form are: date, type of activity, type of participants, and number of participants disaggregated by male and female, anecdotal stories, and remarks. The Pact M&E specialist devoted time to resolving how the number of participants at a mosque could be counted using a good estimate method of number of people per row and number of rows. Time was also devoted to how to derive good estimates from other large group IEC activities. 26 Because of the structure of the form completed by trained volunteers and religious leaders, the data resulting from these include substantial double counting. Also the evaluators learned from their interviews that individuals completing the forms were instructed to avoid counting persons if they had received the same message before; however, if it was a different topic, the person could be counted. Double counting occurs two ways. First, a volunteer or religious leader may count a person whom they have reached with Abstinence and Be Faithful messages more than once during the annual plan period. Second, more than one volunteer (including religious leaders) may be counting the same person. For example, a person might be counted as having received AB messages at the mosque and be counting as having participated in an Anti-AIDS club. Different volunteers reporting on the same individual is difficult to avoid in Prevention activities, however it could be addressed by use of a formula, developed based on evidence, to adjust numbers reported to USAID. Neither Pact nor EMDA kept a master list of staff and volunteers trained. The findings from the evaluators’ interviews indicate that most community level volunteers were unlikely to have received more than one training. Hence, the data on number of people trained in A and B probably includes little double-counting. The youth volunteers were taught both A and B and so A was not included as a separate indicator on number of individuals trained. 2. Rohi Weddu Pastoral Women Development Organization Organizational Governance, Strategy and Structure. Located in the Afar Region, which is predominately Muslim, Rohi Weddu (RW) was established as a legally registered NGO in July 2004. Pact assisted RW to develop a proposed revised structure, which is awaiting approval of RW’s General Assembly (Annex G). Pact also assisted RW with the establishment of its General Assembly, composed of community members, community key stakeholders, and experts. The Board is composed of five members, of which three are females. Board members are knowledgeable persons in diverse professions, who meet twice a year to provide guidance on agreements, examine the annual work plan and budget, and provide general direction and focus. The Board also provides oversight of financial rules, regulations and administration and advises on major issues presented by RW management. It does not have advocacy and fund raising responsibilities. With Pact assistance, RW improved its vision and mission statements and developed a strategic plan, which took into consideration the strategy of the Afar Region. Its mission is: To facilitate and work in collaboration with community, government, and development partners to protect children, girls and women from Harmful Traditional Practices (HTPs), promote participation in education, increase individual and family income and develop decision making power of women at all levels. 27 Its strategic objectives are for 95% of the target population have up to date information on HIV/AIDS and become less vulnerable to HIV infection by the end of 2012,. Related to its HIV/AIDS objectives are strategies on reduction of female genital mutilation (FGM), advocacy for formulation and implementation of anti-FGM law, family law, and land use administration for the benefit of women. Field Structure. RW currently works in four districts, including Awash town and Mille town. Six paid field supervisors live and work in these districts, under the direct guidance and support of a Zonal Field Coordinator. Program/Project Planning and Management. Pact has provided mentoring and training on project preparation. When RW first joined the Project, Pact financed exchange visits to EMDA sites which RW found very useful in planning its program. Annual program implementation plans are developed based on the USAID approved Project annual action plan submitted by Pact. Program management involves monthly staff meetings between the Zonal Field Coordinator and field supervisors. At these meetings the field supervisors submit their monthly reports, discuss challenges and plan actions for the forthcoming month. As a small headquarters office, meetings occur as necessary. Field supervisors, who have received training from Pact on community conversations, serve as a bridge between RW HQ and the communities. They select imams, sheiks, kadis, youth and others for training to become community volunteers and provide support to them through interactive meetings and individual mentoring. The Field supervisors are also engaged in prevention outreach, especially at large gatherings. Monitoring and Reporting. The RW system is similar to the one Pact helped set up within EMDA. Field supervisors were trained in monitoring, evaluation and reporting by Pact. The recently hired Zonal Field Coordinator has not had the opportunity to receive this training, but has documents from the training course. The Program Officer feels that RW needs more support and guidance on program monitoring and reporting. The same weaknesses in the EMDA monitoring and reporting system related to double counting apply to RW. C. Financial Management and Sustainability 1. EMDA Financial Operation Manual, Supervision and Support. EMDA HQ has a financial implementation manual, developed with support from Pact that is used by the finance team. However, district offices do not appear to have the manual. Although financial transactions at the district level are minimal, the Field Coordinators should have received a copy of the manual and been familiar with its provisions, since they engage 28 in small scale procurements and disbursements, and compile financial reports, along with support documents, that are submitted to HQ. PACT has provided various training to EMDA staff at HQ and district levels in financial management. Although EMDA had started to utilize Peachtree before Pact’s support started, Pact gave comprehensive training to staff with financial responsibilities. Even today, the staff can access technical support from the accounting firm that first trained them in Peachtree. EMDA district offices are visited by Pact on a bi-annual basis. These monitoring visits focus largely on technical and organizational aspects. Pact reports that the heavy workload among its finance and grant unit team has prevented them from being more engaged in field monitoring. At the HQ level, Pact conducts a monitoring visit to EMDA HQ one to two months after receipt of its financial report. EMDA HQ visits district and regional offices but the visits are largely related to technical matters. The financial aspect gets supervisory support every year or two because EMDA HQ finance team has only two staff. Budgeting and Fund Flow. All planning and budgeting within EMDA are done at the HQ level with little or no involvement of field staff. The field office prepares an annual implementation plan, broken down by month, which it shares with HQ at the beginning of the year, but they do not have information on the annual budget they are assigned. Fund flow is not regular. In principle, funds should be transferred to district offices on a monthly basis, but at times the funds flow after 4 to 6 months. The delay in funds affects implementation, which sometimes requires the Field Coordinator to liquidate the delayed fund transfer within a short time. The causes of delays include:  Lack of planning capacity within EMDA HQ to request cash in advance;  Shortage of human resources within the finance department of EMDA HQ;  Lack of financial management capacity and overall human resource shortage within EMDA district offices;  Incomplete reporting by EMDA; and  Delays in the approval of financial reports by the EIASC. The evaluators found that the district offices often are faced with dire funds shortages. Often the office runs out of funds needed for photocopying reporting forms and utility expenses and is forced to ask for support from the EIASC Branch Office (Mejlis). In addition, there is a limited budget to pay transportation and subsistence allowances for trainees. The scarcity of funds also limits the number of visits the Regional Coordinator conducts to the field because of a lack of funds for fuel and maintenance for the motorbike. When funds are transferred, a fax message is sent with a detailed disbursement plan. A copy of this fax message is give to the EIASC branch office for counterchecking. As the money is disbursed, the Field Coordinator compiles a financial report as per the 29 communicated detailed financial plan and also attaches supporting documents such as invoices and training attendance sheets. The compiled report is verified by the cashier of the Mejlis and the original copy is sent to HQ with copies kept in the branch office and the EIASC district office. Posting of entries, using Peachtree software, is done at the HQ level and, when the Field Coordinator comes to Addis, he confirms entries. The HQ report is sent to PACT within one to two months, based on the use of disbursed funds. The financial report submitted by EMDA HQ gives details of budget expenditures by project line item and shows expenditures against receipt. In addition, the report also includes a cash request for the next period. The funds utilization rate in EMDA is not so bad given the relatively weak financial management system organization wide. As indicated in Table 5, between 2005 and 2008, budget utilization has averaged at 98% for M-ARCH and 80% for EMDA as a whole. However, there have been large differences between the years. Table 5. Budget Utilization Rate, EMDA Budget utilization rate M-ARCH Project (PACT) (%) Organization wide (%) 2005 101 98 2006 88 88 2007 148 117 2008* 73 34 Average 98 80 * Consumption for 2008 captures the period until July 2008 Cash, Inventory, and Asset Management System. EMDA district offices do not have checking accounts; hence, all money transfer for the office is made through the personal bank account of the Field Coordinator. This is a procedure that opens room for financial mishandling. At the HQ level, EMDA uses a checking account and signatories are two of the three persons authorized to sign: the EMDA Director and two of the Development Sector Heads from EIASC. At the HQ level EMDA keeps petty cash up to ETB 2500 with its cashier. The money is deposited in a safe box and advance is registered in the petty cash register. No petty cash is kept at the district office level. Whenever purchases are required, the Coordinator withdraws money from his personal bank account for the payment. When office supplies are to be procured, the EIASC district office is informed and a copy of the purchase request sent. After procurement, invoices and other supporting documents are sent to the EIASC district office. However, office supplies are not recorded, although there are records for fixed assets that are purchased. Most district offices deal with procurements that involve a small sum of money. The exceptions have been those offices handling sums for building rehabilitation. When this is the case, Field Coordinators set up a procurement committee composed of the Coordinator and representatives from the EIASC district office to help with the selection 30 of the service provider through the evaluation of three proforma bids. Service provider selection decisions are documented rapidly, and a contract is signed. At the HQ level there is a procurement committee composed of Head of the Development Sector in EIASC, Head of EIASC Administration and Finance, EIASC finance person, and accountant of EMDA. Procurements below ETB 500 are done through direct purchase. For all procurements between ETB 500 and ETB 50,000, EMDA HQ collects a minimum of three proforma bids from prospective suppliers, which are reviewed by the procurement committee. Purchases above ETB 50,000 are carried out through open tendering procedure, although this level of expenditure is rare. Assets at the HQ level are registered and labeled and the registry is kept at the EIASC level. However, there is no proper asset registry and labeling at the district office level. Audit and Reporting. The Field Coordinators report to HQ on activities and expenditures, although the pattern may be uneven due to delays in fund release. Since the chart of accounts is set up to respond to the need to have separate donor report and also detailed report by project line item, EMDA HQ does not face a challenge to meet the donors’ reporting requirement. EMDA HQ commissions external audits every year. Audit reports are sent to donors, Ministry of Justice, Disaster Prevention and Preparedness Agency (DPPA), EIASC, EMDA Finance Head, and Director of EMDA. Financial Sustainability. EMDA HQ deals with an average of five donors every year. Most of its donors provide short term support. As a result, the organization’s income fluctuates over the years (Table 6). EMDA has not been able to leverage a significant amount of resource proportional to the size of its constituency. The organizational budget for 2008 is comparable with that of Rohi Weddu which only works in Afar region. EMDA does not have local income generation schemes. Table 6. Financial Sustainability, EMDA Year M-ARCH Project (PACT) (A) Total EMDA annual income (B) Share of PACT funding (%) (C = A/B) Share of local income generation (%) (D) 2005 1,132,764 3,996,050 28 0 2006 1,320,817 2,975,579 44 0 2007 701,656 1,645,021 43 0 2008 1,022,966 3,461,176 30 0 Average 36 0 Currently, EMDA does not have any donor committed to provide it support beyond 2008. This short term natured financial base threatens to upset the capacity built under the M-ARCH project. 31 Some district offices have attempted to raise funds from local sources by writing a proposal to the Bureau of Health, HAPCO, and Youth and Women’s Affairs Office with no success so far. However, the youth clubs that are organized at the district level normally have their own income generation schemes, such as a cafeteria, and/or club that charge a very small amount of money to non-members who attend organized football competitions and drama events. EMDA has been successful in generating in-kind contributions from project communities and local government. Even though the M-ARCH project does not include assistance to PLWHA and OVC, the volunteers and PAC have generated low levels of support for the needy. In one case, the evaluators found that they had secured land from the local government for a youth group cafeteria and a library/meeting room. 2. Rohi Weddu Financial Operation Manual, Supervision and Support. As a result of the support from Pact, RW has developed a detailed financial management manual which covers various dimensions of financial matters. A copy is kept in RW’s Account’s office and the accountant is well versed with procedures described in the manual. Although RW receives supervisory visits from Pact every quarter, it no longer receives much attention in regard to financial management. Four of RW’s staff including the manager, were trained by Pact in financial management. Pact also provided on-the-job mentoring to staff on the financial management manual developed with its help. The trainings and mentoring have helped the organization better organize its financial management system and make it systematic and efficient. Rohi Weddu claims that Pact is the most supportive of all its “donors.” The reluctance of donors to pay the salary of finance personnel is one of the challenges RW faces. Until recently 100 percent of the finance person’s salary was covered by the M-ARCH Project, but recently RW has been able to shift 25% to other projects which they claim as a success. Budgeting and Fund Flow. Rohi Weddu prepares a plan and budget for its programs each year, in line with the projects it has with donors. The plans are prepared with the participation of staff. RW does not face financial shortages. The challenge the organization faces is delay in funds release in January, which is the start of the fiscal year for most of its donors as well as for RW. During this period RW relies on the small amount of income it generates from local sources (i.e. members’ and staff contribution) and funds carried over from the previous year to fill gaps during this period. RW has been using Peachtree accounting software for recording its financial transactions since 2005, before Pact’s support started. Its finance staff was trained in the full package of Peachtree accounting with support from Pact. However, there has 32 been a high turnover of accountants. The current Accountant who joined RW three months ago has not yet been formally trained in Peachtree but is using his prior knowledge of the software from school, and reports that he does not face major problems. The organization prepares a monthly financial report that compares budget against expenditure, which is assessed internally, and action taken when need arises. In addition, quarterly financial reports are prepared, which gives RW an opportunity to review last quarter’s performance and plan for the new quarter. This strong follow up is reflected in the funds utilization rates, shown in Table 7, which averages around 87% organization wide and 88% for the M-ARCH project in particular, taking into consideration the utilization rate for 2008 which is only for six months. Table 7. Budget Utilization Rates, Rohi Weddu Budget utilization rate M-ARCH Project (%) Organization wide (%) 2006 79 87 2007 122 86 2008* 62 54 Average 88 87 * Utilization for 2008 captures disbursement until July 2008 while the fiscal year ends in December Cash, Inventory and Asset Management System. RW follows good, sound procedures for managing its petty cash fund of a maximum of ETB 500. Above this amount, disbursements are made by bank check which is signed by any two of the Manager, the Accountant and the Program Coordinator. Petty cash advance is well recorded. However, there is a flaw because the accountant preparing the check should never be a signatory because it opens room for financial mishandling. Procurement up to ETB 2500 is done through direct purchase, while purchases between ETB 2500 and ETB 50,000 is done through collection of proforma bids from their suppliers. The RW procurement committee, which is composed of the Program Coordinator, Cashier, and Accountant, then makes the selection. Financial transaction at the district level is limited to minor expenditures which are settled at the end of each month. Field Supervisors collect their salary from the HQ when they attend their monthly meeting with the Zonal Supervisor. Inventory is kept in a store and the accountant uses various vouchers to record incoming and outgoing inventory (goods receiving and goods issuing vouchers). In addition, assets are well registered and labeled in Rohi Weddu. Audit and Reporting. As noted previously, RW prepares a monthly financial report and quarterly reports are submitted to donors and government project signatories (Disaster Prevention and preparedness Agency (DPPA), Women’s Affairs Bureau, and Health Bureau of the Region) by the 5th day of the next month. RW reports to donors using the required form and uses a standard reporting format for government signatories. The way the organization has set up its chart of accounts enables it to keep accounts of 33 each donor separately and also by program area; hence, there is no difficulty in tracking expenditure by source of fund or by program area. RW also commissions an external audit each year by certified accounting firms (report seen), which it sends to the same group of stakeholders. Financial Sustainability. Though it is a relatively short time since RW was established, it has been able to diversify its financial sources and strengthen its financial capacity significantly (Table 8). The organization has a good capacity in terms of proposal writing and has been able to win the trust of various donors. As a result, RW has increased its total budget from about ETB 900,000 in 2006 to almost ETB 3 million in 2008. Currently, the organization is benefiting from funds from seven organizations with which it has long term project agreements. Presently it has agreements for funding up to 2012. Table 8. Financial Sustainability, Rohi Weddu (Ethiopian birr) Year M-ARCH Project (PACT) (A) Total Rohi Weddu annual income (B) Share of M-ARCH funding (%) (C = A/B) Share of local income generation (%) (D) 2006 275,420 897,136 27 ~0 2007 290,534 1,521,101 18 1 2008 768,861 ~3,000,000 ~26 NA Average share 24 In addition, Rohi Weddu is attempting to diversify its income source by engaging in local level income generating activities. Currently it gets a small amount of funds from member and staff contributions. RW is presently in the process of acquiring land to construct workshop/meeting centers and guesthouse to generate rental income. In addition, it is finalizing preparations to initiate a community fund with the idea of “1 person contributing at least 1 goat” with the plan to sell the goats to generate funds. D. Technical Capacity and Growth 1. Training, Mentoring and Exposure Visits To build the technical capacity of EMDA and Rohi Weddu, Pact Ethiopia’s approach has been to provide training to partner staff, mentoring, and exposure visits. Training of partner staff. Pact-Ethiopia played a key role in facilitating, conducting and providing training to build the technical capacity of EMDA and RW staff. It provided a number of training workshops on organizational and technical capacity building for partner staff. The technical training included topics such as effective advocacy skills, community conversation approach, and communication. Trainers have been drawn from Pact Ethiopia and, where necessary, consultants were hired through a competitive bidding process. Because trainees for most courses had different levels of capacity, background and work experiences, Pact felt that it was inappropriate to administer a pre- and post-training assessment tool. Rather, at the end of training, participants were asked to evaluate the training, using a format that 34 contained two items related to the training content: important knowledge learned and what they would like to learn in the future. (Pre- and post-test tools were used in some training courses on organizational topics, like strategic planning resource mobilization and sustainability.) Training in the Community Conversation Approach was initially conducted by consultants as a TOT. The one-week training was attended by twenty four participants, out of which ten were staff associated with the M-ARCH project (four staff from EMDA, two from Rohi Weddu, and four from Pact). Consultants developed the training manual and implementation guidelines for trainers’ teaching material. A summary of participants’ evaluation results noted that participants felt that they had obtained the opportunity to know what is being done at community level with the use of community conversation methodologies. Following the TOT in the community conversation approach, a consultant, and two co￾facilitators from Pact trained twenty-four staff (fourteen from EMDA and ten from Rohi Weddu) for five consecutive days. Training on other topics did not use a TOT approach to train community level educators. Instead, the training for imams, women to women peer promoters and youth peer promoters (male and female) organized by EMDA normally involved mobilization of local relevant trainers from the health sector and district HAPCO. Mentoring. Pact-Ethiopia realized that participating in a technical training course does not always provide someone with the necessary skills and confidence to use the information to accomplish the planned tasks. Therefore, one approach Pact-Ethiopia followed in this regard has been improving the technical capacity of field level staff through mentoring. This is accomplished in a planned manner whereby a staff member from Pact spends a few days at a field site to assist staff and provide feedback. Exposure Visits. Exposure visits were used by Pact to build capacity through sharing best practices and holding discussions on challenges in order to learn about solutions. Two exposure visits were organized by Pact: Jimma (in 2006), and Harar and DireDawa (in 2007). The visit to Harrar and DireDawa involved total of thirty participants (eighteen from EMDA, nine from Rohi Weddu and three from the Ogaden Welfare Development Association9 representing technical staff, religious leaders and Anti AIDS club leaders. The visit to Harar created an opportunity to see and learn best practices used by community level volunteers, like the methodology used by Muslim women to teach followers on HIV/AIDS, and a coffee ceremony (an IEC/BCC approach to discussing prevention and other HIV/AIDS-related topics). The evaluation forms completed by study tour participants indicate that they benefited from learning about successful approaches. 9 The Ogaden Welfare Development Association (OWDA) had been included in M-ARCH, but due to political unrest in the Ogaden Pact (in agreement with OWDA) de-obligated activity grant funds. However, the organization was invited to participate in technical and organizational capacity building inputs provided by Pact to M-ARCH grantees. 35 2. Findings from the Focus Group Discussions Imams Four FGDs were held with a total of 29 Imams from three EMDA sites and one RW site. Participants had been trained on modes of HIV transmission, HIV/AIDS prevention, home counseling, use of ART and ART adherence, VCT promotion (especially pre￾marital testing), religious messages promoting AB, and reduction of stigma and discrimination. Although the topics in which they were trained were the same, the duration and the year the imams received the trainings varied greatly between sites Except for five imams in the EMDA DireDawa area who received training prior to the Project, the others had been trained between 2004 until one month before the evaluation (in the case of Rohi Weddu). It was also found that Imams, especially in the EMDA Sokoru area, had received an initial five day training followed by two to three days of refresher training every year until 2008. With the exception of imams from the RW program area, most participants agreed that they did not always feel confident enough to educate others and respond to questions. The three most frequently mentioned topics the imams said they would like to learn more about were: updated HIV information/data, how to look for material and give psychological support, and how to teach followers about taking ART and fasting. All FDG participants reported that they had received an adequate amount of materials and a manual, which were written in different languages, including Arabic. The key message passed by all Imams towards promotion of abstinence was “Away from “Zina”, meaning “Infidelity.” They convey AB messages in the daily mosque prayer, and Friday big congregation of prayer as well as during the Saturday Imam program (for women alone) and the Sunday program (men alone). Each group draws about 40-60 followers. Imams also make home visits to talk with couples and convey AB messages in local gatherings and Muslim religious schools. All Imams encourage their followers to go for VCT service, especially those preparing for marriage, although most do not follow-up on the test results. When they are aware of a follower who is HIV+, most conduct spiritual counseling, advise community and family not to stigmatize, look for economic support for needy ones, encourage the person to take ART and visit the person at home. In the RW FDG, three Imams are providing ART adherence counseling to a total of fourteen PLWHA. In Jimma area, half of the Imams reported that stigma and discrimination has been a problem in the area, whereas in other areas the FGD participants reported that stigma and discrimination is almost nonexistent. All Imams associated with the EMDA program reported that they received guidance and support from their Field Coordinators, largely through review meetings normally held semi-annually and which included a discussion of challenges encountered. In 2008 alone, some areas like DireDawa had held three review meetings. According to FGD participants, the meetings led to improvements in their technical capacity, especially their teaching methodologies. 36 Not all challenges have been able to be addressed within the program’s capacity. Sokoru District has a large population with a high risk of HIV/AIDS due to the presence of large government projects in the area like “Gilgel-Gibe” hydro electric power: these have attracted workers from inside and outside of Ethiopia along with women who offer transactional sex. The low amount of program funds and the size of the risk to the local population make this a frequent topic for discussion. All program-trained imams are expected to submit a report on their activities and numbers reached. The FDGs revealed that the frequency of the reporting varied from a weekly basis to two months. Women-to-Women Peer Promoters. Three focus group discussions with a total of 21 women-to-women peer educators were conducted: one with RW promoters and two with EMDA promoters. The majority of the participants were trained for five days in 2008. The topics most frequently mentioned were: harmful traditional practices (female circumcision, early marriage), as well as modes of HIV transmission and preventive methods, stigma and discrimination, Koran and HIV messages, parental support, VCT, PMTCT and ART. All FGD participants stated that they are comfortable with the knowledge they acquired from the trainings. However, they suggested topics like FGM, MTCT, care to OVC and teaching methodologies/skills as topics for conducting community conversations for future trainings. No training manuals have been distributed to the volunteers in Dire Dawa, whereas the RW volunteers had received a HIV/AIDS teaching guide prepared in Amharic. Two suggestions were provided for improvement of future training session. First, they would like to receive training manuals prepared in different languages (Afar & Oromiffa). Second, the participants suggested that women be trained apart from the men, rather than together, because the presence of males in the training limits the willingness of some families to permit women to attend. All of the FDG women to women peer promoters carry out the following activities:  Promote AB through home visits to couples and small groups;  Encourage, refer or fetch followers for VCT service;  Verify testing and prepare couples; and  For those testing HIV+, provide spiritual counseling, encourage them to go for ART and refer to NGOs for economic and other support. Participants reported regular supervision, guidance and support from their program’s Field Officer. Based on the reporting formats provided during their training, the majority of participants report their activities every month and few of them every two weeks to the supervisor in RW. 37 Youth Peer Promoters (Male and Female). Four focus group discussions with a total of 20 participants (both male and female) were held in EMDA sites. All of them had received five days of training, which covered HIV/AIDS transmission and prevention, VCT, stigma and discrimination, and gender issues. They would like to receive training on sexual reproductive health issues, detailed information on ART, VCT and care and support. With one exception, participants had received brochures and pamphlets but no manual on peer education. The youth peer promoters provide IEC to students in Madrassa during an after-school program, visit families in need of HIV information, and conduct educational activities, such as drama shows and coffee ceremonies. The participants reported regular and sometimes weekly interaction with the Field Coordinator. They had no difficulty completing the monthly reporting forms that they submit on their activities. E. Coordination and Relationships with Other Organizations In addition to Pact-Ethiopia, EMDA has established relationships with a number of organizations working on HIV/AIDS: HAPCO, Action Aid Ethiopia, CRDA, Population Council/USAID, Save the Children and Global Fund are the prominent ones. Networking between field staff and community level educators existed with a few organizations like Save the Children related to OVC support, HAPCO on Community Conversation, government health centers for obtaining BCC messages, and sub-district administration to search for homes for PLWHA, and police administration in cases related to stigma and discrimination. Imams in DireDawa and Sokoru areas mentioned some organizations with which they have interacted, such as being a guest teacher. Community conversation is carried out by several Imams in collaboration with HAPCO on a monthly basis. In comparison, among the recently trained Imams in RW, most know other organizations working on HIV/AIDS, although none of them have had communication, interaction or working relationships. The Rohi Weddu Field Supervisors have established relationships with government officers and participated recently in the Government VCT campaign. One field supervisor reported that as a result of the campaign, 7,000 people were tested.. The Program Officer is a participating member of the district council and has raised the awareness of the regional health office, as well as the district council, about issues related to lack of access to VCT services. At a one-person VCT staffed center, the individual is frequently away from post and hence the center closed, and in another major geographic area there is no VCT center. 38 F. Conclusions  Although there are structural and governance issues at the higher level of EMDA, it has developed a solid sub-structure at the district level, support by Project Advisory Committees, composed of religious leaders and government officers.  Pact’s work with its partners has centered a great deal on organizational aspects, leading to increased knowledge, skills and procedures as well as strategic plans and, in the case of RW, a proposed reorganization structure.  Pact partners received training in monitoring and reporting, and established a program M&R system under Pact guidance. The reporting forms that volunteers submit on their activities has led to double counting of individuals receiving IEC/BCC from the same volunteer during an annual plan period. Although the reported number of persons reached through IEC/BCC includes double counting, there is no doubt that large numbers of people have been reached.  As a result of Pact mentoring and training, gender considerations are reflected in the EMDA program and activities conducted by peer educators  The financial management training and mentoring provided by Pact has contributed to the building of financial management capacity of the EMDA HQ and district staff. The governance structure of EMDA plus its financial situation put at risk the sustainability of the advances made.  EMDA’s centralized system of financial management has affected program efficiency. Delays in financial reporting by EMDA have caused delay in releasing funds by Pact. The delays in reporting have affected the efficiency of financial utilization as there is pressure to liquidate delayed funds within a short period of time.  Strong financial management systems have been evidenced at EMDA HQ level, but not at the district office level. Evaluators found that district offices did not have checking accounts and all money is transferred to, and kept in the personal account of the Field Coordinator.  EMDA as an organization has not yet been able to ensure its financial sustainability. It does not engage in any local income generation schemes; neither does it have donors willing to commit to multi-year funding.  Rohi Weddu has evolved to be a financially strong and sustainable organization. Its small size, governance structure, and openness to improvements has helped it to benefit greatly from the support it received through the M-ARCH Project.  The results of the FGDs indicate that volunteers and religious leaders have received training in technical aspects related to HIV/AIDS, and most feel confident in being 39 able to carry out their work, even though they would like additional training on specific topics. Also, they have been provided with written pamphlets, brochures and, for the imams, manuals, which have been translated into a number of major languages.  EMDA and RW have addressed traditional practices that risk spreading HIV/AIDS. Knife sharing during circumcision ceremonies has been reduced through mobilizing imams and key community stakeholders (RW) and women-to-women peer educators (EMDA) address practices, like sharing of razors and knives.  Although the Project focuses primarily on prevention IEC/BCC, the community level volunteers have also been engaged in assistance to and/or referrals of PLWHA and OVC to other organizations.  At the local level, community educators have relationships with different organizations working on HIV/AIDS. These linkages have been based on ‘need’ rather than being systematic. Also, local level and HQ staff have relationships and contacts with other organizations working on HIV/AIDS, often based on need. G. Major Achievements  Project training and activity grants to EMDA have contributed greatly to recognition by Muslim leaders that HIV/AIDS exists in their communities and that they have a role in addressing the subject. This has resulted in more open discussion about HIV/AIDS and community mobilization to support OVC and PLWHA. Silence on talking about HIV/AIDS in Mosques is broken and Imams began conveying HIV messages openly. Previously, it used to be taboo since HIV was not considered as a problem for Muslim populations.  Use of VCT services has been promoted through Imams being tested in a videoed public forum (EMDA), Imams often require test results of couples wishing to be married (RW), and involvement of staff in a government VCT campaign (RW).  Both EMDA and Rohi Weddu have been successful in strengthening their financial management system at the HQ level through the capacity building support from Pact.  At the community level, EMDA has done an excellent job of generating additional resources from key leaders, business people and general community (to provide food and material assistance to needy PLWHAs and OVC) and from HIV/AIDS related organizations for PLWHAs and OVC, and training of volunteers. This was accomplished through community mobilization, the work of the PACs, and the Field Coordinators. 40 H. Recommendations  Prior to completion of the project, EMDA and EIASC should undertake a facilitated internal participatory analysis of EMDA’s organizational strengths and weaknesses, such as structure, governance, and ability to plan, manage and report on donor funds. Thereafter, it should make changes to reduce limitations to its income generation capacity. In addition, EMDA should encourage and support initiatives by its district offices to generate revenue from local sources.  Prior to project completion, Pact should assist EMDA to institutionalize its financial policies and procedures at the district level. EMDA district offices should immediately open checking accounts in the name of the organization and should identify check signatories.  A deficiency is evidenced in the use of checks at Rohi Weddu whereby the accountant, i.e. one preparing the check is also a signatory. This opens room for financial mismanagement and hence should be addressed immediately.  Rohi Weddu should establish a system whereby organizational memory and skills are transferred on a regular basis by core staff in order to address the skills gap that results due to the high staff turnover, especially accountants. III. GENERAL CONCLUSIONS, LESSONS LEARNED AND RECOMMENDATIONS FOR FUTURE PROJECTS A. Conclusions  The two projects working with faith-based organizations have highlighted the need for attention to organizational capacity building, especially among organizations that are in their development stage, such as EMDA and RW. The positive results of Pact’s work with Rohi Weddu indicates that great progress can be made when the organization recognizes the need to strengthen its organizational capacity and is structured in such a way to act on this.  Both the FBO projects have been involved in strengthening the financial management capacity of their partners. The results have been positive, within the confines of the organizational structures and procedures of DICAC and EMDA. Retention of trained persons is threatened when the salary scales of the FBOs lead to relatively high turnover of persons trained. In some cases, this is causing loss of organizational memory.  Neither project proposal contained a good results framework and no actions have occurred to correct this. This affects a project’s ability to establish a sound monitoring, evaluation and reporting system, and has not contributed to building the capacity of organizations to develop good results frameworks. 41  Both IOCC and Pact provided monitoring and evaluation, training and mentoring, based on the project monitoring and reporting system each established. It included data recording and reporting formats for community volunteers and religious leaders, which are the basis of reports submitted by the field offices. However the Pact data recording and reporting forms for community volunteers led to double counting by the same volunteer, and some of the forms used by volunteers from the IOCC/DICAA program are also susceptible to double counting. Also, the reports submitted by IOCC to USAID indicate problems with accounting for OVC which have directly received three or more core program services, in accordance with PEPFAR guidance. In addition, neither project maintains a master list for those trained nor persons served/reached. The evidence suggests deficiencies in USAID/E’s system for reviewing and assessing data quality in accordance with PEPFAR guidance, as well as weaknesses in the projects’ monitoring and reporting systems.  The field structure of program activities appears to be related to the strength of programs at the district level and below. With a paid staff member and Project Advisory Committee with a proportionally large number of key stakeholders including government officers, EMDA has been very active with outreach into the community, community mobilization, and establishment of linkages with other HIV/AIDS related organizations.  Involvement of program-trained religious leaders, together with the activities of community volunteers, in HIV/AIDS prevention activities has led to greater awareness about HIV/AIDS and a willingness to openly discuss it, promotion of VCT services particularly pre-marital testing, and greater community support for PLWHA and OVC.  Based on the qualitative field findings, it is apparent that both projects have increased awareness and knowledge about how HIV/AIDS is spread and ways to prevent infection (AB and, among Pact partners, traditional harmful practices) among a large number of people. This is due to the involvement of religious leaders and community based volunteers, and is changing social norms, a key factor in behavioral change.  Networking across organizations involved in HIV/AIDS related activities tends to be informal at the district level. At regional and national levels, there are formal NGO and faith-based networks in which the local partners participate.  For technical training, both projects preferred an end-of-training evaluation approach, rather than administration of a more objective method using a pre- and post-test approach for training staff and trainers. However, the end of training evaluations completed by participants inadequately addressed or omitted asking about learning, such as the trainees’ confidence level in using the knowledge taught. 42 B. Lessons Learned  Building organizational capacity within headquarters and at the field level can occur despite difficulties within the organization that are beyond the manageable interests of Lead Partners. However, the organizational capacity building efforts can be more cost-effective and have a deeper impact if the organization does not have difficulties at HQs and with decision-making bodies above its Board, which hinder its ability to act and adopt improvements.  Networking at grassroots level across programs and organizations brings greater efficiency in addressing the needs of OVC and PLWHA and in filling in program/project gaps.  Having received good training, the skills acquired by those providing Home-Based Care are sustainable.  Donor projects which incorporate organizational, financial and technical capacity building/strengthening to their implementation partners/sub-partners are instrumental in strengthening the civil society sector’s ability to respond to HIV/AIDS and other issues.  Programs should include refresher and additional training for community volunteers. Those engaged in IEC/BCC as well as care and support of PLWHA and OVC need to be well grounded in topics central to promotion of health facility services and ART. C. Recommendations to USAID for Future Projects The following recommendations are based on the evaluation results as well as best practices.  Applicants should be asked to present a results framework (or log frame) and budgets should be presented by intermediate results, as well as key standard cross￾cutting categories such as staff and staff support commodities. USAID should request annual action plans and annual expenditure reports in line with these budget categories.  Future projects with FBO partner organizations in the ‘development stage’ should center primarily on prevention (particularly BCC), reduction of stigma and discrimination, and promotion of VCT and PMTCT services, along with organizational (including financial) and technical capacity building. Gender and involvement of PLWHA should be cross-cutting themes. Relatively low targets in the areas of OVC and PLWHA could be included for partner sites with capacity already built in the areas mentioned previously or in the last half of the project. A staged approach should enable organizational capacity to be built in districts prior to expansion of implementation responsibilities. 43 o In each proposed site, a rapid assessment related to knowledge and attitudes might be undertaken on HIV/AIDS, VCT, and PMTCT and on indicators of stigma and discrimination in general and within the household. The results would be used to help guide decisions (budget and strategy) for general IEC and BCC. o A cluster approach should be used to select organizations based on location of their HQs and selection of implementation sites to increase the cost￾effectiveness of supervision and mentoring. Also attention might be given to HIV/AIDS high risk geographic sites.  USAID should fund a project that specifically addresses HIV/AIDS high risk geographic sites where there is an influx of workers from outside the area (for road construction and dam construction), which attract women involved in transactional sex. The project could work with employers to reach the workers, and go into communities to reach the transactional sex workers.  During the annual planning process, budgets, activities and targets should reflect evidence-based costing of activities and supportive elements. Attention should be given to support for volunteers and enable them to carry out their work (e.g. adequate transport allowance, reasonable level of per diem to cover participation in training outside the area where they live, raingear and other basics).  Using a system of quarterly (rather than monthly) funds release and settlement might increase financial utilization efficiency, especially when working with emerging partners that operate with a large geographic coverage and follow a centralized system of financial management.  The capacity of partner organizations should be strengthened to diversify income sources, specifically undertaking income-generating enterprises or activities in line with the new NGO proclamation by the Government of Ethiopia that requires local NGOs to generate a portion of their income from local sources.  After project award, the appropriate person in USAID should help ensure that data gathering and reporting forms enable reporting on PEPFAR indicators according to its guidelines. o The project M&E training should focus on establishment of a sound M&E system, prior to more general training on M&E. o The budget ought to provide ample funds for setting up a sound monitoring, evaluation and reporting system, and addressing reporting gaps and data quality during project implementation. 44  USAID’s new Field Supervisors of HIV/AIDS projects (especially community-based projects in contrast to facility based) and other key staff should receive intensive training on PEPFAR guidelines, different ways that monitoring and reporting systems can be established, so that they can develop and adhere to these. Examples of good data reporting forms by those providing services and BCC outreach, and procedures and approaches in conducting data verification assessments should be provided. o The training might be provided by a knowledgeable, experienced person in REDSO/ESA or a knowledgeable consultant with such experience in the region. 45 ANNEX A. List of Persons Met Full name Position Organization Brad Corner Senior Advisor USAID Karen Towers Interim Evaluation Coordinator USAID Cynthia Shartzer Agreement officer USAID Jamie Browder Logistics and Site Readiness Cluster Coordinator USAID Kibru Mamusha M & E Advisor USAID Kassahun Demeke SI Advisor/Health Team USAID Laurie Rushton Social Mobilization USAID Cassandra Champman Deputy Country Representative Pact Ethiopia Tesfaye Yimer Prog.Officer CB & M-ARCH project Pact Ethiopia Tamiru Lega M-ARCH & CB program manager Pact Ethiopia Fikrte Bekele Health Program director Pact Ethiopia Tenaw Mengist M&E officer Pact Ethiopia Samson Oli M&E officer (out going) Pact Ethiopia Mulumebet Jember Finance Director PACT Ethiopia Addis Getachew Grant Officer Pact Ethiopia Frehun Tesfaye Accountant Pact Ethiopia Abdulhafez Kemal Acting Program director EMDA HQ Haji Teshale Kero Barkea Head of EIASC Relief & Social Development Dept EIASC Haji Mhbube Mohammed Vice Head of EIASC Relief & Social Development Dept EIASC Haji Nuredin Jemal Program officer EMDA HQ Musema Fereja EMDA M-ARCH Project coordinator EMDA HQ Fethudin Ibrahim Project Accountant EMDA HQ Abdizein Yusuf Coordinator EMDA Dire Dawa Zekaria Abdi Ex-Coordinator EMDA Dire Dawa Sherif Abagelan Regional & Jimma coordinator EMDA Jimma Kedir Dekebo Coordinator EMDA, Seqa.Cheqorsa Bedru Abagero Head, Council EIASC, Seqa Cheqorsa Sheik Abbas Abamugi Head of Imams Meti Kebele, Seqa.Che. Mohammed Jemal Coordinator EMDA Sokoru Asmelash Woldemariam Executive Director Rohi Weddu Fatuma Hate Program Coordinator Rohi Weddu Jilani Field Coordinator Rohi Weddu Said Tuba Accountant Rohi Weddu George Antoun Regional Director IOCC Haregewoin M. Desta Proj Officer & Acting Country Rep IOCC Aynalem Tefera Prevention officer IOCC Wolela W/Gebriel Chief Accountant IOCC Solomon Hailu HIV/AIDS Department Head DICAC HQ Binyam Woldu M&E officer DICAC HQ Zelalem Assefa M&E officer DICAC HQ Abebe Demisse Finance Head DICAC HQ 46 Yohannes Ambaw Accountant DICAC HQ Almaz Ayele Accountant DICAC HQ Solomon Ayew Senior Accountant DICAC HQ Derese Kassu DICAC Program Director DICAC HQ Getu Motie Branch Coordinator DICAC Adama branch Mulu Gebre Assistant Branch Coordinator DiCAC Jimma branch Samuel Wondimu Focal Person DICAC Sokoru Woreda Tedla Teshome Vice-Chairman DICAC Board DICAC 47 Annex B Participants in Evaluators’ Debriefing Sessions Leslie Mitchell Pact Country Representative Fekerte Belete Pact Health Program Director Cassandra Chapman Pact Deputy Country Representative Tenaw Mengist Pact Monitoring, Evaluation & Reporting Officer Mussema Fereja EMDA National Pact Project Coordinator Tesfaye Yimer Pact Program Officer Asmelash W/Mariam Executive Director from Rohi Wedu Tamiru Lega Pact Program Manager Nuredin Jemal Program Coordinator EMDA Haji Teshale Kero Head Development Sector, EIASC/EMDA Participants in Debriefing Session for IOCC and DICAC Nigussu Legesse (Dr) DICAC Commissioner George Antoun IOCC Regional Director Solomon Hailu EOC/ DICAC Head of HIV/AIDS Dept. Zelalem Assefa DICAC HIV/AIDS Dept M & E desk officer Aynalem Tefera IOCC Prevention Officer Zelalem Assefa DICAC HIV/AIDS Dept M & E desk officer Aynalem Tefera IOCC Prevention Officer Haregewoin M. Desta IOCC Project Officer Wendwesen Demisie EOC/ DICAC Prevention Desk Name Position Participants in Debriefing Sessions for USAID Glen Anders USAID Director Nancy Estes USAID Deputy Director Karen Towers Interim Evaluation Coordinator Laurie Rushton Social Mobilization Alia El Mohandes Facility Based Care Cluster Coordinator Jennifer Foltz TDY Dawit Abraham QA Specialist Kassahun Demeke SI Advisor/Health T Haile Meskel Balcha Nutrition Advisor Steven Neri HCT Melissa Jones HIV Team Leader Kibru Mamusha M & E Advisor Brad Corner Senior Advisor, FBO projects CTO Henok Amenu A & A Specialist Walelign Mehretu Health Program Network Advisor Garoma Kena Health Network Program Advisor Edson Muhwezi Pediatric HIV/AIDS advisor Jamie Browder HIV/AIDS Logistics and Site Readiness Cluster Coordinator Abeje Zegeye Care & Support Advisor Anteneh Worku SI Advisor Tatek Wondimu Program Advisor Participants in Debriefing Session for Pact and its Partners 48 Annex C Field Trip Schedule Date Sites visited/Type of organizations Accomplishments/activities carried out Mon Aug 11/08 Adama branch office/DICAC Interviews with Branch Coordinator and Assistant Coordinator/Accountant -FGD with trained clergy -FGD trained HBC volunteers Tue Aug 12/08 Awash/ Rohi Weddu HQ Interview Rohi Weddu staff: Executive Director, Program Officer, Field Coordinator, finance officer - FGD with trained Women to Women peer educators - FGD with trained Religious leaders/IMAMS Wed Aug 13/08 Dire Dawa/EMDA Interview EMDA Field Coordinator -FGD with Women to Women educators & Female youth peer educators - FGD with trained religious leaders/IMAMS Thu.Aug 14 /08 Jimma Branch Office/ DICAC DICAC Jimma Diocese Office Coordinator, Assistant Coordinator/accountant - FGD with Para counselors and counselors - FGD with trained HBC providers Morning Shebe/ DICAC - FGD with Youth peer promoters - FGD with Woreda PAC members Fri Aug 15/08 Afternoon Seqa Cheqorsa/EMDA EMDA Field coordinator - FGD with women to women educators and female peer promoters - FGD with trained religious leaders/IMAMS - FGD with trained male peer promoters Morning Sokoru/DICAC Meet with DICAC focal person - FGD with youth peer promoters - FGD with trainer Clergy Sat Aug 16/08 Afternoon Sokoru/EMDA Meet with EMDA Field coordinator and Jimma Regional Coordinator - FGD with youth Peer promoters - FGD trained religious leaders/IMAMS 49 Annex D. Number and types of FGD participants per interview site Number of FGD participants and interview site Types of participants Dire Dawa Seqa Chekora Sokoru Awash Total Imams 9 4 8 - 21 Women to Women 5 8 - - 13 YPP (Male) - 3 9 - 12 EMDA YPP (Female) 4 4 1 - 9 Imams - - - 8 8 RW Women to Women - - - 8 8 Field supervisors +Zonal Supervisor - - - 7 7 Adama Jimma town Shebe Sokoru Total Clergy 9 - - 9 18 Home based care providers 9 8 - - 17 Para-Counselors/counselors - 4 - - 4 Youth Peer Promoters - - 10 8 18 DICAC PAC Members - - 7 - 7 YPP=Youth Peers promoters, RW= Rohi Weddu, 50 Annex E. EOC-DICAC Organigrams 1. EOC-DICAC Structure HOLY SYNOD 51 2. DICAC HIV/AIDS Prevention and Control Department Structure - Senior Secretary - Secretary Pre Department Head Programs Management Division Head Branch Coordination Division Head Psychosoci al Division Head IEC/BCC Division Head Monitoring & Evaluation Desk Officer Branch Coordination Assistance Care and Support Desk Officer IEC/BCC Desk Officer Data Management Desk Officer Data Clerk Counseling Desk Officer Training Desk Officer Media Desk Officer 52 Annex F. Organizational Structure of EMDA Individual also responsible for non-M&E tasks Functions performed by staff with other responsibilities Ethiopian Islamic Affairs Supreme Council Relief and Social Development Sector EMDA Director Finance HIV/ AIDS Department Development Department Monitoring and Evaluation Unit Program officer Program officer National program coordinators (Actionaid/PC) National Program Coordinator for PACT Project Regional Coordinator(s) District Field Coordinators 4 Regional Coordinators 53 Annex G. Rohi Weda Organizational Structure 1. Structure Up to March 2008 Finance & Administration Program Coordination Project Officer General Assembly Board Executive Director Accountant Cashier/secretary Store Keeper Cleaner/messenger Guard Field Supervisors 54 2. Rohi Wedda’s New Proposed Organizational Structure General Assembly Social Development Livelihood Program/Project Board Executive Director District Offices Represented by Field Supervisors Community Development Facilitators (CDF) Capacity Building Programs Coordination Unit Income Generating Rehabilitation Unit Administration and Finance Unit Public Relation and Fund Planning, Budgeting, Monitoring & Evaluation Unit Focus Groups/Targets Secretary 55 ANNEX H Illustrative Results Framework for IOCC-DICAC Project Objective: (impact statement) Indicators: To be determined Intermediate Result 1: Greater awareness and knowledge about HIV/AIDS and ways to prevent transmission Indicators 1.1 Number of individuals trained to promote abstinence and/or be faithful (disaggregated by gender) a. Subset trained re abstinence only (disaggregated by gender) 1.2 Number of persons reached through behavioral change communication (however defined in new PEPFAR guidance) on AB (disaggregated by gender) a. sub-set: reached with abstinence only BCC(disaggregated by gender) 1.3 Estimated number of persons reached with IEC on AB 1.4 Number trained to promote/encourage people to seek VCT and/or PMTCT services In annual plan, list activities activities/support such as TOT, training sessions,, commodities purchased and materials developed/adopted and disseminated Intermediate Result 2. Expansion of care and support of OVC Indicators 2.1 Number of persons trained to provide care and support to OVC 2.2 Number of OVC receiving care and support services Disaggregated by a) number of OVC directly receiving three or more services b) number of OVC receiving one or two services c) gender d) age group e) types of services provided and number receiving each (using PEPFAR categories) 2.3 Number of people trained in advocacy and social mobilization to create a supportive environment for children affected by HIV/AIDS In annual plan, list activities/support leading to the above results (TOT, training of community volunteers and clergy, commodities purchased and materials developed adopted and disseminated) Intermediate Result 3. Expansion of care and support to PLWHA Indicators 3..1 Number of persons trained to provide care and support to PLWHA 3.2 Number of PLWHA receiving Palliative Care/Home Based care support services (as defined by PEPFAR) 56 Disaggregated by type of service (psychosocial, spiritual counseling, economic, Food, care of bedridden, AB, referrals for VCT and/or PMTCT 3.3 Estimated number of individuals reached with reduction of stigma and discrimination BCC messages In annual plan, list activities/outputs leading to the above results (TOT, training of community volunteers, commodities purchased and materials developed adopted and disseminated Intermediate Result 4. Increased ability to report quality accurate data and use data to inform planning Indicators: 4.1 Number of persons trained in new monitoring and reporting system 4.2 Number (and proportion) of districts submitting monitoring reports according to set standards AND on time 4.3 Assessments and evaluations conducted (qualitative) 4.4 Use of data for program planning/management (qualitative) List of key activities including training, commodities, identification of impact indicators supporting achievement of the above results 3. Improved knowledge and skills of staff and managers a. Numbers trained disaggregated a) Headquarters/ diocese/district recorded and by type of training received b. Number of branch offices who have increased their overall score on all Capacity Building items in the Monitoring Checklist In annual plan list activities leading to the results above 57 Select Bibliography Central Statistics Agency and ORC Macro. Ethiopia Demographic and Health Survey 2005, Addis Ababa and Calverton, Maryland. Ethiopian Muslims Development Agency. Organizational Capacity Re-Assessment Report. March 2008 Ethiopian Islamic Affairs Supreme Council. Profile of Ethiopian Muslim Development Agency. Addis, July 2007. Ethiopian Orthodox Tewhido Church, HIV/AIDS Prevention and Control Strategy: Five Years Program (2004/5-2008/9), Addis October 2004. Federal HIV/AIDS Prevention and Control Office, Single Point Estimates of HIV and OVC Indicators, April 5, 2007 McCoy, K . Lynn, Patricia Njeri Ngari and Edwin Krumpe. 2005. Building Monitoring, Evaluation and Reporting Systems for HIV/AIDS Programs. Pact, Washington DC. Tegen, Mandefro and Girma Aberra. Organizational Capacity Assessment of Rohi￾Weddu, Pastoral Women Development Organization. Addis 2006. Tesfaye Yimer, EOC-DICAC Guideline for Job Description, Roles and Responsibilities of the Project Advisory Committee (Branch, District & Parish level), revised August 2000,Addis Ababa (note revised date refers to Ethiopian year) Telikote Smarawi” Participants manual/ trainers guide for basic HIV/AIDS Prevention and Control education for Clergy (Amharic version), EOC/DICAC, September 2006 Addis Ababa Documents and reports in files Training reports, exposure and field visit reports in files of Pact and IOCC/DICAC Annual Reports to USAID, Quarterly Reports, annual and semi-annual PEPFAR reports and annual Plan of Action documents submitted to USAID by IOCC/DICAC and Pact USAID Cooperative Agreements with IOCC and with Pact Powerpoint briefings to evaluation team by Pact, EMDA and Rohi Weddu, Aug 2008 DICAC Quarterly Project Data Collection/reporting formats IOCC/EOC/DICAC HIV/AIDS Project M&E Tool, Addis 2007. EOC-DICAC HIV/AIDS Prevention & Control Department, Checklist to Reporting, January 2006 User guide for M-ARCH Project MER Formats-Revised on August 2007 Sub-Agreement between IOCC and EOC-DICAC under USAID Cooperative Agreement and between Pact and EMDA under USAID Cooperative Agreement