End-of-Project Evaluation “Strengthening the Capacity for Country Ownership” 2 ACRONYMS AIDS Acquired Immune Deficiency Syndrome CAA Catholic AIDS Action CACOC Constituency AIDS Coordinating Committee CCCW Community child care worker CHFL Caprivi Hope for Life CISPS Comprehensive Institutional Strengthening Plans CSO Civil society organization CW Child Welfare ECD Early Childhood Development FTE Fulltime employees GE Gender Equality GHI Global Health Initiative HIV Human Immunodeficiency Virus GBV Gender based violence GRN Government of the Republic of Namibia HBC Home-based care HR Human resources HRH Human Resources for Health IDP Individual Development Plan IP Implementation plan KAYEC Katutura Youth Empowerment Center LAC Legal Assistance Center LOE Level of Effort LL/CL Lifeline/Childline M&E Monitoring and Evaluation MDG Millennium Development Goals MER Monitoring, Evaluation, and Reporting MGECW Ministry of Gender Equality and Child Welfare MLRG Ministry of Local and Regional Government MOE Ministry of Education MOHSS Ministry of Health and Social Services MOU Memorandum of Understanding MSP Ministry Support Plan MTEF Mid-Term Expenditure Fund NANGOF Namibian NGO Forum NCC National Council of Churches NDP National Development Plan MIT Management Implementation Team 3 MOF Ministry of Finance RIT Regional Implementation Team RMP Resources Mobilization Plan NGO Non-governmental organization NIPAM Namibian Institute of Public Administration and Management NPA National Plan of Action for OVC OCA Organizational capacity assessment OD Organizational development OPI Organizational Performance Index OPM Office of the Prime Minister OVC Orphans and vulnerable children PEACE People’s Education, Assistance, and Counseling for Empowerment Centre PEPFAR President’s Emergency Plan for AIDS Relief PMP Performance Management Plan PMS Performance Management System PS Permanent Secretary PSC Public Service Commission PTF Permanent Task Force on Orphans and Vulnerable Children PWC Price Waterhouse Cooper RACOCs Regional AIDS Coordinating Committees RCCFs Residential child care facilities REACH Rapid and Effective Action Combating HIV/AIDS RFA Request for Applications SAHCD Southern Africa Human Capacity Development project SCCO Strengthening the Capacity for Country Ownership TAC Technical Advisory Committee TPM Team Planning Meeting TRG Technical Reference Group UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government 4 CONTENTS I. Executive Summary ......................................................................................................................... 5 II. Background and Purpose of End of Project Evaluation .................................................................. 7 III. Methodology and Limitations ............................................................................................................ 9 A. Methodology ............................................................................................................................... 9 B. Development of Evaluation Questions and Instruments ..........................................................10 C. Limitations................................................................................................................................. 11 III. Findings .....................................................................................................................................12 A. Directorates ..............................................................................................................................13 1. Permanent Secretary ............................................................................................................13 2. Monitoring and Evaluation (M&E) Technical Advisor ...........................................................15 3. Child Welfare ........................................................................................................................17 4. Early Childhood Development ..............................................................................................20 5. Gender Equality.....................................................................................................................23 B. Civil Society Organizations ...........................................................................................................23 1. KAYEK ........................................................................................................................................23 3. Catholic AIDS Action ..............................................................................................................25 4. Lifeline/Childline ...................................................................................................................27 5. Legal Assistance Center (LAC) ...............................................................................................28 6. Caprivi Hope for Life (CHFL) ..................................................................................................30 6. People’s Education Assistance & Counseling for Empowerment (PEACE) ..............................31 V. Summary of Findings ........................................................................................................................33 VI. Recommendations. ..........................................................................................................................36 VIII. Annexes..........................................................................................................................................36 Annex A. Interview Tools ..................................................................................................................36 1. Interview Tool for MGECW ...................................................................................................36 2. Interview Tool for CSOs ........................................................................................................40 Annex B. People Contacts .................................................................................................................44 Annex D. Scope of Work ...................................................................................................................46 Annex E. Schedule of Visits ...............................................................................................................56 Annex F. Mid-Term Recommendations ............................................................................................59 Annex G: PowerPoint Presentation ..................................................................................................60 5 I. Executive Summary This end-of-project report evaluates the final 19 months of a five year project and distinguishes accomplishment in this timeframe from previous accomplishments and activities. The focus of the report is on evaluating Pact’s effectiveness towards strengthening the four directorates of the Ministry of Gender Equality and Child Welfare (MGECW) and six USAID-selected civil service organizations (CSOs). This report continues the USAID evaluation procedure whose last major effort was a Mid-Term Evaluation Report (January 2011). The design of much of the methodology for this evaluation was informed by United States Government (USG) Namibia Global Health Initiative, the President’s Emergency Plan for AIDS Relief (PEPFAR) Namibia’s Gender Challenge Fund, and the PEPFAR Framework for Organizational Capacity Building. The major dimensions of this approach measure effectiveness across definitions of sustainability, organizational capacity building, and systems strengthening and was endorsed by USAID Namibia as the principle framework for this report (see Interview Tools with definitions of these dimensions in Annex A, 1 and 2) . The major limitations of this evaluation effort are limited time and limited access to people within the MGECW, and at regional and constituency level, as well as limited access to staff in different roles within CSOs. The conclusion that Pact fulfilled all of its contractual obligations at a standard that is defined as “satisfactory” ( i.e., fulfilling its contractual obligations of Award Number 690- A-00-07-00104-00) is supported by primary collection of evidence via document review and key informant interviews. This evaluation also identifies strengths and weaknesses as reported by both Pact advisors and Ministry and CSO partners. The following summative, high level dashboard table reflects what is detailed in the report itself and in the accompanying PowerPoint presentation (Annex G). The quantitative ratings used as scores in the table are defined as follows: Level 4= “Excellent” - Pact’s effectiveness provided a significant added value to the contractual obligations. Level 3 = “Very Good” - Pact’s effectiveness provided a modest added value to the contractual obligations. Level 2 = “Satisfactory” - Pact’s effectiveness met the contractual obligations. Level 1 = “Unsatisfactory” – Pact’s effectiveness did not meet some or all of the contractual obligations. 6 The following text abstracted from the Modified Agreement of November 2010 outlines the expected results of the contract: II. III. IV. Though both the Child Welfare Directorate (CW) and the Early Childhood Directorate (ECD) received a Satisfactory rating (Level 2= Satisfactory”, i.e., Pact’s effectiveness met the contractual obligations), the effectiveness of assistance within the Gender Equality Directorate (GE) is “Unsatisfactory”, i.e., Pact’s effectiveness did not meet some or all of the contractual obligations. Part of the reason for this lower rating is due to the limited direct assistance and time provided this directorate, which only received some OD-related training provided across all directorates. It was also reported that GE wanted technical assistance only (not OD – organizational development assistance); yet technical assistance was not part of Pact’s mandate per the Modified Agreement of November 2010. Note that the fourth directorate (General Services) is not listed here as the schedule did not allow for evaluators to interview staff from this directorate. It should also be noted that a rating of “Satisfactory” (Level “2”) is largely determined by the difficulty of achieving IR1.1 above (i.e., strengthening planning, management, and implementation capacity across all four directorates) within a 19 month timeframe. There are many reported instances of high quality inputs provided by Pact, especially via Pact’s technical/OD advisors; those who were most effective were those who were assigned to specific directorates for the longest time and who also worked at higher levels (such as within the Planning Services Commission and Office of the Prime Minister) to effect systems-wide change. These ratings represent a “dashboard “level analysis. Documentation and justification for each rating is based on the in depth narrative presented in “Section II. Findings, A. Directorates” of this report. Dimensions: MGECW Directorates: Civil Society Organizations (CSOs): CW ECD GE 3 Graduated CSOs 3 Non-graduated CSOs Sustainability 2 2 1 2 1 Organizational Capacity Building 2 2 1 2 1 Systems Strengthening 2 2 1 2 1 Strategic Result 1. Strengthened capacity of MGECW to effectively coordinate activities relating to gender equality and orphans and vulnerable children. Intermediate Result 1.1 Build the planning, management and implementation capacity of all directorates. Intermediate Result 1.2 Strengthen the human resources management capacity at the Ministry, allowing the institution to improve its efficiency. Intermediate Result 1.3 Contribute to the Child Directorate’s continued progress in assuming its role as the national leader of the National Plan of Action (NPA) and the implementation of OVC-related policies. Strategic Result 2. Strengthened six civil society organizations to be eligible for US direct funding while maintaining high standards of existing services. 7 II. Background and Purpose of End of Project Evaluation USAID awarded a five-year Local Field Support Cooperative Agreement to Pact Inc. (CA Number 690- A-00-07-00104-00) on September 26, 2007, with a planned life of project of US$40 million through the PEPFAR. The goal of the Pact program, Community Rapid and Effective Action Combating HIV/AIDS (REACH) Namibia program”, was to scale up and sustain a comprehensive response to HIV/AIDS services (including prevention, care, and support for orphans and vulnerable children) through Namibian and international sub-grantees. The six key program objectives that guided the program through October, 2010 were as follows: 1. To provide an effective and transparent grant award and administration system for the provision of responsive, fast-track grant making assistance to organizations responding to the Emergency Plan. 2. To provide implementers with access to financial resources and high quality technical expertise to assist in achieving and effectively reporting results while complying with USG financial and administrative requirements. 3. To expand civil society’s response by providing capacity building to local, regional, national and international organizations resulting in increased capacity of organizations and networks to provide and sustain HIV/AIDS services. 4. To strengthen the linkages between civil society, host country government and private sector with the aim of promoting longer term sustainability and capacity building in-country. 5. To support the Ministry of Gender Equality and Child Welfare through targeted technical assistance and capacity building, and implementation of an OVC database that enables registration of OVC nationwide, links country partners to the Government of Namibia’s (GRN) social welfare grants system, and allows Pact to support Ministry in their overall goal of implementation of the National Plan of Action for OVC. In November 2010, following a mid-term evaluation, USAID entered into a Modification of Assistance agreement with Pact Inc. to realign its program to GRN and PEPFAR phase II priorities. The emphasis shifted from a focus on the emergency scale-up approach of PEPFAR I to the Pact “Strengthening the Capacity for Country Ownership (SCCO)” programmatic themes of country ownership, long-term sustainability, and systems strengthening. In relation to this new direction, a USAID audit of Namibia’s Human Resources for Health (HRH) submitted in February 2011 delineated the need to address issues of sustainability, particularly with respect to strengthening the local sub￾grantees. Subsequently, USAID revised the total estimated value of the Cooperative Agreement to US$31,822,981 (with an additional cost share valued at US$460,780). The two key strategic results (and summarized Intermediate Results) that reframed the modified Pact program from November 2010 – September 25, 2012 are as follows: 1. Strengthened capacity of the (four Directorates of the) Ministry of Gender Equality and Child Welfare (MGECW) to effectively coordinate activities relating to gender equality and orphans and vulnerable children. 8 IR1.1 Build the planning, management and implementation capacity of all (four) directorates. IR1.2 Strengthen the human resources management capacity at the Ministry, allowing the institution to improve its efficiency. IR1.3 Contribute to the Child Welfare Directorate’s continued progress in assuming its role as the national leader of the National Plan of Action (NPA) and the implementation of OVC￾related policies. 2. Strengthened six civil society organizations to be eligible for US direct funding while maintaining high standards of existing services. The following USG PEPFAR Principles were integral to the modification of the Award and are foundational to the purpose of the end-of-project purpose and approach: Sustainability: PEPFAR programs are designed to strengthen host country stakeholders (including GRN and civil society) to mobilize financial, human, and organizational resources as well as plan, implement, and manage their own activities to continue once USG funding has ended. Country Ownership: Under PEPFAR II, the Global PEPFAR Strategy emphasizes “strengthening partner government capacity to lead the response to this (HIV/AIDS) epidemic and other health demands”, including government capacity to lead efforts to evaluate and define the needs and roles in the national response. The participation of local organizations is critical to achieving results of a national response. Systems Strengthening: PEPFAR II calls for an integrated approach to systems strengthening to ensure all processes and procedures for financing, planning, implementing, and monitoring services are in place. The overall purpose of this End-of-Project Evaluation is to determine the extent to which Pact has achieved the Strategic Results stated in its Modified Agreement (see above) during the (19 months) period from November, 2010 through June, 2012. During this period, Pact Inc. applied an average of 17 full time equivalents (FTE) and expended US$7,763,714 as key inputs to implement the SCCO program. The evaluation will also capture “lessons learned” from the Pact technical and organizational assistance model to inform the design of USAID’s follow-on award within Namibia. The team’s evaluation is also conducted within the context of the following other key directives: a) The vision of the USG Namibia Global Health Initiative (GHI) Strategy (2011-2015/6), and specifically PEPFAR Namibia’s Gender Challenge Fund in partnership with the MGECW. The GHI focuses on a “transitioning approach” to increase both government and civil society capacity to manage, coordinate, and finance the health sector in support of Namibia’s progress towards achieving the Millennium Development Goals (MDG) for health (see Results Framework of the GHI). This approach encompasses the following core principles: country ownership; “smart integration” (i.e., integration of services where it makes technical, financial, and cultural sense); systems strengthening; strategic coordination; leveraging resources; improved monitoring and evaluation (M&E); a focus on women and girls; and an emphasis on innovation. 9 b) Redirection of USAID assistance to a “technical assistance only” role. Prior to the Modification Agreement, USAID Pretoria conducted a “Pre-Award Assessment” of civil society organizations in Namibia in order to: a) review policies and procedures related to financial and procurement operations to ensure USAID funds would be properly expended and accounted for in compliance with the terms of the award; b) review relevant U.S. Federal Government accountability and fund management rules; and c) review generally accepted accounting/financial management practices. The financial, accounting, human resources/personnel, and procurement elements of this Pre-Award Assessment informed Pact’s approach to strengthening the MGECW and (six) CSOs (see Strategic Results 1 and 2 of the Modification Agreement dated November, 2010). For instance, Pact’s tool and approach incorporated these elements and added management, strategic planning, resource mobilization, and other elements. c) Emphasis during the evaluation effort on assessment of Pact’s effectiveness in strengthening the organizational development and capacity of the MGECW (75% weight) and of six civil society organizations (25% weight).1 In view of upcoming USAID programming to provide direct government-to-government technical assistance, the focus of the evaluation highlight especially the ministry’s strides forward in organizational capacity building and systems development. III. Methodology and Limitations A. Methodology 1. Documents review: See Annex C for all documents that were consulted, e.g. documents that indicate degrees of progress of institutional improvements: Systems strengthening, Organizational capacity building, and Sustainability. For example, a. OD Roadmap Scores (as documented by Pact) that demonstrate improved changes in organizational capacity using Pact program components; b. MOU or other documentation of collaboration or referrals with the GRN or private sector organizations; c. Resource mobilization plans (RMPs) showing funding strategy for the future to diversify funding portfolios (including USAID). 2. Key informant interviews – with Directors of Ministry Directorates, CSO directors, Pact advisors and staff (Chief of Party) and USAID staff – These are listed in Annex B. 3. Interview tools: Since key informant interviews were so critical to this review, see the section below on the design approach and framework for interview tools development and Annex A for copies of each Interview Tool, one specifically for informant interviews at the Directorate or PS level and one for informant interviews with CSOs. 4. Surveys- Previous surveys of Pact success were also consulted including the REACH Midterm Evaluation., Comprehensive Institutional Strengthening Plans (CISPs) used for baseline data and to track CSO progress, the Ministry Support Plans, etc. 1 This weighting of evaluation effort (i.e., 75/25) was suggested by USAID/ Namibia during an initial Team Planning Meeting. The relative importance of MGECW/CSOs for the entire evaluation exercise is based on future plans for USAID assistance in Namibia. 10 5. Team Planning Meeting – Over the course of the first three days in-country, a team planning meeting (TPM) was held with USAID staff. This time was used to clarify team roles and responsibilities, deliverables, development of tools and approach to the evaluation, and refinement of the agenda and subsequent schedules. In the TPM the team: a. shared background, experience, and expectations for the assignment; b. formulated a common understanding of the assignment and clarified team members’ roles and responsibilities; c. held in-depth discussions with USAID/Namibia about the context for the evaluation; d. agreed on the objectives and desired outcomes of the assignment; e. established a team atmosphere, sharing working styles; agreed on procedures for resolving differences of opinion; f. developed data collection methods, instruments, tools and guidelines; g. and agreed upon an evaluation approach, tools, timeline, and strategy for achieving deliverables. B. Development of Evaluation Questions and Instruments Key Guiding Evaluation Questions These three dimensions represent universals of organizational performance. In particular, for the context of HIV/AIDS and OVC service delivery for Namibia; these dimensions support USAID/Namibia transition to its anticipated “technical assistance only” role. The dimensions and the answers to the questions posed in the Interview Tool (guided by the following three overarching evaluation questions) accurately describe the challenges of support from USAID/Namibia and any other donors who might serve the same government and civil service organizations. All questions below are specific to the time frame of November 2010, which is the period of time following the Midterm Evaluation. This End of Project Evaluation builds upon the Midterm Evaluation, which covered the period October 2007 through March 2010. The Midterm Evaluation considered results against the six original objectives of Pact’s agreement and included determining sustainability of activities and long term impact of investments in human resources. The End of Project evaluation focuses on the two Strategic Results that constituted a Modified Agreement with Pact in November 2010. The intended results centered on issues of strengthening the MGECW and six civil society organizations (as described in the “Background and Purpose” section of this report). Also see “C. Response to Selected Recommendations from the Midterm Evaluation” in this report. 1) In what ways has sustainability been achieved (by MGECW and the 4 Directorates; by the 6 CSOs), due to Pact’s assistance? 2) What evidence exists for organizational capacity building (of the MGECW and the 4 Directorates; of the CSOs), as a direct result of Pact assistance? 3) What evidence exists for systems strengthening (within the MGECW and the 4 Directorates; within the 6 CSOs) as a direct result of Pact assistance? Construction of Interview Guide for Key Informant Interviews Two versions of the Interview Guide are being developed, one to be used with Directors and their designees within each Directorate within the MGECW, and one to be used with 11 representatives of the six CSOs. Additionally, interviews were conducted with the 4 Pact Advisors and the Pact/Namibia Chief of Party. The following four key resources represent the major framework for the design of the Interview Tool(s) which expands the three dimensions of assessment. Additional resources will be listed in the report bibliography as well as persons contacted: 1) PEPFAR Framework for Organizational Capacity Building – USAID’s AIDSTAR II Technical Brief No. 2, January 2011: “Organizational Capacity Building Framework: A Foundation for Stronger, More Sustainable HIV Programs, Organizations, and Networks” 2) USAID Performance Monitoring Plan (Pact Namibia): Strengthening the Capacity for Country Ownership Program, April 2012 3) PEPFAR Draft Guidance: Next Generation Indicators Reference Guide, April 3, 2009 (Appendix 2: Health Systems Strengthening) 4) USAID/Namibia: Community Rapid and Effective Action Combating HIV/AIDS (REACH) Midterm Evaluation, January 2011 C. Limitations 1. One possible weakness for 100% verification of Pact self-reported accomplishment, for the limited time we had, was to accept Pact’s self-report of task accomplishment as valid. The alternative would have been to corroborate huge quantities of mostly Pact￾produced documents that prove every Pact statement of activity and task completion. For the Findings section below, CSO interviews both supported and provided different viewpoints about how Pact self-evaluation instruments describe degree of successes. 2. The entire focus of this evaluation was, as stated above, exclusively on Pact’s effectiveness in developing the institutional capacity of the six CSOs and four directorates of the MGECW. Conclusions can only be made by inference that when all of the managerial, financial and HR. functions are improved, the output of the organization will also improve. But without attention to the services and recipients of these services, this is just an inference. This evaluation did not include the tasks of evaluating the quality or quantity of health services delivery and/or the impact on the recipients. This will be addressed further in the section on Recommendations. 3. This evaluation was only tasked to evaluate the last 19 months of the 4.5 years of Pact technical assistance under this Award. Recall was a recurring issue. Many respondents found it difficult to separate out the last 19 months of Pact support from the larger periods of assistance and partnership with Pact that preceded November 2010. 4. Activity Based - Usually, an evaluation of progress is to measure improvement of institutional performance as it moves from point A to point B., where point A is a performance standard described in measurable terms or metrics. This would have enabled this report to present a more rigorous picture of Pact’s “effectiveness”. Instead, the various OD Roadmaps and CISP and MSP instruments focused on systems, procedures, and opinions of CSO or Ministry staff about activities. The only focus available for CSOs and Directorates has been the completion of activity and creation of documents and systems, most notably, procedures for M&E and conformity to USG 12 financial administration regulations. The Limitations section of Methodology addresses the fact that evaluation is limited to completion of activity, and that “accomplishment” is also usually reported only on the activity level, irrespective of whether or not the given “activity” led to improved organizational performance or improved delivery of service. 5. The evaluation has considered a plethora of documents that have models and frameworks that are not aligned with one another in order to ensure consistent monitoring of the same performance areas. Also some documents must have had multiple authors (like this one), but wherein, authors did not collaborate to use all terms in the same way so that terms are variably defined, particularly terms key to this evaluation such as “capacity”, “strategy”, “result”, “objective”, “system”, etc. 6. Only a small subset of CSOs was interviewed, compared to the much larger group that was recipients of USAID assistance through Pact prior to the Modified Agreement. The group interviewed included three CSOs that had “graduated” and three CSOs that had not graduated to direct USAID funding. Therefore recommendations for an improved follow-on project may have limited application or are not entirely generalizable to the rest of the original 21 CSOs that were sub grantees to Pact (through a competitive bidding process supported by USAID funding). The concern is that former sub grantees’ non-participation in the current project might actually be an indication of greater weakness in all of the categories of our assessment. These weaknesses would have “provoked” other, additional recommendations for Pact’s improved support and for improved models of institutional capacity building. 7. Observation of the CSOs functioning and determining how these functions could be attributed to Pact support would have been ideal, but with the limited time, this could not be scheduled in addition to the meetings we had with CSO directors. III. Findings The OD roadmap assessments were rigorously developed and applied but used only for reporting to USAID and not that the methodology was used by the graduated CSOs as a managerial tool. To graduate: Except for passing the Price Waterhouse Cooper (PWC) auditing standards, there is no cumulative scoring system that describes a “pass” to graduation status. The OD Roadmap scores might be included in some weighted proportion for the total “pass” criteria. The following “findings” are organized by data and comments collected from each Pact technical advisor, three of the four MGECW directorates (note that meetings with the General Services Directorate were not scheduled), the views/experiences of the former Permanent Secretary of the MGECW, and each of six civil society organizations. The Pact advisor and directorate “Findings” document those most noteworthy contributions and areas where more effort is needed and/or problems are noted, from the perspective and recall of each interviewee (as noted above each table). Note that each civil society organization table specifically documents the three major evaluation parameters (i.e., Pact’s effectiveness in contributing to sustainability, organizational capacity development, and/or systems strengthening) in accordance with the questions in the Interview Tools (see Annex A). Refer to Annex B for list of persons interviewed per agency listed below. 13 Rather than presenting the answers provided by key informants to every single question of the Interview Tool, the answers are amalgamated, condensed and organized in such a way as to indicate which specific questions yielded the most information. The “other” category captures answers to additional related questions, e.g., the usefulness of the technical assistance “model” as a way to maximize effectiveness; or questions about use of the OD Roadmap for more than reporting to USAID and its uptake by “graduated” civil society organizations (CSOs) as a managerial tool beyond the point of graduation. A. Directorates 1. Permanent Secretary The following findings are from key informant interviews (see titles as indicated in tables below) and review of primarily annual plans and Ministry Support Plans. Perspective of Pact Advisor to MGECW Pact’s Primary Contributions: - Helped at the PS and OPM levels to finalize the proposed organizational restructure, including facilitating the review by the Regional Councils. - Revised the already-costed new HR structure (salary and allowances) (Note that original costing of the MGECW HR structure was undertaken by another USAID regional project, Southern Africa Human Capacity Development, SAHCD; however, the same consultant who implemented SAHCD also provided assistance under the Pact agreement and was able to respond to requests for revisions from OPM). - Worked with the OPM to increase the cost figures (to include housing and transportation allowances). - Helped MGECW improve reports – formats, how to handle emails and the guidelines to ensure that email communication was two-way. Before Pact involvement, the staff of the MGECW did not answer their emails as part of “ongoing business”. - Wrote an internal/external communications strategy for the entire MGECW, based on a need identified through HR Strategic Planning (that had also been developed by SAHCD). This strategy for decentralized communication seeks to improve decision making, the authority hierarchy, external communication, and websites. One direct result is that the PS immediately implemented the “Frequently Asked Questions” on the MGECW website. - Organized a training plan for individual development plans (IDPs) and started its implementation. This included identifying formal training needs and outsourcing some training to local trainers or to the Public Service Commission - Helped develop a framework for staffing the new HR structure once it is approved, including a HR recruitment strategy with two levels - This Pact Advisor effectively linked an overarching, guiding system (OPM/PSC) and a smaller one (MGECW). As a result people feel like they really belong to some place where progress happens. - Evidences of “success” include recognition that the MGECW and PS improved processes and operations (they were awarded a trophy). Limitations and that which was left undone: - The government needs exponentially increased labor and time resources in order to sustain managerial skills critical to delivery of social services (which, by definition, only occurs at the people￾to-people interface). These include leadership and transformation skills and service improvement skills, as stated in the Public Service Charter - There needs to be a process to facilitate and agree upon priorities of Advisor activities in order to reduce tasks to a more manageable level; currently, some of these tasks become overwhelming. 14 Failed to get everything done because of too many competing priorities. - Some overlapping roles between these 4 Tech Advisors, e.g. M&E and PMS might have been more integrated and coordinated. - Understanding and acceptance of external advisors differs. Some public sector staff believes that some advisors do nothing. Others would like to let advisors do their work; and others feel it is appropriate to learn skills from them in order to do their own work better. - There are concerns about how HR recruitment and deployment will actually be implemented. - Most of the HR issues are handled by the OPM; and the Ministry (MGECW) does not feel it has the capacity to handle this. The Ministry must first develop the recruitment strategy in full and then submit it to the HR chief for review and submission to the PS before it is submitted to the PCS for formal approval, which is a legislative process. - Only one directorate (CW) was awarded the trophy which both elevated the success possible in other directorates and caused some competitive tension and resistances among other directors. Perspective of (former) Permanent Secretary of the MGECW Pact’s Primary Contributions: - MGECW consulted all 13 regions during restructuring and needed Pact technical support at certain key junctures, e.g., when signing performance agreements. Pact helped the MGECW to ensure that regional staff in all 13 regions has the planning skills to carry out performance management, annual planning, and strategic planning. The PS is confident that efforts to decentralize will continue. Pact facilitated as the MGECW reworked its structure, including consultations with OPM and the Regional Councils. - The Decentralized Enabling Act provides a framework if the Ministry wants to strengthen operations during decentralization (though it is only a plan now). - The PS was “stuck” due to the level of resistance by directors and other staff, but one of the Pact advisors helped to convene a meeting to air differences and problem solve; this turned into a planning workshop to implement the approach, together. This key support helped the PS learn how to resolve conflict in such a way as to build teams all identified with the mission of the organization. Now the PS can say with confidence: “We agree to disagree.” - Pact assisted with facilitating a participatory process for development of the MGECW HR Strategic Plan. - The Pact Advisor lobbied for the importance of OVC Standards for RCCFs. The advisor facilitated GRN communication with networks of CSOs and GRN is now paying some CSOs managing RCCFs. OPM has signed an agreement with the Council of Churches to work with all ministries to identify CSOs and outsource to them as appropriate. - The Prime Minister is also involved – and 50 – 60% of progress in OVC programming is attributed to Pact (particularly regarding the RCCFs). - GRN is now subsidizing places of safety, for which Pact helped in developing the criteria. The shelters are now either registered or in the process of registration. - At community level, there is more knowledge about MGECW’s work/services and how to qualify for social grants (which was not true before Pact’s assistance). - With Pact support, MGECW influenced Home Affairs to help improve issuance of birth certificates to children (who as a result are now in better positions to receive services). - Before Pact involvement, the PS did not get support from the Prime Minister (and every directorate was operating very independently). There was no common vision until the PS helped directorates embrace themselves as part of a greater whole; this sense of common identity was made possible 15 due to Pact’s OD Advisor. Limitations and that which was left undone: - Remaining work needs to be done in order to decentralize the MGECW down to the Regional Councils; it is critical that staff be able to report to the Chief Regional Officers. - Sometimes there is a tension between the central directorate offices and the decentralized levels. There could be fear of the unknown and doubts about how to decentralize in a way that builds in cooperation and coordination and actually clarifies roles. Much work remains in this regard. 2. Monitoring and Evaluation (M&E) Technical Advisor Note that the M&E Advisor to the MGECW began working in October, 2008, seconded only to the Child Welfare Directorate. From November 2010 (date of Modified Agreement), the M&E Advisor’s scope was expanded to work with all four directorates of the MGECW. The focus of the following data is limited to November, 2010 – June 30, 2012. Perspective of Pact M&E Advisor to MGECW Pact’s Primary Contributions: - Prior to the Advisor’s activity, there were many reports but they were not systematic and were not linked to planning or budgeting. The Advisor began a system of reports from regional offices so that data could be used at the MGECW level (including personnel data) and could also be entered into the MARS database. At deputy director level (within directorates), the data is now reviewed as part of the planning cycle to set targets for programs and budgets. It is also starting to be used for planning at the regional level. - Via Pact assistance, residential child care facilities (RCCFs) are defining standards, and implementation of services is managed within a MGECW-led network. - With Pact assistance, MGECW was able to develop self-assessment tools to examine quality of services (particularly within RCCFs). - The Pact advisor was partially involved with work to support WACPU (Women and Children’s Protective Units) to define protocols of reference and how to manage child protection cases, etc. Some of this is being done with 16 UNICEF and RIT while looking at issues of quality also. - The Pact M&E Advisor worked closely within OPM planning frameworks as a first step to developing stronger monitoring systems within MGECW. - The Performance Management System (PMS) is based on the Logical Framework format (with help from Pact) and on what had been previously accomplished during 2008-09. Pact helped MGECW use the hierarchical alignment of objectives for purposes of performance management as well. - PACT and the M&E Advisor worked with MGECW as a team, including cross-directorate staff in trainings, planning, etc. - The quality of data reporting is now mature; note that it required 3 years of effort as it was begun through Pact assistance in July, 2009. - The Pact Advisor worked with and trained two M&E counterparts in the directorate. As of November 2010, the Advisor worked to expand the M&E systems to support all 4 directorates of MGECW. - Organizational Development (OD) at the Regional Level: The advisor brought together Regional Implementation Teams (RITs) from 3 regions into a workshop with personnel from each major functional area in order to help develop individual performance agreements that were also within shared “job categories”. This standardized job tasks and also allowed staff to task shift or task share when appropriate. Regional Performance Agreements now exist for all staff. - At the central MGECW level the Advisor worked with the OPM which maintains the server and system for the OVC database to improve tracking of number of children receiving maintenance grants, foster care grants, and disability grants. - The Advisor supported two sections of the Child Welfare Directorate – to better administer child welfare services and child allowances. - The Advisor helped the Community Child Care Workers (CCCWs) start a filing system for grants and to build their skills to maintain the grants area of the overall database. Pact has also integrated and improved communication between this database at OPM and the MGECW, with support of previous M&E/IT consultant. - The Advisor also created a team approach with another Advisor so that the contribution was wider than either of their specific job descriptions. This broadened roles and contributions within the MGECW. For instance, working together, they created a system so children could have access to grants based on their status as orphans and/or their income levels rather than qualifying (only) as “OVC”. - The advisor produced the format used for M&E-related trainings. The Advisor also planned a reporting format that could be a major contribution to M&E systems being more stable and sustainable. (The system was actually first begun at MoHSS and then was migrated to MGECW and adapted so that eligibility criteria could be used to identify the appropriate children for services). Both the planning and M&E systems are responsive and can adapt to changes in strategic objectives. The MGECW M&E system is also aligned with the OPM system. - Examples of wider contributions to planning, data collection, and data utilization by the Pact Advisors working as a team: - Contributed to planning in order to then move to indicator construction, outputs, timeframes, persons responsible, and other elements of the LogFrame that could be adapted to meet the needs of the MGECW; - LogFrame planning was then introduced to other directorates who were at first reluctant to adopt the methodology but who eventually found it useful - The first experience in developing the MARS reporting system (including forms, indicators, and the corresponding database) was successfully instituted at the Child Welfare Directorate beginning in 2009 and became the basis for similar activity with the other directorates of MGECW. - Directorates had never done anything on their own as they always had an advisor or a ministerial level person when decentralizing the PMS. The Pact Advisor held workshops in conjunction with a MGECW counterpart who became almost like another technical advisor in the process. - The M&E system is now able to monitor beneficiary levels of data and activity, but the data is not yet included in the Performance Management Plan ( PMP). So while Pact can and has reported against Strategic Result 1 (i.e., 80% of directorate’s planned activities implemented), the directorate now also has more specific information on activities specific to the job descriptions of volunteers and CCCWs, etc., down to community level. 17 - Ministry (MGECW) Support Plans (FY11 and 12) were developed by Pact as a team and submitted to USAID which then discussed them (targets, etc.) with MGECW before finalizing them with Pact. This caused a slight delay, but it was thought that it is better to have full team discussions and reviews. - The Advisor initiated regular meetings for feedback from MGECW to Pact about each technical advisor, progress, and to resolve any problems and make improvements as part of the process. This was really important for the first year to develop MOUs and build a relationship, trust, and understandings about intended contributions of Pact advisors. - Though there is no guarantee that the Pact-facilitated processes and structures (teams) will persist (due to the very rapid transition of a new PS for the MGECW), the staff are equipped to discuss strategy and have improved technical abilities to monitor progress on the activities level. It is a pivotal moment to be able to build the M&E system towards monitoring quality/impact as well. Limitations and that which was left undone: - After the Modified Agreement, the M&E Advisor started working with 2 monitoring persons per directorate as counterparts, who were finally chosen by the PS last month. The M&E counterpart staff within the ECD only had a 1 month exposure to the Pact M&E advisor for training before these appointments. - The MGECW counterpart can now revise indicators and MARS to encompass the larger activities of WACPU and is trying to develop a monitoring tool that measures “quality” similar to the self-assessment tool used by the Residential Child Care Facilities. The other 3 directorates are just starting to receive regional level activity reports and similar reports from the General Services & Administration Directorate. - The MGECW databases still do not collect data that supports interpretations critical for strategy and other decision-making about impact, quality of services, coverage achieved, or comprehensiveness of service delivery. There is an outstanding need to help the MGECW with development of monitoring tools to measure quality, extent of services, “gaps” in unmet needs, etc. - MGECW-wide consulting was delayed along with (MGECW) appointments of M&E counterparts. - Some project implementation has been delayed about 6 months for each of the 3 directorates because of many, competing priorities; the very recent change at the PS level further delays the implementation of the MGECW￾wide M&E system. - In May and June 2012, the revision of performance agreements carried out on regional level s was very time and labor-consuming. - The monthly and quarterly reports only summarize at the “activity” level, including percentage activities completed, and do not measure results or impact. - Pact had two years (from 2008 – Nov 2010) to achieve capacity building within just one directorate (Child Welfare); it might be an unrealistic expectation that Pact could replicate this level of assistance to 3 additional directorates in an 18 months timeframe. - Pact’s M&E technical assistance helped to improve M&E and annual plans, but the new MGECW overarching 5 year strategic plan will be developed next year. Technical assistance will be necessary to align the improvements in annual planning back “up” to strategic planning levels. - This is in an environment of tremendous change, with the passage of the new Child Care Protection Bill and new National Agenda for Children (both of which enormously expand the number of beneficiaries to be served by the MGECW). 3. Child Welfare Perspective of Pact Child Welfare Directorate (CW) Advisor Pact’s Primary Contributions: 18 - On the basis of findings of the 2007 HR Gap Analysis (2007), the advisor helped the Child Welfare Directorate (CW) to extend annual planning to the regional level and to align budgeting to planning. - Following the Modified Agreement (November 2010), the advisor helped CW shift to begin annual planning cycles at the regional level (through Regional Implementation Teams, RITs); subsequently, regional representatives informed and participated in national annual planning workshops (via the Management Implementation Team (MIT). - This inclusion of regional planning within annual cycles of central planning appears to be sustainable because the finance team relies on needs-based regional budgets as part of the national budget planning exercise. The RIT and MIT have budget line items to support future regional-to-national planning and budget exercises every year. - The advisor helped to introduce a transport policy to increase dedicated resources for vehicles and mileage, based on activities in the annual work plan. Prior to this, the regions did not have a sufficient transport budget or subsistence and travel to conduct workshops that were in their plans. The new transport-specific budgets are needs-based (e.g., depending on number of employees in that region, the extent of the geographic territory of each region, and the population to be served). - The advisor facilitated improved funding and resource decisions within the CW’s Administration and Management Team. In 2008, CW retained approximately NAM$100M in reserve while there was no money reaching CW staff at constituency level (and activities were not being implemented). The funds retention was due to CW’s fear of depleting funds without having any budget tracking capabilities (except to request expenditure reports from the MoF), Due to Pact assistance, the CW now has a 3-year rolling budget and project-specific budgets, and the ability to review budgets against annual work plan targeted activities. - The advisor conducted an assessment of 106 constituency level OVC Forum functioning and presented findings to UNICEF and CW. - The advisor helped CW align to more Mid-Term Expenditure Fund (MTEF) targets, such as child justice, child protection, juvenile courts, etc. (which were used to lobby for funding from the Ministry of Finance (MOF). - The advisor helped to develop the Terms of Reference (TOR) for constituency, regional, national level (to strengthen processes and participation in the Permanent Task Force (PTF)). - What monitoring tools needed to track progress of these forums – there is a monitoring tool being piloted - While this Pact input was ongoing well before the Modified Agreement, Pact supported CW by helping to task shift and transition volunteers Child Care Community Workers (CCCWs) who are being absorbed by MGECW. This included training volunteers in planning and reporting (based on their job descriptions job description) as well as in basic principles of customer care and counseling skills. At least 40 volunteers were directly hired as CCCWs. - Since 2010, the Pact advisor spearheaded the performance management system across all 4 directorates. This included 1-on-1 mentoring of supervisors to develop performance agreements for all administration and management staff in a way that was aligned with the annual work plan. The advisor first focused assistance at MIT and then allowed the MIT to transfer skills to the RIT (only attending the first such training workshop to quality control the transference process). The EU conducted an evaluation of the performance agreements across 4 ministries for the OPM; the findings were positive for the MGECW. - The advisor reviewed all plans of action for gender, OVC, and ECD and incorporated priorities into the National Development Plan (NDP IV) by working closely with the MIT. - Per the Modified Agreement, peer-to-peer learning among one network was facilitated by the advisor, based on a review of most successful models of Residential Child Care Facilities (RCCFs). The advisor helped develop action plans for those facilities not meeting recognized minimum standards and trained the RCCF network to train other RCCF networks. CW was assisted in budgeting and creating a tool to monitor RCCF networks. Limitations and that which was left undone: - There was a very slow process of acceptance of (an external NGO) advisor by the CW; this model of technical assistance requires a long timeline to include the first year of acceptance. - CW needs to conduct a beneficiary level assessment in order to determine to what extent planned activities are actually reaching most vulnerable children and those most in need of protection. 19 - In order to fully operationalize regional to national budgeting exercises, finance staff in MGECW and budget directors need to be trained to track their budgets and expenditures every quarter, every region, per directorate. - There was a lot of resistance to instituting the Performance Management System (PMS) for CW from the directors. It is not clear that this system will persist in view of the recent change of Permanent Secretary who is being “lobbied” by directors to roll back some gains in this area. - While the CW budget includes support for the OVC Forums, it is not clear to what extent and rigor they are monitored for degree of contribution and impact. - Though the advisor helped the MIT to develop a support plan (including aspects of succession planning), this is undermined by the rapidity with which the PS was replaced. This in turn threatens the degree of systems uptake and sustainability of all gains achieved in this ministry through Pact assistance. There is also still some residual resistance (by directors) to the MIT (consisting of deputy directors). - There remains a critical shortage of social workers, yet it is unclear the extent to which CW will outsource to the civil society organizations positioned to implement activities at constituency level. - The advisor has advocated with the MIT that the next round of (5 years) MGECW strategic planning develop strategic objectives that will then be aligned with objectives of annual work plans; but this has not yet been carried out and will occur after the end of the Pact Award. - Even with the strongest organizational structures and management capability, the shortage of skilled staff (particularly social workers at community level) will compromise impact. Government salary structures are not competitive (so Namibia is hiring social workers from Zimbabwe and Zambia). There should be additional government grants for social worker students, especially given that the new “target population”, according to the National Agenda for Children (2012 – 2016) constitutes 46% of the population under 18 or an estimated 885,973 children (2006 estimate). Perspective of Child Welfare Directorate in MGECW (including Pact counterparts) Pact’s Primary Contributions: - CW counterparts believe that the capacity building received from Pact assistance at the central level will be transferable to the regional and constituency levels “eventually”. - CW staff interviewed was optimistic that the new PS is currently continuing the MIT to receive progress reports from deputy directors and about initial impressions that the new PS “has a belief in decentralization” and “seems action oriented”. - Pact trained senior CW staff in outcome mapping (for use by the PTF during development of the National Agenda for Children). They have identified the CSOs as “boundary partners” with whom CW would like to work more closely. - A major Pact contribution was help to strengthen the Performance Management System (for CW and the entire ministry), including Individual Development Plans (IDPs). - CW staff highlighted the importance of Pact’s help in registering Child Care Facilities (CCFs) and developing and training 76 RCCF staff and 60 social workers in self-assessment tools used by CCFs to measure service against minimum standards and by the CW to conduct inspections of the 38 RCCFs. - Pact’s assistance with social worker internships and bursaries added (two) social workers to CW staff – but this is unclear as it is not certain if these social workers are bonded or not. - (Probably accomplished prior to 2010, with Pact’s assistance, CW was able to identify the causes/bottlenecks for distributing social grants and improved their core business of grant administration.) Limitations and that which was left undone: - Because of the critical shortage of social workers, CW has requested that more CCCW positions be created during restructuring. This is a stop gap measure that does not adequately address a staff shortage critical to fulfilling the directorate’s mission. - There may be over-reliance on the assumption that through restructuring the gains in capacity building at the national level will be replicated at regional level. 20 - CW does not have an electronic HR MIS -- all personnel files are maintained as hard copies. They are in the process of requesting desktops so may be able to create e-versions; regions have 3Gs (as of last year) and are only in planning stages for a network that would enhance the Performance Management System (PMS). - CW is struggling with managing budget and expenditure streams (and reported being in danger of running out of funds within the next two months). However, the end of the fiscal year is in March and there seems to be a couple months “gap” in renewal of funds that has to be accounted for in future planning. They do not feel confident that they can continue budgeting to achieve performance objectives or track distribution and utilization of funds at decentralized levels without further capacity building assistance. - The CW staff reported the need to do an “impact study” defined as the cost versus benefits of services delivered via social workers in order to justify salary (and CW budget) increases. - One CW staff who has studied how CSOs work in the US stated that the government has not yet developed a mechanism for how to work optimally with CSOs. The CW has a plan (only) to approach the National Partnership Framework (NPF) to engage and outsource to more NGOs (currently this arrangement is only with the National Council of Churches). - There is a perceived need for advisors to be placed at “ministry level to work with and push directors” in order to institute lasting organizational change. 4. Early Childhood Development Perspective of Pact Early Childhood Directorate (ECD) Advisor Pact’s Primary Contributions: - The Advisor devised criteria for the allocation of ECD funds to the Regions (used during FY2011/12 and FY2012/13). - The advisor worked to adapted regulations of the Child Care and Protection Bill, with reference to the ECD provision. - Assisted with preparations for the MTEF Budget review in October 2011, which determined budget allocations for ECD in 2012/13. - The Advisor has facilitated budget deliberations (still not fully based on strategic objectives) and planning for greatly increased ECD budgets within the MGECW. (The ECD budget was 2010-11 NAM$400,000 for 2010-11; then NAM$5M in 2011-12; then NAM$10.5M for 2012-3, just for activities (no staff)). - Assisted with shifting fund distribution decisions to include allocation based on access, quality, and equity, with a high degree of ownership for this (new) decision making process. - Pact facilitated a shift in distribution of grant funds through Regional Councils for the first time. - The advisor conceptualized and wrote the TORs for two additional USAID consultancies to develop an ECD 2-4 curriculum and an ECD qualification on the National Qualifications Framework for Human Resources. (These are expected to result in a career path for ECD teachers but have not yet been implemented.) And the advisoralso helped the Ministry with the selection and managing of two other consultancies funded by ETSIP – the ECD Standards and the ECD database. - Assisted ECD to deliver a successful Cabinet Submission and Implementation Plan for subsidies for ECD centers (eliminating dependence on families’ fees to send children to kindergarten and resulting in the first time GRN funds are planned to be used to fund ECD caregivers). - Assisted the Director in finalizing a report with recommendation from the High Level Technical Committee on ECD to formalize ECD provision. - Assisted in conceptualizing the MOU between Saskatchewan and the MGECW to train assessors of ECD centers (yet to be implemented in order to operationalize new ECD Standards). - Assisted with finalization and launch of the National Agenda for Children, an inclusive framework for all children (which expands the “target population” for the MGECW). - Assisted in encouraging a private –public partnership with a private foundation operating in Otjozondjupa Region to build and support ECD centers. 21 - Due to severe limitations in staffing, the advisor has not trained people but rather identified junior staff within ECD and helped provide on-the-job training and involving them in more critical tasks. - Pact helped design and institute MARS (Monthly Activity Reporting System) that helps the MGECW report against such PMP targets as “Number of MGECW Directorate accomplishing at least 80% of the planned activities from previous year that are funded through the MGECW and in line with the Strategic Objectives” (PMP, April 2012, pg. 21). - The advisor helped to introduce a health information system (the ECD-specific MIS); Pact also ensured that an external consultant was hired to develop a very simple, user-friendly ECD-specific database with kindergarten data; it is expected that ECD has the capacity to continue operation of the database and use of the data to track kindergarten services by location, number of ECD teachers, and line item budget. Limitations and that which was left undone: - The ECD Directorate has huge capacity issues that cannot be addressed by 1 person in 5 years, including a director who needs significant ongoing mentoring as a manager. - The ECD director is not confident, “I often have good ideas and the right instincts but people don’t listen to me….” this has meant that much of the Pact advisor’s contribution has been to encourage and validate the director. Much assistance will be needed to capacity build executive manager capability. - The MOF’s (activity-based) budget is not aligned with (General Services of the) MGECW and doesn’t allow directors of directorates to oversee expenditures. - With the anticipated huge growth in ECD budget and need to distribute NAM$4M out of the $10.5M for 2012- 3 to ECD centers, government is facing a critical HR issue around the enormous growth of the public sector force. - The proposed MGECW restructuring, during which ECD will become its own directorate with much more staff, is evolving very slowly. The current lack of staff limits an advisor’s ability to have impact through counterparts; this has necessitated an approach focused on standards setting and HR frameworks, in anticipation of added staff. - While activity-level data gathering and reporting has been improved (MARS), it has not achieved linkage of activities to Strategic Objectives with a MGECW-wide strategy. - Support and involvement at the PS level is needed to bring each directorate’s staff at constituency level together and to build their capacity ( CCCWs, SWs, ECD liaisons, etc.) in order to comprise operational ”child care team”. While some Pact-assisted training of local ECD counterparts has occurred, this falls short of step￾wise functional team building to provide a well defined and quality controlled set of services and support at community level. - The (new) ECD budget distribution decision making practice (which considers number of children underserved, etc.) is a “high tech” process that ECD will not (yet) be able to sustain on their own without further assistance. - Outstanding “to do’s” within ECD include: o Implementing roll out of the subsidy payment process (which starts in September, 2012, coinciding with end of project); o Reviewing the 12 week basic ECD course to improve content and delivery o Reviewing the Family Visitor Program to align it within the subsidy system. o Aligning the overall MGECW Strategic Plan (in development) with the ECD annual work plan and with the NDP4, both within the MTEF. Perspective of Early Child Development Directorate in MGECW (including Pact counterparts) Pact’s Primary Contributions: 22 - The most significant Pact contributions have been:  Organizing the work and prioritizing activities to focus better. Most impactfully, Pact helped ECD strategically plan as a team that included team planning of regional annual workplans (so organizational capacity building was delivered to build a collaborative, decentralized team within ECD). The team was self-identified and wore (Pact provided) t-shirts that said “Every Child Counts” during the ECD baseline exercises.  Help with advocacy need to improve ECD, specifically with regards to improving financial allocations, preparing ECD to participate in a (MOED) national education conference last year, and successfully lobbying the Cabinet for increased budget to support ECD volunteer teacher subsidies plus small allowances for caregivers. - Received training on Excel and on strategic planning (LogFrame methodology) for annual work planning; Pact also provided training via 3 workshops to train staff to carry out baseline survey for the ECD database (including developing a TOR for an outside consultant to develop the ECD database and conduct evaluations and determine scoring). Data is collected by location of kindergarten center; number of children enrolled; number of caregivers; whether it is a private, community-based organization (CBO), or faith-based organization (FBO) – run kindergarten; and number of children progressing through or dropping out of kindergartens. The director stressed the importance of this data for allocating resources. - Pact helped redesign the MGECW’s overall organogram. The director would like one ECD officer in each of the 107 constituencies. - Pact helped ECD “focus on what activities they were supposed to carry out in a given month”. This ability to implement and report on activities is especially critical during monthly Administrative and Management meetings at MGECW in which each directorate reports activity-based progress. - Also had to report these during monthly Admin and Mgt meetings – and on the reporting format is a column for what do you do to correct and get activities done Limitations and that which was left undone: - Pact assistance to ECD has been very limited ( one advisor from April 2011 - present) - Needs additional assistance to pilot implementation of the ECD teacher subsidies and caregiver allowances and measure impact. - change management question: Working on a plan to outsource distribution of the teacher subsidies to civil society - With only 10 staff (for the currently combined ECD and Community Development directorate), the director felt “they can’t do everything” with regards to developing new mechanisms to manage in a changing environment. There is not yet a vision that mechanisms for actively involving and supporting CSOs is a directorate responsibility. The director felt that it might be a regional ECD function to develop systems for outsourcing to CSOs (in order to play a very important role in distributing ECD subsidies within the new program or to help train caregivers or manage orphanages) or maybe via the Church Alliance for Orphans or DAPP Development from People to People. - There are limitations to providing (only) regional trainings; while Pact provided training to the RITs (Regional Implementation Teams), at this point there is no “train-the –trainer” or other trickle down mechanisms to Regional Councils but has been delivered to CW regional staff and other government regional staff. - Outstanding needs include:  Help with community development and mobilization programs (e.g., income generating activities (IGA) like providing seed money for uniform sewing groups and other community projects) - Though ECD received Pact assistance to develop job function categories, there is an outstanding and urgent need for technical assistance to develop job descriptions for all the new staff expected as ECD becomes a separate directorate. The director felt this assistance was necessary for Public Service Commission (PSC) approval. 23 5. Gender Equality Note that there was no Pact Advisor seconded to this Gender Equality Directorate. All contributions listed as “findings” here are the reports of a Gender Equality counterpart who participated in some (limited) Pact trainings that were provided cross-directorate. Perspective of Gender Equality Directorate (GE) in MGECW (including Pact counterparts) Pact’s Primary Contributions: - From this person’s perspective, Pact’s greatest impact is across the entire MGECW in M&E training and systems development. Pact worked with the CW directorate first and then all of MGECW, though there has not been significant organizational capacity building within this (Gender Equality) directorate. - Pact helped with M&E and worked with the Management Implementation Team (MIT), which includes the Gender Equality director level. - Pact has helped to identify opportunities for improved integration of OVC issues so that they are seen as cross￾cutting issues for each directorate, including Gender Equality. - The Gender Equality directorate tries to align with the National Agenda Policy and the 10 year strategy on Gender Based Violence. (It is unclear the extent to which Pact contributed to this alignment however.) Limitations and that which was left undone: - This technical specialist did not work directly with the Pact Advisor, but attended some workshops of generic managerial interest. - Pact had no technical gender expertise so GE received (ongoing) help from the UNFPA. The Gender Equality directorate has different issues and so it brought in the UNDP, but GE wants an expat expert in gender issues to also help with organization capacity, etc. - No direct support from Pact in areas of organizational capacity building or systems strengthening. - Though Gender Equality asked Pact for assistance with a gender-based violence (GBV) database, it was not forthcoming. - The Pact M&E Advisor could not design gender-specific indicators; this limited Pact’s contributions, even to the M&E component of the Gender Equality directorate. - Since gender is also a cross-cutting issue, much work remains to increase collaboration between Gender Equality and NDP, AIDS/MOHSS, etc. - Whenever the Gender Equality directorate wants to engage with NGOs, it goes through NANGOF – Namibian NGO Forum. It was unclear if this included formal funding mechanisms yet. - The National Council of Churches (NCC) has a MOU with GRN as a mechanism for ministries to outsource gender services to CSOs, but this hasn’t really started working yet. Yet the Gender Equality directorate relies on the church and its networks to support campaigns against “baby dumping”, passion killing, GBV, and human trafficking. B. Civil Society Organizations 1. KAYEK Note: KAYEC was established in 1993 by the Anglican Diocese in Namibia and the Evangelical Lutheran Church. In 1999, KAYEC was registered as a Trust with the High Court of Namibia with establishment of a vocational training center in Katutura, and in 2002 a second Vocational Training Centre was built in the Oshana Region in the town of Ondangwa on the premises of Rossing Foundation. KAYEC’s current programming includes: a) enabling out-of-school youth to access vocational skills training, and b) a Youth Development (KYD) Program addresses educational, 24 psychosocial support and HIV prevention needs of youth aged 12-18 who are in–school. They have a total of 1400 graduates in 9 vocational trade areas. Within the Modified Agreement timeframe, Pact provided assistance to KAYEC from November 2010 to March 2011. It was the fourth round of USAID-funded assistance that KAYEC has received during its history. KAYEC attributes its success with “graduating” to direct USAID funding (in April 2011) directly to the quality of Pact assistance received. SUSTAINABILITY PEPFAR –supported institutional strengthening translates to increased donor recognition and future support. Pact assisted in “upgrading” KAYEC’s staff pride and loyalty, control systems and organizational capacity to meet USG standards. Along with a unique mission related to vocational training and youth development, this organization-wide “upgrade” provides a substantial competitive advantage for capturing funds from other donors, thereby increasing chances for sustainability. Increased appreciation for sustaining human resources. Pact assisted with developing a strategic plan and a HR Strategic Plan using a Balanced Scorecard approach that improved the organization’s impetus to sustain human resources. Pact also assisted with a systematic review of KAYEC’s 18-months work plan. Building sustainable relationships. While KAYEC’s core business is not in the health sector per se, Pact has assisted them with longstanding participation on the Permanent Task Force of the MGECW. ORGANIZATIONAL CAPACITY BUILDING Improved staff development and performance appraisal processes. Pact provided substantial assistance to help strengthen these processes and as a result KAYEC now allocates funds specifically for staff development. As the director stated, “….with the new performance appraisal system, the main thing you have is you know where you are trying to go….” However, Pact did not have the technical ability to help set performance objectives or provide training (in the vocational training field. Building internal systems for career planning and staff promotions. Though they did not previously develop career paths, due to Pact assistance, KAYEC has promoted senior advisors to a managerial level, increasing the number of managers overseeing the vocational training program within the organization over the last year. Rigorous strategic planning processes instituted, including strategic hiring and succession planning. *was instrumental in helping KAYEC set targets linked to strategic objectives. *helped significantly with developing an organogram (inclusive of 48 staff positions) and bringing on a full complement of staff. *helped the executive director plan a six months transition and overlap period into a process of hiring his own replacement. Step-wise completion of institutional strengthening plan. *provided quality staff or consultants to work in a very practical, systematic, and collaborative manner with KAYEC to assess organizational weaknesses and implement a comprehensive institutional strengthening plan (CISP). However, this tool seems time-restricted, used primarily to monitor Pact assistance and KAYEC’s improvements during one limited period of time (before “graduation” to direct USAID funding). KAYEC did not incorporate the tool as an ongoing part of organizational assessment and development. SYSTEMS STRENGTHENING Significant improvement in M&E processes and systems. With Pact’s assistance, the director of the educational training program began to collect student achievement data and progress in grades. KAYEC has just hired a M&E advisor, so is just beginning to utilize this data in order to improve programs for school age youth. Pact also assisted the KYD program to conduct focus group discussions in order to ensure the program was meeting the priority needs of participating youth. Increased ability to track financial resources to beneficiary level. Through Pact assistance, KAYEC has been able to compile organization-wide budget and expense sheets, while still accounting for separate funding streams for each of their (two) campuses. “Due process” has been integrated into financial systems and the increased accountability was one factor in KAYEC’s successful “graduation”. However, there was lack of continuity or quality control in Pact-provided financial advisors. Improved policies and procedures. KAYEC described Pact’s assistance as a “systematic approach” to developing policies and procedures (for procurement, finance, travel, and human resources). (See “Policies and Procedures of the KAYEC Trust, July 2010”. Ongoing improvement in data management and procurement systems, during and after “graduation”. Pact provided direct assistance to KAYEC during development of the proposal and budget that was acceptable for direct USAID funding. It is notable that even during the first year of direct USAID funding, KAYEC retained a line item in the budget to be able to obtain periodic Pact assistance, specifically for setting up data systems and data analysis; training in Excel, and developing procurement documents for new tool kits for a program. 25 OTHER Key Concern and Opportunities: - Unionization movements are introducing unknown external factors into staff hiring, career development, and retention. - Different size organizations are impacted differently in a climate of “USAID direct funding”. While it is apparently cost effective that USAID will reduce the administrative costs of an “intermediate” funding and technical assistance group (like Pact, Inc.), there is increased vulnerability for smaller organizations that also benefitted from Pact capacity building assistance but who are not guaranteed ongoing USAID direct funding and are facing an overall environment of diminishing resources - There is a relatively lengthy learning curve and adjustment period needed for an organization to conform and adhere to USAID requirements. Considerable concern was voiced among multiple “graduates” that it requires additional human resources to meet all the financial and reporting requirements and that the general perception is that there is no room for failure or non-compliance at any level. One area where Pact could have provided more assistance is in the area of preparing USAID-acceptable financial reports. - There is an opportunity for USAID to build a “learning environment” into the contractual relationships with newly￾graduated CSOs. 3. Catholic AIDS Action Note: Pact provided assistance to CAA from 2006 – June 2011, when CAA “graduated” to USAID direct assistance. Therefore, questions were directed only for the “slice” of time corresponding to this evaluation’s timeframe (November 2010 – June 2012). CAA has 15 field offices that provide home-based care to 7500 clients and provide primarily psychosocial support and support for school attendance to 12,000 OVC per month. SUSTAINABILITY Diverse resource mobilization: While CAA does not yet have a written resource mobilization plan, considerable strategic thought has been focused on the fact that USAID funding comprises 75% of current funds. CAA is a longstanding organization so has qualified assistance from Pact as “minimal” for strategizing the following: a) transferring OVC activities to local churches to fund and sustain; b) joining alliances of CSOs (e.g., the Prevention Alliance Namibia) to seek funding together; c) strengthening arrangements with the private sector for in kind donations (e.g., fish for OVC food programs); d) designing a program of home based care client-to-client support, linked to (sustaining) health facilities; e) reducing CAA’s strategy to OVC-related work only in view of severe expected reductions in donor funding; f) outreach to MGECW in anticipation of direct USAID funding to GRN in the near future; g) shift in training focus to build psychosocial support skills of direct caregivers, instead of more sporadic staff and/or volunteers. Ability to implement change management: Due to Pact’s training on child participation and children’s rights, CAA participates more actively on the (national level) Permanent Task Force (PTF) as well as regionally on Regional Child Care and Protection Forums/MGECW; Regional Home-Based Care Forums/MoHSS; and RACOCs; and at constituent level on DACOCs. Pact’s contribution to strengthening CAA management was especially on the national level. Pact played a significant role in linking CAA with the (Ugandan) Africa Palliative Care Association; while this occurred in 2007, this organization is still currently funding CAA’s palliative care program, including integration of palliative care into basic health services. Linkages to corporate responsibility programs: Pact linked CAA with corporations like Daimon, DeBeer, Standard Bank, and Namibia Dairies. CAA is receiving direct and in kind resources to support OVC soup kitchens, etc. (level of funding not specified). ORGANIZATIONAL CAPACITY BUILDING Arranging for technical skills training: CAA reported that Pact’s biggest contribution was providing funds and access/referrals to training to increase staff technical abilities; Pact provided direct training in M&E and financial controls Building human resource capacity (and also sustainability): 26 Pact’s provision of funds to CAA enabled training, reward, and retention of community volunteers (i.e., a base of 2400 volunteers built over the course of 14 years ); this training was vital in enabling many/most volunteers to become and be recognized as leaders at community level (i.e. sources of health advice, referrals, etc.). The presence, skills levels, and roles of these volunteers sustain CAA’s efforts and would remain even if the organization no longer existed. Building performance standards for administration, implementation, and capacity building plans: Pact did not assist CAA to develop a HR Strategic Plan. Pact did assist with OD Roadmaps and CISPs that were used to report results and to determine progress towards institutional systems and structures for capacity building. Strategic planning: Pact assistance was key to formalizing and improving the CAA strategic plan. CAA rated Pact’s assistance a “7” out of 10 in terms of achieving “graduation” to direct USAID funding, primarily because Pact helped CAA develop a strategic plan and be responsive to USAID’s reporting requirements. However, Pact provided moderate (a “5” out of 10) assistance in helping CAA identify program priorities. In 2010, Pact started to help CAA try to measure impact of services and support delivered, which has continued to the present (post-“graduation”). HR Management Systems, Career Planning, and Task Shifting: Staff salaries have been flat-lined since 2009 and staff retention is an issue (14 out of 100 staff positions are “vacant”) as staff are seeking government positions “due to the donor climate”. However, CCCWs have been promoted internally to administrative or regional CAA positions. The current annual performance appraisal system is not very effective, so CAA is now working with a (non-Pact provided) consultant. HR MIS: Prior to Pact assistance, CAA used an Excel spreadsheet for MIS; CAA successfully adopted and is utilizing an electronic “VIP salary package” with a backup system of manual personnel files. SYSTEMS STRENGTHENING M&E Systems: Pact’s strongest contribution to CAA was the technical and financial support to help CAA staff improve their M&E system. Assistance was provided in various ways: via workshops, and 1-on-1 with CAA staff to engage in problem solving specific to the needs of the organization. System to support HR at social worker/community child care worker/volunteer levels: CAA already had a strong RN, HBCW, CCCW and volunteer base of support and continuity. Little Pact assistance was required in this area. System for collection and utilization of data for key decision making: Pact provided very effective assistance (a “9” out of 10) to help CAA design a database (which CAA is maintaining and improving now). The database is used for coverage and comprehensiveness of OVC and home-based care services and is in a position to also report (to the MGECW) on a quarterly basis. Financial Accountability Systems: (Prior to Pact assistance, CAA received significant assistance from Family Health International and from the European Union to strengthen financial reporting.) Pact helped CAA improve their accounting software (Pastel Evolution) so the reports more explicitly and accurately document expenditures by activities for USAID. CAA stated they are the only NGO that has this ability to analyze expenditures by region and by field office. There has been a change in executive directors at CAA which has led to a need for additional support at that level to be able to supervise financial system functioning. OTHER Key Concern and Opportunities: CAA is: - not aware of any concrete plan by MGECW to actively collaborate, engage, or fund CSOs. - may be in a position to technically assist the MGECW with measuring impact but there is no mechanism by which to accomplish this - very aware of an upcoming (September 2012) USAID competitive bid process and will need to purchase external consultant assistance for this. - stated Pact’s assistance was a “6” out of 10 over the entire five years effort. - believes a longer timeframe than 3 – 4 years is needed for strong organizational sustainability planning, given the complex nature of their programming to support OVC and provide palliative care; and also due to complex underlying social issues, e.g., adherence to medication and other long-term care issues that, by definition, must be addressed at community/constituency level where CSOs play a vital role. 27 4. Lifeline/Childline Lifeline/Childline (LLCL) – NOTE: LLCL has been in operation for 32 years and consists of staff at a Windhoek-based central office, the New Start program and counseling center, and 3 regional offices. The organization began as a completely voluntary Lifeline/hotline for people suffering from depression or suicidal ideation; it then merged with Childline and registered as a welfare organization under the name “Lifeline/Childline” in 1998. LLCL’s OD Roadmap Scores show minimal improvements because LLCL focused on implementing a Pact child protection program as a “critical partner” with two other CSOs for one and a half years, reducing the actual amount of assistance to about half a year before LLCL “graduated” to USAID direct funding in July 2011. Only a small portion of this assistance (i.e., the 7 months from November 2010 – June 2011) constitutes the timeframe for this evaluation. The potential impact of Pact effectiveness might be reduced due to the fact that this longstanding organization already had many systems and considerable organizational capacity before Pact assistance began. LLCL also received IntraHealth assistance during the same timeframe. SUSTAINABILITY Participation in regional coordination and planning entities. LLCL does not participate in Regional Child Care Protection Forums and does not believe they are functional. Creation of sustainable business units. LLCL has started to utilize its strong training department on a fee for service basis in order to cover overhead costs of the organization and (possibly) improve funding streams. Issues with staff retention and change management abilities in this environment. The specter of dramatically diminishing donor funding directly, negatively impacts staff retention. There are 6 vacancies out of a total of 65 staff (almost 10%), including one crucial senior position. LLCL is responding to this environment by refocusing on its “core business” and redesigning its strategic plan to ensure this part of the organization will persist, regardless of the possible funding scenarios. Task shifting and sharing is planned in anticipation of further staff losses. Management is reviewing how to be more cost efficient and is increasing the number and frequency of planned fundraising campaigns, based on the organization’s strong brand and name. LLCL is also attempting to diversify its donor fund portfolio and is currently preparing 7 proposals and responding to government tenders by the (training) business wing. ORGANIZATIONAL CAPACITY BUILDING Evidence of strong planning, administration, and performance management systems. LLCL reported that Pact’s strongest contribution was in the area of management advice. Pact helped staff develop general but rigorous performance objectives and Pact staff contributed their technical competence. LLCL has a clear strategic plan articulated down to the strategic objective level. The organization has an organogram with associated job descriptions for each staff and volunteer position. Human resource management including professional development and career planning. LLCL advertises and recruits using standardized interview panel and questions. LLCL has established an annual employee appraisal system that links performance results to objectives; however, employees’ performance is not “scored” for quality, efficiency, productiveness, etc. and a salary-based incentive system has not been instituted (salaries are adjusted for inflation only, regardless of performance). Career planning and step-wise progression is minimal and only three staff has been promoted from within during the last year. The organization is beginning to engage in succession planning since the executive director will retire in two years. LLCL does not have a strong system of individual development plans (IDPs) but is seeking assistance from KPMG on a pro bono basis (corporate responsibility program) to leverage needed capacity building support (apparently not provided through earlier Pact assistance). HR MIS: LLCL only maintains manual records systems and did not receive Pact assistance for this. SYSTEMS STRENGTHENING Financial system strengthening: LLCL did not find Pact’s approach to financial training useful as it emphasized Pact and USAID rules and regulations rather than providing hands on help with establishing financial systems, financial management processes, and organizational-specific problem solving. LLCL effectively used IntraHealth’s assistance for financial systems strengthening instead. Institutional Strengthening Plans and Proposal Development: – LLCL reported that Pact was helpful in developing the CISP as an important adjunct to already-established internal planning. Pact’s role was as “critical friends and advisors who helped to identify weak areas, though Childline/Lifeline did most of the work themselves.” In addition to helping LLCL complete the CISP (required for “graduation” to direct USAID funding), Pact worked very closely with LLCL staff during proposal 28 development for USAID ; some skills transferred during that process still remain in the organization, but skills transference was limited by the degree of direct Pact proposal writing necessitated by a short timeframe. Establishment of an OVC database and use of the data for critical decision making. Operation of the OVC Warehouse seemed to depend on one particular Pact staff person. The quality of data on child protection issues is poor but LLCL is still able to input their data for a clearer national “snapshot” of child protection services and gaps. LLCL was mentioned by name in the National Agenda for Children as a key entry point for child protective services; its helpline is one of 15 child protection indicators. M&E System Strengthening. LLCL was very positive about the Pact assistance received for improving their M&E system and described the Pact “supportive management style”. Pact M&E assistance was so strong that they helped LLCL begin to gather input from program beneficiaries and analyze the data for program adjustments. Decentralized resource tracking systems. LLCL reported no Pact assistance to improve the ability to track resources (financial, human) down to the impact (beneficiary) level. OTHER Concerns and Opportunities: The fact that LLCL “graduated” according to the most rigorous USAID standards has a positive spillover effect as it is seen as a good credential for other funders. The timing of “graduation” coincided with the departure of many donors so that strides made in organizational capacity and systems did not necessarily translate to sustainable funding sources. There is great uncertainty about whether funding will continue to be available (or not) as USAID transitions to a “technical assistance only” role. Since 90% of this organization’s current funding is from USAID, there is organization-wide concern of closing or rolling back to a volunteer-only organization. 5. Legal Assistance Center (LAC) Note: LAC was established in1988, before Independence. The organization is comprised of 34 staff in the central office, 2 staff in the north, 4 consultants, and a stream of expat interns (usually from Canada or the U.S.). The Gender Research and Advocacy Project (GREAP) and the AIDS Law Unit are flagship programs that have worked with the MGECW and other parts of government, with USAID and other donor assistance. The majority of LAC’s financial assistance is from international donors: European donor, Embassy of Finland, Canadian International Development Agency (CIDA). Only the AIDS Law unit (established in 1999) is supported by USAID funds, which comprise about 25% of the total funds.) It was challenging during this evaluation to separate the contributions Pact made to LAC as part of a joint child protection program versus specific Pact provision of organizational strengthening assistance to LAC during 2011. SUSTAINABILITY Strengthened internal processes for key decision making and continual resource mobilization. Pact inputs helped to strengthen LAC’s “coordinating committee” (a managerial planning committee) to better analyze and make critical decisions about sufficiency and continuity of financial and human resources. Development of sustainability and resource mobilization plans and their limitations. Pact helped with development of these documents which outline potential new avenues for resources, but did not help in practical terms with actually finding additional resources. LAC characterized Pact’s help in this area as “minimal”, which might be expected of such a mature, longstanding organization that has already had considerable success in proposal development and favored well during many competitive bids. Some of the drawbacks of this assistance include: Pact’s interventions are specific to meeting USAID rules which is of minimal assistance/impact to an organization (such as LAC) required to respond to so many different funder requirements. While Pact was open to helping the entire organization, the limited systems required for enhancing organizational capacity and sustainability of the AIDS Law Unit were not appropriate/applicable to the rest of the organization. The AIDS Law Unit did not receive all the financial-related interventions needed for sustainability. LAC understands the need to operate like a business in order to be sustainable but needs additional funds to strengthen HR and other resource management systems. Organizationally, they have reached a point where they believe they need to hire a 29 full-time, dedicated development officer for fund-raising in order to allow other staff to focus on technical areas and program outputs. Pact helped LAC decide to develop promotional materials that might strategically position LAC to capture private (corporate) funding support. Perspective on change management and sustaining critical programming in an environment of diminishing resources. LAC participates in a child protection program (with other CSOs, under Pact funding). Forecasting the end of funding, they are working closely with the MGECW to institutionalize this program as a national government program. LAC’s role would then shift to a consultative role to the MGECW to help expand the pilot program beyond 3 regions currently being served. The underlying assumption is that highest levels of government will be willing, and deem it necessary and advantageous to begin outsourcing consulting services to Namibia-based CSOs. Sustainable participation in Regional Child Care and Protection Forums. Pact assistance was not necessary to improve LAC’s participation on RCCFs; however, they did facilitate LAC becoming a part of the Permanent Task Force (PTF) and LAC is now reporting on a child protection indicator in quarterly reports. LAC reported high levels of participation (“80% of inputs from CSOs”) within current gender and child protection frameworks, but felt that sustainability was highly dependent on relationships with current MGECW staff. The condition lacking for fully sustainable contribution is any financial mechanism for direct funding agreements with government. ORGANIZATIONAL CAPACITY BUILDING Short-term utilization of OD Roadmap and Comprehensive Institutional Strengthening Plan (CISP) to facilitate organizational progression. These tools were developed in a consultative manner but using an already-set list of areas for capacity building, not geared to more mature organizations like LAC. The process was somewhat different as it meant opening LAC’s reporting and other systems to Pact’s external review and assessment (which LAC did not always agree with….but eventually realized gains from in terms of how they conduct business). They were valuable for focusing on areas where strengthening structures and processes was needed and for conducting ongoing progress check ins. At this juncture, only the OD Roadmap baseline has been completed (not the progress measures). LAC is uncertain about using any of these tools past the Pact assistance period. Increasing financial and technical inputs to plan for and improve capacity building. LAC reported that though Pact provided funding and technical assistance only to the AIDS Law Unit, the technical organizational development assistance provided, also impacted the entire organization in terms of improving program implementation and financial and M&E systems. Gains in strategic planning and internal policy development. Pact has provided input on HR policies and financial policies. A Pact consultant reviewed all conditions of employment and flagged areas for improvement. Pact also trained LAC on LogFrame methods and assisted with (moderate) revision of the 3 – 5 year strategic plan. Human resource management inclusive of performance objectives linked to strategic objectives. Previous increases in salary were linked to performance objectives at LAC; with a staff of only 32 people, they decided to shift to more informal performance reviews by unit. These reviews are not yet formally linked to metrics for salary increases. The HR MIS system is still a manual system but LAC didn’t feel the need to use resources to create an electronic database for such a small staff size. SYSTEMS STRENGTHENING M&E system strengthening. With Pact assistance, LAC is in the process of expanding the M&E system. A Pact-provided M&E “101” is planned for August/September 2012. In the meantime, Pact is providing technical assistance by reviewing and improving every report, even quarterly reports. Utilization of data from OVC Warehouse. Pact has helped LAC understand how to access, input, and utilize data from the OVC Warehouse at the MGECW. LAC is using data from the OVC Warehouse as baselines to justify the importance of planned interventions for cases of violence during proposal development. Pros and cons of Accounting System development. Pact helped to improve financial processes by working with LAC’s 3 accounting staff to review financial policies for gaps and improvement. Pact also provided 1-on-1 and workshop training on elements of USAID-specific financial reporting requirements and structures, and Pact’s consultant has been very available to assist with problem solving (which has not yet been actually needed by LAC). One challenge for LAC is that as an organization it has established one overarching accounting system but now is required to collect additional data for different time periods specifically for one department (the AIDS Law Unit). This is a very real human resource issue and reporting burden with such limited staff and for the “coordinating committee” who are asked to report across many different frameworks. OTHER Key Concern and Opportunities: 30 - As a smaller organization, LAC anticipates increased vulnerabilities due to gaps in funding (between the end of the current funding period and before the next tranche of money has been approved by Congress). It does not have a substantial enough financial “reserve”, so this threatens sustainability. - Organizational capacity development assistance should be more tailored to widely different organizational needs. This should begin with a more specific organizational needs assessment rather than interventions that are generalized and assumed to “fit” organizations across all levels of development. - The nature of some “core business” is very difficult to quantify in a way that is readily apparent to donors. For instance, it is relatively easy to show “cost effectiveness” by distributing uniforms or school supplies or other easily quantifiable units of service. But interventions that are labor-and time-intensive (such as child protection interventions that result in children re-entering and completing school) seem less cost effective. Donors need to consider the value of every child’s life and future potential, though they are challenging to quantify. 6. Caprivi Hope for Life (CHFL) Note: CHFL was established in 2001 to deliver HIV related information at community level (e.g., via meetings, drama). The organization only has 9 staff members. It was registered in 2006 as a welfare organization. CHFL received organizational strengthening assistance from Pact Namibia from November 2010 – June 30, 2012 (19 months) with an additional 15 days no cost extension. SUSTAINABILITY Limited strategic and financial sustainability planning. CHFL did receive Pact assistance to develop a strategic plan, but reported that it was developed too quickly, needs another round of revision, and needs to include a fundraising plan. CHFL has not yet been able to strategize priority actions. Because CHFL just recruited a resource mobilization person (during the last 4 months), the organization has not had the capacity for marketing, writing proposals, and implementing fundraising events, all necessary for sustainability. Increased ability to financially plan has not (yet) translated into sustainability. Pact has enhanced CHFL’s capability for fundraising through competency-based training workshops to align them to USAID rules/policies/regulations. However, this assistance has fallen short of actual financial planning or help with prioritizing and implementing actions. Enhanced visibility and creative collaborations with potential for sustainability. CHFL’s prominence has increased due to Pact’s assistance with strategic positioning. CHFL is the chair of a newly-created Regional Advocacy Forum for NGOs that targets government funding and provides a mechanism for creating sub-agreement partnerships with other local NGOs. The organization also regularly reports to RACOCs, and their volunteer health promoters participate with MGECW by referring potential beneficiaries to social grants. The MoHSS and Parliamentary Committee have been interested in assessing CHFL’s sustainability. ORGANIZATIONAL CAPACITY BUILDING Monitoring and reporting capacity building. Pact assisted CHFL by providing skills-based training (for one dedicated staff and the program coordinator) in M&E planning, developing a database, and developing a data quality management plan. As also captured by the OD Roadmap scores and the CISP, Pact assisted CHFL in better defining open staff positions and with identifying/addressing compliancy issues and improving reporting to better align for direct USAID funding. Strategic planning and policy development. As a result of Pact assistance, CHFL has developed a long-range strategic plan (2011 – 2016) with all the elements necessary to match strategic and performance objectives. With Pact support, CHFL also has Policies and Procedures manual and a succession plan. Insufficient human resource assessment and management strengthening. While the executive committee did receive training, one shortcoming was that a salary evaluation assessment to benchmark CHFL’s human resources to performance and market rates was never carried out. There are also a number of staff positions to fill; they feel that revising the salary structure is critical to hiring qualified staff (particularly because equivalent staff from other regions is paid twice higher salaries, which CHFL cannot afford). SYSTEMS STRENGTHENING Participatory development of decentralized exchange processes (with competently skilled volunteers and other community￾based workers). Pact facilitated cross-training events to support CSO-to-CSO skills development. For example, a training for grassroots health promoters involved C-Change, the MoHSS, Lifeline/Childline, and CHFL (who staff trained on issues of community mobilization, administration, drivers of the HIV epidemic, M&E, male engagement, and male circumcision). 31 Though this is not systemized, Pact’s inputs to CHFL are transferred to other stakeholders and CHFL’s role and presence is expanded in the process. Improved data for evidence and decision-making. Through Pact assistance, CHFL established a database, with data inputs from field promoters, and provided reviews and suggestions for improvement. CHFL reported using the data as evidence for proposals. They also use the data for internal programmatic decision making; e.g., the data revealed low participation by male counterparts which initiated additional studies that were funded by Pact. Systematic, routine inputs from beneficiaries to improve programming. A major area of organizational behavior change was that through Pact encouragement, CHFL began “quarterly sharing meetings” directly with beneficiaries to review their progress, achievements, and costs together. As a result of this more inclusive process, various volunteer challenges were addressed and beneficiaries understand the value of receiving HIV prevention and other education, resulting in increased appreciation of CHFL’s programs. Improved aspects of budgeting and accounting. Pact provided significant inputs to identify cost drivers and to help CHFL conduct a cost analysis to track how much was spent per cost driver (e.g., transport, training, special meetings). OTHER Key Concern and Opportunities: - CHFL is unclear about whether the fact that they have achieved a level “3” (for effective delivery, finance, management, and M&E systems) qualifies them to “graduate” to direct USAID assistance or what the process for learning this is. - CHFL and other similar small community-based CSOs need international donors to engage government for a better understanding of the “value added” in terms of better coverage of beneficiaries at constituent level. They are hoping government and donors will award grants to the collective of CSOs that has formed the Regional NGO Advocacy Forum. -There is an inherent problem in that training/capacity building staff inevitably leads to high staff loss from CSOs and those staff take government or private sector positions (due to increased skills and marketability). -The timeframe for organizational support of small NGOs is insufficient; more mentoring time is needed to actually develop and carry out a resource mobilization plan (which CHFL does not believe is possible, without Pact’s continued assistance). - CHFL staff did not have access to outside (expat-delivered) trainings due to constructing an inadequate budget. - The (VSO?) person who developed CHFL’s website did it very hurriedly and not professionally. 6. People’s Education Assistance & Counseling for Empowerment (PEACE) Note: The Peace Trust was established in 1996; the mission is to assist people to address the impact of all forms of violence and other events that result in trauma, including trauma of children due to rape, abuse, and other violence. Services have included central and decentralized counseling support and training for other service providers, including the MGECW. PEACE’s model for providing OVC support and child protection was being pilot tested in 4 (out of 13) regions. PEACE is a very small organization that relies on strengthening networks of child care advocates and watch dogs (such as police, nurses, and prison officers). Most recently it had a total staff of 10 (including psychologists, 2 counselors, 1 director trained in social work, 2 RN project managers, 1 administrative assistance staff, 1 financial officer, and 1 support staff; additionally, it has a Board of seven members. PEACE serves approximately 58 new clients per quarter; typical services include conducting a minimum of 4 progressive counseling sessions per client and additional interventions with caretakers and parents as well as facilitating parent support groups. In 2009, PEACE became one of 3 CSOs within the Pact child protection program; it played a unique role in supporting traumatized children. That was also the first time that the organization had received PEPFAR funding and technical assistance (from Pact). PEACE then received Pact assistance for capacity building from November 2010 through June 2012 (19 months). Assistance ended on June 30, 2012, though the organization did not “graduate” to direct USAID assistance. SUSTAINABILITY Sustainability has not been realized; instead, this capacity building assistance model has led to an extremely vulnerable situation for the organization where it may cease to exist. PEACE cited unrealistic expectations for achieving sustainability in a timeframe that is a little over 1.5 years. The organization reported that the strategic planning assistance they received from Pact did not help them begin to plan for and take steps in the immediate future in order to ensure sustainability 3 to 5 years from now. Sustainability has been built through transference of skills and training but 32 this has not translated into organizational sustainability for PEACE. In addition to Pact assistance (which was not focused on technical program delivery or promotion of technical areas of expertise), PEACE has attracted pro bono assistance from worldwide leading authorities to provide high level training on approaches that have transferred skills for helping children cope with and survive trauma. For instance, PEACE hosted a joint venture, a mini-conference, with the Rehabilitation Center for Victims of Torture; this conference featured pro bono psychologists from Kosovo who trained sociologists and psychologists in Namibia using “story grafting” theory to work with traumatized children. This built a network of trained specialists. Inadequate financial resource base sustained to be able to meet the needs of beneficiaries. Pact advised PEACE about the importance of seeking and fostering partnerships with government (which the organization believes is one of the most difficult things to do). Pact also advised about developing a “business unit” to bring in income and about the importance of diversifying their financial resource base. However, Pact did not provide necessary and sufficient assistance to ensure that the organization planned 3 – 5 year sustainability plans or implemented them. The ongoing needs of beneficiaries (in this case, traumatized children and their caregivers and families and others) are not projected to continue. As an example, PEACE said that the “MGECW called yesterday and asked to refer a traumatized child, but PEACE can’t accept a new case due to their situation and there are no other resources to which to refer child trauma cases, so they remain unaddressed.” As a result of lack of financial stability/sustainability and also due to the fact that Pact training has positioned key staff to find better paying positions outside of the organization, PEACE has lost key staff and currently has 4 staff openings. Visibility vis-a-vis participation with government in Regional Forums. PEACE is an active member of the Child Protection network (coordinated by the MGECW) and helped to design the national flowchart for entry points of children in need of protection services. The majority of their referrals are directly from the 15 Women and Children Protection Units (WACPUs) which are maintained within the government (sustainable) system (Ministry of Safety and Security). ORGANIZATIONAL CAPACITY BUILDING Pact’s use of a standardized tool for organizational assessment. PEACE found the Comprehensive Institutional Strengthening Plan (CISP) very helpful in order to identify areas of strengths and areas needing improvement across the organization. However, the tool and resulting plan and use of Organizational Development (OD) Roadmap scores (for progress on specific capacity building components) did not provide a “warning signal” within the organization that alerted management and actually provided assistance when issues with losing a key financial staff person led to more system-wide problems. Policy development and improvements. Pact assistance helped PEACE develop HR, financial, and procurement policies with guidance for regulating internal processes and procedures. SYSTEMS STRENGTHENING Financial point person trained instead of a lasting financial system being built. Pact assisted with strengthening the skills of PEACE’s financial officer, ensuring that (one) person was conversant with USAID requirements and reporting systems. However, the training also positioned this key staff person for upwardly mobile job opportunities. The loss of the person significantly and negatively impacted the organization’s ability to create a sustainable financial system responsive to USAID administrative and financial rules. Strengthening M&E system. The strongest reported area of strengthening was in improvements of an already existing M&E system. Specific Pact contributions including assistance in developing M&E plan, tools, and data quality monitoring. Pact provided general M&E training in 2010 and then specific training in conducting end-of-project evaluations. Pact also helped PEACE produce reports that met USAID requirements. OTHER Key Concern and Opportunities: - It is clear that this organization is severely negatively impacted by receiving “some” capacity building but not the full complement required in order to help it “graduate” to direct USAID funding. The consequences are dire in terms of staff loss….and even more dire in terms of government and other stakeholders losing PEACE’s unique service delivery and support for traumatized children/youth, caregivers, and frontline child workers. The unmet need in this area is still great and PEACE’s programs actually need to expand coverage to include training of volunteers at constituency level, police, nurses in casualty and emergency services, and prison officers to better understand the signs and symptoms of child trauma, how to report it, and support the child (or caregiver) at every stage of the process. Other agencies (including the MGECW and members of the public) are still referring 33 child abuse and foster care abuse cases to PEACE that they cannot address nor refer. Because of a May 2012 radio show that PEACE produced, they are also receiving cries for help directly from foster children and neighbors that they cannot address nor refer. - It is a weakness of this technical assistance model that particularly when capacity building smaller organizations, training is person-specific; when only one person is trained in an area key to organizational functioning (like finance), this approach falls short of building institution-wide capacity and ensuring key - When initial institutional system assessments were done (CISPs), it must have been evident that some CSOs require more technical assistance and longer-term technical assistance than others in order to achieve capacity. PEACE is a small organization comprised of mostly direct, technical service providers. Support plans should have taken this into consideration in terms of Pact’s time and labor allocations for assistance. - Ongoing Pact assessments, scoring, and capacity building assistance must have better identified the critical point for the organization’s livelihood when both the financial and project coordinator left. Rather than hesitating to “ interfere in the organization’s internal affairs” (which is what organizational development interventions do by definition), Pact should have increased technical assistance efforts to help PEACE re-strategize in order to plan change management strategies and/or to attract and hire the right people and/or plan for interim task sharing, etc. - Pact’s institutional strengthening plans (CISPs) must address issues of staff retention and go beyond planning to assistance with implementing those plans in some (critical) cases. While this may be reported as a “challenge” in a report to USAID, the level of challenge and vulnerability of the CSO as a result of that challenge should be rated so that resource are focused on most critical junctures of an organization’s capacity building. The M&E coordinator left (In addition to the finance officer) after being trained by Pact; and the person who took over the M&E role is currently seeking other employment due to lack of funding that threatens to shut down this organization. - PEACE might play a critical role in the future of the MGECW’s entire service delivery system in that they have the skills to help ensure “staff retention” (of volunteers, CCCWs, ECD teachers and others in that system) by providing support counseling, given the fact that these frontline workers are often traumatized by being involved in reporting violence or abuse of children. - PEACE staff was unclear about whether the goal of qualifying to “graduate” to USAID direct assistance was included in the CISP and Pact’s planned contributions to their organizational capacity building from the beginning. They did not feel they had (yet) received sufficient assistance to help them comply with requirements for direct USAID funding. V. Summary of Findings The following summary is organized according to the 3 parameters used to guide this evaluation. It provides summary highlights of the strengths and weaknesses of Pact’s effectiveness in addressing: sustainability, organizational capacity building, and systems strengthening. One caveat is that a “summary” is extremely challenging, given the disparate (financial and human) resources across CSOs and directorates; and the variable duration of Pact technical assistance to different CSOs and directorates. For that reason, the “summary” is a presentation of highest level evidence for Pact’s relative effectiveness (i.e., “relative” in terms of the inputs provided to Pact such as timeframe since the Modified Agreement, a maximum of 19 months, and funding). “Effectiveness” is also multi￾variable: it includes social value, moral imperative, economic issues, etc. that in this case should answer the question, “is USAID getting value for money through the deliverables as planned with Pact assistance to the intended recipients?” The summary analysis is drawn from interviews with key informants (i.e., CSO staff, directorate staff, the former Permanent Secretary, and Pact advisors and their directorate counterparts). It is also drawn from our review of key documents (e.g., directorate annual work plans, Ministry Support 34 Plans, electronic databases used for various functions of monitoring and evaluation, the MGECW Five-Year Strategic Plan, as well as a multitude of Pact reports to USAID and such Pact-specific tools as the OD Roadmap, OD Roadmap Scores, and Comprehensive Institutional Strengthening Plans (CISPs). While many interviewees have referenced regional activity, planning, and strengthening, this evaluation has not had access to region-specific documents. (See Bibliography.) Sustainability At the higher level of the MGECW, an OD specialist advisor aligned the ministry PMS within the framework of PSC’s mandate. Pact’s effectiveness is evidenced in its strategic positioning of the PMS in a way that ensures ownership by a higher authority and therefore both accountability upwards and sustainability at the ministry level. Though Pact advisors were not “mandated” to expand their reach to the “higher” levels of the PSC or OPM, two of the advisors initiated this reach and working relationship with higher levels in order to leverage greater systems-wide impact. Pact was also highly effective and instrumental in assuring increased financial and human resources for ECD which will be sustained at a directorate level now. However Pact’s effectiveness regarding providing value towards sustainability is limited to results that are immediately realized and may not be long lasting. We did not find evidence for Pact building capacity for long term change management processes in any directorate (a result which may not have been possible due to the short duration of assistance). Example: If succession planning had been institutionalized on a sustainable basis, the impact of a new PS for the MGECW would be marginal. It is difficult to capture the extent of Pact’s effectiveness and value added regarding improving sustainability of the 3 CSOs that were already mature and positioned for direct USAID assistance prior to the Modified Agreement. However, Pact’s assistance was ineffective in assuring adequate, continued funding and human resources for the 3 non-graduates. In one instance, a non-graduate CSO felt that they did not have sufficient staff resources to replicate the capacity building and systems strengthening that Pact’s assistance had brought to the only USAID-funded department. That CSO also did not think it a good investment to scale up to the level of the Pact requirements to increase funding for only one department. Example: There were a number of reports from CSOs that Pact initiated helpful but generic training workshops (activities accomplished) but found no data that Pact would then revisit each program in order to introduce systems-level change. Every time an individual system is changed it will also affect the entire organizational system which needs to be revisited, over and over, in partnership with each CSO. Organizational Capacity Building Effective organizational capacity building is evident when human beings in an organization have learned to do something that they could not do before that is vital to carrying out the organization’s mission (i.e., core competencies). Pact was generally effective in facilitating early stages of organizational capacity building in some directorates (i.e., primarily Child Welfare and General Services and to a lesser degree with ECD). There is evidence that Pact built capacity in: individual and team planning; standards for service delivery (e.g., standards for RCCFs); administration of the grants system; administration of databases (to some degree) and training workshops; using and translating data into useful information to support the appropriate resource allocation; and senior management skills (e.g., advocacy, budget planning, development of policies and procedures). 35 Pact’s effectiveness did not reach as far as building the capacity to work in teams, building internal cross-training capacity, building mechanisms to conduct business with CSOs, building capacity of leaders who can independently manage the current organization and lead it into the future. Example: The former PS of the MGECW was very enthusiastic and appreciative of Pact’s assistance. However, that level of capacity must be balanced by her reported lack of resiliency and ability to replicate what was learned in one ministry during her transfer to another ministry. Pact’s mandate for organizational capacity building of CSOs was with one end point in view, i.e., building primarily financial management capacity to transition to USAID direct funding agreements. The trade off with this approach was evident from our interviews with all CSOs. They reported the need for organizational capacity assistance at critical junctures and to meet urgent problems, such as when key staff left, leaving a number of organizations without an entire system, after having received Pact training. Per Strategic Result 2 of the Modified Agreement, Pact is to “strengthen six civil society organizations to be eligible for USAID direct funding while maintaining high standards of existing services”. Pact’s organizational capacity development assistance should have been more flexible, especially with small CSOs, in order to effectively meet this part of the Strategic Result. For instance, in the example above, Pact could have helped the CSOs seek pro bono services in the interim, identify another donor to hire replacement staff, obligated that key (usually finance) staff to train others before their date of departure, or create any number of other back up plans. Pact was not effective in actively supporting CSO partners to establish partnerships with the private sector, both to generate funding and, where appropriate, to establish joint implementation of activities….” (pg. 12, Modified Agreement) The tools Pact used to monitor organizational improvement were also never fully utilized for purposes of building organizational capacity to problem solve. Example: One CSO that has not yet graduated reported that though their management capacity was strengthened by Pact assistance, their needs are in another direction. They are expanding programming and need immediate help with benchmarking salaries, creating job positions, and developing a recruiting strategy in order to attract much needed staff in order to “maintain high standards of existing services” (implied) (Strategic Result 2). Systems Strengthening Credibility and relationship building, with technical competence predicts effectiveness of systems strengthening. Pact’s technical advisors are all well qualified people who were consistently characterized as “helpers, doers, facilitators, teachers and trainers, validators, and role models”. All advisors were effective to the extent that: a) counterparts, when appointed, could absorb the skills being transferred; had minimal “other duties”; and were stable staff who did not leave the directorate; and b) that sufficient time and trust-building occurred to be able to support directors under conditions of doubt or resistance from staff. However, we observed the most value for investment at the Permanent Secretary level. Pact’s effectiveness was hampered by being minimally assigned to the highest levels (PS, OPM, PSC) that are the real systems drivers for change. Example: For instance, the Pact M&E Advisor began working at one directorate (prior to November 2010, i.e., for approximately two years). But his greatest impact in terms of systems development was at the ministry-wide level, with support of the PS, during which time he was able to plan and institute an M&E system and an aligned Performance Management System (PMS) that at least involved 3 regional levels (RITs) during planning and some reporting. As a note, even at that level, Pact’s assistance did not result in building a user-friendly, cross-directorate database for reporting, budgeting, and strategic planning. 36 The combination of CSOs managerial maturity (pre-Pact assistance), motivation to “graduate” to USAID direct funding, and ability to retain staff were major predictors of Pact effectiveness. Example: By definition, the three CSOs that “graduated” to USAID direct funding have successfully demonstrated systems strengthening, primarily in financial and M&E systems. They must sustain their systems in order to maintain their direct USAID funding status. VI. Recommendations. VIII. Annexes Annex A. Interview Tools 1. Interview Tool for MGECW I. Sustainability NOTES: “Sustainability” is measured across 3 parameters: a) resources (funds); b) adequacy of human resources; c) change management processes and systems “Sustainability” answers the question: will you continue to operate? Timeframe for purposes of this evaluation is at least the next 2 years. Each question should be answered through the viewpoint of Pact’s assistance in helping to achieve sustainability. 1) In what ways has Pact helped you improve organizational sustainability? 2) Describe your approach and level of success to: a) sustain current funding; b) obtain increased funding from MoF to be able to meet strategic objectives and needs of your beneficiaries now and in the immediate future? c) What of this was due to Pact assistance? 3) Based on your HR Strategic Plan and/or organogram, to what degree does your current staffing allow you to continue to deliver services at a “satisfactory” standard? 4) What preparation have you made to anticipate and address environmental changes (e.g., social, financial, 37 political) that may change the nature of your business (e.g., services delivered, the mandate, the targeted beneficiaries)? 5) How have you ensured the sustainability of the Regional Child Care Fora (RCCFs)? 6) How have you ensured the sustainability of the social grants system? ` II. Organizational Capacity Building NOTES: “Organizational Capacity Building” is measured across 4 parameters: a) organizational strategic planning; b) HR; c) HR management systems to achieve intended performance objectives; d) (at least) adequate resources to achieve intended performance objectives. “Organizational capacity building” answers the question: will you be able to do more and better? Timeframe for purposes of this evaluation is at least the next 2 years. Each question should be answered through the viewpoint of Pact’s assistance in helping to achieve organizational capacity building. 1) How has Pact helped you expand your organizational capacity for achieving the results outlined in your strategic objectives? 2) How has Pact helped your organization improve your resource base? 3) Checklist with Corroborating Evidence/Documents specifically relating to: a) Strategic Planning (Including partnerships and linkages.) Mission Vision Goals Objectives Core Values Core Competencies Organogram (with job titles, including volunteers and CCCWs and SWs) (Includes regional and decentralized levels of the directorate.) b) HR Job Descriptions All of this depends on “IRI.2” (Modified Agreement): “Strengthen the human resources management capacity at the Ministry, allowing the 38 institution to improve its efficiency. Pact will collaborate with the Ministry to implement a performance based human resources management system based on the anticipated findings of the MGECW’s USAID-funded HR assessment. Pact will support the definition of performance based management parameters and facilitate the building of the skills and systems required to implement it successfully. Pact will also continue to support volunteers and student bursaries based on commitments from the MGECW to expand the community child care workers (CCCW) cadre and recruit the social worker graduates.” (All of the HR section refers to direct hires, volunteers, and students with bursaries, etc.) Recruiting (advertisements, screening, interviews, job offers) New Employee Orientation Career Planning Training Individual Development Plan (IDP) for New Employees c) HR Management Systems Create Performance Objectives for quarterly performance management reviews Salary increases (if merited based on performance) Training (if needed) Succession Planning Assess candidate for next promotion – align with succession planning and employee’s career plan HR MIS d) Resources needed for achieving performance objectives (What percentage of the MGECW’s overall budget is allocated to this directorate?) 4) Did you receive adequate technical and financial resources from Pact to achieve your (directorate’s) performance objectives? (Explain successes and failures here.) 5) To what degree was task shifting employed to implement the Directorate’s mandate, with assistance from Pact? 6) Have you accomplished “80% of the activities outlined in your 39 annual work plan” (according to both the PMP and the CISP)? And to what degree were activities accomplished with Pact assistance? (Describe most helpful forms of assistance.) III. Systems Strengthening (through Human Resources for Health – HRH) “Systems strengthening” is measured across 4 parameters of the (6) PEPFAR “core areas” of Systems Strengthening plus a parameter that measures effectiveness and contributions of the 4 Pact advisors to the MGECW: a) HRH – task shifting to allow for a broader pool of workers within the system; b) health information for reporting and decision-making; c) governance in terms of engaging community in designing service delivery systems responsive to their needs; d) finance in terms of accounting for expenditures and improving resource flows to decentralized levels of the system and to ultimate intended beneficiaries. (Note that we have intentionally not included the 2 PEPFAR core areas of service delivery or medical products/technologies, since they do not fall within the scope of this evaluation.) “Systems strengthening” answers the question: to what degree is government enabled to serve all its internal stakeholders effectively, efficiently, and productively? (This limits the definition. “Systems strengthening” should also be realigned with external stakeholders and extent of access and utilization of services during service delivery.) 7) To what degree has each Pact advisor been accountable for transferring skills to at least one counterpart who remains within the directorate and who can sustain performance at a level pre-defined as at least “satisfactory”? (OD Advisor; ECD Advisor; IT Advisor; OVC Advisor) (Determine the length of service to each directorate.) (Use advisor job descriptions as a checklist of their progress and success.) 8) For each advisor and each director: Please describe the process for engagement and the exit process. How might have your attachment to the directorate have been more effective? 9) How effectively has Pact’s assistance enabled the directorate to task shift to the 40 following positions: a) Social workers b) Community Child Care Workers (CCCWs) c) Volunteers d) Other 10) How effectively has Pact’s assistance enabled the directorate to improve use of data from the OVC database and other data sources for key decisions such as appropriate allocation of resources? 11) How effectively has Pact’s assistance enabled the directorate to democratically engage community in a transparent and accountable process? (Examples.) 12) Describe how Pact’s assistance has helped you improve your ability to track resources to decentralized levels of your system and to intended beneficiaries. 2. Interview Tool for CSOs IV. Sustainability NOTES: “Sustainability” is measured across 3 parameters: a) resources (funds); b) adequacy of human resources; c) change management processes and systems “Sustainability” answers the question: will you continue to operate? Timeframe for purposes of this evaluation is at least the next 2 years. Each question should be answered through the viewpoint of Pact’s assistance in helping to achieve sustainability. 7) In what ways has Pact helped you improve organizational sustainability? 8) How has Pact assistance supported your ability to: a) sustain current funding; b) Obtain increased funding from diverse sources to be able to meet strategic objectives and needs of your beneficiaries now and in the immediate future? 9) Based on your HR Strategic Plan 41 and/or organogram, to what degree does your current staffing allow you to continue to deliver services at a “satisfactory” standard? 10) What preparation have you made to anticipate and address environmental changes (e.g., social, financial, political) that may change the nature of your business (e.g., services delivered, the mandate, the targeted beneficiaries)? 11) How have you ensured your sustainable participation as a CSO in the Regional Child Care Forum (RCCF)? V. Organizational Capacity Building NOTES: “Organizational Capacity Building” is measured across 4 parameters: a) organizational strategic planning; b) HR; c) HR management systems to achieve intended performance objectives; d) (at least) adequate resources to achieve intended performance objectives. “Organizational capacity building” answers the question: will you be able to do more and better? Timeframe for purposes of this evaluation is at least the next 2 years. Each question should be answered through the viewpoint of Pact’s assistance in helping to achieve organizational capacity building. 13) How has Pact helped you expand your organizational capacity for achieving the results outlined in your strategic objectives? 14) How has Pact assisted you in achieving the 3 accountabilities outlined in Strategic Result 2 of the Modified Agreement? a) Assess and prioritize needs against the performance standards necessary to administer US direct funds; b) Address priority capacity building needs by implementing a capacity building plan; c) Participate with Pact in monitoring progress against measureable performance Pg. 1 of Revised Program Description, Modified Agreement, Nov. 19, 2010 - Refer to OD Roadmap scores - Refer to CISPs 42 benchmarks for capacity building. 15) Checklist with Corroborating Evidence/Documents specifically relating to: e) Strategic Planning (Including partnerships and linkages.) Mission Vision Goals Objectives Core Values Core Competencies Organogram (with job titles, including volunteers and CCCWs and SWs) (Includes regional and decentralized levels of the CSO?) f) HR Job Descriptions Recruiting (advertisements, screening, interviews, job offers) New Employee Orientation Career Planning Training Individual Development Plan (IDP) for New Employees g) HR Management Systems Create Performance Objectives for quarterly performance management reviews Salary increases (if merited based on performance) Training (if needed) Succession Planning Assess candidate for next promotion – align with succession planning and employee’s career plan HR MIS h) Resources needed for achieving performance objectives 16) Did you receive adequate technical and financial resources from Pact to achieve your (CSO’s) performance objectives? (Explain successes and failures here.) 17) To what degree was task shifting employed to implement the CSO’s mandate, with assistance from Pact? 18) To what degree did you accomplish the activities outlined 43 in your annual work plan or CISP? And to what degree were activities accomplished with Pact assistance? (Describe most helpful forms of assistance.) VI. Systems Strengthening (through Human Resources for Health – HRH) “Systems strengthening” is measured across 4 parameters of the (6) PEPFAR “core areas” of Systems Strengthening: a) HRH – task shifting to allow for a broader pool of workers within the system; b) health information for reporting and decision-making; c) governance in terms of engaging community in designing service delivery systems responsive to their needs; d) finance in terms of accounting for expenditures and improving resource flows to decentralized levels of the system and to ultimate intended beneficiaries. (Note that we have intentionally not included the 2 PEPFAR core areas of service delivery or medical products/technologies, since they do not fall within the scope of this evaluation.) “Systems strengthening” answers the question: to what degree is civil society enabled to serve all its internal stakeholders effectively, efficiently, and productively? (This limits the definition. “Systems strengthening” should also be realigned with external stakeholders and extent of access and utilization of services during service delivery.) 19) How effectively has Pact’s assistance enabled the CSO to support and utilize the following positions: e) Social workers f) Community Child Care Workers (CCCWs) g) Volunteers h) Other 20) How effectively has Pact’s assistance enabled the CSO to improve use of data from the OVC database and other data sources for key decisions such as appropriate allocation of resources? 21) How effectively has Pact’s assistance enabled the CSO to democratically engage community in a transparent and accountable process? (Examples.) 22) Describe how Pact’s assistance has helped you improve your 44 ability to track resources to decentralized levels of your system and to intended beneficiaries. Annex B. People Contacts NAME ORGANIZATION /TITLE Andjama, Helena MGECW, Dir. Of Child Welfare Directorate Ausika, Sirrka MGEWC, Previous PS Botha, Danie Chairman of the Board, PEACE Bruce, Kerry Pact, M&E Director – results and measurement Chisenga, Godwin Catholic AIDS Action (CAA), Executive Director, Casimir Chipere Pact’s OVC Advisor De, Susna USAID/Namibia, Health Office, Systems Strengthening and Capacity Development Advisor Dashtur, Sarah, Pact OVC Specialist Dr. Ibe, Ochiawunma USAID, Health Office, Senior Technical Advisor-HIV/AIDS Care and Nutrition Eysselein, Malka PEACE, Clinical psychologist Bauer, Father Rick CAA ,Previous Director Felton, Silke, OVC and Child Protection Expert USAID, HIV/AIDS Office Festus, Robert USAID, Strategic Information Program Specialist Giombini, Elio Pact, M&E Technical Advisor to MGECW Hancox, Toni Angavetene, Amon Legal Assistance Center (LAC), Director AIDS Law Unit Hlahla, Donald Pact Consultant, Pact’s OD Advisor (part-time) Jones, Mellissa USAID, Director Karures, Vicky PEACE , Operation Coordinator, Lambert, Otilie Pact, organizational strengthening coordinator Munsu, Victor CHFL, Director Natuka, Joyce MGECW-Child Welfare Directorate Nghiitwikwa, Elina MGECW- Early Childhood Development Nshimyimana, Brigitte MGECW, Chief Control Social Worker Posner, Stephanie Pact, Chief of Party Rosenthal, Matthew USAID Shikongo, Lydia MGECW, Deputy Director - Child Allowances, Child Welfare Directorate, Shitjuwete, Jane Lifeline/Childline, Director Yates, DeeDee Pact, ECD Advisor Yates, Mike Katutura Youth Education Center 45 (KAYEC), Director Annex C. Documents Audit of the USAID Resources Managed by the Catholic AIDS Action Trust (CAA) Under Cooperative Agreement No. GPH-A-00-01-0007-CAAT with Pact & IntraHealth for the 30 months ended 31 December 2008 CAA Transition Plan to Direct Funding (May 2010 - Sept 2011) Capacity Development Global Indicator: Organizational Performance Index (Capacity Development Indicator Protocol: Dimensions of "effectiveness", "efficiency", "relevance", and "sustainability" (email from Kerry Bruce, Pact 7/5/12) Catholic AIDS Action (CAA) - Comprehensive Institutional Strengthening Plan (Final CISP) (June 30, 2011) and Transition Plan (Oct 2010 - June 2011) CHFL CISP 18 June 2012 (November 2010 - June 2012) Developing Capacity, Improving Performance - Pact Organizational Performance Index (OPI) (sent via email 7/5/12, from Kerry Bruce, Pact) Email traffic - Revised Workplan May 2012 - from Susna De FY12 Q1 Pact SCCO Quarter 1 Report Final - Excel FY12 Q2 Pact SCCO MGECW Indicators Final (SAPR FY12 SAPR=Q1 + Q2)(October 2011 - March 2012) KAYEC CISP (no dates?) KAYEC CISP Final COP10b (May 2010 - March 2011) LAC CISP Final 18 June 2012 (October 2011 - June 2012) Lifeline/Childline (LLCL) CISP (December 2010 - Sept 2011) Lifeline/Childline (LLCL) CISP (December 2010 - Sept 2011) Lifeline/Childline OD Roadmap Scores (2010 - June 2011) and Fund-raising Sub-Committee Deliverables (Resource Mobilization) LLCL CISP30-06-2100 Final (December 2010 - June 2011) Measuring Our Capacity Development Results", Prepared by Capacity Development/Program Advancement, April 2012, sent via email from Matt Reeves at Pact DC MGECW (Child) Grant Payment Summary 2 - Excel MGECW Five Year Strategic Plan 2010 - 2014 46 Modification of Assistance - Pact MOD 6 MOU between the MGECW of the GRN and Pact Namibia (Jan 8, 2009) NAMIBIA - GLOBAL HEALTH INITIATIVE - 2011-2015/16 - FINAL DRAFT STRATEGY Namibia - Global Health Initiative -2011-2016 - Final Draft Strategy Namibia looking forward: Effectiveness of capacity development support to local partners" - Stakeholder data validation workshop, July 2, 2012, Windhoek (PowerPoint Presentation) Namibia's National Agenda for Children - 2012-2016 National Health Policy Framework 2010-2020 National Strategic Framework for HIV and AIDS Response in Namibia (saved ecopy only - no hard copy) OD Roadmap CAA June 2011 Final OD Roadmap CHFL Sept 2011 Final Annex D. Scope of Work V. 21 June 2012 I. TITLE: USAID/Namibia: Pact, Inc. “Strengthening the Capacity for Country Ownership” End-of Project Evaluation. II. PERFORMANCE PERIOD: Not including time for preparation and completion of report, two to three weeks in-country, On/about, May, 2nd 2012 through mid-June, 2012. III. FUNDING SOURCE: This assignment will be funded by USAID/Namibia IV. PURPOSE AND OBJECTIVES: Goal of the evaluation: The evaluation is designed to evaluate the outcome of the Pact award and inform the design of the follow-on award. Objectives of the evaluation: To determine results achieved primarily against the objectives in Pact’s modified agreement; To determine the sustainability of their activities (including sub grantees); To determine if investments in human resources have had a long term impact and are sustainable; To develop recommendations for consideration in the follow-on award Key implementation issues: The evaluation will require concurrence of Government of the Republic of Namibia (GRN) counterpart Ministry, namely the Ministry of Gender Equality and Child Welfare and will focus on the six Pact sub-grantees outlined in the Modified Agreement. USAID/Namibia will 47 inform relevant GRN ministries about the evaluation and request their concurrence and cooperation during the implementation of the evaluation. In addition, per a recent audit by the Office of Inspector General, (See “Audit of USAID/Namibia’s efforts to build health workforce capacity” Audit Report No. 9-000-11-001-P, February, 24 2011), this evaluation will be required to define the Pact project’s contribution to systems strengthening through Human Resources for Health (HRH) investments, as well as identify Pact’s contribution to sub-partners or to the government of Namibia in the area of organizational capacity. Period under review for the evaluation: From start of the modification (November 2010) to April 2012. Note, this is slightly prior to the end of the project (which is September 25, 2012) as USAID/Namibia, in consultation with GRN plans to explore ways to continue critical activities to protect the projects achievements. V. BACKGROUND: In 2007, USAID awarded Pact Inc. an Associate Cooperative Agreement to provide financial support, largely through the President’s Emergency Plan for AIDS Relief (PEPFAR), for the program named “Community REACH Namibia Program.” USAID awarded the prime partner, Pact Inc., a five-year Local Field Support Cooperative Agreement, (Contract Agreement Number: 690- A-00-07-00104-00) on 09/26/2007, with a planned life of project of $40,000,000 ending on 09/25/2012. In November 2010, following a mid-term evaluation, the agreement was modified to reflect greater alignment to GRN and PEPFAR phase II priorities. The rationale for this Modification was rooted in PEPFAR’s new emphasis on country ownership, long-term sustainability, and systems strengthening - -- representing a sizeable shift from emergency scale-up approach of PEPFAR I. In addition, the Modification took into account decreasing donor funding, including USG, in Namibia and the critical need to ensure sustainability of the HIV/AIDS response sooner rather than later. Consequently, the name of the program was changed to “Strengthening the Capacity for Country Ownership.” Prior to the modification, the Pact award outlined 6 objectives, which were as follows: To provide an effective and transparent grant award and administration system for the provision of responsive, fast-track grant making assistance to organizations responding to the Emergency Plan. To provide implementers with access to financial resources and high quality technical expertise to assist in achieving and effectively reporting results while complying with USG financial and administrative requirements. To expand civil society’s response by providing capacity building to local, regional, national and international organizations resulting in increased capacity of organizations and networks to provide and sustain HIV/AIDS services. To strengthen the linkages between civil society, host country government and private sector with the aim of promoting longer term sustainability and capacity building in-country. To support the Ministry of Gender Equality and Child Welfare through targeted technical assistance and capacity building, and implementation of an OVC database that enables registration of OVC nationwide, links country partners to the Government of Namibia’s social welfare grants system, and allows Pact to support the Ministry in their overall goal of implementation of the National Plan of Action for OVC. To implement a gender sensitive approach to HIV/AIDS programming that uses complementary and interrelated steps to arrive at appropriate programming interventions. 48 Following the Modification, the Pact Award aims to achieve two main results: Strengthened capacity of the Ministry of Gender Equality and Child Welfare (MGECW) to effectively coordinate activities relating to gender equality and orphans and vulnerable children. Strengthened six civil society organizations to be eligible for US direct funding while maintaining high standards of existing services. Through the course of this award, Pact has worked with 21 sub-grantees; this was reduced to 6 sub￾grantees by the time of modification. Currently, Pact Inc. has three remaining sub-grantees, namely Legal Assistance Center, Lifeline/Childline, and PEACE. Pact works currently at the national level as well as the regional level for both its MGECW institutional strengthening activities as well as for its civil society work (which is currently focused on Caprivi, Hardap, Kavango, and Khomas Regions). Illustrative Key Questions to be addressed by the team: Guiding evaluation questions: 1. What was the performance and effectiveness of Pact as measured against stated objectives in the modified agreement? 2. What are the critical recommendations and lessons learned that should inform future activities in this area (that improve effectiveness) and how resources should be allocated? 3. What has been Pact’s contribution to health systems strengthening through investments in HRH? 4. How sustainable are the activities of PACT and its sub grantees, including activities related to HRH? 5. What are Pact’s contributions in the area of organizational capacity strengthening for the relevant GRN institutions, current and former sub-grantees and other? Performance information sources— 1. Baseline assessments for program implementation (if any). 2. Mid-term assessment 3. Country Operational Plan FY07, FY08, FY 09, FY 10, FY 11 narratives. 4. Work plans and PMPs. 5. Quarterly, semi-annual and annual progress reports 6. Financial reports and pipelines. 7. Key line ministries, MGECW and the Ministry of Health and Social Services (MoHSS), reports on Pact activities 8. Any signed agreement with local partners 9. Key informants interviews 10. Field visits and direct observations Evaluation questions will be finalized during the team planning meeting and will be provided to USAID/Namibia for review. VII. METHODOLOGY The evaluation team will use a variety of methods for collecting and analyzing qualitative and quantitative information and data. The methods to be used in completing this evaluation will include, but not be limited to: reviewing documentation, interviews, site visits, stakeholder 49 meetings, etc. Drawing on experiences in other PEPFAR countries, USG Namibia will seek the assistance of external consultants, headquarters, host country and local USG counterparts to conduct the assessment. The following essential elements should be included in the methodology as well as any additional methods proposed by the team: Document Review Prior to arriving in country and conducting field work, the team will review various project documents and reports. A list of key documents is included in Section XIII. As stated previously, The USAID/Namibia team will provide the relevant documents for review as soon as possible. Team Planning Meeting A three-day team planning meeting (TPM) will be held during the evaluation team’s first two days in￾country with USAID staff. This time will be used to clarify team roles and responsibilities, deliverables, development of tools and approach to the evaluation, and refinement of agenda. In the TPM the team will: o share background, experience, and expectations for the assignment o formulate a common understanding of the assignment, clarifying team members’ roles and responsibilities o hold in-depth discussions with USAID/Namibia o agree on the objectives and desired outcomes of the assignment o establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion o develop data collection methods, instruments, tools and guidelines, and methodology and develop an assessment timeline and strategy for achieving deliverables This methodology should elaborate on the quantitative methods that will be used, how the data will be analyzed in relation to the qualitative data that will be collected, and how the data collected will be used to assess performance and effectiveness. Strengths and limitations of the chosen methodological approaches for each research question should be articulated. o develop a draft report outline for Mission review and approval Key Informant Interviews The team will conduct structured interviews with the project staff, and key partners including the MGECW, NGOs, other donors (as relevant), implementing partners, and other stakeholders (especially with stakeholders who are no longer receiving Pact support, such as former sub-grantees and community child care workers). To ensure that comparable information is collected during interviews, the team will develop standard guides reflecting the questions posed by the evaluation scope of work. Field Site Visits The team will coordinate with USAID/Namibia to prepare for and conduct site visits while in-country, and to interview key informants at these sites. These site visits will be focused primarily on those 50 activities relevant to the modified award. Site visits will be conducted in three regions as well as in the Windhoek area (Khomas Region). These regions will be chosen in consultation with Pact, based on their current activities (that were also in operation during the period of evaluation) and locations of current and former sub-grantees. Assuming a two person team, the team will conduct site-visits in one, possibly two regions (three days in each Region, with three days travel). USAID/Namibia will arrange for all required in-country transport from Windhoek to site visit locations. USAID/Namibia will make arrangements for accommodations as needed. Quantitative Data: To extent possible, the methodology will use quantitative data to support evaluation findings. Possible sources for qualitative data will include progress reports, data sources from relevant MGECW sources as well as sub guarantees. Stakeholders and “consumers” of Ministry and CSO services: This evaluation was only for looking at the increased institutional capacity, systems strengthening and sustainability of the Ministry and CSOs and therefore, these groups were not interviewed. VIII. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORT USAID anticipates that the evaluation team will consist of the following individuals and groups (TBD): Profile of Evaluators (see descriptions below for a two person team): o Should be external evaluators, at least one being an international consultant. o Should have expertise in HIV/AIDS programs—in particular care and support (incl. home based care, OVC). o Should have expertise in organizational development – ideally with respect to civil society as well as Government strengthening. Team Leader: In-depth expertise and experience in organizational development, with an emphasis on strengthening the capacity of both civil society as well as Government agencies in care and support (incl. home based care and OVC). Should be knowledgeable in capacity building for local, regional, national and international organizations for increased capacity of organizations and networks to provide and sustain HIV/AIDS (including effective and transparent grant award and administration systems, and effective reporting results while complying with USG financial and administrative requirements). Experience with PEPFAR funded programs a plus. Minimum Masters, preferably a management (e.g. MBA) related area. Must have a proven track record participating in evaluations with ability to assist in the synthesis of findings within a short time frame. Team member: Should have extensive expertise in the mid- and end-of-project evaluation of PEPFAR funded HIV/AIDS programs designed to build Government Ministry, NGO and civil society agency capacity for care and support (incl. home based care, OVC). Minimum Masters, preferably a Doctorate in Public Health related area. Should have proven track record leading evaluations with ability to synthesize findings into high quality final report within a short time frame. USAID/Namibia and/or team members: The local team members (which may include appropriate USAID/Namibia) staff members will participate with the two external evaluators as needed. The role of USAID staff members will be limited to accompanying consultants on site visits and introducing them to national and local key informants. USAID/Namibia staff members will not be responsible for the drafting of the evaluation report. These roles and responsibilities will be clearly communicated during the TPM. . All assignment-related costs for local team members will be covered by USAID/Namibia. USAID will notify Pact as soon as possible with local team member information. 51 USAID/Namibia will provide a detailed contact list of key informants and key points of contact to the consultants during the document review period, so planning can begin for appointments, interviews, and site visits can be set up for the team’s arrival in-country. USAID/Namibia will also provide a draft schedule for field visits including duration of stay at various sites to inform the team’s time in￾country. Key contacts include: Head of the Directorate of Child Welfare, MGECW, Dr. Stephanie Posner, (Pact, Inc.), and appropriate USAID/Namibia staff members. ( ESTIMATED LEVEL OF EFFORT (LOE): A six-day work week will be approved when the consultants are working in country. Task/Deliverable Team Leader LOE Second team member LOE Read Background Documents 3 days 3 days Travel to Namibia 2 days 2 days Team Planning Meeting 2 days 2 days Assessment work 15 days 15 days In-briefing with USAID HIV/AIDS team (and partner(s) as needed) (1 day) (1 day) Conduct site visits and key informant interviews (includes in-country travel days) (9 days) (9 days) Discussion, analysis and draft report preparation (3 day) (3 day) Mission (and partner debriefing) (1 day) (1 day) Complete report draft – revise report & incorporate debriefing comments into draft report (1 day) (1 day) Return travel 2 days 2 days Mission sends technical feedback/comments on draft report to Pact (within 5 days of submission) 0 0 Consultants revise/finalize report 5 days 3 days Mission reviews/signs off on final report (within 5 days of receipt) 0 0 Pact edits and finalizes report – approx. 30 days after mission 0 0 52 approval Total LOE 29 Days 27 Days IX. LOGISTICS Pact will provide: International travel to and from the consultant’s point of origin and Namibia. Pact will provide full-fare economy. Business Class is NOT authorized. Pact consultant per diem and lodging expenses. Country cable clearance. Please note: a formal electronic country clearance (ECC) request is not necessary; instead, an informal email request directly to Melissa Jones, Director of HIV/AIDS and Health Office, USAID/Namibia will suffice. Ms. Jones will provide an e-mail concurrence upon receipt of this request. USAID/Namibia will provide: Visitors will not have an EA and therefore will need to work out of their hotel/lodging or a designated work space (TBD). They will need prior approval to bring any laptop into the USAID office for any meetings or briefings. Cell phone, but consultant(s) will purchase air time Pact will submit a list of all stakeholders and beneficiaries for field visits and USAID will provide logistical support for the team in country assessment. Arrangements/logistics for in-country site visits. Reserve hotel and guest house accommodations in country. USAID/Namibia will provide a USAID/Namibia car and driver for use by Pact Consultants only when other USAID staff members accompany them. When no USAID staff members accompany consultants, they will use taxis. X. DELIVERABLES AND PRODUCTS 1. A written methodology/work plan (Evaluation design/operational work plan) prepared during the TPM and submitted to the Mission for review and approval before field work and key informant interviews begin. 2. A draft report outline prepared during the TPM. 3. A Mission and partner debrief meeting that will be held before the team’s departure and prior to the submission of the draft report. The team will prepare a PowerPoint presentation for this event. 4. Prior to departing Namibia, a draft report addressing key performance findings, conclusions, recommendations and lessons learned will be submitted. The mission will have 10 days following the submission of the draft report to respond and provide written comments and feedback to Consultants. 5. Final Report: The team will incorporate all feedback provided by USAID/Namibia reviewing team. A final unedited draft report will be submitted 5 days from the date of receipt of USAID/Namibia’s feedback on the draft report. The report should conform to USAID Evaluation Policy “Criteria to Ensure the Quality of The Evaluation Report” (please see Appendix 1). 6. Conditional upon receipt of comments from USAID/Namibia five days beforehand, the final report will be due on [Date to be determined]. It will be the property of USAID. Dissemination of relevant findings will occur through official channels at local (Mission, USG 53 and stakeholders) as well as Washington level. Some of the findings may be used for country operational planning. The report shall not exceed 30 pages, excluding the annexes o The revised final unedited report will be provided to the mission 5 days after the comments are received. o Only if the final draft is approved by USAID/Namibia prior to May 16, 2012, consultants will provide the edited and formatted final document approximately 30 days after USAID provides final approval of the content. Otherwise, USAID/Namibia will need to go through another mechanism to finalize the report. Procurement sensitive information will be removed from the final report and incorporated into an internal USAID Memo. The remaining report will then be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the consultants will provide the final report to USAID/Namibia for distribution (5 hard copies and CD ROM). XI. RELATIONSHIPS AND RESPONSIBILITIES Prior to in-country work: Consultant Conflict of Interest. To avoid conflicts of interest (COI) or the appearance of a COI, review previous employers listed on the CV’s for proposed consultants and provide additional information regarding any potential COI. Background Documents: Identify and prioritize background materials for consultants and provide them to consultants as early as possible prior to team work. Key Informant and Site Visit Preparations: Provide a list of key informants, site visit locations, and suggested length of field visits for use in planning for in-country travel and accurate estimation of country travel line items costs (i.e. number of in-country travel days required to reach each destination, and number of days allocated for interviews at each site). Lodging and Travel: Provide information as early as possible on allowable lodging and per diem rates for stakeholders that will travel/participate in activities with the evaluation team. Also, provide guidance on recommended secure hotels and methods of in-country travel (i.e., car rental companies and other means of transportation) and identify a person to assist with logistics. During in-country work: USAID/Namibia will undertake the following while the team is in country: Mission Point of Contact: Ensure constant availability of the Mission Point of Contact person(s) to provide technical leadership and direction for the consultant team’s work. Meeting Space. Provide guidance on the team’s selection of a meeting space for interviews and/or focus group discussions (i.e. USAID space if available, or other known office/hotel meeting space). Meeting Arrangements. While consultants typically will arrange meetings for contacts outside the Mission, support the consultants in coordinating meetings with stakeholders, Formal and Official Meetings. Arrange key appointments with national and local government officials and accompany the team on these introductory interviews (especially important in high-level meetings). 54 Other Meetings. If appropriate, assist in identifying and helping to set up meetings with local professionals relevant to the assignment. Facilitate Contacts with Partners. Introduce the team to project partners, local government officials, and other stakeholders, and where applicable and appropriate, prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. Following in-country work: USAID/Namibia will undertake the following once the in-country work is completed: Timely reviews: Provide timely review of draft/final draft reports and approval of the deliverables. XII. MISSION CONTACT PEOPLE/PERSONS Nabil Alsoufi Health Officer Health Office USAID/Namibia Tel. + 264 (0) 61 273730 Fax: + 264 (0) 61 273756 Cell: + 264 (0) 00811278236 e-mail: nalsoufi@usaid.gov Ms. Susna De Systems Strengthening and Capacity Advisor Health Office USAID/Namibia Tel. +264 61 273 723 Fax +264 61 273 756 Cell. +264 81 129 7720 email: sde@usaid.gov Ms. Silke Felton OVC Advisor Health Office USAID/Namibia Tel. +264 61 273 713 Fax +264 61 273 756 Cell. +264 81 127 8572 email: sfelton@usaid.gov Melissa Jones Director Health Office USAID/Namibia 55 Private Bag 12028, Ausspannplatz, Windhoek, Namibia Tel. + 264 61 273715 Fax: + 264 61 227006 Cell: + 264 81 127 8428 e-mail: mejones@usaid.gov XII - COST ESTIMATE - TBD The estimated budget for this evaluation includes, but is not limited to: 1. Minimum Level of effort: (29 days Team Leader, 27 days Team Member) 2. Two return full-fare economy flight tickets Washington-Windhoek 3. Lodging for two evaluators (up to 17 days) 4. Meals, incidental expenses 5. Per diem costs 6. In-country travelling for sites visits (transportation and lodging will be provided by USAID) 7. Pact Washington-based technical assistance 8. Pact Edit and Finalize Report 9. Pact Overhead Costs VIII. REFERENCES (PROJECT DOCUMENTS) -- to be provided to Pact as soon as possible. Original RFP/RFA Original contract/agreement Modified Agreement Work plans Namibia GHI Strategy SAPR and APR Namibia PFA Namibia NSF 2010-2016 Draft National Prevention Strategy Other key national policy documents APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. Evaluation reports shall address all evaluation questions included in the scope of work. The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. Evaluation findings will assess outcomes and impact on males and females. 56 Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. Sources of information need to be properly identified and listed in an annex. Recommendations need to be supported by a specific set of findings. Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. Annex E. Schedule of Visits Schedule (17 July Version) Date and time Activity Venue/Contact person Comment Saturday 07 Jul 2012 Evaluation Consultants arrive. HKIA Airport pick up and hotel reservation made by Pact Monday – 09 Jul 2012 08:00-10:00 (met at 7:45 at Hilton Hotel by Matthew, USAID) In-Brief with USAID Namibia Health Office USAID Namibia 5th Floor Include discussions on methodology, workplan, visit schedule, logistics Tuesday – 10 Jul 2012 08:00-17:00 Team planning meeting Consultants Meeting Place / Hotel Finalize methodology / work plan and review current literature Wednesday – 11 Jul 2012 08:00 – 13:00 14:30 – 16:00 Team planning meeting continues Present revised workplan and methodology to USAID Consultants Meeting Place / Hotel USAID Namibia 5th Floor Finalize methodology / work plan Discuss revised workplan / methodology and all logistics Thursday – 12 July 2012 09:00 – 12: 30 (Pick up by Pact at 8.45; all subsequent site visit transport provided by Pact) 12:45-14:00 Meet Pact Chief of Party & Team Pact offices Assess all sub-objectives and review additional literature 57 14:15 – 15:30 DeeDee Yates KAYEC (DID MEET) Meet Catholic Aids Action DID NOT MEET. ARRIVED TOO LATE, RESCHEDULE FOR FRIDAY AT 1530 ECD Advisor CAA Offices DIDN’T HAPPEN Godwin Chisenga Friday – 13 July 2012 08:00- 1000- Cassie 1000 to 1200- 14:00 – 15:15 Elio Lifeline Childline LL/CL Offices Jane Shitjuwete 15:30- 17:00 CAA CAA offices Godwin Chisenga WEEKEND Monday – 16 July 2012 09:00-11:00 11:15 – 13:00 14:00-17:00 Legal Assistance Center PACT PACT LAC Offices PACT Offices PACT Offices Tonie & Amon PACT Advisors PACT Advisors & Staff Tuesday – 17 July 2012 9:30 – 10:45 14:30-15:30 (TBC)_ PEACE Center CHFL PEACE Center Offices By Phone to CHFL Hettie Rose-Junius Victor Munsu, CHFL Wednesday – 18 July 2012 8:15-8:50 9:00– 10:45 11:00-12:45 Contextual Meeting with AOR MGECW – Child Welfare Directorate MGECW – Early Childhood Development USAID Offices MGECW Offices MGECW Offices Susna De Joyce Nakuta Lydia Shikongo Brigitte Nshimyimana 58 14:30-15:30 – TBC 15:45 – 17:00 MRLGH – OLD MGECW PS MGECW – Gender Directorate MRLGH Offices MGECW Offices Elina Nghiitwikwa & Team Sirrka Ausiku Ms Shililifa Thursday – 19 July 2012 Team put compile notes & start draft report Consultants Meeting Place / Hotel Note compilation and draft report Friday – 20 July 2012 Saturday – 21 July 2012 Team put compile notes & start draft report Consultants Meeting Place / Hotel Note compilation and draft report SUNDAY Monday – 23 July 2012 08:30-10:00 12:00-13:00 OUTBRIEF Draft Report finalized and to be presented to USAID – preferably key findings in Powerpoint format OUTBRIEF Draft Report finalized and to be presented to PACT – preferably key findings in Powerpoint format USAID Health Office, 5th Floor Conference Room USAID Health Office, 5th Floor Conference Room Powerpoint Presentation 3 PACT staff members to come to USAID for Out brief Tuesday – 24 July 2012 Comments incorporated into draft report / present Consultants Meeting Place / Hotel Draft Report Wednesday – 25 July 2012 Thursday – 26 July 2012 Friday – 27 July 2012 WEEKEND – Consultants depart Windhoek 59 Annex F. Mid-Term Recommendations 1. Pact should revise its indicators to ensure that there is consistent evidence that measures progress toward each of its major objectives, particularly capacity building. This may require independent capacity assessment of continuing sub-grantees by an OD professional. 2. Pact should work with continuing sub-grantees to create personal and organizational transition plans. These should include a stakeholder analysis, identification of donor trends, and creative problem-solving about how to minimize their costs and obtain alternative funding, and strategies for specific actions they can take to identify and advocate for resources. Transition strategies should consider two scenarios: the first guiding what shape the organization will take if it finds funding to continue operations, and what its goals and programs will be; the other guiding how it will end activities responsibly if funding is not forthcoming, including how to help staff find employment or educational opportunities and limit their stress during the process. Transition plans should examine how to support volunteer motivation and skill-building over the next two years to create opportunities for income or educational attainment. 3. In conjunction with continuing sub-grantees, Pact should develop a strategy and specific action steps to engage the private sector with the aim of obtaining continuing private sector support and resources for their programs. 4. Pact should think strategically about how it can play a role with other parts of the Ministry (MGECW). This levels the playing field in terms of technical support, and working with the Gender Equality Directorate has the potential for increasing the emphasis given to gender issues. 5. Pact should work with the Ministry to develop specific strategies for retaining those who have been recruited and supported through the bursary and internship programs as well as how the Ministry can continue those programs. 6. Pact should revisit with the Ministry the responsibilities of supervisory personnel within the Directorate of Child Welfare to determine what factors are limiting provision of effective supportive supervision so that remedial action can be taken. 7. Pact should enhance its M&E support to civil society sub-grantees to ensure that they understand its uses for planning and program adjustments. 8. Pact should take steps to ensure that the M&E support it has provided to the MGECW results in data that are linked to other relevant databases and are used strategically by the Ministry for planning and assessing programs as well as for supportive supervision. Pact should work with all directorates of the Ministry on M&E to ensure that the resulting capacities and data are comprehensive and sustainable. 9. Pact should explore ways of continuing collaboratives or similar entities, perhaps as adjuncts of the PTF, technical advisory committee (TAC), and technical reference groups (TRGs), to work on specific issues to continue to engage government with civil society. 10. Pact should enhance its emphasis on gender issues, through – Working with the MGECW to share good models for male engagement from other countries widely with CSO sub-grantees. – Initiating discussions with the Directorate of Gender Equality to ensure there is common agreement on what the key gender issues are that affect HIV and AIDS prevention, care, and support in Namibia and to develop “best practices” on gender issues for broad application. – Continuing to place emphasis on child welfare and women’s empowerment. 11. Pact should hire a well-qualified OD officer as soon as possible to oversee its continuing capacity building efforts. 60 12. In its technical assistance to the MGECW, Pact should emphasize collaboration with existing regional structures, such as the RACOCs and CACOCs, to ensure coordination. Annex G: PowerPoint Presentation 61 62 63 64 65 66 67 68 69 70 71