SANDEEP Final Evaluation Report Final Evaluation of Strengthening the NGO Sector in Nepal (SNSN) Project September 01, 2008 By Mohan M. Thapa, Team Leader Kedar P. Badu, Team Member Shanker R. Pandey, Technical Advisor i EXECUTIVE SUMMARY The Save the Children US (SCUS) has been implementing the five year long the Strengthening the NGO Sector in Nepal (SNSN) project, locally known as SANDEEP (Sanstaghat Diego Pariwartan – Sustainable Organizational Change) since November 2003. The SANDEEP project aimed to strengthen the capacity of local non-government organizations (NGOs) to improve delivery of health, education, and economic services in the conflict affected regions of Nepal by strengthening their operational and financial capabilities and expanding linkages with and enhancing program learning and sharing among NGOs, government and private sector institutions. SANDEEP did this through a standard set of modular training and coaching support to 5 Intermediary Support Organizations (ISOs): who in turn, applied their expertise within their own organizations and provided similar support to 35 local district partner NGOs (PNGOs)1. The evaluation assessed whether the project achieved its objectives and the net impact of changes in the capacity of Intermediary Support Organizations (ISOs) and PNGOs in terms of service delivery and linkages with and effects on key stakeholders. It identified how the conflict situation in Nepal affected ISOs and PNGOs and also evaluated the viability of sustaining the capacities developed within SC/US and partners. Besides, the evaluation assessed unintended results and identified challenges, lessons learned and specific organizational development (OD) issues and needs of PNGOs. Finally, the evaluation made appropriate recommendations with regards to replication and improvement of prevailing OD practices applied by SANDEEP and also suggested advocacy measures targeting potential donor agencies which can support SANDEEP’s second phase design and implementation. The evaluation, which was guided by a detailed ToR, employed both secondary data compiled by the Project Management Team (PMT) and primary data collected by the Final Evaluation (FTE) team during the field visits. Out of 5 ISOs and 30 PNGOs supported by the project at the time of the final evaluation, 3 ISOs and 7 PNGOs, spread in the eastern, mid western and far western development regions, were selected as samples for in-depth interactions after consultation with the SANDEEP’s PMT. Relevant project documents, Organizational Assessment (OA) data from the Institutional Development Framework (IDF) collected by the PMT and interactions and interviews with executive committee (EC) members, staffs (senior and field level), Organizational Capacity Building Officers 1 Though 35 PNGOs were selected at the outset, the project phased out 5 PNGOs. These PNGOs were unable to comply with primary selection criteria i.e. role separation of EC and staff and unavailability of even a single project. ii (OCBOs/As) and Financial Organizational Capacity Building Officers (FOCBOs), district level stakeholders, community people, SANDEEP PMT staff, SCUS directors and representatives of USAID and AIN and donors supporting the ISOs and PNGOs were the key sources of data. The implementation of the project has largely followed the planning matrix. Based on the available OA and IDF data compiled by the PMT, the project was successful in achieving its strategic objective of developing “capable NGOs”. It has met or exceeded the target against the Planning Matrix with respect to majority of the indicators of the three intermediate results of the project. The pre-award survey, training, mentoring and coaching activities and OA exercises using the IDF tool led to ISOs and PNGOs initiating actions (such as action plans, road maps and long term strategic plans) to address their gaps in capacity (human resources development, effective management systems, external linkages and resource mobilization). The project’s hypothesis of “enhancement of NGOs capacity to work in current conflict situation” was relevant and the project’s approach and interventions supported this aspect of the organization’s capacity. Improved organizational capability enabled ISOs and PNGOs to develop coping strategy to work in conflict situation. All the ISOs and majority of the PNGOs were able to sustain development interventions while many of them were even able to grow during these times. There is positive correlation between NGOs capacity enhancement to "improvements in the delivery of health, education and economic services". However, it was difficult to establish the extent of its influence on service delivery as it is one among several that influences service delivery in terms of quality and coverage. Though the extent of the project’s link to "mitigate the effects of conflict and poverty" cannot be ascertained, project inputs helped to mitigate the effects of the conflict at the micro level by emphasizing on transparency, accountability and participation of community people in planning, implementation and monitoring of projects. The significant positive impact of the project on management efficiency was one of the major achievements of the project. Changes were more visible in smaller PNGOs which had fewer systems already in place and thus were more receptive to SANDEEP’s inputs in comparison to ISOs, and larger NGOs with established structure, system and mind set. ISOs and PNGOs were aware about the fundamental elements and characteristics of a NGO as an organization and were also equally concerned about strengthening organizational management through policy formulation and reform, strategic planning and clear delineation and separation of roles and responsibilities. SANDEEP inputs had increased the professional capacity and self-confidence iii level of staff and EC members enabling them to contribute to management efficiency. The project facilitated in-depth discussion on OD issues by OD teams which supported the EC to take timely decisions. As a consequence, coordination with stakeholders improved while organizational issues were being addressed through timely decisions and better understanding. Overall, the organizational public image of ISOs and PNGOs was enhanced because of higher level of transparency and accountability in their functioning. ISOs and PNGOs improved their organizational structure by defining clear roles and responsibilities and enhancing reporting channels. Strategic plans provided a distinct direction to their actions and increased motivation of staff and EC to fulfill organizational objectives. Strategic plans were increasingly used as an analytical tool to assess gaps between strategic objectives and operational plans as well as for developing new programs to address these gaps. Owing to SANDEEP’s financial management training, financial recording and reporting improved significantly in the ISOs and PNGOs. The ISOs and PNGOs developed, revised and implemented a set of financial and personnel policies. Donors and stakeholders appreciated the financial and administrative systems of SANDEEP affiliated NGOs. Another significant achievement of SANDEEP intervention was the realization among ISOs and PNGOs that resources can be accessed from local sources and private institutions in addition to existing funding agencies. ISOs and PNGOs diversified their funding portfolio that now includes the private sector. Resource management team has been formed and had started taking fund raising responsibilities. Despite these positive changes, linkages with private sector at the district level were modest and smaller PNGOs still face difficulties in accessing donor resources. ISOs and PNGOs leaders have adopted participatory approaches in the program cycle, particularly in participatory planning, monitoring and evaluation and delegation of authority. They have started planning interventions with increased community participation in project development and review. Monthly work plans and clear cut guidelines for delegation of authority improved staff performance and facilitated prompt decisions that led to more efficient service delivery. Coordination with other NGOs, stakeholders, government agencies and community had improved in recent times which had helped in reducing duplication and overlap. The practice of social audit by ISOs and PNGOs increased transparency and accountability leading to an increase in communities’ and stakeholders’ trust of the project partners. Social audits were being replicated across many NGOs in the respective regions. Donors had begun to allocate more funds in iv NGO program budgets to cover the costs of social audits while the Social Welfare Council (SWC) is contemplating on making the social audits mandatory in the new NGO Act. ISOs and PNGOs refined their administrative and personnel policies to incorporate new learning from the Human Resource Management (HRM) Module. Fair, transparent and practical policies and systems not only enabled SANDEEP’’s partners to better manage available human resources but also infused higher levels of confidence in the staff and EC members motivating them to improve their performance. The project approach of working through ISOs using modular training was effective and the contents of the modules were highly appreciated by the ISOs and PNGOs. ISOs were mostly dependent on OCBO/As and FOCBOs for SANDEEP’s project activities. The Detailed Implementation and Monitoring Plan (DIMP) and review and reflection processes were effective to monitor project progress and develop action plans. The development of the IDF tool with the five training modules and administration of the first four modules took up the most of the project time. The HRM module had less time compared to other modules. As the designated ISO staff (OCBO and FOCBO) had to begin administrating the modules to the PNGOs immediately after receiving their training, the uptake time was not adequate for them to add value while transferring the knowledge gained became a challenge at the initial stage. SANDEEP processes and changes are being institutionalized by ISOs and PNGOs who have benefited from the implementation of their new policies and systems. With fully functional OD teams, ISOs and PNGOs developed a roadmap for sustaining the SANDEEP processes and changes. Conducive environment for replication of SANDEEP learning across the NGO sector has been promoted by sharing of SANDEEP learning with Association of INGOs in Nepal (AIN) and SWC. Despite retention of OCBOs and Organizational Development Facilitators (ODFs) posing problems to ISOs and PNGOs due to financial constraints, 9 out the 12 OCBOs/FOCBOs in ISOs and 22 out of the 33 ODF in PNGOs have been retained in their respective organization. Besides being assigned other organizational responsibilities, they will also partially continue their organization capacity building function. The sustainability of SANDEEP will, primarily, be contingent on the availability of projects to practice SANDEEP’s learning and processes, especially in the case of small PNGOs whose major concern is how they can tap donor and local resources. Another major concern regarding further replication of SANDEEP’s learning is the depth of ISOs’ commitment to perform their role as OD facilitators. v The recommendations for SANDEEP/SCUS to consider are as follows: • There is a need for continued support to the ISOs and PNGOs in terms of information, follow￾up and linkage for at least two years after the project phase-out to make sure that SANDEEP learning is institutionalized. The support should be in the form of: i) biannual review and reflection sessions and reporting; ii) follow up and monitoring of the Road Map; iii) linkage to SC Alliance and AIN members; and iv) provision of adequate annual budget for the implementation of these activities through the respective ISOs. • A Second Phase SANDEEP project, in collaboration with the SC Alliance, AIN and the SWC, should be developed and implemented. • SANDEEP learning and experience should be extensively disseminated through appropriate sharing forums targeting donors, NGO networks, relevant government agencies, private sector and OD experts at the national and regional level. • Advocacy strategy for policy change in INGOs and bilateral and multilateral development agencies in favor of support to OD processes in civil society organization and funding specific OD focused projects should be formulated and implemented. • The SC Alliance, which has many SANDEEP affiliated NGOs as partners, should be lobbied for a joint initiative to specifically monitor service delivery in projects implemented by SANDEEP partners for a period of three years. The findings should be used to support advocacy efforts for OD focused initiatives in the future. • Potential donors should be consulted and lobbied for imparting full or partial course on SANDEEP to train their partners for a period ranging from one to two years. • SANDEEP modules should be supplemented with sections on social mobilization and proposal development. 1 TABLE OF CONTENTS Page # INTRODUCTION 3 1. PROBLEM STATEMENT 5 2. HYPOTHESIS AND THEORY OF INTERVENTION 6 3. EVALUATION OBJECTIVES AND METHODOLOGY 7 4. PROJECT ACHIEVEMENTS 8 4.1 ASSESSMENT OF ACHIEVEMENT OF PLANNING MATRIX 9 4.2 EVALUATION OF THE NET CHANGE 14 4.3 IMPROVEMENTS IN SERVICE DELIVERY OF ISOs AND PNGOs 25 4.4 IMPLEMENTATION AND CHALLENGES 27 ƒ Projects approaches and interventions ƒ Contribution of ISO approach to achieve strategic objective ƒ OD team approach ƒ Addressing the variation in the nature and size of PNGOs ƒ Effectiveness of OCBOs, FOCBOs and ODFs ƒ Suggestions for alternative approaches / interventions ƒ Ability of ISO and PMT to convey the modules ƒ Comment on support chain: PMT-CBPO-OCBO-ODF ƒ Mechanisms for reporting, feedback and decision making 5. EVALUATION OF USAID PMP INDICATORS 31 6. ASSESSING THE EFFECTIVENESS OF LINKAGES 33 7. IMPROVING CAPACITY TO DELIVER SERVICES IN CONFLICT 34 8. SUSTAINABILITY OF SANDEEP 35 9. ADDRESSING UNINTENDED CONSEQUENCES 37 10. CONCLUSIONS 37 11. RECOMMENDATIONS 40 12. LESSONS LEARNED AND CHALLENGES 41 ANNEXES ANNEX 1: TERMS OF REFERENCE FOR FINAL EVALUATION 44 ANNEX 2: QUESTIONNAIRE FOR STAKEHOLDER INTERVIEWS 50 ANNEX 3: LIST OF PEOPLE INTERACTED 71 ANNEX 4: PLANNING MATRIX – ACHIEVEMENT 76 ANNEX 5: PMP CHART DEVELOPMENT EXPLANATION 82 ANNEX 6: LIST OF DOCUMENTS REVIEWED 84 ANNEX 7: LIST OF SANDEEP ACTIVITIES 85 ANNEX 8: SUMMARY OF STAKEHOLDER INTERVIEWS 88 ANNEX 9: LIST OF ISOs AND PNGOs 112 ANNEX 10: LIST OF SAMPLE ISOs AND PNGOs 114 2 ABBREVIATIONS AGM Annual General Meeting AIN Association of International NGOs Bandh General Shutdown of transport, strike BASE Backward Society for Education (Dang) CBM Capacity building manager CBPO Capacity Building Program Officer DDC District Development Committee DEO District Education Office DIMP Detailed Implementation and Monitoring Plan DPHO District Public Health Office EC Executive Committee/ Governing Board of NGO ED Executive Director EOP End of the Project FCBO Finance Capacity Building Officer (SCUS RO staff member) FNCCI Federation of Nepali Chambers of Commerce and Industries FOCBO Finance Organizational Capacity Building Officer (ISO staff member) FWR Far Western Region HR Human resource HRD Human Resource Development IDF Institutional Development Framework INGO International Non Government Organization IR Intermediate Result ISO Intermediary Support Organization JD Job description M&E Monitoring and Evaluation MOU Memorandum of Understanding MTE Mid-Term Evaluation MWR Mid Western Region NFN NGO Federation of Nepal NGO Non Government Organization NICAT NGO Internal Capacity Assessment Tool NNSWA Nepal National Social Welfare Association NRCS Nepal Red Cross Society OA Organization Assessment OCB Organizational Capacity Building OCBA Organizational Capacity Building Assistant (ISO staff member) OCBO Organizational Capacity Building Officer (ISO staff member) OD Organizational Development ODF Organization Development Facilitator (PNGO staff member) ODS Organizational Development Specialist OM Operation Management PD Project Director PMT Project Management Team PNGO Partner NGO PPME Participatory Project Monitoring & Evaluation R&R Review & Reflection RO Regional Office RPM Regional Program Manager SAC Social Awareness Centre (Surkhet) SANDEEP Sansthagat Deego Pariwartan (Sustainable Organizational Change) SCUS Save the Children-US SNSN Strengthening the NGO Sector in Nepal SWC Social Welfare Council, Government coordinating body for I/NGOs in Nepal Terai Fertile plains that lie across southern part of the country TOR Term of Reference ToT Training of Trainers USAID United States Agency for International Development VDC Village Development Committee VMGO Vision Mission Goal and Objectives WRO Western Regional Office 3 INTRODUCTION The Strengthening the NGO Sector in Nepal (SNSN) project, locally known as SANDEEP (Sanstaghat Diego Pariwartan – Sustainable Organizational Change), was a 5-year USAID-funded project implemented by Save the Children Federation, Inc. (SCUS). SANDEEP, initiated in November 2003, aimed to strengthen the capacity of local non-government organizations (NGOs) to improve the delivery of health, education, and economic services, especially for women, children and disadvantaged groups in the conflict-affected regions of Nepal. SANDEEP envisaged accomplishing this through the following three objectives: 1. Strengthen the operational and financial capabilities of individual partner NGOs (PNGOs) and intermediary support organizations (ISOs); 2. Expand the linkages among and/or between PNGOs and government and private sector institutions; 3. Enhance the program learning and sharing among and/or between NGOs, government and private sector intuitions. The project’s philosophy is based on the hypothesis that “Strengthening the capability of the NGOs in terms of improved operational and financial systems, expanded linkages and increased learning and sharing will strengthen their governance and leadership, human and financial resource management systems, and program development and monitoring and evaluation capabilities. As a consequence, NGOs will increase their effectiveness in program delivery, and thus, achieve desired impacts in the communities they serve.” SANDEEP works with 35 NGOs (initially 40) comprised of 5 ISOs and 30 PNGOs (initially 35) in 17 districts in the Mid and Far Western Regions and parts of the Central and Eastern Regions. The list Program Mgt Team NGO & ( Sectoral ) Networks Community members, especially women, children & other disadvantaged people, in conflict affected areas CBO Networks Local Government Agencies & Local Institutions 35 Partner NGOs (PNGOs) in 17 districts 5 Intermediary Support Organisations (ISO) Program Outreach to NGOs and Communities 4 of NGOs and their location is available in Annex 10. The program model is based on providing support to 5 ISOs through SANDEEP’s Program Management Team (PMT) and to develop them as organizational development and capacity building support organizations. The ISOs, in turn, apply their increased expertise within their own organizations while supporting 30 local district PNGOs to strengthen internal organizational capacities and thereby improve services to target communities. The project uses a standard set of modules to facilitate training and technical support to the ISOs and the PNGOs. At the outset, the PMT staff members are oriented in the modules, then the PMT eastern and western regional offices staff, Capacity Building Program Officers (CBPOs) and Finance Capacity Building Officers (FCBOs) provide training of trainers (TOT) training to the ISOs’ SANDEEP staff, the Organizational Capacity Building Officers (OCBOs) and the Organizational Capacity Building Assistants (OCBAs). Next, the ISOs’ staff conduct training within their ISOs with the support from the CBPOs and FCBOs and finally to their respective PNGOs. Upon completion of the training, the organizational development (OD) team of ISOs and PNGOs, which was made up of representatives from executive committees, management and SANDEEP ISO staff facilitated the SANDEEP interventions and built an internal capacity building action plan drawing on the knowledge gained form the module trainings. SANDEEP’s major activities from November 2003 through July 2008 consisted of two major phases: i) set-up (November 2003 to February 2005) and ii) “project -set interventions" with ISOs and PNGOs (February 2005 onwards). The major accomplishments dueing the set-up phase were the Selection of ISOs (completed in March 2004), completion of the initial baseline cum needs assessment (completed in April 2004), finalization of the Detailed Implementation and Monitoring Plan (DIMP) (completed in September 2004 after the workshop held with 5 ISOs in May 2004), selection of 35 partner NGOs (completed in two phases in July/August 2004 and May 2005), organizational development workshops with 5 ISOs (completed in August/ September 2004), strategic planning for ISOs (completed in December 2004) and in-service training and orientation of PNGOs. "Project -set interventions" commenced on February 2005 with the application of the IDF in 5 ISOs, which was subsequently applied in the 35 PNGOs. At the outset, the organizational assessment tool was the basis for NGOs to assess their organizations. The DIMP followed an "organizational development" issue based approach with the organizational assessment tool forming the basis for PNGOs to self-assess their organizations and formulate activity plans. However, this was constrained by the PNGOs lack of adequate knowledge on how to address the issues they identified. Further, the approach was also constrained by the need for the project to have consistent activities across the PNGOs to ensure that common indicators in the planning matrix were addressed and that the PMT manage the range of intervention activities required. Thus, in the fiscal year 2005, the PMT in collaboration with ISOs revised the project approach and identified a new tool, the Institutional Development Framework (IDF), to more effectively help PNGOs assess their current status and future ambitions. A set of major modular training interventions were identified and initiated based on the IDF assessment. These modular trainings included: Governance and Leadership (including strategic planning and management); Operational Management; Human Resource Management; Financial Management; and Resource Mobilization. In addition to the internal strengthening of NGOs, the project also addressed NGO linkages with other stakeholders and the sharing of learning about organizational strengthening, which was integrated into the Resource Mobilization module. The major activities of SANDEEP in relation to its three intended results (objectives) were: • IR 1: The major activities under IR 1 included the organizational assessment (OA) exercises (baseline, mid-term and final) using the Institutional Development Framework (IDF) and modular 5 trainings on governance and leadership, financial management, resource management and operational management (social audit). A strategic planning exercise, involving the development of strategic plans, was also introduced to supplement the governance module. These activities included all 5 ISOs and 30 PNGOs. Pertinent organization development issues were identified and action plans were developed to address capacity gaps. The development and practice of compliance guidelines by SCUS was another important activity under IR 1. • IR 2: The major activities under IR 2 included the development of a resource directory, a study on linkages with the private sector and orientation for ISO leaders on private sector linkages, initiation of dialogues with the NGO Federation in the eastern and far western regions, dialogue with AIN and the formation of a capacity building task force, dialogue with SC alliance on common partnership guidelines, coordination with ISO donors and PNGOs at the national level and collaboration with SWC to disseminate the governance and leadership module to NGOs in the mid-western region. • IR 3: The major activities under IR 3 included review and reflection sessions, discussions with AIN (including taskforce formation) on AIN partnership guidelines development, meeting of ISO leaders and the PMT, ISO exposure visit to India and the collection of case studies from ISO/PNGOs, disseminating effective practices through NFN in the east and far west regions and sharing SANDEEP learning at the SCUS regional office in Bangkok and Bhutan. (For details of the major activities and timeframe, please refer to Annex – 7) 1.0 PROBLEM STATEMENT Nepal has witnessed rapid and unprecedented political changes in recent times. The April 2006 pro￾democratic movement ended the King’s direct rule and reinstated the dissolved house of parliament. This was followed by the comprehensive peace treaty between the Government and the Communist Party of Nepal - Maoists (CPN (M)), formation of the interim government with the participation of the CPN (M), promulgation of the interim constitution, elections for the constituent assembly and the end of the 240 year old monarchy with the formal declaration of Nepal as a federal republic on 28 May 2008 by the first meeting of the Constituent Assembly. Owing to these political changes, the conflict situation in Nepal has subsided and undergone major changes in past two years, resulting in increased access and mobility of the government, non-governmental organizations and civil society groups to most of the areas area that were most affected by the conflict. Despite these positive changes, access to certain areas of the eastern and central terai region (especially areas that lie south of the East – West Highway) is still hindered due to the presence of more than a dozen armed groups and the deteriorating law and order situation. Structural causes of poverty and injustice, such as entrenched social exclusion, under-representation of certain groups in the development mainstream, lack of proper access to justice, uneven geographical development, caste- and gender-based discrimination and poor governance still persist. Human rights abuses in the form of threats to human life, abduction, extortion and arbitrary physical actions still continue. The peace and reconciliation process is also still fragile due to political uncertainty. Civil strife and protests accompanied by frequent bandhs (closures) and violence are still common feature as various groups, often based on ethnic and geographical identities, demand recognition and inclusion of their agendas in the new constitution. The problem statement of the SNSN program that “the conflict situation of Nepal resulted in deteriorated government mechanisms for service delivery on health, education and economic empowerment” is still partially valid. The NGO sector is in a position to complement the efforts of the government in terms of service delivery as implementation of Government programs are being revived with increasing vigor in conflict affected areas. Government service delivery in rural areas in partnership with NGOs, which was largely unheard of before the conflict, has now gained acceptance following the crucial role played by NGOs in delivering services in conflict affected rural areas that were largely inaccessible to the State’s development interventions. As a consequence, district based 6 civil society organizations are now increasingly viewed by Government agencies as partners for the implementation of development interventions. The crucial role played by NGOs in the past decade in providing services to the community is acknowledged by the Government and international development agencies. The Interim Plan of the Government stresses effective mobilization of NGOs and strengthening collaborative works and coordination of the government and NGOs for the overall economic and social development of the country. The Interim Plan also states that it is necessary for NGOs to own the beliefs of transparency, accountability, participation and inclusive development and translate them into practice. Good governance in NGOs has been gaining significance as NGOs have been criticized for lacking sound organizational, financial and managerial systems, especially as related to transparency and accountability. It has been commonly assumed that improved internal organizational capacity will lead to more effective service delivery. Thus, many development agencies supporting NGOs have emphasized strengthening the internal organizational capacity of NGOs to deliver effective services. The majorities of local NGOs rely heavily on donors for funds and are usually implementing “donor led” programs, having a low capacity for resource mobilization, which results in competition for limited external funds. NGOs have inadequate linkages to the private sector where opportunities to avail support for health, education and economic activities exist. As NGOs, operating without proper administrative, financial and operational polices and systems, find it very difficult to deliver quality services, the challenge facing all stakeholders is to improve and institutionalize good governance systems and enhance coordination, linkages and networking among local NGOs and promote better coordination among NGOs, donors, the private sector and the Government. 2.0 HYPOTHESIS AND THEORY OF INTERVENTION The SNSN project aimed to test the hypothesis that "Enhancing the organizational capacity of NGOs and sharing this experience across the NGO sector will lead to improvements in the delivery of health, education and economic services. It will also increase the NGOs’ capability to work in the conflict situation in Nepal and also help mitigate the effects of the conflict and poverty.” Hypothes is E nhancing the organizational capacity of NGOs, and sharing this experience across the NGO S ector Will lead to improvements in the delivery of health, education and economic services And increas e the NGOs’ capability to work in the current conflict situation in Nepal Will help mitigate the effects of the conflict & poverty The findings, with regards to the project’s hypothesis, are as follows: • The hypothesis is relevant in terms of the “enhancement of NGOs’ capacity to work in current conflict situation” as the project’s approach and interventions support this aspect of the organization’s capacity. ISOs and PNGOs stated that improved organizational capability enabled them to develop coping strategies to work within the conflict situation. All the ISOs and a majority of the PNGOs were able to sustain development interventions while many of them were even able to grow during these times. • The direct link of NGO capacity enhancement to "improvements in the delivery of health, education and economic services" is difficult to establish. The direct beneficiaries of services 7 delivered by the ISOs and PNGOS were not able to validate this. To validate this linkage, the project relied on feedback (a questionnaire survey) from donors who were supporting the ISOs and PNGO. About 47% of the donor sample reported improvement in service delivery due to capacity enhancement of the NGOs. Strengthening organizations’ internal capacity can play a significant role in the improvement of service delivery, but it is one of several aspects of an organization that influences service delivery in terms of quality and coverage. • The extent of the project’s link to "mitigate the effects of conflict and poverty" is difficult to ascertain as the project’s design, approach and interventions did not clearly define and establish the linkage in term of processes and outputs. What is noticeable is that the project inputs contributed towards enhancing ISOs and PNGOs capacity to work within the conflict situation and also helped to mitigate the effects of the conflict at the micro level by emphasizing transparency, accountability and the participation of community members in the planning, implementation and monitoring of projects. 3.0 EVALUATION OBJECTIVES AND METHODOLGY The FTE assessed whether the program achieved the three intermediary objectives which lead to the strategic objectives, the successes and failures of the project, as well as the project's influence in NGO capacity building. In addition, the FTE assessed the impact of capacity building on service delivery, evaluated how the project mitigated the conflict through capacity building interventions and made recommendations to the development community on the value of conducting and supporting OD amongst NGOs in Nepal. (The ToR for the Final Evaluation is available in Annex – 1) The key objectives and expected answers of the Final Evaluation were to: 1. Evaluate whether the project achieved its planning matrix including PMP and whether the project achieved its strategic objectives. 2. Determine 'if' and 'why' the project's objectives were achieved fully, partially or not at all 3. Evaluate net impact (net change between the baseline, midterm, and end of project) regarding the status of project interventions, capacity of NGOs to deliver development services in the areas involved and expanded linkage among NGOs, networks and public and private sector organization 4. Identify whether there are indications that these changes are resulting in improvements in the ISOs and PNGOs' service delivery 5. Identify what effect and impact the project has had on the other stakeholders (other NGOs, NGO networks, donors, AIN and other government organizations which were not direct beneficiaries of the project) 6. Identify how the conflict situation in Nepal affected ISOs and PNGOs, and whether the project responded appropriately to mitigate these effects. 7. Evaluate the viability of sustaining the capacities developed within SC/US and partners 8. Assess the unintended results of the project 9. Identify the challenges/lessons learned during the project period 10. Identify specific OD issues and needs of PNGOs in the health, education and livelihood sectors and make appropriate recommendations for replication and improvement of prevailing OD practices applied by SANDEEP as well as other donor agencies for effective service delivery by their respective partner organizations working in these sectors. 11. Recommend specific advocacy measures targeting potential donor agencies who can support SANDEEP’s second phase design and implementation The FTE adopted a pragmatic approach targeting ISO and PNGO executive committee members, staff and beneficiaries and relevant stakeholders (DDC NGO desk, NGO networks and relevant line agencies) along with donor agencies for the purpose of primary information collection. The FTE team had regular consultations with SANDEEP PMT team in Kathmandu prior to the finalization of the evaluation objectives, processes, tools and methods. 8 Sources of Data Sampling: Purposive sampling method was employed to select sample NGOs. Out of 5 ISOs and 30 PNGOs supported by the project, with 3 ISOs and 7 PNGOs selected as targets for field visits. Visits were also made to the remaining 2 ISOs for a half hay interaction program with executive committee members and staff. (The list of sample ISOs and PNGOs, OA and final evaluation dates are available in Annex – 10.) The sample of PNGOs, which was discussed with and agreed by the SANDEEP PMT, took into consideration the following variables: • Location of PNGO (whether it is accessible location or a more remote and heavily conflict affected location, type of service delivery in geographic locations – hill and terai) • Composition of PNGO board and staff by ethnicity and gender • Size of the PNGO (mainly determined by the staff number) • Number of years the PNGO had been in operation • ISOs or PNGOs selected for measuring PMP The FTE employed both secondary data compiled by the PMT and primary data collected by the FTE team during the field visits. The secondary data was primarily comprised of data collected by the PMT for monitoring purpose. The data were collected from the following sources: Project Documents: Relevant project documents, Monitoring and Evaluation (M&E) manuals, annual progress reports and donor questionnaires were key sources of data. (The list of documents reviewed is available in Annex - 6.) IDF Scoring Data: The baseline, mid-term and final term OA (Organization Assessment) data from IDF collected by the PMT were used for comparison and analysis of change. ISOs and PNGOs: Executive committee members, staff (senior and field level), OCBOs/As and Finance Organizational Capacity Building Officers (FOCBOs) and relevant project documents were the key sources of data. District Level Stakeholders of ISOs and PNGOs: District level stakeholders of the sample ISOs/PNGOs, mainly representatives of District Education Offices, District Health Offices, District Development Committee/Planning Sections and Women Development Offices/Child Sections, were interviewed to collect information on changes they perceived and their impression on the NGOs’ performance. Beneficiaries of ISOs and PNGOs: The community level beneficiaries were the main sources of data to identify and assess changes in service delivery of the ISOs/PNGOs. Monitoring Donors of ISOs and PNGOs: The representatives of donors supporting SANDEEP’s ISOs and PNGOs were the main source of data in assessing the organizational changes taking place within ISOs and PNGOs and the subsequent effects on service delivery. SANDEEP PMT Staff Members in Kathmandu and Regional Offices, SCUS directors and representative of USAID and AIN were also interviewed. Data collection methods and tools Review of relevant documents, development of checklists and instruments, focus group discussions (FGDs), interviews/interactions, observations, data analysis and presentations of evaluation findings were the key methods applied in the FTE. The details of methodologies employed are presented below. Review of key documents and records: The FTE reviewed relevant documents and records to further the evaluation in order to accomplish its objectives. Data collection at field level: FGDs were conducted for data collection from executive committee members and staff of ISOs and PNGOs while individual and group interviews were employed to collect data/information from OCBOs/As, FOCBOs, ODFs, PMT staff, and district and Kathmandu based stakeholders and donors. Observation as an additional verification tool was employed throughout the information collection process. (List of stakeholders interviewed and 9 questionnaire guidelines and checklists, developed for individual and/or group interviews and FGDs are available in Annex – 3 and Annex – 2 respectively). Review of ISO and PNGO Organizational Assessments using IDF: During the evaluation ISOs and PNGOs conducted organizational assessments with the PMT to assess changes taking place within their organizations and service delivery quality. This information was used by the FTE to assess changes and progress across ISOs and PNGOs. Sharing of preliminary findings and validation: The data collected by the FTE team and OA by PMT Kathmandu were analyzed. The preliminary findings were shared in 2 presentations with a) PMT, SCUS Senior Staff, USAID representative and members of the SC Alliance in Kathmandu and b) a Reflection and Review sharing session in Chitawan participated by ISOs’ executive committee and staff members, PMT and Regional Office staff, SCUS directors and a representative of USAID. The feedback and discussions from the presentations clarified and to a large extent validated the data collected. SANDEEP periodic reports and OA outcomes also validated the data collected. Data Analysis, report write-up and feedback/comments: The data was further analyzed following the FTE ToR and the feedback received from the sharing presentations. The draft report was shared with PMT Kathmandu and Regional Offices and SCUS directors and representative of USAID for feedback and comments, which were incorporated prior to final report submission. 4.0 PROJECT ACHIEVEMENTS 4.1 Assessment of achievement of Planning Matrix The assessment of achievement against the Planning Matrix has been mostly moving in the right direction. A majority of project targets and indicators of the three intermediate results were either met of exceeded.. The results indicated that the pre-award survey, training, mentoring and coaching activities and OA exercises using the IDF tool had led to ISOs and PNGOs initiating actions (specific actions such as action plans, road maps and long term strategic plans) to address the various capacity gaps (human resources development, effective management systems, external linkages and resource mobilization). 4.1.1 Strategic objective: Enhance the capacity of the NGO Sector to improve the delivery of health, education and economic services, especially for women and children and disadvantaged groups, in the conflict affected Mid, Far Western, and parts of Central and Eastern Regions The project was successful in achieving its strategic objective on the basis of its Planning Matrix indicators. (Planning Matrix and EOP Achievements are available in Annex – 4 and Annex - 5) The indicators and targets, presented in the form of charts, are as follows: Enhance Capacity of NGO 20 00 20 00 40 40 13 80 40 34 80 90 47 0 20 40 60 80 100 % of ISO % of PNGOs % of PNGOs who receive Positive impact Assessment Score Baseline MTE Target MTE Achievement FTE Target FTE Achievement 10 The project achieved or exceeded the targets for developing “capable NGOs”2 at the end of the project (EOP) period as it shows 80% of ISOs (4 /5) and 90% of PNGOs (27/30) have become "capable NGOs". Likewise, in comparison to the target of 34% set by the PMT, 47% of the PNGOs (14 out of the 30) received positive impact evaluations of their health, education or economic opportunities programs by their respective donors. These donors identified the positive impact to be as a result of SANDEEP’s intervention. Donors were of the opinion that there is a positive correlation between service delivery and improvement of internal organizational culture and systems (policies and practices, expanded linkages and increased learning and sharing) resulting in effectively run programs. However, the donors also pointed out that it would be difficult to ascertain to what extent the SANDEEP intervention had actually contributed in the enhancement of service delivery without a detailed assessment of the correlation after the conclusion of the OD intervention due to the fact that SANDEEP’s focus on internal organizational capacity strengthening, though very important, is one among several factors that influence result achievement at the service delivery level. 4.1.2 Differential Progress in ISOs and PNGOs on indicators related to Intermediate Results (IRs) IR 1: Strengthen operational and financial capabilities of individual PNGOs and ISOs. Project experience and learning indicated that it was easier to initiate changes in smaller and newly formed NGOs in comparison to larger and older ISO/NGOs whose operational and financial policies and practice have been in place for a longer period of time. This was validated by speedier and more effective changes experienced in new and smaller PNGOs and ISO in comparison to the four larger ISOs. The FTE team also observed that the IDF revealed changes but does not indicate whether such changes have really been consolidated or reinforced and subsequently developed into institutional systems. ISO level progress: Progress was made in ISOs on the different capacity areas related to IR 1 during the project period. The EOP targets were met in terms of effective management, planning, monitoring and evaluation and financial management systems along with the establishment of an internal system and usage for measuring quality of service with beneficiaries’ participation. The target related to good governance and leadership and practice of participatory human resource management fell short in case of one ISO.3 2 The project’s M&E Manual defines “capable NGOs” as ISOs and PNGOs that are able to reach the ”Expansion/Consolidation” stage in the IDF tool for the following areas: Oversight/Vision, Participatory Management, Human Resource Management and Financial Management and are able to reach the development stage for the area of “External Relations”. 3 SANDEEP Project defines: i) “Good Governance & Leadership” as those ISOs and PNGOs who achieve the”Expansion & Consolidation” stage of the IDF tool when combining the scores of the following components: Board, Mission and Leadership Style. ii) ISOs and PNGOs that have an “Effective Management System” achieve the “Expansion & Consolidation” stage of the IDF tool when combining the scores of the following components: Participatory Management and Management Systems. iii) ISOs and PNGOs that include a “Participatory Human Resource Management System” as those who achieve the ”Expansion & Consolidation” stage of the IDF tool when combining the scores of the following components: Staff Development & Organizational Diversity. iv) ISOs and PNGOs that have an “Effective Planning, Monitoring and Evaluation System” as those who achieve the ”Expansion & Consolidation” stage of the IDF tool when combining the scores of the following components: Planning, Community Participation and M & E. 11 Strengthen Operational & Financial Capabilities of ISOs 40 20 40 40 40 0 60 60 40 60 80 0 40 60 40 40 40 0 100 100 100 100 100 60 80 100 80 100 100 80 0 20 40 60 80 100 120 Good Governance & Leadership Effective Management System Participatory HR Management Effective Planning & ME System Financial Management System. System for Measuring Quality of Service Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievement PNGO level progress: The progress of PNGOs, a majority of which did not have systematic operational and financial policies and practices in the initial stage, was visible in all indicators related to IR 1 though interventions in PNGOs started later. In the case of PNGOs, the project has been able to achieve the targets relative to all indicators. . Strengthen Operational & Financial Capabilities of PNGOs 000000 3 6 0 9 11 0 20 14 60 11 17 0 89 97 71 80 77 48 100 97 90 100 100 100 0 20 40 60 80 100 120 Good Governance & Leadership Effective Management System Participatory HR Management Effective Planning & ME System Financial Management System. System for Measuring Quality of Service Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievement IR 2: Expand linkages among and/or between PNGOs and government and private sector institutions. The PMT has prepared a guideline for developing linkages between the private sector and NGOs with the purpose of improving NGOs’ access to support (funds, technical expertise, moral support) from the private sector, raising awareness of private sector opportunities for market expansion through contribution to the alleviation of poverty in Nepal and enhancing relations between NGOs and the private sector leading to joint actions to alleviate poverty. Workshops on private sector linkages, facilitated by an external consultant, were also conducted for all the ISOs. The evaluation team observed that at the central level, the PMT has had some success in soliciting private sector support4 but this was reported, by ISOs and PNGOs, to be modest at the district level. The FTE team felt that the conflict environment was not conducive for the expansion of linkages between NGOs and the private sector. The private sector, which was adversely affected by the v) ISOs and PNGOs that have an “Internal System for Measuring the Quality of Service with Beneficiaries Participation” as those who achieve the ”Expansion & Consolidation” stage of the IDF tool based on the criteria “Beneficiary Participation”. 4 “Own a School” targeting commercial banks and dialogue targeting large corporate houses 12 conflict in the form of frequent bandhs (closures), labor unrest, extortions, abductions, threats to life and arbitrary actions, faced the most challenging and difficult time for sustaining their operations. As a result many business enterprises had to either close or scale down their operations. Despite the official end of the Maoists insurgency and their election to the Constituent Assembly, the security situation in recent times has not significantly improved in some parts of the country. The central and eastern terai region still experience armed conflict while closures, protests and an environment of impunity continue to prevail in some parts of the country. Further, inculcation of corporate social responsibility and its ensuing benefit is yet to find a firm foothold in the business community of Nepal. NGOs have also not been able to convince the private sector that linkages with NGOs and their support for social justice will enhance their public image and ultimately benefit them in product/services marketing and acceptability. Thus, initiation of linkages with the private sector by some ISOs and PNGOs, in spite of the difficult environment under which they operated, can be considered to be a significant, though modest, achievement. ISO level progress: The ISOs have achieved targeted progress in all capacity areas related to IR 2 during the project period in accordance with the Planning Matrix. The level of progress in different stages of the project is presented in the graph below. Expansion of Linkages of ISOs 0 20 0 20 60 0 60 80 100 80 0 20 40 60 80 100 120 Received not more than 40% of Funds from One Source. Received Consultancy Service & Private Sector Support Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievement In alignment with the project target all ISOs are receiving a maximum of 40% of funding from one source and 80% of the ISOs (4 out of 5) have received mentoring or pro-bono consultancy services or in kind donations from the private sector to expand their delivery of services to communities in need. PNGO level progress: The progress of PNGOs in respect to indicators related to IR 2 has progressed well with achievements beyond project targets for all indicators. The level of progress in different stages of the project is presented in the graph below. 13 Expansion of Linkages of PNGOs 00 11 0 23 3 11 0 100 57 46 0 57 66 46 70 100 70 43 100 0 20 40 60 80 100 120 Initiation of Meeting with other PNGO Received not more than 60% of Funds from One Source Received Consultancy Service & Private Sector Support Participation in Yearly Training/Workshop/Learning Fora Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievement Executive Directors or EC members of all PNGOs (100%) are initiating meetings with other PNGOs to develop standards in management practices. Likewise, 70% of PNGOs are receiving a maximum of 40% of funds from one source while 43% of them received mentoring or pro-bono consultancy services or in-kind donations from the private sector to expand the delivery of services to communities in need. All the PNGOs (100%) are participating in learning fora with other NGOs. The FTE team felt that progress based on the relatively few indicators employed by the PMT cannot be conclusive for judging the actual expansion of linkages with the private sector and government. This was due to the absence of indicators related to linkages with government line agencies and the modest private sector funding support for projects implemented by the ISOs and PNGOs. IR 3: Enhance the program learning and sharing among and/or between NGOs, government and private sector institutions. ISO level progress: The ISOs have achieved or exceeded targeted progress in all capacity areas related to IR 3 during the project period. The level of progress in different stages of the project is presented in the graph below: Enhancement in Programme Learning and Sharing of ISOs 00 40 00 0 80 80 100 80 0 20 40 60 80 100 120 Publications on Effective Organizational Practices Replicating Effective Organizational Practices Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievem In alignment with the project target, all ISOs (100%) are producing publications on effective organizational practices that help to build service delivery capacity in health, education, and economic opportunity programs. In addition, 4 out of the 5 ISOs (80%) are replicating effective organizational practices, which resulted from participation in learning fora with other ISOs and PNGOs in order to build their service delivery capacity in health, education, and economic opportunity programs. 14 PNGO level progress: There has been partial achievement of PNGOs in respect to indicators related to IR 3. The level of progress in different stages of the project is presented in the graph below: Enhancement in Programme Learning and Sharing of PNGOs 00 00 00 50 50 23 100 0 20 40 60 80 100 120 Publications on Effective Organizational Practices Replicating Effective Organizational Practices Score in % Baseline MTE Target MTE Achievement FTE Target FTE Achievement Though 50% of the PNGOs were targeted to produce publications on effective organizational practices that help to build service delivery capacity in health, education, and economic opportunity programs, only 23% (7 out of 30) of the PNGOs have produced such publications. However, while 50% of the PNGOs were targeted to replicate effective organizational practices resulting from participation in learning fora with other ISOs and PNGOs in order to build their service delivery capacity in health, education, and economic opportunity programs, in fact all the PNGOs (100%) were replicating effective organizational practices by the end of the project. 4.2 Evaluation of the net change: The status of project interventions, capacity of NGOs to deliver development services in the areas involved and expanded linkages among NGOs, networks and public and private sector organization. Based on the situation analysis of the NGO sector in the Mid and Far Western Regions (2003) and the Baseline Study (2004), the SANDEEP project identified 5 key areas of organizational development support for ISOs and PNGOs. These key areas were (i) Governance and Leadership, (ii) Financial Management, (iii) Human Resource Management, (iv) Resource Mobilization and (v) Operational Management. The project developed related Institutional Development Framework (IDF) tool and training modules on each of these thematic areas. The training modules were administered by the PMT staff to the selected five ISOs, who in turn provided similar training to the selected PNGOs under their portfolio. The achievements of the project through these interventions were as follows: 4.2.1 One of the major achievements of the project was the significant impact it generated towards the management of an organization. Besides becoming aware of the fundamental elements and characteristics of an NGO as an organization, the ISOs and PNGOs reported that they were also equally concerned about organizational management, particularly in reference to the sustainability of activities and programs. Consequently, the organizational assessment (OA) workshop, using the OA tool (IDF), was highly effective. First of all, the workshop assisted ISOs and PNGOs to discover and appreciate the vital ingredients of an organization, and secondly, it helped the participants to see the organization beyond the project on a path of sustainability. “OA exercises helped us to realize the importance of division of role and responsibility and the value of individual contribution and participation to achieve the goals of the organization. This realization led to the practice of regular joint meetings between staff and EC members and inter￾project “synergy” meetings to discuss and address organizational and program issues in a coordinated manner.” Often repeated view of EC members and staff of ISOs and PNGOs interacted 15 It is notable to mention here that one of the ISOs had even initiated a two day annual retreat (with EC, staff and life members as the participants) for team building exercises as a follow-up on the OA workshop. According to the ISOs and PNGOs, there has been an increase in staff and EC participation in management and program decisions within the organizations. The overall commitment of EC members also increased, as evidenced by the fact that EC members were spending more time on organizational matters. Owing to the IDF’s emphasis on inclusion and diversity, the composition of the EC and staff also became more inclusive in terms of gender and caste. It was observed that the project successfully instilled in the ISOs and PNGOs a need to have standardized filing, proper documentation, transparent bookkeeping and proper store management. Though some of these systems were present in some of the ISOs prior to SANDEEP, these were not comprehensive and were not systematically implemented. During the interaction with the FTE team, ISOs and PNGOs reported that SANDEEP inputs had increased the self-confidence level of staff and EC members enabling them to clearly express their opinions and ideas to relevant stakeholders. Stakeholders, donors and line agencies, reported that the quality of reports submitted to them had improved in recent times. ISOs and PNGOs had published and distributed brochures and annual reports to disseminate information about their organization to stakeholders. The staff and ECs stated that meetings are now more focused on the agenda, which are now communicated to the participants prior to the meetings. As a consequence of these activities and changes, coordination with stakeholders has improved while organizational issues are being addressed through timely decisions and better understanding. There was consensus among the staff and EC members that the IDF tool was instrumental in helping them to make a realistic assessment of their organizational capacity through participatory review and reflection process. However, what needs to be understood is that the IDF tool, which reveals organizational capacity at a particular time, does not necessarily reflect the depth and sustainability of changes in ISOs and PNGOs in terms of whether such changes were just introduced and applied recently or have been developed into habits. Given the limited resources of the project, a standard approach with common indicators and compliance was used for all sizes of NGOs. A more flexible approach would have required more staff with higher qualifications and technical skills in specific sectors (education, health etc.) and more staff training – all of which was beyond the project budget. However, the ISOs and PNGOs were able to apply some degree of flexibility by following their own action plans that allowed them to adjust the modular approach to address their specific needs. During their interaction with the FTE team, the OCBOs, FOCBOs and ODFs suggested that some indicators (such as formation of various committees, use of charts of accounts, use of FAMAS software, integrated reporting and filing systems) need revisiting to align them to the size of NGOs and taking into consideration their practical application. Case Study - 1 PNGO Reflects on SANDEEP Experience “Our donor helped us to become established and gave us project, but did not give us skills on how to run an organization” says Sitaram Chaudhary, President of KPUS, a PNGO in Kailali. “If you don’t know how to run an organization, how can your programs be effective?” Kamaiya System Eradication Society (KPUS) was established a decade ago by federating Kamaya groups in 12 VDCs of Kailali district. Sitaram says that SANDEEP taught them what good governance is and helped revise their constitution and policies, helped separate dual roles and established finance systems in their organization. “Due to SANDEEP learning we have now delegated responsibilities to the VDC level committees and clarified their roles and responsibilities” says Sitaram, “now these committees have started taking independent initiatives and have started visiting VDCs with their own plans”. He says that staff and EC need to work jointly to improve the programs. “Just having constitution is not 16 enough, we equally need guidelines to check the practice of ad-hoc decision-making” argues Sitaram. With SANDEEP’s help, KPUS conducted a social audit and received feedback from Kamaya groups and stakeholders for improvement. Before SANDEEP, they used to have top down type of planning system. KPUS has now started making action plans against audit report recommendations. Sitaram says that all planning before SANDEEP was done by donors, but now after strategic planning exercise, they are able to develop annual plans independently. Sitaram believes that SANDEEP helped them to improve their quality of service. SANDEEP training helped them to understand how to facilitate community groups and planning. They have also been able to link the VDC-level committees to VDCs and have encouraged them to participate in the VDC planning process. Some of the committees have successful obtained resources from VDCs to build schools in their community. Sitaram believes that SANDEEP has changed the behavior of both the NGO and the donor. “Our donor did not like SANDEEP in the beginning, because they thought SANDEEP training only diverted our attention from project implementation. But, now, our donor is satisfied with SANDEEP, because our organization has become much more systematic in terms of planning, accounting and reporting etc.” says Sitaram. Regarding the changes in his own attitude, Sitaram says “before SANDEEP I used to think that organization’s internal affairs should be kept a secret, but now, after SANDEEP, I know that we need to be as transparent as possible. This is the effect of the OA exercise”. 4.2.2 The governance module was instrumental in enabling the EC and staff to understand their roles and responsibilities. As a result, JDs provided to the staff and the EC members eliminated dual roles and increased individual accountability. OD teams, comprising key staff and board members, had been formed in all ISOs and PNGOs. The OD Team, which generally meets once a month to discuss organizational issues, was highly valued by the EC and staff as it has been instrumental in facilitating faster decision making processes within the EC. ISOs and PNGOs reported that they had revised their personnel policies to reflect separation of roles within the organization and delegation of operational responsibilities/authority to staff, which expedited the decision making process. Subsequently, most ISOs and PNGOs “enshrined” the separation of dual roles in their “Constitution”. Training on the governance module also led to the formation and/or review of the composition of sub-committees for purchasing, staff recruitment, and resource mobilization etc. in these organizations along with the development of related guidelines for the sub-committees. Some PNGOs also developed EC member selection criteria, EC members’ performance evaluation tools and organizational codes of conduct as suggested by the module. ISOs and PNGOs took major steps to reduce the practice of nepotism by appropriate changes in their recruitment policies making these more transparent. This training module was reported to have been particularly effective in PNGOs as these, unlike in ISOs, did not have any resistance to change created by the organizational structure and organizational culture. The FTE assessed that the governance module increased the technical skills of staff and EC members to formulate and revise various policies. General members of the organizations have also started to actively participate in different sub-committees. Both ISOs and PNGOs conveyed that administrative work became more efficient after the formulation of organizational policies. The policies were also revised in accordance with organizational needs thus suggesting the application of some sort of an evaluation process to assess the policies from time-to-time. There are other positive impacts that the Governance module had on the ISOs and PNGOs. Despite initial fears, ISO and PNGOs pointed out that clearly defined roles and responsibilities of staff and EC 17 members contributed towards management efficiency owing to the division of workloads in a transparent and accountable manner. In PNGOs, the ECs were reorganized to make them gender and caste inclusive. Development of various policies also helped ISOs and PNGOs to better negotiate with donors. Overall, the public image of the participating organizations was reported to have been enhanced because of the higher level of transparency in the functioning of ISOs and PNGOs. Case Study - 2 INGO Reflects on SANDEEP Impact “SANDEEP’s partner NGO (PNGO) is most responsive to our comments and feedback about their organization and performance whereas many NGOs take our feedback with skepticism” says the chief executive (CE) of one of the INGOs working in Nepal. The PNGO, which works in the terai of central development region, was a small organization in 2002 when the INGO started supporting it with an education project targeting conflict affected children. “Donors only commit short term (generally one year) projects with such small NGOs, which is a bottleneck for their growth and sustainability. As projects supported by other donors had just been completed, we were the only donor of this NGO.” he recalled. In 2005, the SANDEEP project selected this PNGO and provided training on OD and good governance. This training was effective in persuading the Chairperson, who was also working as the executive director, and other EC members, to end dual roles in the organization. Attitudes within the organization started changing. The staff and the EC members became clear about their roles, strategies were formulated and the PNGO was able to build rapport with district stakeholders. Impressed by the social audit conducted by the PNGO, the CE stressed the need for all NGOs to institutionalize social audits. The CE shared: “OD support has definitely contributed to the improvement in the quality of services provided by the PNGO but it is very difficult to ascertain how much of this is due to such an intervention. In recent times, the PNGO has improved the quality of its reports. It has increased its interaction and rapport with community members and district stakeholders and it has been able to mobilize community structures more effectively. The PNGO also received a new project from the ILO. We don’t have funds for dedicated OD support to NGO partners, but in the future, we can mobilize the NGOs that have become capable due to SANDEEP’s support”. 4.2.3 Prior to the implementation of SANDEEP, the strategic plans of ISOs were largely donor driven and in many instances unable to address the socio-economic needs of the people, particularly those in conflict affected districts. Under the governance module, SANDEEP, through its strategic planning workshops, provided an opportunity for the ISOs and PNGOs to align organizational elements (structure, culture, program, policies and human resources) and develop a shared vision of the organization. ISOs and PNGOs reported that the strategic plans developed in the workshops identified resource gaps to meet the agreed organizational goals. As a result, many of them improved their organizational structure by defining clear roles and responsibilities. Internal communication was greatly enhanced with reporting channels clearly delineated. The new structure facilitated ISOs and PNGOs to make timely decisions. Strategic plans also supported their resource mobilization efforts. Some of the ISOs and PNGOs expanded geographical coverage, while others were using their strategic plan to internalize processes, and align programs with their structures. The strategic management module helped the ISOs and PNGOs become more inclined towards their target groups/working areas and organizational initiatives rather than being guided entirely by donor requirements. According to the ISO and PNGOs, they now give more emphasis to consultation with targeted communities to planning, project design, implementation and monitoring programs. Context analysis and needs assessments were regularly employed during the project planning and design stage. ISOs and PNGOs also made planning and review systems more systematic by having daily, quarterly 18 and annual plans in line with their strategic plans. There was a general agreement among ISOs and PNGOs that strategic plans provided a distinct direction to their actions and increased the motivation of staff and the EC to fulfill organizational objectives. The most important achievement of the strategic management module was that the ISOs and PNGOs started to see the value of strategic plans as an analytical tool to assess gaps between their strategic objectives and operational plans as well as develop new programs to address these gaps. 4.2.4 Following the pre-award surveys and organizational assessment workshops, financial management training was identified as a major intervention for building the organizational capacity of ISOs and especially PNGOs. The majority of PNGOs had absolutely no systems or policies for financial and store management before the SANDEEP project. ISOs, on the other hand, reported that they had financial policies and systems before the SANDEEP project, but these systems were neither comprehensive nor consistently followed. The ISOs and PNGOs informed the FTE team that prior to the project they were unable to manage financial resources even when they had willing donors to fund their activities for lack of financial policies and systems. As a result of the financial management training, financial recording and reporting greatly improved. The EC and staff members of all the ISOs and PNGOs consulted during the FTE pointed out that they had developed, revised and implemented a set of financial and personnel policies. The organizations developed the practice of using separate ledgers and reporting systems for each project they undertook. Procurement guidelines were developed and implemented. Unlike in the past, when most of the transactions were done in cash without adequate control, a system of handling cash through a “petty cash” system was developed and practiced. The ISOs and PNGOs developed financial and administrative formats for the control of advances, travel of staff (timesheets), and leave records. A system of annual physical verification was developed, and for the first time, physical verification of materials in the store was done. All ISOs and PNGOs prepared detailed terms of reference for external auditors. It was reported by the ISOs and PNGOs that their organizational budgets are now being reviewed semi-annually and meetings between finance and program staff are being held on a regular basis. They now also have specialized sub-committees for auditing and procurement. Systems including a log book for telephone and motorbike use, among others, are now in place and monitored on an on￾going basis. Both ISOs and PNGOs stated they were strictly following the policy provision on the clearance of advances with appropriate supporting documents before settlement of such advances. Cash transactions have been greatly reduced. Donors and stakeholders also appreciated the financial and administrative systems of SANDEEP￾affiliated NGOs. Community leaders, donors and district based stakeholders pointed out that the ISOs and PNGOs were now more transparent and accountable, which increased the trust of donors, beneficiaries and other stakeholders. Case Study – 3 CECI/Sahakarya Reflects on SANDEEP Experience Among the 21 partner NGOs of CECI’s Sahakarya Program, 10 are SANDEEP partners. These PNGOs partners were selected during the conflict period because working through NGOs was the best strategy at the time. “Many NGOs in Nepal have received donor funding, but they lack management and delivery capacity. We understood that the SANDEEP project was initiated to fill this gap – to develop the capacity of NGOs (i.e. help them to establish systems, policies and guidelines) so that they could improve the quality of services provided to the target communities” says Pawan Karki, regional coordinator of CECI/Sahakarya project based in Surkhet. 19 Pawan saw notable changes in SANDEEP partner NGOs. “Before, these NGOs did not have much work (projects). They did not have administrative or financial policies and systems and their ECs met rarely. There was no system of vouchers and the leaders of the NGOs used to spend project funds like they spend money at home. Now all these practices have changed. PNGOs have developed systems and policies. Their EC has become active and motivated. They have developed good financial systems and all payments are made through bank cheques. “Before, we used to return their financial report 5-6 times, but now their reporting has improved. They have become transparent and don’t hide problems and issues from us” says Pawan. CECI/Sahakarya officers have also observed some improvement in the quality of service. “All the staff and EC members have been provided with JDs, which have clarified the duty of every person. Social audits are conducted regularly due to which the community is clear about the budgets of the projects. Before SANDEEP, only the Chairperson of the NGO could give information about the projects, but now these PNGOs look like real organizations.” Pawan believes that SANDEEP project’s impact can also be evaluated by comparing the performance of SANDEEP PNGOs with other NGOs they work with. In SANDEEP partners, dual roles have ended, operational guidelines are developed and followed, a system of vouchers has been established, selection of staff is done through public advertisement and decisions are made through collective meetings. “We don’t see such elements in other NGOs we work with. Therefore, we have imparted training to other NGOs using resource persons developed by SANDEEP” says Pawan. In order to measure the quality of program delivery, CECI uses some key indicators: a) delivery in time, b) transparency at the user group level at all stages of the project cycle, c) community capacity for transparency and accountability (transfer of these skills from NGO to CBO) and d) community capacity in project monitoring. “SANDEEP partners have fulfilled these criteria during project implementation. They also take CECI’s feedback very positively and act on them.” Pawan has also observed some areas for improvement in SANDEEP partner NGOs: “a) they have not been able to fully implement their policies, and b) advertisements are published during staff selection, but in some cases, we have seen manipulation, and financial settlement is slow in some cases. The NGOs have been capacitated, but due to a lack of adequate programs, they have not had the opportunity to fully implement/ practice their learning. As CECI and SANDEEP programs phase out around the same period, the PNGOs may have no projects.” Therefore, Pawan believes that SANDEEP needs to assist in resource linkages as well. Another important area of improvement is that the EC needs to be further strengthened. “Many donors and stakeholders just want to see the results, but they don’t see the actors and processes from beginning to end. SANDEEP has done a good work in building the foundation for this process. We want to give the following suggestions for the possible II phase of SANDEEP: ™ Provide continuous follow-up on implementation ™ Help develop programs and link them to potential donors ™ Help transfer the systems from PNGO to CBOs (PPME, OM) ™ Establish innovation funds to support those NGOs who have innovative ideas SANDEEP needs to a link rights based approach with OD” says Pawan Karki, Regional coordinator, CECI in Surkhet.” 4.2.5 One of the major capacity building needs of ISOs and PNGOs identified during the baseline was on enhancing .organizational skills on resource mobilization amidst conflict. INGOs in these districts were adopting a position of “wait and see”, forcing some of the NGOs to close their offices 20 and programs. To cope with the situation, most of the project partners felt the need to diversify their funding sources to include government, the private sector and local institutions. To address this need, the PMT developed a training module on resource mobilization and conducted trainings for ISOs and PNGOs in different phases. According to the ISOs and PNGOs consulted, the training was an eye opener. They were able to understand the importance of persuading donors with not only their sincerity but by making innovations in program design and implementation along with greater transparency in their functioning, which improved their organizational image. They also learned that PNGOs should proactively identify and access resources rather than waiting for visits from donors. Another significant achievement of the module was the realization among ISOs and PNGOs that resources can be accessed from local sources, like VDCs, DDCs, and from private individuals in the community in addition to existing funding agencies. The participating organizations now have an understanding that their voluntary contribution of time and effort could be translated into local resources generated from within the organization. The module also helped to identify the resource gaps between the strategies and goals of all project partners. Based on the identified gaps, the resource mobilization plans were prepared and implemented. Case Study – 4 UNESCO Club Banke: Initiates Collaboration with Private Sector The UNESCO Club, located in Nepalgunj of Banke district, is a SANDEEP PNGO supported by the ISO, BASE. This PNGO has recently initiated a joint collaboration with Jaffer Carpet Company in Kathmandu which is providing on the job carpet weaving training to the first batch of 11 persons (6 muslims and 5 kamiayas). The training will then be extended to additional batches. All the trainees are to be brought back to Nepalgunj to work in a joint venture carpet factory to be established in Nepalgunj by some UNESCO EC members and the Kathmandu based Jaffer Carpet Company. An understanding to allocate 25% of the net profit of the factory has been made for UNESCO social activities. The legal agreement for the establishment of the factory is to be formalized after the completion of the training of the first batch towards the end of September 2008 The evaluation found that ISOs and PNGOs were trying to contact donors by short listing them according to their mission using the Resource Directory developed by the project. They diversified their funding portfolio so that it now includes the private sector. A resource management team was formed and started taking responsibility for fundraising. ISOs are taking the initiative to link PNGOs to donors. Some ISOs and PNGOs reported that they have submitted joint proposals thereby taking their partnership to a new level. All the ISOs and PNGOs visited by the FTE team indicated that that they had received new projects enabling them to increase coverage of their overall program. 4.2.6. During the final year of the SANDEEP Project (2007-8), the remaining two modules were implemented, namely, operational management and human resource management. The operational management training was important for sensitizing ISOs and PNGOs leaders on the program cycle, particularly in participatory planning, monitoring and evaluation and the delegation of authority. Since NGOs were implementing programs in partnership with donors, they therefore had less control over program decisions. However, given the fact that there is an increasing demand from communities, government and donors for sustainable impacts of project activities, these NGOs' leaders and staff members need to involve stakeholders, including beneficiary communities, in program management, specifically in planning, monitoring and evaluation and thereby contributing to a greater degree of transparency in their activities. As a result of the training and the implementation of the action plans, ISOs and PNGOs stated they had prepared monthly work plans and developed guidelines for the delegation of authority. This was reported to have improved staff performance, facilitated prompt decisions and led to more efficient service delivery. District based stakeholders (related line agencies, donors, NFN and community leaders) pointed out that SANDEEP ISOs’ and PNGOs coordination with other NGOs, stakeholders, 21 government agencies and communities had improved in recent times which had helped in reducing duplication and overlap. ISO and PNGOs also reported that their donors have acknowledged and begun to accept the recently developed participatory M & E procedure and guidelines. Another significant area of improvement as a result of this training was that the EC members were sensitized and had started visiting communities for monitoring and feedback. Another major impact of the module was the replication of social audits across many NGOs in the respective regions. Some donors said that they had begun to allocate more funds in NGO program budgets to cover the costs of social audits. Appreciating the added value of social audits as demonstrated by SANDEEP, SWC is currently contemplating on making the social audits mandatory in the new NGO Act. Case Study – 5 SIMI-OVC Evaluates SANDEEP Impact Small Irrigation Market Initiative (SIMI) is implementing Other Vulnerable Children (OVC) project in Surkhet with support from USAID. SIMI has also been working with one of the SANDEEP PNGOs over the past year on a program related to child health, education and nutrition. “We have understood that SANDEEP is about building staff capacity, sharing learning and supporting institutional development of NGOs. In a very short period of time, we saw some remarkable results from SANDEEP PNGO. The PNGO is totally different from other NGOs!” says Kabita Shrestha, Project Coordinator of SIMI-OVC in Surkhet District. Due to PNGO’s performance, Surkhet district program was evaluated as the best by SIMI head-office in comparison to similar programs in Banke and Bardiya. Kabita believes that the credit goes to the PNGO and ultimately to the SANDEEP process. The PNGO completed the project on time and provided timely reports. SIMI monitored their activities in the field and found that they have properly used project resources practicing full transparency with user groups. SIMI also provided training to some of their staff and they immediately conducted sharing training for their rest of the staff team in their office. “We have learnt that for improvement in quality of service, two things are important: value chain and capacity of partners!” says Kabita. SIMI has a positive impression of SANDEEP due to the performance of this PNGO. If this PNGO will be able to deliver good results as it has done so far, SIMI will continue its partnership with the PNGO in the future as well. Kabita thinks that in order to improve performance further, PNGO needs some physical support as well. “They need a computer and printer. They don’t have even a bicycle and spend lot of time on traveling by foot. Such kinds of support save time.” says Kabita. 4.2.7. The Human Resource Management (HRM) Module was the sixth and final capacity building intervention of the SANDEEP project. The module covered the overview of human resource management, HR planning, recruitment and selection, HR orientation, performance management systems, performance planning and performance evaluations, performance support systems, HRD and HR empowerment. As the module was recently implemented (Dec. 2007 – Jan. 2008), the ISOs and PNGOs are in the process of implementing the action plans. ISOs and PNGOs, which revised their policies immediately after the governance module, felt the need to further refine their administrative and personnel policies to incorporate new learning from the HRM Module. The senior staff and OCBOs of ISOs suggested that the HRM Module should have been sequenced immediately after the governance and strategic management training. 4.2.8. Changes in ISOs and PNGOs ability Donors, district stakeholders, staff and EC members of the ISOs and PNGOs acknowledged that that there was significant improvement in the overall functioning of the SANDEEP affiliated NGOs. The organizations now have sufficient capacity to conceptualize and develop and review administrative, 22 financial and program policies, systems and guidelines. Employees in collaboration with the EC members were able to develop strategic plans identifying the gaps between the goals of their organization and their operational plans. The ISOs and PNGOs developed and implemented effective and transparent financial systems. As part of the M & E process, these organizations institutionalized social audits for greater transparency and accountability. As part of a long-term sustainability plan these organizations also developed resource persons who conduct OA workshops for their organization and for other NGOs. Overall, at the end of the project, the HR capacity in these ISOs and PNGOs had increased to a point where they can now effectively dialogue with stakeholders (GO, INGO and district stakeholders) for resource sharing and advocate for good governance and management practices within the NGO sector in various district level forums. 4.2.9 ‘System approach’ to service delivery: Out of the five modules developed by SANDEEP, there are two modules that are directly related to a systems approach to service delivery - namely, the strategic management and operational management. The strategic management module gave the ISOs and PNGOs a greater sense of their mission through an increased focus their programs on target groups. The operational management module gave skills to ISOs and PNGOs to better manage the project cycle and PPME. The evaluation found that both ISOs and PNGOs were occupied in developing internal systems and policies and did not have sufficient time to reflect on how to use these systems and policies systematically to improve service delivery. Some improvement in service delivery is observed due to the participation of staff in decision making and the delegation of authority as well as increased participation of community in the project cycle and improved coordination with other stakeholders. 4.2.10 Changes in relations between ISOs and PNGOs There were occasional contacts between ISO and PNGOs before SANDEEP. At the outset, the ISOs and PNGOs were hesitant to talk to each other and share their organizational shortcomings. Facilitated and capacitated by the SANDEEP modules, they are now openly discussing issues and sharing experiences. PNGOs have started viewing ISOs as an ally that helps them in times of need by giving them relevant information and advice. ISOs, for their part, have begun to feel that they have some kind of responsibility in initiating PNGOs’ linkages to donors. Some ISOs and PNGOs have even started submitting joint proposals, which donors are encouraging. Some ISOs and PNGOs have also jointly conducted trainings on modules to other NGOs and networks, which were greatly appreciated. These activities demonstrate the level of capacity development as a result of the SANDEEP initiative. Though the collaborative approach was picking up between the ISOs and PNGOs, there also existed, in some instances, an environment of competition between ISOs and the larger established PNGOs that work within the same district. The evaluation team observed that PNGOs that work in very remote areas had more trust in ISOs as these ISOs are, sometimes, the primary channel for obtaining information and support. It was also noted that the political inclination of the ISOs and PNGOs also determined the strength of their relationships. 4.2.11 Influence on NGO sector and donors The SANDEEP approach appeared more “ecological”5 as all modules and interventions gave top priority to dialogue with stakeholders as a basis for resource sharing and organizational sustainability. The modules on OA, governance, strategic management, resource management and operational management were particularly useful in building the capacity of ISOs and PNGOs to link with other NGOs, networks, donors and government stakeholders. Because of the SANDEEP intervention ISOs and PNGOs conducted social audits with the participation of donors, district stakeholders and 5 Mutual learning and sharing process 23 beneficiary groups. These audits established a framework for stakeholder consultation on one hand and on the other hand provided valuable impact-based feedback for improvement in implementation.6 This pressured other non-SANDEEP NGOs to follow suit and be transparent.7 Donors have also now started emphasizing good governance practices in NGOs.8 Staff and EC members of ISOs and NGOs stated that the SANDEEP modules helped reduce conflict within organizations and helped create a conducive environment for tapping resources from different donors. Therefore, other organizations, seeing positive changes in SANDEEP partners, have demanded coaching from them on modules, especially the governance and policy development ones.9 One general overall impact of SANDEEP modules is that contacts of ISOs and PNGOs with other related NGOs, donors and GO line agencies have increased. For example, they have started working in collaboration with government agencies on issues like polio eradication, school enrollment and child labor (CFWA, SAC, NNSWA, BASE). However, collaboration with the private sector at the ISO and PNGO levels is still nominal because ISOs and PNGOs still need to give more time to establishing internal systems and could not give sufficient effort to dialogue with the private sector. NGO partners have started sharing project learning with other NGOs through FEDEN and NFN. An ID forum has been facilitated by ISOs as a mechanism for sharing among district NGOs (e.g. by ISO NNSWA in Kanchanpur). These forums were organized on a rotating basis by leading district NGOs, including ISOs, on quarterly basis with the goal of sharing SANDEEP learning. However, ISOs have not been able to give continuity to these sharing forums due to political divisions within the district NGO networks in Nepal (NFN and FEDEN) among party lines. Though sharing of SANDEEP experiences and best practices was taking place with other NGOs, sometimes in an organized way, most of it was done spontaneously and on a more personal and individual basis. Diverse attitudes and practices of donors have been assessed as a bottleneck as ISOs and PNGOs have reported that some donors appreciate the practice of dual roles on the ground, which in their opinion makes project implementation faster, while a number of other donors have no trust in the policies and systems of ISOs and PNGOs. They use SANDEEP partners only for social mobilization and purchase project related hardware and equipment by themselves. Donors like Save the Children Norway, CARE, and CECI, however, have shown more interest in SANDEEP learning. Therefore, there is a need to continue the initiative of the PMT to organize interactions between SANDEEP partners and INGOs/ donors through AIN and other forums to sensitize these donors on best practices and lessons learnt. At various district level stakeholders’ meetings, SANDEEP partners have highlighted their best practices in respect to issues on dual roles, the need for second line leadership and ending nepotism in the NGO sector. ISOs reported that these public discussions have helped district NGOs to improve their staff hiring procedures. Collaboration between SANDEEP partners and other organizations have evolved in a multi-facet dimension. SANDEEP partners share their annual reports with VDC, NGOs, DDC and donors. Other NGOs have begun to visit ISOs and PNGOs to either obtain a copy of organizational policies, or to learn about their filing systems or to get some insights into the operational modalities of their 6 One of the SANDEEP PNGOs that is implementing education program for children of poor families received feedback during social audit that the NGO needs to provide livelihood projects together with education support. The community argued that, as the children go to school, the parents have to make additional expenses for clothing and stationery and also lose working hands. Therefore, in order to continue sending the children to school, the NGO needs to support the families IG programs for generating more income, not just the education program. 7 See stakeholder interviews, Annex--8, 8 E.g. SC Japan, SC Norway, PAF, CECI. For more see stakeholder interviews, Annex-8, 9 Stakeholder interviews (Annex-8) 24 programs. They also ask for different formats, copies of constitutions and operational guidelines. Another notable point is that ISOs’ and PNGOs’ participation in networks has increased since SANDEEP interventions were initiated (RHCC, District Development Fund). One of the staff of a SANDEEP partner organization in Darchula highlighted the changed context as “The attitude of stakeholders towards us has changed after the success of our electrification project that gave light to 110 households. We send monthly reports to DDC, and now they have started to call us for meetings. However, they still feel that they are the boss, they still do not see us as partners working hand in hand.” SANDEEP partner organizations conducted joint training with PAF, World vision and CECI partners on governance, strategic planning and policy development, and these entities have replicated these modules in their implementation modality. Training was also conducted with SWC. SWC, which has appreciated the SANDEEP modules, have shown an interest in publishing these modules jointly. There is also the possibility of incorporating the SANDEEP learning and best practices in the upcoming new NGO Act. Sharing of the social audit modules has also taken place within the Save the Children Alliance. Currently, discussions are ongoing on creating an OD cell to institutionalize learning within the alliance. SANDEEP learning is also shared with the SCUS regional office and headquarters, which have exhibited interest in the project outcomes for replication among its country programs. SCUS has already incorporated partner OD as an important strategic focus in their new strategic plans (2007- 12). Similarly, SANDEEP worked with AIN on uniformity of partnership policies. In the course of this association, AIN has also taken note of the good governance practices propagated by SANDEEP and has formed a task force to study SANDEEP learning. The task force has developed a common partnership policy for 80 INGOs affiliated with AIN and has been advocating the AIN members to implement the policy. Drawing on the experience of SANDEEP, the AIN has assisted NFN in the formulation of Good Governance Act of NGOs. The task force is also working with bilateral donors to change their grant policies in line with SANDEEP learning.10 4.2.12 Demand from NGO sector for OD All ISOs and PNGOs reported that the demand for OD support by the NGO sector in the districts and regions has increased significantly. Although the issues of separation of dual roles and nepotism have been a taboo for the NGO sector, many district level NGOs requested ISOs and PNGOs to facilitate training on good governance, resource management, strategic planning and coaching support to develop policies and systems. However, only a few of such demands were actually met by the project partners due to lack of staff time as the ISOs and PNGOs were mostly engaged in developing and practicing their own systems and policies. 4.2.13 Value added by SANDEEP There have been several valuable additions through the SANDEEP interventions. First and foremost, the development of the training modules itself is significant. The support provided to train and coach the PNGOs through ISOs is noteworthy. The project ensured the development of various organizational policies and guidelines.. The project helped open up immense opportunities to the partner organizations to tap into new program areas and funding sources. SANDEEP introduced a very sophisticated concept of OD in a very simplified manner to ensure sustainability of programs 10 Currently bi-lateral donors have a policy that doesn’t require separation of dual roles for grants less than 100,000 USD. 25 and organizations. The project created a pool of local community based experts to undertake such assignments, and this may yet be another milestone that the project has been successful in creating. Other valuable additions can be grouped as influences it has generated to outside organizations and governing institutions. SWC has shown a positive inclination to incorporate the elements related to governance in national policies. Discussions are already underway in and among the NGO/donor community on issues brought to the forefront by SANDEEP, including transparency, accountability, ending dual roles, social auditing etc. Finally, SANDEEP contributed to the overall development of the SC alliance in particular and AIN in general as many of the best practices and lessons learnt are finding space in organizational policies and programs. 4.3 Improvements in the ISOs’ and PNGOs' service delivery: To identify whether there are indications that these changes are resulting in improvements in the ISOs and PNGOs' service delivery The evaluation did not find a direct link between SANDEEP interventions and improvement in service delivery. The evaluation team is of the opinion that more time will be needed to see the tangible impact of SANDEEP intervention on service delivery of NGO to communities. However, there are indirect signs (links) that SANDEEP has definitely contributed in the improvement of service delivery. First and foremost, the ISOs’ and PNGOs programs started taking place in a planned way as many PNGOs did not have periodic action plans before the SANDEEP intervention. Project partners started developing plans and implemented them with the participation of communities and stakeholders. Through the process of OA, ISOs and PNGOs now understand the significance of participation within their organization and therefore are encouraging community participation to ensure community ownership of programs. Second, the ISOs and PNGOs have given high priority to coordination with relevant stakeholders and share information with them for improved coordination and monitoring of projects. They have become more transparent and accountable through social audits and have started incorporating feedback of community and stakeholders to improve programs. Third, ISOs and PNGOs have developed continuous contacts with communities even project completion because they have realized that continuity of support is one of the important indicators for quality of service. The policies and systems developed by ISOs and PNGOs have increased trust by donors resulting in many of them have expanding their coverage in terms of geography and population. It was also observed that ISOs and PNGOs have become more serious in the selection of target groups through community participation. Fourth, some of the ISOs and PNGOs have started transferring some of their management and technical skills to CBOs, including book keeping skills, how to organise meetings, how to develop CBO policies and constitutions, clarity of roles within CBOs, participatory monitoring and reporting, among others. Community demands for projects were followed up with further interactions to clarify issues and ensure community participation and contribution. Apart from regular technical monitoring by staff, EC members also now make periodic monitoring visits to communities and provide feedback to staff for improvement while stakeholders are also involved in monitoring of ISO/PNGO projects. Participation of communities has increased due to inclusion provisions (for dalits, women and other disadvantaged groups) in NGO policies and program guidelines. Community trust has increased due to increased transparency of programs and budgets. Regular review and reflection within ISOs and PNGOs has helped to identify program issues and then address them in time to improve program effectiveness. 26 Due to delegation of responsibilities within the organization, plans are implemented on time and the budgets are effectively used, resulting in a significant reduction in the number of budgets being returned to donors. Inter-sectoral meetings within the ISOs and PNGOs have increased coordination and monitoring of programs. Improved financial systems (obtaining price quotes, providing quality goods to communities) has also helped improve the overall quality of service. The practice of transferring funds from one project to another has stopped, increasing the percentages of work that is completed on time. Owing to increased trust of communities and transparency of NGOs, VDCs, (excluding Project areas in the eastern Terai) have started funding NGO projects. Case Study – 6 DPHO in Dhanusha Reflects on PNGO’s Quality of Service “SANDEEP PNGO CFWA has improved coordination with us in recent years. We have participated in many of their health related activities. They have applied their capacity, knowledge and skills to improve their programs. CFWA is actively participating in district level health networks like RHCC, DACC and DICC. They monitor our immunization programs and give reports to us. CFWA recently developed a musical cassette with songs and drama in local language. This helped to increase our coverage for immunization. It has also conducted social audits due to which stakeholders are informed about their activities. It’s very difficult to pinpoint how much SANDEEP contributed to improvements in service delivery of PNGOs. But there is definitely improvement in the quality of service. DPHO has provided health￾related technical trainings to CFWA and SANDEEP provided management training. As both of these activities contributed to the improvement of program delivery, it’s very difficult to segregate the share of these programs on improving service delivery. CFWA has improved their capacity, now they need projects related to the health sector, which is their organizational mission. There are many issues and opportunities in health related programs, such as awareness on potential bird flu epidemic and reproductive health (RH), in which CFWA could play a leading role. CFWA could collaborate with UNFPA to work on these issues in the future. HIV infection in this area. We have realized that HIV patients also need care similar to DoTs procedure. This is also an opportunity for the CFWA. If SANDEEP can help in linking CFWA to donors, the PNGO definitely could be a leading organization in health sector in the district”. ~ Bijay Kumar Jha, District Public Health Officer, Janakpur, Dhanusha. The evaluation team found that the communities are not in a position to indicate the specific improvements in service delivery as SANDEEP is not a direct service delivery project.11 Some of the indicators of service delivery commonly raised by key stakeholders (targeted communities, ISOs, PNGOs, NGOs, line agencies and donors) are presented in the box below. Consolidated Findings of Indicators of Quality Quality in relation to achieving program goals: • Setting achievable goals and their realization within a scheduled timeframe. (done through strategic and annual planning) • Reviewing and resetting intermediate goals after review (done through periodic program review) 11 School teachers in Suda VDC in Kanchanpur said that the PNGO with whom they work on education project has become highly committed to the needs of the communities. They said: “this NGO is quite different from others, they are much more serious to our needs and concerns and works with the target groups in an impartial way”. 27 • Performing and measuring up to set criteria (partly done through social audit, but this is area for improvement for both donor and NGO) Quality in relation to program impact: • Ability of target groups (community people) to dialogue with wider audiences and decision makers at the district level (more work is required) • Realization by decision and policy makers of the actual reality at the grassroots level. (Policy dialogue with AIN and donors has started, but the NGO sector is still struggling to make a meaningful dialogue with government) • Improved access of target groups (community people) to resources (NGOs need to work more in this indicator, as generally target groups don’t access resources meant for them) • Improvement in livelihood status of target groups (community people) (partly done, NGOs and donors need to work more on PPME) • Ability of target groups (community people) to plan, implement and manage programs that affect their lives. (NGOs need to work more, they have not fully practiced OM module) • Strong institutional structure of community based organizations (How to sustain community organization after the project is still a challenge for ISOs and PNGOs. Need to work more on this) • Positive changes in the attitude of donors towards target group issues and problems. (How do donors identify community needs when their interface with communities is low? A major area of improvement on donor behavior) Quality as based on process: • Work culture that carries the ideology of social justice and not monetary gain (It takes time for NGOs to become fully Mission-oriented) • In-depth understanding of the fundamental issues, context and the intensity of the local problems before actual planning of the program intervention. (In-depth baseline survey) (More inputs needed to make NGOs capable to assess community issues) • Formulation of program plans with intensive and wide scale participation of targeted groups enabling them to identify programs most beneficial to them. (partially through upgraded participatory planning processes but need to work more on this) • Ensuring transparency and accountability. (done through social audits and policies / systems) • Ensuring the participation of target groups (community people) and that they are involved in every phase of the program cycle. (regular community meetings and social audit) • Ensuring quick dissemination of relevant information from the top to the bottom and vice versa. (there is a gap here, strategies and action plans need to take this seriously) • Cost effective approach (This is a complex issue that needs specific and clear indicators but tenders, quotations and timely completion of activities have made ISOs and PNGOs activities more cost effective) • Learning from each other, building on the programmes strengths and correcting weaknesses. (ID forums have not been regular. NGOs need marketing skills to sell OD) • Good and factual documentation of actual experiences (this is being practiced but needs to be upgraded) • Bottom up partnership approach (This is an interesting idea to implement through the OM module, but needs a paradigm shift in the attitude of donors, NGOs and government. • Ensuring micro and macro linkages (multi-stakeholder dialogue at different levels) • Autonomy of the NGO sector (remember the link dependence – independence – interdependence). 4.4 Implementation and Challenges: To assess whether the project's current program approach is appropriate to achieve the project's strategic objective and effectiveness of the interventions within that approach. 4.4.1 Projects approaches and interventions The ISO approach and the intervention modules have been effective in achieving the strategic goals of the project. According to the project strategy, the ISOs received training first and later they provided the same training to PNGOs. This strategy created a condition where the ISOs had to be a role model for PNGOs, for which ISOs had to introduce the changes (walk the talk) before they 28 recommended such changes in PNGOs. The change process was more challenging in ISOs as they already had a deep-set organizational culture and hierarchy whereas PNGOs were at the initial stage of their growth. Therefore, the change process in PNGOs was faster than in ISOs. However, the time gap between the training for ISOs and PNGOs was very small with the result that ISOs did not get sufficient time to digest the modules and then facilitate the PNGOs based on their own experience. Instead, the change process in ISOs and PNGOs occurred simultaneously, and sometimes the PNGOs changed first, putting ISOs under tremendous pressure to change, especially on the issues of nepotism and dual roles. However, many PNGOs think that PMT’s direct support to them might have been effective as it takes time to reach PNGOs through ISOs. This feeling may be due to the intention of PNGOs to have direct contacts with the donor (SCUS) so that these contacts could result in project funding in the future. Other issues raised by PNGOs were that there was not enough institutional sharing between ISOs and PNGOs as there was no provision in the project strategy on inter EC sharing and ISOs mostly depended on the ability and initiative of OCBO/FOCBOs. in sum, most of the PNGOs think that the ISO approach had both positive and negative aspects. Without ISOs, some PNGOs assumed that they could have received more direct funding and training from the PMT, which would have resulted in better comprehension of the modules, but others opined that if trainers came from the national level (instead of ISOs, which were locally based), these trainers might not have understood the local context as the ISO did (knowledge of local language and culture), making it more difficult for them to comprehend the modules. 4.4.2 Contribution of ISO approach to achieve strategic objective Five ISOs have successfully built the capacity of 30 PNGOs. These ISOs and PNGOs have improved their management efficiency through systems and policies. They have become more accountable to the community and more serious in identifying the real target groups. However, the project duration was too short to see real and tangible improvement in the delivery of health, education and livelihoods of the target communities. Therefore, the evaluation concludes that the project has partially achieved the strategic objective. 4.4.3 OD team approach After the training on the governance module, the ISOs and PNGOs formed OD teams with participation of key staff and EC members. ED, OCBO, the president, secretary and one EC member participated in OD team meetings that met once a month to discuss the implementation of action plans on the modules. Both ISOs and PNGOs have realized the importance of continuation of OD teams upon project completion, as it helps to discuss OD issues in-depth and provides recommendations to the EC for timely decisions. However, in PNGOs, as they generally have a small number of staff and too many subcommittees in the beginning, the same individual had to participate in various committees, including the OD team. Therefore, in order to make the OD team more effective, some PNGOs decided to merge the OD team with the resource management sub￾committee so that internal and external resources could be effectively mobilized. It was also observed that OD team members are actively participating in the meetings, which is reflected in the fact that they bring OD agendas for discussion in the meetings. All ISOs and PNGOs have given priority to continue the OD team in the future as reflected in their SANDEEP phase-out Roadmap. 4.4.5 Addressing the variation in the nature and size of PNGOs SANDEEP partners felt that the modules were better suited/ tailored to meet the needs of ISOs than PNGOs as the latter are generally small organizations with a small staff contingent and small sized programs. Therefore, the OCBOs had to modify some of demands of IDF indicators and help the PNGOs make action plans suited to the nature and size of PNGOs. The EC members of some of the 29 PNGOs don’t have much in terms of educational qualification and the ODF and OCBOs had to give extra time to make them understand the contents of the module. 4.4.6 Effectiveness of OCBOs/As, FOCBOs and ODFs In the beginning, the OCBOs/As and ODF did not have much knowledge about OD and were not confident in their ability to facilitate PNGOs through the process. Another challenge was that the OCBOs had to convince ISO’s EC (actually their bosses) to develop action plans and then get true cooperation from senior members of the organization in their implementation. Due to these challenges, some OCBOs left SANDEEP in the early stages. However, constant coaching from CBPOs and PMT staff as well as rehearsal of the modules (ToT) prior to the workshops at ISOs and PNGOs greatly enhanced the knowledge, skills and confidence of OCBOs. Now OCBOs have become confident in their ability to facilitate workshops not only within their own organization, but also with organizations in the districts. The increased capacity of OCBOs and ODF is reflected in the fact that some of them are now working as ED or team leader in their own organization. Those OCBOs who have left the organizations have also continued working as OD professionals with other donors. All ISOs/PNGOs appreciated the critical role of OCBOs/As and FOCBOs in supporting the OD process in their organizations. OCBOs/As and FOCBOs were observed to be highly motivated and took up their roles seriously. ISOs reported that OCBOs/As and FOCBOs have increased their conceptual knowledge, presentation and facilitation skills as well as coordination skills. The CBPOs and FCBPOs provided intensive coaching to this group in developing understanding, clarifying issues and also monitoring progress. After the MTE, CBPOs were placed at ISOs to provide more consultation and advice on the implement the action plans. OCBOs, with multi-dimensional responsibilities, faced more challenges than the FOCBOs whose jobs are more technical and confined to a specific area. Time management appears to be their major challenge as they usually support the ISOs’ regular project works in addition to writing SANDEEP reports and conducting ToT for ODF, mentoring and coaching PNGOs and facilitating SANDEEP’s activities with the ISOs. The ODF also appeared to be involved in all aspects of the PNGO work, including writing proposals. As internalization of SANDEEP’s concepts and modules took some time, OCBOs, FOCBOs and the ODF reported that it was a case of “learning by doing and enhancing skills through experience” for them. 4.4.7 Suggestions for alternative approaches / interventions SANDEEP, implemented as an independent project, was not integrated with specific service delivery programs and project-specific indicators for monitoring ongoing projects were not incorporated. An alternative approach would be tying OD elements to ongoing projects and assessing the impact of OD on the quality of service delivery through OA, review and reflection and monitoring and evaluation processes. OD interventions require close coordination with targeted communities and donors to assess their impact on service delivery so that there is synergy between programs and OD. Thus, there is a need for specific indicators in projects for measuring quality of services and more coordination within different sectors within SC/US and with ISOs and PNGOs program donors. Regarding future capacity building needs/modules/interventions, the ISOs and PNGOs expressed that they need support in resource linkages, proposal development, community research, social mobilization and on some specific technical aspects based on their missions (health, education or livelihoods). Therefore, preparing new modules on grant proposal writing, community research and social mobilization might be useful for ISOs and PNGOs. ISOs and PNGOs also feel that advisory 30 support would also be useful in some technical aspects where both the ISOs and PNGOs lack sufficient expertise on that specific theme (e.g. SNV Ilam capacity building approach).12 4.4.8 Willingness of partners to play ISO role ISOs expressed a commitment to support the PNGOs under their portfolio in the future. However, the evaluation team is of the opinion that the attitude of “Let’s focus on accessing easy money for implementing donor projects instead of entering uncharted waters of OD consultancy” has a high probability of prevailing despite great potential for ISOs to play this role for the NGO sector. However, if opportunities arise for donor funded OD projects, ISOs will readily take them up. 4.4.9 Ability of ISO and PMT to convey the modules Both the ISOs and PNGOs expressed a high level of satisfaction on the delivery of the training modules and coaching they received from PMT and ISOs. They were also highly satisfied with the content of the modules and they believe that if the content is fully implemented, the organisations will become more sustainable. However, ISOs realized that delivering the same module to the PNGOs is very challenging because of the size of the PNGOs. Some PNGOs argued that ISOs got more intensive training (9 days) then PNGOs and the time provided to them was not sufficient. They also felt that ISOs only conveyed the modules without adding much value to the content. 4.4.10 Comment on support chain: PMT-CBPO-OCBO-ODF The project had a core team of three persons, a Project Manager, a Monitoring and Evaluation Officer and an Organization Development Specialist in the Kathmandu office of PMT, who were responsible for overall management and the provision of technical support to ROs for Project implementation, program development, production of modules, and training of CBPOs/FOCBO and OCBOs/FOCBOs through TOTs, The PMT was also responsible for providing coaching and backstopping support and field level monitoring and reporting and developing linkages and coordination with national and international NGOs / INGOs/Donors and SWC/GO. Development of modules was outsourced. The entire process of understanding the SANDEEP hypothesis and the development and internalization of modules took about 1.5 years. The PMT had to then give more time to develop modules, organize research, develop strategies, and monitoring and reporting, resulting in less time being available to link PNGOs with donors at the national level. SCUS's ROs provided coordination and administrative support such as: signing agreements (MOUs) with ISOs, transferring budgets, providing feedback on the effectiveness of the project, supporting training on financial management, establishing finance and store systems and installing financial management software in ISOs. At the RO level, three CBPOs in western region and one in eastern region were responsible for providing training and coaching to OCBOs. After the MTE, one CBPO was placed for each ISO for intensive consultation, coaching and support. One FOCBO was also placed in each RO to provide financial training and coaching to FOCBOs. They also helped the PMT design and deliver trainings to ISOs, and improve regional level coordination. They also provided follow-up, and monitoring and reporting. The CBPOs received technical support from the PMT on concepts/ modules and in solving critical issues raised by ISOs. The EC members of ISOs and PNGOs acknowledged the vital role played by the OCBOs and ODFs in the formulation and revision of policies. The OCBOs and ODFs conveyed that they lack resource tapping skills. There was uncertainty about their role extension after SANDEEP phase-out due to lack of resources. 12 SNV has a capacity building project in Ilam, where they have seconded technical advisors to the partners to support them on specific technical aspects like support to develop programs and grant proposals. 31 4.4.11 Mechanisms for reporting, feedback and decision making The planning matrix, DIMP and the M&E manual were developed as the basis for monitoring project progress and reporting. The M&E manual provided various tools and formats as well as the indicators for monitoring. The IDF tool provided indicators that helped measure changes in ISOs and PNGOs in the key capacity development areas. ISOs and PNGOs received continuous orientation on monitoring and evaluation through the DIMP process. Review and reflection workshops at thr ISO and PNGO levels were found to be effective for assessing project progress and identifying learning. The workshops provided mutual feedback between PMT-ISO-PNGO. Modules were shared with ISOs before finalization, to incorporate their experience and promote ownership. Project related documentations were well maintained by the PMT. The modules were published in both Nepali and English for better comprehension by ISOs, PNGOs, PMT and other stakeholders. However, the modules were not published in adequate quantity to meet the demand from the NGO sector. All the mechanisms for reporting, feedback and documentation helped ISOs and PNGOs to improve their planning and reporting skills. Many donors appreciated this improvement in ISOs and PNGOs. Constant changes in reporting formats created challenges. Uneven understanding of the formats among the PNGOs occasionally led to delays in reporting. The reporting process initiated from the ODF who reported to the OCBOs. The CBOs then reported to CBPOs who in turn reported to PMT. ISOs and PNGOs felt that the planning process was more of a top down process (PMT first made its action plan, then the ISOs and finally the PNGOs) and their demand for activities were not always accepted. 5. USAID PERFORMANCE MEASUREMENT PLAN INDICATORS The USAID Performance Measurement Plan (PMP) Indicators, which have been administered annually since 2004, are based on direct answers to a set of questions by the target group of NGO partners, ISOs and networks identified by SANDEEP. The PMP comprises three indicators, which are internal operations, external relations and financial resources. (The detail analysis and summary of all the indicators generated are available in Annex - 6.) Analysis of PMP Indicators reveals mixed results within the sample NGOs during the project period (2004 – 2008). Though it is difficult to establish a clear trend owing to inconsistent results during the initial three years, the improvement in internal and external operations does appear visible in the final two years of the project period in all the samples except for SAC which recorded a progressive decline in the fourth and fifth years with regards to improvement in external operations indicator. Likewise, a similar result was also apparent with regards to the improvement in financial resources indicator with the initial three years showing inconsistent trends and improvement in the final two years for the entire sample NGOs except TWOF which showed the same result for all the five years. The graphs generated for the three indicators are presented below. 32 Improvement in internal operation 34 32 29 37 30 30 31 33 32 33 34 29 32 34 34 31 38 33 35 31 34 36 36 38 33 35 36 36 37 38 40 37 37 36 33 0 5 10 15 20 25 30 35 40 45 SAC BASE NRC- Kailali NNSWA TCDF TWUC TWOF Assessment Score YR 2004 YR 2005 YR 2006 YR 2007 YR 2008 Improvement in external operation 21 28 16 23 14 25 13 23 25 20 24 17 23 21 26 23 21 18 25 27 22 23 26 22 20 25 27 21 20 29 24 26 27 27 23 0 5 10 15 20 25 30 35 SAC BASE NRC￾Kailali NNSWA TCDF TWUC TWOF Assessment Score YR 2004 YR 2005 YR 2006 YR 2007 YR 2008 Improvement in financial resources 17 19 22 19 16 16 16 17 21 20 20 18 20 16 20 21 21 19 17 17 16 19 20 22 22 17 17 16 22 23 22 23 20 21 16 0 5 10 15 20 25 SAC BASE NRC￾Kailali NNSWA TCDF TWUC TWOF Assessment Score YR 2004 YR 2005 YR 2006 YR 2007 YR 2008 The aggregate of all indicators shows also indicates mixed results. While ISOs show mixed results in first four years with improvement in the final year, PNGOs demonstrate steady improvement from the first year onwards. When the three indicators are further aggregated across all the samples (ISO and PNGOs), the trend of steady improvement from the first year onwards is visible. 33 Consolidated Improvement 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% SAC BASE NRC- Kailali NNSWA TCDF TWUC TWOF Aggregate Acheivement Assessment Score YR 2004 YR 2005 YR 2006 YR 2007 YR 2008 6. ASSESSING THE EFFECTIVENESS OF LINKAGES Other NGOs in SANDEEP districts have realized that developing systems and policies are important to avoid conflict within the organization and to win the trust of donors. They have also understood the importance of separating dual roles, but they believe that this should be given adequate time to materialize as changing ingrained attitudes and human behavior takes time to alter. Advocacy work of ISOs and PNGOs to reduce dual roles, nepotism and increase transparency and similar demands from AIN has put tremendous pressure on other NGOs to improve governance. District based NGOs, having realized that the public image of PNGOs has increased within a short period of time due to SANDEEP support, have been attracted towards these PNGOs to learn about their systems and policies. District NGOs have requested ISOs and PNGOs to provide training on governance, resource mobilization, strategic management and OA. They have also requested ISOs and PNGOs to provide them templates of policies, guidelines and other administrative formats to improve their systems and policies. The NGO Federation of Nepal has also conducted governance workshops for its members in the Eastern and Mid Western regions jointly with ISOs and have developed guidelines for NGOs incorporating SANDEEP learning. After the resource management workshops, the ISOs and PNGOs started identifying local resources and have increased contacts with local stakeholders (VDC, DDC and line agencies). This has led to the increased participation of local stakeholders in the planning and monitoring and coordination of programs implemented by ISOs and PNGOs. ISOs and PNGOs now also submit periodic progress reports and invite the stakeholders to take part in social audits. The ISOs and PNGOs have become role models for the stakeholders who refer them to other NGOs as organizations that are based on policies and systems rather than the individual discretion of NGO leaders. This has led to increased trust of local stakeholders. For example in Darchula, the government handed over a rural electrification project to SANDEEP PNGO because the other NGO could not implement it. The practice of social audits developed by SANDEEP has now been widely replicated by other NGOs and donors. PMT in Kathmandu have been able to sensitize the AIN members on the need for OD of partner NGOs and have formed a task force to implement the common partnership guidelines. Common partnership guidelines have also been developed for the SC alliance with SANDEEP facilitation. The PMT also organized interactions between AIN members and ISOs for the sharing of learning. The SC alliance also organized interfaces between ISOs and other NGOs for mutual learning. The net impact is that there is growing interest among both INGOs, government (SWC) and NGOs on SANDEEP learning. As the ISOs and PNGOs have started developing systems and guidelines, some INGOs have also started reviewing their systems. 34 In short, the SANDEEP project has brought a paradigm shift in INGO - NGO relations. In the past OD of partners was a non-issue for many INGOs but they have now slowly started to realize the importance of OD for partners as well as for their own organizations. INGOs have started to realize that they need to understand the process of OD to improve the quality of service and sustainability of their interventions. 7. IMPROVING CAPACITY TO DELIVER SERVICES IN CONFLICT The SANDEEP project supported ISOs and PNGOs to continue working within the conflict environment of Nepal. The conflict demanded more transparency and accountability from NGOs and the SANDEEP modules provided relevant conceptual and technical skills to become transparent and accountable to communities and the stakeholders. The project provided training on conflict management based on the “Do no harm” principles to the ISOs and PNGOs selected in the first phase, which gave them tools and the confidence to dialogue with stakeholders, including the rebels. In the course of their interaction with the FTE team, ISOs and PNGOs revealed that though they were denied access to communities at the beginning, they were allowed to work in conflict affected areas later on when they shared their fact sheets, budgets, programs, policies and guidelines to the communities and stakeholders. At a later stage, social audits also helped increase the trust of stakeholders of the ISOs and PNGOs and helped to mitigate the conflict. It was also reported that project activities had to be rescheduled due to frequent bandhs (closures), with NGO workers at times intimidated by both sides to the conflict. They were asked to seek permission from the local Maoists leadership to conduct their activities, register their NGOs with the Maoists’ “People’s Government” and pay taxes to them. This reduced their presence in the rural areas and many field offices had to be relocated to the central office though projects were not closed down and implementation continued through local staff. ISOs and PNGOs reported that they coped with these challenges by introducing changes in the way they managed and implemented their projects. They emphasized dialogue with the conflicting parties and clarified their expectations. Meetings were conducted behind closed doors during bandhs. Local staffs, familiar with the local context, were hired. Transparency in every aspect of the project was emphasized while orienting communities about their programs. To promote community ownership, MOUs with beneficiaries groups were signed. NGOs kept a very low profile (such as using bicycles instead of vehicles and not carrying laptops to the field). VDCs near army barracks were generally not selected. NGO solidarity and joint press releases to declare schools and health posts as ‘Zones of Peace’ helped to reduce tensions in communities. The communities were encouraged to be united in the face of intimidation. Sources of project funds were kept a secret and maintenance of neutrality was strictly adhered to. On many occasions, CBO leaders had to come all the way to the district headquarters to attend planning meetings while monitoring responsibilities was primarily with the field staff as the frequency of office staff visits to the field was reduced. ISOs and PNGOs also started to delegate planning and monitoring functions to local clubs and committees. The EC and staff members of the ISOs and PNGOs reported that the timing and the relevancy of the Governance Module helped them face the many challenges during the conflict period. Besides helping them to become transparent; the Module helped them to prepare codes of conduct, communication strategies and an overall strategy for working within the conflict situation. Fact sheets and policy documents were shared with stakeholders. Establishing computer backups and using pen-drives were other coping strategies. Some NGOs organized meetings with the participation of staff, EC, stakeholders and Maoists to share project information, which helped reduce the conflict. The most important learning is that during conflict, NGOs need to centralize their structure and provide information through a “one door” system. Some NGOs issued ID cards for security of staff. Sharing 35 of experiences for working in conflict during the review and reflection workshop also helped the project partners develop strategies more effectively. ISOs and PNGOs learnt that the root cause of conflict were exclusion and marginalization of communities. They have also learnt that conflict can be reduced by ensuring large scale participation of communities in all phases of the project cycle. SANDEEP contributed to the reduction of conflict by building strong civil society organizations that function on improved systems/processes. After the comprehensive peace accord in 2007, national level political conflict subsided in many parts of the country. However, conflicts still prevail in the terai region while extortion, intimidation and kidnapping are still a common occurrence. Despite these challenges, ISOs and PNGOs have been continuing their operations by being accountable to the communities and transparent to the stakeholders. 8. SUSTAINABILITY The SNSN project proposal envisaged that over time the key to NGO sector sustainability would be learning, adaptability and performance. It was believed that the Project’s initiatives would increase the NGOs’ sustainability and impact as they would develop a reputation for technical expertise that would attract support from government, external agencies, and the private sector that are interested in a particular sector or sub-sector. Some NGOs were expected to graduate to ISO status during the project period. Collaborating with other agencies and linking with networks was considered as one of the major ways in which the grassroots experience and best practices of SNSN Program would be shared. All the ISOs/PNGOs visited by the FTE appreciated and acknowledged the significance and benefits derived from the practical application of SANDEEP’s intervention. Realizing the need to sustain the process, they expressed a firm commitment and enthusiasm for continued application of the established OD policies, systems and processes. However, as this is contingent on availability of projects, smaller PNGOs were of the opinion that it would be difficult to sustain SANDEEP’s learning and experience as they face a lot of difficulty in getting new projects owing to inadequate linkages with donors and nominal access to local resources, especially the private sector. Smaller PNGOs expressed that they have not been able to adequately practice many of the policies and systems due to the short time span. They stressed on need for follow up support to reinforce new practice into habits. . As SANDEEP has been streamlined in the recently amended partnership and compliance guidelines of SCUS, it has benefited both SCUS and its partners. For SCUS, partnership will now be based on firm commitments of its partners towards good governance systems and practices in alignment with SANDEEP’s values. This can also be expected to contribute to improvement in planning, implementation and monitoring of projects supported by SCUS. Further, SCUS now has a country wide network of capacitated district based NGOs it can rely on to implement its programs and effectively serve its targeted communities. For the partners, SANDEEP’s learning and sharing processes, both with and without SCUS, along with advocacy efforts targeting AIN members, provide them opportunities to enhance their governance systems and processes. A more positive and favorable attitude towards OD components in project funding has been created in SCUS and the SC alliance in particular and AIN members in general, though this may take some time to produce tangible results. In the long run, partners who have VMGO focused approaches and have developed good governance systems, improved the capacity of their human resource systems and processes, adopted transparent and accountable financial systems and practices and strived for enhanced linkages will be in a more advantageous position to compete for donor funded projects and mobilize local resources. The sustainability of SANDEEP is also supported by the internal review and implementation system initiated and established by the project. ISOs and the PNGOs demonstrated strong commitment and 36 practices for the continuation of OD Team along with Organizational Assessment and Strategic Planning and Management Processes. A road map was also prepared and implemented to sustain project initiatives. The ISOs were confident that they would retain the OCBOs and FOCBOs in a suitable position in their organization despite financial constraints. The scenario was different in the case of PNGOs despite the commitment to, and realization of, the benefit of retaining the ODFs as financial constraints make this primarily contingent on the receipt of new projects. However, sustainability should not be judged in terms of retention of the OCBOs, FOCBOs and ODF. What is important is that the OD Team and OA and Strategic Planning processes are being institutionalized in the SANDEEP network of NGOs. Further, a pool of OD-based human resource, who are confident enough to continue independently with OD or a career in a related field and are in demand in the NGO sector, have been developed to carry on the SANDEEP learning and experiences. SANDEEP emphasized sharing its learning and experience with key stakeholders at the district and national levels, which among others includes members of district based chapters of the NGO Federation, donors, AIN members and the Social Welfare Council (SWC). The SWC, which collaborated with SANDEEP in disseminating the Governance Module to NGOs through a training program, can be a potential partner in imparting SANDEEP’s modules to a wider audience at the national level. Besides sharing of its learning and experience among AIN members, SANDEEP has also contributed to the development of AIN’s INGO - NGO Partnership Guidelines and is working with AIN to implement this by building the capacity of NGOs. These collaborative efforts are strong advocacy measures that can produce wider results for sustainability of SANDEEP’s initiatives as they emphasize the need for Government, INGOs and donors to take more responsibility for motivating and supporting NGOs to improve governance and management systems. Furthermore, to mitigate the effects of the conflict, AIN and bilateral donors are stressing the achievement of high standards of accountability and transparency. As this is contingent on NGOs improving their management and governance systems, intensive OD focused interventions could gain in significance. The OD team of ISOs and PNGOs has strong support and involvement of EC members. Knowledge transfers from SANDEEP staff to senior management staff appear more effective than to EC members as many EC members, especially new ones inducted to accommodate mandatory SANDEEP provisions (inclusion and avoidance of dual role) still lack OD knowledge and appear to be placed in the EC to complete formalities. Another area of concern is the ability and willingness in terms of availability of time, resources and commitment by EC members who have to work voluntarily without direct financial benefit. Though the approach of "facilitating organizational capacity building of PNGOs by ISOs" is appreciated, the sharing and interaction between the ISOs and PNGOs at the organization level is primarily limited to the PNGO and ISO SANDEEP team, informal sharing over the telephone, attendance of common meetings/forums and submission of a few joint proposals to donors that have not materialized. The initiation of networks such as internal discussion for a (IDF) Fora (by NNSWA) and SDN (by BASE) for sharing/learning was a move in the right direction for sustaining SANDEEP interventions. However, these have lately become inactive owing to a lack of initiative by the ISOs and supportive inputs by the project. Institutionalization of the SANDEEP network of NGOs is yet to be seen. The phase out of the project has been initiated through the road maps developed by all ISOs and PNGOs. Sustainability of SANDEEP initiatives will primarily depend on the level of change in the attitude of NGO leaders to positively adapt and accept changes in their organization, which the project has been able to achieve to an appreciable extent. It will also depend on funding awards being made to the NGOs. The major concern in this regard affects smaller PNGOs who worry that they may not be able to convince donors to support them despite their efforts. More changes were apparent in PNGOs than in ISOs. Among ISOs too, new partners showed more commitment towards institutionalization of SANDEEP in comparison to SCUS’s traditional partners. 37 Thus, changing the partnership approach to induct new partners and focusing on alliances of NGOs rather than on continuing partnership with individual traditional isolated partners could be a move in the right direction. 9. ADDRESSING UNINTENDED CONSEQUENSES During the evaluation, the FTE team came across some specific outcomes/consequences that were unintended as a result of the project intervention. The project’s approved budget for the fifth year was significantly reduced by USAID which affected the planned activities related to the HRM and OM modules. During their interaction with the FTE team, ISO and PNGOs reported that much of the project timeframe was taken up by the development of the IDF tool with the five Training Modules and administration of the first three modules (especially the Good Governance Module). As a consequence, they felt that administration of HRM and OM modules was rather rushed and fell short of the required level of inputs, which would have further contributed to the effectiveness of these modules. Some of the smaller PNGOs felt assured that just being SANDEEP’s beneficiary would by itself open up linkages to donor agencies and give them an immediate competitive edge over other NGOs. However, in reality many donors still rely on their network of established partners and give limited priority to induct new partners. In such an environment, smaller SANDEEP PNGOs in remote districts can be de-motivated as they have experienced losing out to other NGOs they consider less endowed in terms of SANDEEP’s IDF indicators. In a majority of the ISOs and many PNGOs, the separation of roles (doing away with dual roles of EC members) met stiff resistance and changing the mindset of EC members (primarily founding chairpersons and active EC members) became the most challenging proposition for SANDEEP. Thus, many fresh EC members without adequate experience and qualifications were inducted as EC members in many ISO/PNGOs while active EC leaders became senior staff members to accommodate SANDEEP’s requirements. New EC members, a majority of whom have not been exposed to the project’s inputs, are primarily passive contributors while the control of the previous leadership (especially in most ISOs and larger PNGOs) remains intact. However, what is positive is that in recent times even in organizations where a single person’s control was visible and apparent, there has been slow but noticeable shift towards development of second line of command and delegation of authority. 10. CONCLUSIONS 10.1 Hypothesis and theory The capacity of NGOs has been enhanced in terms of their ability to work within the conflict situation. There is a positive correlation between NGOs’ capacity enhancement to "improvements in the delivery of health, education and economic services". It is difficult to establish the extent of its influence on service delivery as it is one among several that influences service delivery in terms of quality and coverage. Though the extent of the project’s links to "mitigate the effects of conflict and poverty" cannot be ascertained, project inputs, besides contributing towards enhancing ISOs and PNGOs capacity to work within the conflict situation, helped to mitigate the effects of the conflict at the micro level by emphasizing transparency, accountability and participation of community people in planning, implementation and monitoring of projects. 10.2 Achievements: The implementation of project activities has largely followed the plan though many of the activities were cancelled and staff out placed due to a sudden reduction in project budget from USAID in the last year of the project period. Precise measurement of project contributions at the strategic level 38 was difficult as indicators had not been set to assess improvement in the quality of services in the health, education and economic opportunities sectors. Though no direct link between OD of NGOs and quality of delivery has been observed, there are indirect indications which show that SANDEEP interventions have contributed to the improvement in service delivery capacity of ISOs and PNGOs. The project partners have become more transparent to stakeholders (through their policies, systems and social audits). They have started planning interventions with community input (action plans are prepared before implementation) and community participation has increased in project development and review (social audit). Likewise, owing to improvements in the monitoring system (EC visiting communities, regular review meeting with staff and EC members), program related problems and issues have been addressed in a timely manner. Utilization of project budgets has improved while coverage has also increased. ISO and NGO relationships with communities and CBOs are generally project guided and primarily dependant on the availability of donor funded projects. CBO continuity after project phase-out is vulnerable as ISOs and PNGOs have not been able to adequately transfer SANDEEP’s learning to the community level. ISOs and PNGOs (especially the smaller ones) are compelled to accept diverse sectoral projects as coping strategies to retain their staff despite focused VMGO and an inclination towards specialization with a focused sectoral approach. ISOs and PNGOs have not been able to mobilize adequate resources to fulfill community needs. In many instances, donors have not supported system based NGOs that follow participatory planning processes. Significant progress has been made to meet the IR level (planning matrix) targets. Both ISOs and PNGOs have separated roles of EC and Staff, administrative, financial and program policies and guidelines have been established, project partners now have a clear sense of their mission with clear VMGO and strategies, organizational structure is clear with clarity in roles and responsibilities through JDs. This has led to the EC being active in policy making and oversight. The established systems and policies have also led to delegation of authority which has made decision making more efficient (timely). Participation of staff has increased in decision making process due to regular sectoral and inter-sectoral review meetings. Similarly, ISOs and PNGOs have been able to increase coordination with and participation of district level stakeholders in their projects’ activities. ISOs and PNGOs are actively participating in district level networks and are accessing resources that benefit the targeted communities. The practice of social audits by ISOs and PNGOs has increased transparency and accountability and this has led to increases trust from project partners. Due to this, many ISOs and PNGOs have been able to tap donor resources to expand their program coverage. However, linkages with the private sector at the district level are nominal. ISOs have expanded their linkages with donors at the national level but PNGOs have limited national level linkages resulting in difficulty accessing donor resources. Due to a lack of projects, some PNGOs are also unable to practice various policies and systems. Professional capacity as related to HR has improved. Establishment of fair, transparent and practical policies and systems to better manage available HR has created a higher level of confidence in staff and EC members (particularly in terms of organizational support and impartiality) motivating them to improve their performance. There have been several value additions through the SANDEEP interventions. First and foremost, the development of the training modules itself is significant. The project has also created a pool of local community based experts to undertake such assignments, and this may yet be another milestone that the project has been successful in creating. Discussions are already underway in and among the NGO/Donor community on issues brought to the forefront by SANDEEP—the issues of transparency, accountability, ending dual roles, and social auditing, etc. Due to the interest of other funding agencies, the project has also started to share best practices and assist in replicating the work elsewhere. Finally, SANDEEP has contributed to the overall development of the SC Alliance in particular and AIN in general as many of the best practices and lessons learnt are finding space in 39 these organizational policies and programs. Although the issues of separation of dual roles and nepotism have been a taboo for the NGO sector, many district level NGOs are interested on availing training programs on good governance, resource management, strategic planning and coaching support to develop policies and systems. Changes were more visible in PNGOs which started afresh. These originations were more receptive to SANDEEP’s inputs owing to the absence of entrenched policies and systems. In the case of ISOs, which had established hierarchies and mindsets combined with underdeveloped systems and policies, it took a longer time to achieve the required level of changes. 10.2 Program approach and interventions: Given the number of ISOs and PNGOs and the extent of project coverage, the project approach (working through ISOs and interventions through modular training) was effective. The ISOs and PNGOs were satisfied with the level of facilitation and support from the PMT and ISOs, while the contents of the modules were highly appreciated. After the MTE, the project provided intensive coaching and advice to ISOs and PNGOs by using CBPOs, OCBO/As at the ISOs and PNGOs which provided timely support to partners to implement the action plans they had developed. Institutional dialogue between ISOs and PNGOs was found to be nominal and the ISOs were mostly dependent on OCBO/As and FOCBOs for SANDEEP’s project activities. OD teams have been established and are functioning effectively, which has facilitated in-depth discussions on OD issues, which has supported ECs in making timely decisions. The DIMP and review and reflection processes have also been effective for monitoring project progress and developing action plans. Social audits have been widely used by ISOs and PNGOs. Much of the project timeframe was taken up by the development of the IDF tool with the five training modules and administration of the first four modules (especially the Good Governance Module). As a consequence, the administration of the HRM module, implemented in the last year, was rather rushed and fell short of the required level of inputs for maximum module effectiveness. As it took some time for the designated ISO staff to understand and internalize the SANDEEP concept and modules, providing training immediately after the completion of their ToT was a challenge initially. Due to this, ISOs could not add much value and just conveyed the module to the PNGOs 10.3 PMT management systems and processes: The conceptual and technical support/coaching from the PMT to CBPOs and from CBPOs to ISOs was adequate. ISOs and PNGOs participated in project planning and monitoring (bi annual and annual reviews and reflection workshops). Adequate monitoring systems were established with formats and reporting requirements at different levels. However, assessments of the contributions of the output level changes to the strategic objective were left to the donors without any assessment by the project itself. The project did not incorporate any internal mechanisms to monitor the quality of service delivery at the field level through specific indicators, but relied entirely on donors’ feedback for judging improvements in service delivery. 10.4 Sustainability: ISOs and PNGOs are already benefiting from the implementation of their new policies and systems (increased participation, reduced workloads, increased delegation and efficiency in decision making, increased trust from communities and stakeholders). OD teams have become fully functional and have developed a roadmap for sustaining the SANDEEP processes and changes (such as commitment to continue OA, social audits, reviews of policies and systems, coordination with stakeholders and strategies reviews). EC members and staff are empowered owing to clear roles and responsibilities 40 and established policies and systems. These changes, along with SANDEEP partners image as system guided NGOs, will make it easy to sustain the institutional changes initiated by SANDEEP. Further, sharing of SANDEEP learning with AIN and SWC has created a conducive environment for the replication of SANDEEP learning throughout the NGO sector. The major sustainability issues are that capable and educated EC members became staff to accommodate SANDEEP’s requirements. The new EC members who replaced the original ones have not been sufficiently trained and are unable to provide sufficient guidance and oversight. This creates a situation where staff will be dominant over the EC for some time to come. As PNGOs need projects to practice their learning, the major concern now is how they can tap into new donors and local resources. Retaining OCBOs and ODF by ISOs and PNGOs can also pose a problem due to financial constraints in the absence of donor funded projects. Another major concern regarding sustainability and replicating SANDEEP’s learning is: Will the ISOs be willing to perform their role as OD facilitators or will they focus more on project implementation? Thus, much will depend on supporting follow-up activities to ensure that SANDEEP’s learning is institutionalized. 11. RECOMMENDATIONS The recommendations for SANDEEP/ SCUS to consider are as follows: 1. There is a need for continued support to the ISOs and the PNGOs in terms of information, follow-up and linkages after the project phase-out. It should be ensured that this support is provided for a period of at least two years to make sure that SANDEEP learning is institutionalized by the ISOs and the PNGOs. In this regard, SCUS regional offices need to take up ownership to carry forward the SANDEEP initiative and also coordinate with donors at the regional level to improve linkages of PNGOs with donors. The FTE suggests that support be in the form of: i) biannual reviews and reflection sessions and reporting ii) follow up and monitoring of the Road Map iii) linkages to SC Alliance and AIN members and iv) provision of adequate annual budget support for the implementation of these activities through the respective ISOs. 2. There is high demand for CB focused projects from stakeholders and NGOs. All the 100+ INGOs and bilateral/multilateral development agencies need capable partners. Thus, there is a need to develop and implement a Second Phase SANDEEP project, in collaboration with the SC Alliance, AIN and the SWC, as there is more to be done in this field. 3. There is a need for extensive dissemination of SANDEEP learning and experience. This should be done through appropriate sharing forums targeting donors, NGO networks, representatives of relevant government agencies, private sector and OD experts at the national level in Kathmandu and all the regional level headquarters. The SANDEEP OD Modules should be published and made available for distribution at subsidized rates through the NGO Federation and/or other appropriate channels. 4. Advocacy for policy change in INGOs and bilateral and multilateral development in favor of support to OD processes in civil society organizations and funding specific OD focused projects needs to be formulated and implemented. 5. Advocacy for OD initiatives can only succeed when it can show tangible results in terms of enhanced service delivery and tangible benefits at the targeted community level. In this context, there is a need to support SANDEEP initiatives with actual project implementation and closely monitor service delivery through specific criteria and indicators. As many SANDEEP partners are already working in partnership within the SC Alliance, the Alliance could be lobbied for a joint initiative to specifically monitor service delivery in projects implemented by SANDEEP partners for a period of three years. The findings could be used to support advocacy efforts in the future. 41 6. AIN members demonstrating strong OD support and the SC Alliance can be consulted and lobbied for full or partial course support on SANDEEP to train their partners for a period ranging from one to two years. 7. There is a need to supplement the SANDEEP Modules with sections on social mobilization and proposal development, which would fit into OM and RM modules respectively. 12. LESSONS LEARNED AND CHALLENGES 12.1 LESSONS • ISO and module approach: For a dedicated OD project like SANDEEP, where there are diverse partners in terms of size, ethnicity and cultural affiliation and geographic locations, working through the locally based ISOs is the right approach. Similarly, identifying the common needs of the NGO partners and developing modules to address these needs is more cost effective rather than trying to address the needs of these partners separately. • Time commitment: The process of OD is about changing individual and collective attitudes and behavior of people in the organization, and this takes time. As OD is an “inside￾out” process of change, it takes time and lot of patience to identify organizational strengths and weaknesses, develop and practice new systems and policies to address organizational issues. The OD facilitator has to be emphatic (has to be “on the shoes” of the NGOs) and understand the organizational boundaries and context and coach the leaders for change, which is time consuming. Creating consensus for change through participation also takes time. • Addressing diversity: Though common needs of NGOs are addressed through the modules, the specific needs of each NGO needs to be addressed through intensive coaching. Therefore, modules must be implemented in conjunction with intensive coaching. • Leadership commitment: OD is about changing organizational culture to make the organizations democratic, pro-poor and sustainable. As the leaders of organizations set the tone of the organizational culture, changing the attitudes of those in leadership positions is extremely important in driving forward organizational change. • Focused OD Interventions o Focused OD support, which is rarely recognized by most donors and not being advocated for by NGO networks, is essential to equip NGO leaders to manage and improve organizational systems and polices. o Without strong local organizations sustainable development remains unachievable and without OD organizations themselves will not be sustainable. 12.2 CHALLENGES • Institutionalization of changes: The ISOs and especially the PNGOs do not have sufficient time and resources to implement the systems and policies they have developed during the project period and institutionalize the changes fully. Another challenge for institutionalization is retaining the staff of ISOs and PNGOs (OCBO, FOCBO, ODF, Accountants) who have facilitated the organizations on organizational change. Trained OD personnel are in high demand by INGOs and donors and ISOs and PNGOs face the threat of losing these talented staff to larger organizations (who can offer better financial and career development opportunities) despite the effort and environment created by the leadership of 42 these organizations to retain these staff. Finally, another challenge for institutionalization involves motivating the EC members who have to give time voluntarily to guide the NGOs. • Marketing SANDEEP to Donors/INGOs: Donors and INGOs have diverse policies, practices and priorities. Marketing the SANDEEP learning to AIN and donors through the implementation of AIN’s partnership policy guideline is the immediate challenge to replicate and sustain the benefits of the project. • Collaboration with the private sector: Collaboration between NGOs and the private sector is still “uncharted territory” in the development context of Nepal. ISOs and PNGOs must work towards creating trust and common understanding with the private sector for mutual benefit. The private sector could learn social responsibility from NGOs and NGOs could learn (social) entrepreneurship skills from the private sector. • Measuring quality of service: Quality of service is affected by too many factors and these factors could range from the local to global levels. Therefore, it is a challenge for ISOs and PNGOs to develop participatory mechanisms to measure changes in the quality of service and identify issues for program development. 43 ANNEXES 44 ANNEX - 1 Terms of Reference For The Final Evaluation of Strengthening the NGO Sector in Nepal (SNSN) Project BACKGROUND SANDEEP (Sanstaghat Diego Pariwartan) project is a USAID funded program which is implemented by Save the Children US. The project aims to strengthen the capability of local NGOs to improve the delivery of health, education, and economic opportunity services to conflict affected Mid and Far Western Regions of Nepal and strengthen the capacity of the NGO sector in the region. SANDEEP envisions accomplishing this by achieving results in three key areas: 4. Strengthen the operational and financial capabilities of individual PNGOs and ISOs; 5. Expand the linkages among and/or between PNGOs and government and private sector institutions; 6. Enhance the program learning and sharing among and/or between NGOs, government and private sector intuitions. The project's philosophy is based on the hypothesis that strengthening the capability of the organizations in the above ways is necessary if NGOs in Nepal are to increase their effectiveness in program delivery, and therefore, achieve desired impacts in the communities they serve. MAP OF WORKING DISTRICTS GEOGRAPHIC AREA AND COMPOSITION OF SANDEEP NGOS SANDEEP is a five year project that was initiated in October 2003. The project works mainly in Mid, Far Western and parts of Central and Eastern Regions of Nepal. The project covers 17 districts and 45 works with 35 NGOs (Initially 40 NGOs) in total. The NGOs are divided into 5 ISOs and 30 (initially 35) PNGOs (see Annex 1. for a list of PNGOs). PROGRAMMATIC STRUCTURE The program model provides support through the Program Management Team (PMT) of SANDEEP to 5 Intermediary Support Organizations (ISOs) which in turn support 30 local district PNGOs to strength their internal organizational capacities so that they can improve services to target communities. The ISOs are developed as organizational development and capacity building support organizations by PMT, applying their expertise both with PNGOs and within their own organizations. Program Mgt Team NGO & (Sectoral) Networks Community members, especially women, children & other disadvantaged people, in conflict affected areas CBO Networks Local Government Agencies & Local Institutions 35 Partner NGOs (PNGOs) in 17 districts 5 Intermediary Support Organisations (ISO) Program Outreach to NGOs and Communities PROGRAM APPROACH The SANDEEP program started in November 2003. Intermediary Support Organizations (ISOs) were selected in March 2004, and an initial baseline cum needs assessment was conducted in April 2004. In May 2004 a Detailed Implementation and Monitoring Plan (DIMP) workshop was held with 5 ISOs, and from this a draft DIMP was prepared and finalized in September 2004. At the same time the selection of 35 partner NGOs was initiated. The DIMP followed an "organizational development" issue based approach where by the organizational assessment tool was the basis for NGOs to assess their organizations and formulate activity plans to address their issues and build on their strengths. This approach was constrained by the NGOs lack of knowledge of how to address the issues they identified, and by the need of the project to have consistent activities across the NGOs to ensure common indicators in the planning matrix are addressed, and to ensure that the project management team (PMT) can manage the range of intervention activities required. During FY 2005, the PMT with ISOs revised the approach and identified a new tool, Institutional Development Framework (IDF), to more effectively help NGOs assess where they are now and where they want to reach. Based on the tool and the assessment of NGOs a set of major interventions were identified using a training approach and the development of modules was initiated. These modules are: Governance and Leadership (including strategic planning and management); Operational Management; Human Resource Management; Financial Management; and Resource Mobilization. Since August 2005 SANDEEP has been implementing 'project-set 46 interventions' to all PNGOs and ISOs that have not reached "capability" in the five organizational categories as defined by the project. In addition to the internal strengthening of NGOs, the project is addressing the NGOs linkages with other stakeholders, and the sharing of learning about organizational strengthening. The development of linkages is integrated with the Resource Mobilization module. The project uses a standard set of modules, to facilitate each training. First, the PMT staff is oriented in the modules, and then the PMT regional staff will give a TOT training to the ISOs’ SANDEEP staff - the OCBOs and the OCBAs. Next, the ISOs staff will conduct the training within each of their ISOs, and finally to their respective PNGOs. Once the training has been delivered to the ISOs and PNGOs, they are asked to build an internal capacity building plan drawing on the knowledge gained form the module trainings. Though this is a tighter approach than a traditional OD approach this new approach ensures that the PMT, the OCBOs, OCBAs, and the PNGOs are all working within the same conceptual framework. This makes it easier for the project to coach the NGOs to implement their activities, and to monitor the progress of outputs and outcomes of those activities, as summarized in the project's common planning matrix. PURPOSE AND OBJECTIVES OF EVALUATION As SANDEEP is in its last year of program implementation, the project is required to conduct a Final Evaluation facilitated by external consultants. The project is seeking consultants to conduct SANDEEP's Final Evaluation, which will require 43 days of work – March to August 2008. Purpose: The purpose of the Final Evaluation is to assess whether the program achieved the three intermediary objectives which lead to the strategic objectives, and what are the success and failure of the project along with the project's influence in NGOs Capacity Building. In addition to that the evaluation will find out the evidence based impact of capacity building on service delivery, evaluate how the project mitigates the conflict through capacity building interventions and make recommendations to the development community on the value of conducting and supporting OD amongst NGOs in Nepal. This information is required by the participating PNGOs and ISOs, by the donor USAID, by the Program Management Team (PMT) of Save the Children, and by other stakeholders, in particular the Government of Nepal and PNGO donors. They want to know the achievements, lessons and effective practices of the program, and how these might be further strengthened. This information will enable the PNGOs, ISOs and PMT to make program adjustments in consultation with the USAID and to inform other stakeholders of how other NGOs' capacities can be enhanced. Some of the information may require further investigation to ensure adjustments are well designed and targeted. Objectives: The key objectives and expected related answers of the Final Evaluation are: 12. To evaluate whether the project achieved its Planning Matrix including PMP and whether the project achieved its strategic objectives. 13. To determine 'if' and 'why' the project's objectives were achieved fully, partially or not at all 14. To evaluate net impact (net change between the baseline, midterm, and end of project) regarding the status of project interventions, capacity of NGOs to deliver development services in the areas involved and expanded linkage among NGOs, networks and public and private sector organization 15. To identify whether there are indications that these changes are resulting in improvements in the ISOs and PNGOs' service delivery 47 16. To identify what effect and impact that the project has had on the other stakeholders (other NGOs, NGO networks, donors, AIN and other government organizations which were not direct beneficiaries of the project) 17. To identify how the conflict situation in Nepal is affecting ISOs and PNGOs, and whether the project responded appropriately to mitigate these effects. 18. To evaluate the viability of the sustainability of capacities developed within SC/US and Partners 19. To assess the unintended result of the project 20. To identify the challenges/lesson learned during the project period 21. To identify specific OD issues and needs of PNGOs in the health, education and livelihood sectors and make appropriate recommendations for replication and improvement of prevailing OD practices applied by SANDEEP as well as other donor agencies for effective service delivery by their respective partner organizations working in these sectors. 22. To recommend specific advocacy measures targeting potential donor agencies who can support SANDEEP’s second phase design and implementation EXPECTED DELIVERABLES OF THE EVALUATION: At the end of the final evaluation SANDEEP projects expected outputs are: ¾ Compilation of questionnaire responses ¾ Separate PNGO, ISO, PMT(SC/US), donor and other stakeholder level analysis and recommendations ¾ Preliminary finding presentation ¾ A draft and final Evaluation report (hard copy and soft copy into a CD) RECOMMENDED METHODOLOGY: Since much of this evaluation will be conducted to evaluate the net impact so that the SANDEEP project is positioned to achieve its strategic and immediate objectives, evaluators should use various methods to assess the different aspects of the project to see all the objectives outlined above. Though the evaluators have full leeway to design and use methods they feel are appropriate at different stages of the assessment, the evaluation should be participatory in its design and implementation. The evaluation team should choose purposive sample of ISOs and PNGOs for the evaluation: The following methods for the assessment are highly recommended: 1) Literature Review: The evaluator should review all major program documents such as the: The project RFP; The SNSN Proposal; Detailed Implementation and Performance Plan (submitted to USAID June 2004); Baseline Report 2004; Annual Reports; Baseline Report 2005; DIMP 2005; Mid Term Evaluation Report; All Monitoring and Evaluation Reports; the Organizational Assessment Packet; and Overview History of Project; The illustrative Questions on approached to OD used in Conflict Vs. Non Conflict situations; Reporting Guidelines; etc. to achieve the objective of the evaluation, Organization Capacity Building Research Report. 2) Observation and Field Visits to sample ISOs, PNGOs and Beneficiaries: The evaluation should include a visit to a sample of the project's PNGOs, ISOs and their beneficiaries while the ISO's OCBOs (Organizational Capacity Building Officers), and OCBAs (Organizational Capacity Building Assistants) and the project's CBPOs (Capacity Building Program Officers) are conducting Organizational Assessments for the Final Evaluation. The sample selection of PNGOs should take into consideration the following variables: • Location of PNGO (whether it is accessible location or a more remote and heavily conflict affected location, type of service delivery in geographic locations – Hill and Terai) • Composition of PNGO board and staff by ethnicity and gender • Size of the PNGO (mainly determined by the staff number) • Number of years the PNGO has been running the organization 48 • ISOs or PNGOs selected for measuring PMP 3) Checklists or Questionnaire: Checklists or questionnaires be developed to ensure there is consistency in the collection of information 4) Focus Group Discussions: The evaluation should include focus group discussions with a sample of the ISOs and PNGOs: • Board Members, Senior Staffs and key field staffs • OCBOs, OCBAs and FOCBOs • Target Beneficiaries 5) Individual Interviews using Checklist or Questionnaire: The evaluation should include individual checklist or questionnaire to interview with: • Target Beneficiaries • Chairperson or president • Executive Director or Chief of Staff • PNGOs OD Facilitators • Project's PMT CPBOs and RPTL • Project's PMT at Katmandu • Other stakeholders (including USAID, PNGO donors, district/regional NFN) 6) Review of ISO and PNGO Organizational Assessments using IDF: During the evaluation ISOs and PNGOs will conduct organizational assessments with the PMT to assess what changes are taking place in their organizations and service delivery. This information will be used by the Final Evaluation to help assess changes and progress across all ISOs and PNGOs. All questionnaires and methods should be accepted by the SANDEEP PMT, especially the Monitoring and Evaluations Officer, before conducting them. Analysis: the information collected will be analyzed by the Evaluation Team to identify correlations and establish what are the major trends and issues. The basic unit of data will be each PNGO and ISO. Data will be disaggregated by gender, caste and ethnicity to identify how program inputs are benefiting disadvantaged and advantaged groups. The data collected will be both quantitative and qualitative. The major quantitative data will come from questionnaire survey by evaluator and Organizational Assessment, which will be collected by evaluator and the M & E officer respectively, who will also support the team in its processing of the data. COMPOSITION OF EVALUATION TEAM The Evaluation Team will be composed of the following team members: • an external consultant, who will be team leader, that specializes in Organizational Development, and evaluations of OD and service delivery projects. • an external consultant that specializes in Organizational Development and experienced on evaluation of project to support team leader The local USAID Mission, SC/US home office and the Government of Nepal's Social Welfare Council will be approached to see if they wish to appoint a team member from their organizations – either full time or part time. MANAGEMENT OF EVALUATION The evaluation team will report to SANDEEP's Monitoring and Evaluation Officer (M&EO) in Kathmandu. The M&EO will be designated as the key contact person for the final evaluation, he/she 49 will make all decision regarding the final evaluation in close consultation with the project's ODS (Organizational Development Specialist), the Project Manager and Evaluation Advisory Team. TENTATIVE TIMELINE Activities Level-of-Effort Timeline Stag e No . Particulars TL TM-1 1.0 1.1 Preliminary consultation 0.5 0.5 March 25 2.1 Literature review 1.0 2.0 March 26- 27 2.2 Design of evaluation framework 3.0 3.0 April 8 -10 2.3 Meeting on design and finalization 0.5 0.5 April 11 2.4 Preparation of questionnaires / checklists 1.0 2.0 April 16-17 2.0 2.5 Meeting on questionnaires / checklists 0.5 0.5 April 22 3.1 Field Visit 1: ISOs and PNGOs OAs, OCBO/As, ODFs, CBPOs, and Beneficiaries 11.0 11.0 3.2 Field Visit 2: ISOs and PNGOs OAs, OCBO/As, ODFs, CBPOs and Beneficiaries 8.0 8.0 May 12 – 22 June 5 - 9 3.0 3.3 Meeting with PMT and stakeholders (in KTM/field) including extensive discussions with development partners whose programmes are being impacted by SANDEEP 5.0 5.0 June 23 - 27 4.0 4.1 Preparation of preliminary findings 2.0 2.0 July 7 - 8 4.2 Presentation of preliminary findings to PMT, ISOs, SMT and collection of feedback 2.0 2.0 July 10 - 11 5.1 Prepare draft report 5.0 4.0 July 22-26 5.2 Meetings with PMT & SC to share draft report and collect feedback 0.5 0.5 August 6 5.0 5.3 Finalization of report 3.0 2.0 By August 15 TOTAL 43.0 43.0 50 ANNEX - 2 QUESTIONNAIRE FOR INTERVIEW WITH STAKEHOLDERS FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2008 Questionnaire Guideline for EC Members of PNGO/ISOs Name of ISO/PNGO Address Phone/ email Date: Respondents Designation 1 2 3 4 5 6 7 1. Please explain why your organization joined SANDEEP Project? What were the objectives and needs? 2. Has these objectives been fulfilled? What are the major achievements and changes? 3. How has the role of EC changed over the past 4 years? 4. Please explain how these changes and achievements contributed to fulfill the goals of your organization? 5. Are the supports from ISO/PMT SANDEEP in line with your organization’s OD needs? Which inputs (training/ coaching/modules) were effective to bring change? Why? 6. Has the organizational changes brought improvement in service delivery? 7. Have you developed any best practices where OD helps improvement in the delivery of health, education and livelihoods of poor and marginalized communities that could be replicated? 8. Please provide your view on SANDEEP's approach (mode) of providing support to PNGOs through the ISOs? Any other comments and suggestions to enhance the Support Approach to strengthen ISOs/PNGOs? 9. Has your organization developed any strategies to sustain the benefits of SANDEEP? Can your organization sustain an ODF/OCBOs/As/FOCBOs financially? 10. Please mention if there are other NGOs/donors/INGOs in your district interested in ISO/SANDEEP support (OD support)? Why? 51 11. Has there been any change in the relationship with other stakeholders (GO/NGO/ Donors)? If yes, can this be attributed to SANDEEP support? What are the results and benefits of these linkages? 12. What is the impact of SANDEEP on the NGO sector? Has the capacity of the sector enhanced? How? 13. What linkage does your organization have with private organizations (corporate)? For what purpose? What are the results of linkages? 14. Please mention the SANDEEP learning (Governance/ management systems/ HRD/ resource mobilization) which you have shared most? What is the mechanism to share the learning? 15. In your opinion, why should development community support OD of NGOs? 16. How can OD help mitigate the effects of conflict? Which OD interventions are effective in conflict environment? (For PNGOs) 17. In your view are the ODF capable to continue to providing “OD support service” after project phase out? (For ISOs) 18. In your view are the OCBOs/As and FOCBOs capable to continue to providing “OD support service” after project phase out? • Any other comments /suggestions: 52 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2008 Questionnaire Guideline for ISO/PNGO Senior Staff Name of ISO/PNGO Address Phone/ email Date: Respondents Designation 1 2 3 4 5 6 7 8 9 1. Please explain how your organization looked like before SANDEEP Project and what changes have you achieved now? 2. What can your organization do now that you could not do before joining SANDEEP? How? 3. What contributed to these changes? Which inputs and support from PMT/ISO were most effective and why? 4. Please provide your view on SANDEEP's approach (mode) of providing support to PNGOs through the ISOs? 5. How is OD team functioning and would it be continued after the project and how? 6. Is the capacity of ISO staff adequate to handle the number of PNGOs they are supporting? 7. Please provide your view on the appropriateness of the training module approach including content of the module? 8. Any other comments and suggestions to enhance the Support Approach to strengthen ISO/PNGO? 9. Has the organizational changes (governance, management systems, HRD, financial management, resource mobilization) contributed to the improvement in the delivery of services to the communities? How? What mechanisms have you developed to improve service delivery as a result of OD support from SANDEEP? 10. Please suggest other approach / interventions (support) that can support to enhance service delivery? 53 11. What are the learning and challenges (planning and implementation) in the organizational development of NGOs? 12. Considering phase out of SANDEEP, what will your organization give continuity to the “the change processes initiated”? 13. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 14. Has the PNGO/ISO intervention helped to mitigate the conflict? Give reasons for your answer? Does OD help to mitigate conflict? Given the conflict situation, what adjustment in planning, implementing and monitoring programs did you makes/should be made by NGOs to improve its contribution towards conflict mitigation? To PNGO Staff 15. Please explain the relationship with ISO before SANDEEP? How has the relationship evolved (changed)? Why? 16. Any suggestions to strengthen your organization’s relationship with the ISO? (From your organization side and/or from the ISO side) 17. How is your working experience with ISO and with PMT (SANDEEP – Regional/KTM) 18. How has your relationship with other NGOs, NGO networks, donors, AIN changed in the project intervention period? What was before and what is now? Why change? Result of change? 19. How frequent is ISO support to the PNGO? What type of support does your organization usually demand for? Is the response of ISO adequate? 20. What is the process adopted by ISO and your views in providing the support to your organization? 21. Is the capacity of the ISO adequate to provide the support? Is there a need to strengthen this further? If yes, please mention areas and provide suggestions. 22. Any other comments /suggestions: 54 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2008 Questionnaire Guideline for field Staff of ISO/PNGO Name of Organization Address: Date: Respondents Designation 1. 2. 3. 4. 5. 1. Please briefly explain about SANDEEP and its objectives. Why did your organization partner with SANDEEP? What were your objectives? Have these objectives been met? 2. Please mention what benefits your organization has had (if any) from the SANDEEP partnership? (changes in governance, management, HRD skills, linkages financial management) 3. What has been the communities' response to your organization’s services (programme and activities) over the past 4 year, 2 year and now? 4. What are the key strengths and areas of improvement of your organization? Are there efforts taking place to address the areas for improvement? Who? How? 5. Is there any change (improvements) in the internal operations of your organization - 4 years ago and recently? If yes, please mention the main changes and what influence this has had in your work area? 6. Have you noticed any change (improvement) in the capacity of your colleagues (OCBOs/As and FCBOs)? If yes, what contributed to this change? 7. Are there other NGOs that are informed of SANDEEP support and demand for capacity strengthening inputs? 8. What has been your major learning in the past 4 years? What do you think has been the major challenge for ISO/PNGO in the past 4 years? 9. What should the ISO/PNGO focus on to develop itself further? 10. In what way has the conflict affected your organizations (ISO/PNGOs) activities/ programme delivery? Please mention these, and the coping mechanisms and responses adopted: Any other comments /suggestions: 55 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2008 Questionnaire Guideline for ODF (PNGO) Name of PNGO: Name of ODF: Address: Phone/ email: Date: 1. What do you think are the major achievements of SANDEEP? 2. What is your experience (positive/negative) of working as ODF? (support, relation with PNGO EC, staff)? 3. How is the relationship with OCBO/As/FOCBOs/CBPOs and other SANDEEP PMT staff in terms of their support? What do you suggest for improvement in relationship between PNGO, ISO, PMT for PNGO capacity building and improved service delivery? 4. How do you feel about your workload as an ODF? How do you deal with demanding situations? 5. Any suggestion to improve the role & effectiveness of ODF after end of project? 6. In which areas do you solicit support from OCBOs/As/FOCBOs and CBPOs? Do you get support in time? 7. What are the mechanisms for communication (including reporting), monitoring, feedback and documentation? Any suggestions to enhance it? 8. What is your view on SANDEEP approach? 9. How do you find the OD team approach? What are the challenges, if any? Please suggest how to improve this. 10. Please provide your view on how the PNGO can give continuity to learning and other inputs from SANDEEP? 11. How has conflict influenced in PNGOs capacity strengthening process? 12. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 13. Has the PNGO/ISO intervention helped to mitigate the conflict? Give reasons for your answer? Does OD help to mitigate conflict? Given the conflict situation, what adjustment in planning, implementing and monitoring programs did you makes/should be made by NGOs to improve its contribution towards conflict mitigation? 14. Any other comments and suggestions: 56 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2008 Questionnaire Guideline for OCBOs/As and FOCBOs Date: Name of ISO Address Phone/ email Date: Respondents Designation 1 2 3 4 5 1. What have been the major achievements of SANDEEP? 2. What are your roles/functions in SANDEP project? 3. Please briefly explain your working experience (level of support, inputs, communication, feedback) with PNGOs/ISOs and PMT (regional/KTM) 4. In which areas do you solicit support from regional PMT and KTM? Do you get the support in time? 5. What are the challenges in executing your role/functions? 6. How was the relationship with PNGOs before SANDEEP? How has the relationship changed? In which areas? 7. Please provide your views on SANDEEP's approach/model of providing support to PNGOs through ISOs? Are there other viable alternatives? Have there been any adjustments made to the project approach in the past 5 years? 8. What is your opinion of project approach in addressing the variation in size and nature of PNGOs? 9. Do you find the project's approach and interventions clear and in alignment with the projects assumptions and hypothesis? 10. How do you find the OD team approach? What are the challenges, if any? Please suggest how to improve this. 11. What is your understanding of OD concept? 12. Which intervention modules and programme initiatives do you find challenging to implement? Why? 13. What change do you feel in your confidence level before (start, 2 year back and now)? What capacity do you need to strengthen further to enable you carry out your functions effectively? 57 14. How is the work distributed among the OCBOs/As and FOCBOs? How do you deal with demanding situations? 15. What are the reporting, feedback, and decision making mechanism for making programmatic and managerial decisions in the SANDEEP project? What needs to be improved? 16. Do you think your support will be demanded (internally / externally) after project phase out? 17. How has conflict influenced in extending your support to the ISOs/PNGOs? 18. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 19. Has the PNGO/ISO intervention helped to mitigate the conflict? Give reasons for your answer? Does OD help to mitigate conflict? Given the conflict situation, what adjustment in planning, implementing and monitoring programs did you make/should be made by NGOs continue to work in the conflict situation? 20. Any other comments and suggestions: 58 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project April – August 2006 Questionnaire Guideline for CBM/CBPOs and RFM (SCUS RO staff) Name Address Phone/ email Date 1. What is the major achievement of SANDEEP? 2. How do you perceive SANDEEP's approach, process and interventions? Any suggestion? 3. What adjustments have been made to the approach in the past five years? 4. What is your suggestion on SANDEEP's approach/model of providing support to PNGOs through ISOs? Are there other viable alternatives? 5. What is your experience of project approach in addressing the variation in size and nature of PNGOs/ISOs? 6. What is your comment on training module approach? Are the content of the modules appropriate? How sustainable are the project approach? 7. What is your experience of conveying the content of the 'intervention module' and other programme initiatives? Which intervention modules and programme initiatives do you find challenging to implement? Why? 8. Do you find the project's approach and interventions clear and in alignment with the projects assumptions and hypothesis? 9. How do you find the OD team approach? How do you think it can be sustained? 10. Please assess the capacity of the OCBOs/As, FOCBOs – what are the strengths and weaknesses? To what extents have OCBOs /As, FOCBOs internalized the OD concepts and skills? 11. How effective do you find the OCBOs/As, FOCBOs in transferring the knowledge /skills and facilitating the change process as per need of ISO/PNGOs? 12. What change do you feel in your confidence level before (start, and now)? What capacity do you need to strengthen further to enable you carry out your functions effectively? 13. How is the work distribution vis a vis PMT KMT? How do you deal with demanding situations? 14. In which areas do you solicit support from PMT KTM and do you get the support in time? 15. What are the mechanisms for communication (including reporting), monitoring, feedback and documentation? 16. How likely will the OCBOs/As/FCBOs continue to fill this function post programme? 59 17. To what extent to you think the ISO/PNGOs value OD in order to continue to drawing upon knowledge of OCBOs/As in the future? 18. How has conflict influenced in extending your support to the ISOs/PNGOs? 19. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 20. How has the project interventions helped the PNGO/ISO helped to deliver services in conflict situation? Give reasons for your answer? Does OD help to mitigate conflict? Given the conflict situation, what adjustment in planning, implementing and monitoring programs did you make/should be made by NGOs to continue to work in the conflict situation? 21. Any other comments and suggestions: 60 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project March – August 2006 Questionnaire Guideline for Community People (FGDs with PNGO/ISO Beneficiaries) VDC/Ward District Programme Date: Respondents Designation 6. 7. 8. 9. 10. 11. 1. What are the activities/services undertaken by PNGO/ISO in your community? Please mention the activities of the PNGO that you find most useful and why? 2. Over the years, is there any change (improvements) in the way the PNGO/ISO deals with, facilitates and implements activities in your community? If yes, what are the changes (improvements) and how were they changed? In your opinion, what made the PNGO/ISO change its way of dealing and implementing its activities? 3. In recent times, has the PNGO/ISO services (e.g. training conduction, facilitation and community mobilization, communication and coordination, increased activities and resources, more field visits), been more effective and helpful for the communities' development? If yes, which services? 4. How does the PNGO/ISO interact with you (regular meetings, visits by PNGO/ISO staff)? Have you participated in or been consulted during planning, implementation and monitoring of PNGO/ISO activities/programmes or asked for feedback on PNGO/ISO work? Have PNGO/ISO accepted your suggestions/feedbacks and made suitable changes in their planning, implementation and monitoring processes? 5. What are the strengths and weaknesses of the PNGO/ISO? 6. Is the PNGO/ISO transparent about budgetary allocation for project activities in your community? Are social audits/public hearing organized by the PNGO/ISO? 7. How many NGOs other than the PNGO/ISO are active in your area? How do you compare the quality of services of PNGO/ISO with other NGOs? Is it better or the same as other NGOs? If it is better, what makes you say so? E.g. more resources, more effective services, better transparency and accountability? 8. What do you think has been the challenges in planning, implementing and monitoring programs for the PNGO/ISO in the past 4 years (in terms of community mobilization, imparting awareness, conducting consultation session with community people, motivating community people to take ownership of program, effectively participating and supervising program activities etc) 61 9. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 10. Has the PNGO/ISO intervention helped to mitigate the conflict? Give reasons for your answer? Given the conflict situation, what adjustment in planning, implementing and monitoring programs should be made by NGOs to continue to work in the conflict situation? 11. What should the PNGO/ISO focus on to develop itself further to enhance its service delivery? 12. Any other comments /suggestions: 62 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project March – August 2006 Questionnaire Guideline for district level stakeholders Name of Organization Address Programme Date: Respondents Designation 13. 14. 15. 16. 17. 18. 19. 1. Are you aware of SANDEEP activities and programmes? 2. What is your view of the PNGO /ISO and the services it provides in the district? (Usefulness, adequacy, timeliness and gender sensitivity) 3. In your opinion, which activities/programs of PNGO/ISO do you think are most effective? Why? 4. In recent times, have you noticed any positive changes in the way the PNGO/ISO deals with, facilitates and implements activities in the community and or deals with you/your organization? If yes, what and how? 5. If the response to question 4 is yes, what in your opinion has influenced the PNGO /ISO to change its way of dealing and / or implement its activities? 6. How does the PNGO/ISO interact with you (regular meetings, visits by PNGO staff)? Have you been consulted during planning, implementation and monitoring of PNGO/ISO activities/programmes or asked for feedback on PNGO/ISO work? If yes, have you provided feedbacks and suggestions? Which areas of organizational development aspect are your feedbacks and suggestions generally concerned with? Do you find any change after your feedback? 7. What are the strengths and areas of improvement of the PNGO/ISO (governance, management systems, human resource, planning & monitoring evaluation, financial management)? What are your suggestions to facilitate improvement? 8. Are the services of the PNGO /ISO better or the same as other NGO in the district? Please provide reasons for your answer? E.g. more resources, PNGO/ISO staff attend more trainings/workshops outside the district; PNGO/ISO staffs are busy training other CBOs/local NGOs. 9. Are you satisfied with collaborative and coordination effort of PNGO/ISO with other stakeholders? If yes or no, please provide reasons. 63 10. Considering conflict situation: how did the PNGO/ISO (and other NGOs/government) work in the district/VDC in the conflict situation? Did it have any effect on the PNGO/ISO activities e.g. programme reduction and less field presence? Were there any programs/activities that had to be slowed down, postponed or closed due to the conflict? 12. Has the PNGO/ISO intervention helped to mitigate the conflict? Give reasons for your answer? Given the conflict situation, what adjustment in planning, implementing and monitoring programs should be made by NGOs to continue to work in conflict situation? 13. What do you think has been the major challenge in planning, implementing and monitoring programs for the PNGO/ISO in the past 4 years (in terms of community mobilization, imparting awareness, conducting consultation session with community people, motivating community people to take ownership of program, effectively participating and supervising program activities etc) 14. In your opinion, which operational/management aspects of your PNGOs need to be further developed for effective service delivery to your target group? 15. How would you be able to collaborate/support a program that primarily focuses on the organizational development of civil society organizations? 16. Any other comments /suggestions: 64 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project March – August 2006 Questionnaire Guideline for Donors Name of Organization Address Programme Date: Respondents Designation 1. How long have you been collaborating with the PNGO/ISO? What is your support? 2. What are your primary reasons for collaborating with the PNGO/ISO? 3. Are you satisfied with the service delivery of the PNGO/ISO? Please provide reasons for your response? 4. In your opinion, which activities of the PNGO/ISO are the most effective? Why? 5. During the period of your support have you noticed any positive changes in (governance, management systems, human resource, planning & monitoring evaluation, financial management) the way the PNGO/ISO deals with, facilitates and implements activities in the community and or deals with your organization? If yes, what and how? 6. If the response to question 5 is yes, what in your opinion has influenced the PNGO /ISO to change its way of dealing and / or implement its activities? 7. Has PNGO/ISO asked feedback on their program activities? If yes, have you provided feedbacks and suggestions? Which areas of organizational development aspect (governance, management systems, human resource, planning & monitoring evaluation, financial management) are your feedbacks and suggestions generally concerned with? Do you find any change after your feedback? 8. What are the strengths and areas of improvement of the PNGO/ISO? What are your suggestions to facilitate improvement? 9. What should the PNGO/ISO focus on in the future and what capacity does it need to strengthen for this? 10. Are the services of the PNGO /ISO better, the same as or not at par with other NGOs under your portfolio? Please provide reasons for your answer? E.g. more resources, PNGO /ISO staff attend more trainings/workshops outside the district, PNGO/ISOs staff busy with training to other CBOs/local NGOs 65 11. How has PNGO /ISO (and other NGOs / government) worked in the district/VDC in the conflict situation? What where the major effects of the conflict on the service delivery of the PNGO/ISOs? e.g. programme reduction and less field presence? 12. What were the major steps taken to mitigate the effects of the conflict? 13. How is the work of NGOs affecting the root causes of the conflict? Given the conflict situation, what adjustment in the approach should be made by NGOs to continue to work in conflict situation? 14. What do you think were the major challenges in planning, implementing and monitoring programs in partnership with NGOs in recent years? 15. In your opinion, which operational/management aspects of your PNGOs need to further developed for effective service delivery to your target group? 16. Does your organization support organizational development activities of PNGOs? 17. If yes, what proportion of your program budget do you allocate for OD? 18. What are the best practices in formulating, implementing, monitoring and evaluating a capacity development project? 19. Are you interested in supporting a program that specifically focuses on organizational development of your PNGOs? If yes, what type of support? 20. Any other comments /suggestions: 66 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project March – August 2006 Checklist for interaction with PMT Staff Name of Respondent: Designation / Function Date: General Opinion • Objective and rationale of SANDEEP as per project document approved by USAID • Changes/amendment made to approved project objectives and targets and rationale for doing so. • Major achievements of SANDEEP • Major challenges in implementation of the project. • Successes cases • Major learning: What are the main lessons that can be drawn from the project experience that may have generic application? What are the best practices in formulating, implementing, monitoring and evaluating a capacity development project? Role and responsibilities • PMT structure and various role/functions/responsibilities • Communication, reporting and coordination: within PMT (KTM/Region) and with ISO/PNGO PMT (OCBOs/As/FOCBOs, ODF and others), community members, SCUS staff, donors and USAID/Nepal & Washington. • Challenges and constraints in execution of role/functions/responsibilities Programme: process and approach • Achievement of strategic objectives (full or partial). Reasons for opinion. • Feedback from the direct "beneficiaries" of SANDEEP’s support (regional PMT, ISO/PNGO SANDEEP staff), satisfaction/dissatisfaction about programme approach and processes. • Project approach of ISO strengthening PNGOs – "what has worked", what can be enhanced and the approach to address the challenges. (relative to PNGOs delivery of services and linkages to networks and alliance) • Partner NGOs initiative /role in accessing different support from SANDEEP in accordance with their OD needs 67 • Changes made in programme approach, and implication on SANDEEP’s activities thereafter (functions /responsibilities of staff (OCBOs/As, FOCBOs, FCBO, CPBOs and PMT KTM) – please specify these. "Benefit" of the change in the approach – e.g. responses from the partners etc. • OD team approach? (Rationale, advantages and challenges) Management systems and processes • Programme planning and decision making (participation PMT team and partners) • Reporting and communication: current system (guidelines and manuals) , major modifications if any and reasons for such modification and response of partners and donors • Effectiveness and timeliness of monitoring, documentation and reporting of programme and learning • Major lessons learnt, sharing and feedbacks (within PMT team, PMT and ISO/PNGOs, PSO/PNGO & beneficiaries and PMT, ISO/PNGO, targeted community people & stakeholders) • Reporting and feedback from donors Support to PMT regional and partners of SANDEEP • Support to regional PMT and partners – mostly on which areas, why? • Taking up requests from regional and field staff, and ISO/PNGOs • Changes (improvements) observed on ISO/ PNGO functioning • ISO/PNGO service delivery effectiveness • Progress in terms of expanded linkages between PNGOs, Govt., & Private sector • Learning & sharing between NGOs, Govt., & Private sector institutions as a result of expanded linkages Capacity of PMT staff • Capacity of CBPOs/FCBO to strengthen OCBOs/As and FOCBOs • Capacity of OCBOs/As and FOCBOs in facilitating change in PNGOs/ISOs ISO/ PNGOs OD needs and service delivery in conflict situation • Major difficulties and challenges posed by the conflict to service delivery (activities that were severely affected) • Major OD needs to face difficulties and challenges • Adjustments/changes made in programme and organization to face challenges and difficulties. • Contribution of SANDEEP to the reconciliation and recovery process 68 Sustainability of program • Major issues for sustainability • Phase out strategy • Future role of OD team: continuity, demand assessment, adaptation of techniques to serve other organization in the future. • Future focus of ISO/PNGO focus to develop itself • Advocacy focus for the support of future OD program • Actions recommended to follow up or reinforce the benefits from the project? • Any other comments /suggestions: Thank you for your cooperation 69 FINAL EVALUATION OF Sansthagat Diego Pariwartan (SANDEEP) Project March – August 2006 Checklist for interaction with SCUS & USAID Name of Respondent: Designation / Function Date: • Organizational policies and strategies on OD intervention • Major achievements of SANDEEP • Major challenges (factors affecting success) of OD program • Major learning from SANDEEP/ OD program. What are the main lessons that can be drawn from the project experience that may have generic application? What are the best practices in formulating, implementing, monitoring and evaluating a capacity development project? (For SCUS only) Lessons learnt in application of OD program and how they have been applied (For USAID). • OD aspect focused on while supporting program. How is blended to service delivery? (For USAID) • Specific OD issues and needs of PNGOs that need to further strengthened. • Major issues related coordination, reporting and feedback • Effectiveness and timeliness of monitoring, documentation and reporting of programme and learning • Introduction of “new” non-traditional components into the OD strengthening program (for example, conflict analysis or assessment, planning for conflict mitigation, conflict resolution or mediation skills, psycho-social or other forms of support for program staff). • Biggest challenge to working with strengthening of local partners in a conflict affected environment • Unique opportunities available for doing NGO capacity building that are present in a conflict environment. If so, what are some of these opportunities? • Particular OD interventions well suited for conflict affected environments and why? • OD interventions and approaches that are NOT appropriate to working in a conflict-affected environment and why? • Major difficulties and challenges posed by the conflict to service delivery (activities that were severely affected) 70 • Major issues for sustainability • Phase out strategy • Future role of OD team: continuity, demand assessment, adaptation of techniques to serve other organization in the future. • Advocacy focus for the support of future OD program • Corrective actions recommended for the design, implementation, monitoring and evaluation of the project (For SCUS only) • Influence of SANDEEP on the NGO sector and their donors and how it has been applied at the country, regional and international level (For SCUS only) • Actions recommended for following up or reinforcing the benefits from the project? • Any other comments /suggestions: Thank you for your cooperation 71 ANNEX - 3 Organizations, groups and individuals interacted for data/information collection during field visit. Date Time Activity People Interacted Position Organization Janakpur May 14, 2008 10 - 18 Observation of OA exercise Staff and EC members of CFWA CFWA May 15 10 - 12 Interview with EC 6 EC meembers of CFWA 12-14 Interview with Staff 4 staff members of CFWA Meeting with DPHO Bijay Kumar Jha DPHO 14-16 Interview with ODF Laxmeshwor Mandal ODF 16 FGD with School teachers and parents 10 participants Interview with NFN Ram Chandra Sah President NFN FGD with adolescent girls of GATE Class 14 particiapnts Bageda-4 Interview with OCBO Rambabu Shah OCBO/SCDC 17 Interview with ISO staff 10 staff members SCDC Choharwa May 18 7-9 Interview with CBM Rabindra Thapa PMT SCUS/RO 10-12 Interview with ERO Manager Bhim K Pun Regional Coordinator SCUS BRT 13-15 Interview with Regional NFN Mr Shankar Thapa Resional President NFN BRT Darchula May 22 10-12 Meeting with EC Sundar S Budathoki Sarita Budathoki V. President Secretary RYC 13-15 Meeting with Staff 5 staff members 23 Meeting with ODF Gokarna Badu ODF Meeting with LDO Mr. Danda Pani Baral LDO FGD with Community Meeting with NFN Mr Narhari Thekre NFN secretary; Program Manager PNGO NFN Darchula PNGO Markende Meeting with Helvetas WAMP 72 Date Time Activity People Interacted Position Organization Meeting with BDA Kanchanpur May 25 Meeting with EC 7 EC members of NNSWA Meeting with staff 11 staff members of NNSWA Meeting with NFN Bhawa R Regmi Krishna Joshi Surendra Bista Dharma S Kunwar V. Presidnet Member President Treasurer NFN 26 Meeting with Child care centre Bhan B Bista Min B Dhanuk Ganesh B Negi Dev B Singh Ratan Kadayat Madhavi Kunwar President Parent Parent Parent Parent Facilitator Jhalari W-2 Simalphanta Meeting with DEO Lok Mani Joshi DEO Kanchanpur Meeting with OCBO/ FOCBO Hansha R Joshi Karna Sunar Meeting with PNGO Bhawa R Regmi President NEEDS 27 Meeting with EC 7 EC members Tharu Mahila Manch Meeting with staff Meeting with 8 staff members TMM Meeting with WDO Vishwo M Joshi Maya Joshi DWO Supervisor DWO 28 Meeting with DDC Bhagirath Bhatta Bhanu D Bhatta DDC Advisor Advisor DDC Meeting with Child Club 22 participants Haldukhal village Meeting with teachers and parents 6 participants Suda High school Meeting with ODF Hema Bhatta ODF TMM Dadeldhura 29 Travel to Dadeldhura 30 Meeting with sr. staff Ghanshyam Poudel Program Coordinator NRCS 73 Date Time Activity People Interacted Position Organization Mahadev Awasthi Prog Coordinator Meeting with field staff Harendra K Paneru Sarswati Khadka Iswari D Joshi Store Keeper Social mobiliser Counselor Meeting with EC 6 EC members Meeting with Junior Redcross Prem B Dhanuk Sher B Saud Student class X Volunteer JRC. Kailpalmandu-6 Meeting with DPHO 31 Meeting with ODF Jog Raj Air ODF Travel to Nepalganj June 1 Travel to Kathmandu Surkhet 5 Observe OA Staff and EC of SAC SAC Surkhet Observe OA Staff and EC of SAC 6 Meeting with EC 6 EC members Meeting with staff 7 staff members Meeting with DEO 7 Meeting with community Meeting with CECI Pawan Karki Maya Nath Reg. Coordinator Livelihood officer Sahakarya Project Meeting with Donor B kram Rana Tharu Project Coordinator WARNP Helvetas Meeting with OCBO Prakash Subedi OCBO SAC 8 Meeting with EC of PNGO SDC 7 EC members l SDC Meeting with Staff 5 staff members SDC Meeting with ODF Hasta Sunar ODF/TL 9 Meeting with Donor Kabita Shrestha P Bishwokarma District In-charge Marketing supervisor SIMI-OVC Meeting with women group of SIMI, OVC 12 participants Dang 74 Date Time Activity People Interacted Position Organization 10 Meeting with Staff 10 staff members BASE Meeting with EC 3 EC members Meeting with community 25 participants Meeting with Child Group 10 participants Meeting with OCBO Tilak Tharu 11 Meeting with WDO Rashi Pande Dhital Ishwara Rawal Program Officer Staff member Ghorahi Meeting with DDC Bal Krishna Khanal Social development officer Ghorahi Meeting with NFN Suman Ghale EC Member NFN Dang Travel to Nepalganj Kailali 13 Meeting with Staff of ISO 11 staff members NRCS Meeting with EC 7 EC members Meeting with Mercy Corps Intl Meeting with Lutheran World Federation Rajan Niraula Program Coordinator LWF Meeting with Chamber of Commerce and Industry Gopi Hamal Binod Bhandary Meeting with NFN Mr Prakash Sah President Regional chapter Meeting with OCBO Tirtha Raj Joshi OCBO 14 Meeting at Ex-Kamaya Camp Masuriya Meeting with staff of PNGO 4 staff members KPUS Meeting with EC S EC members KPUS Meeting with ODF Banke 12 Meeting at DEO Mr. Dabal B. C, Planning Officer UNESCO Club Meeting with EC 6 EC members 13 Meeting at SC, Norway, Npj. Pasupati Sapkota Prakash Acharya Dev Ale Regional Manager Meeting at DDC Dev Bdr. Adhikari Sharad Paudel Program Officers 75 Date Time Activity People Interacted Position Organization Ramesh Shah Meeting at UNDP Mr. Meeting with staff 6 staff members UNESCO Club Meeting with ODF Shova Laxmi Joshi ODF UNESCO Club Meeting with OCBO Prem Chadhari OCBO 14 Meeting at SCUS, Banke Lila Mani Sharma Regional Coordinator Ratna Tandulkar OCBPO Kathmandu 16 World Education Manoj Silwal Program officer WE office Meeting with SCN Norway Bhola Jagat Khadka Deputy CR 16 Meeting with USAID Linda 17 Meeting with SCUS directors Tory Uday Manandhar ?? 18 Meeting with PAF Rajesh Babu Shrestha Project Manager 19 Meetingh with SCUS Japan Eiichi Rajendra Manandhar Chandra D Sah Country Director Program Officer Prog Officer Meeting with AIN Sony AIN Meeting with PMT Deepak Koirala Program Manager SCUS 25 Meeting with CECI Sahakarya Mahendra Laxmi Sharma 76 ANNEX - 4 Planning Matrix and EOP Achievement Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities Goal: Improved health, education and economic well being for communities, especially for women and children and disadvantaged groups, in the conflict affected Mid, Far Western and parts of the Central and Eastern Regions of Nepal Strategic Objective: Enhance the capacity of the NGO Sector to improve the delivery of health, education and economic services, especially for women and children and disadvantaged groups, in the conflict affected Mid, Far Western, and parts of Central and Eastern Regions of Nepal. Risks and Assumptions: Improved internal organization, expanded linkages and increased learning and sharing means more effectively run programs, which in turn means improved service delivery. SNSN PNGOs continue to receive donor support from their respective donors for health, education or economic opportunity programs during the life of the project. Building capacity of 35 NGOs and 5 ISOs will help enhance and Data for indicators 1.1 & 1.2 will be collected during midterm and final evaluation by IDF Data for indicator 1.3 Interviews will be conducted with the PNGOs’ respective donors. Secondary data will be collected via PNGOs’ impact evaluation reports ISOs: 1.1 %(#) of capable ISOs PNGOs 1.2 %(#) of capable PNGOs 1.3 # of PNGOs who receive positive impact evaluations of their health, education, or economic opportunity programs by their respective donors (who identify the impact as a result of SANDEEP intervention) 20%(1) 0% (0/35) % ( ) MT: 20%(1/5) EOP: 80%(4/5) MT: 0% (0/35) EOP: 40% (14/35) EOP: 34%(12/35) MT: ISO 40% (2/5) EOP: 80% (4/5) PNGO 11% (4/35) EOP: 90% (27/30) MT: (12.5% in aggregate) EOP: 47% (14/30) 77 Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities influence the NGO sector in the Mid, Far Western, and parts of Eastern Region. NGOs in Mid and Far Western Region are able to reach to their target groups in conflict situation 35 PNGOs =100% 5 ISOs =100% 1. Strengthen the operational and financial capabilities of individual PNGOs and ISOs. Risks and Assumptions: SNSN ISOs and PNGOs will be able to reach defined capability areas of the Oversight/Vision Management Resources, Human Resource Management, and the Financial Resources sections in the IDF within the project period. All SNSN ISOs are organizationally developing at the same rate. All SNSN PNGOs are organizationally developing at the same rate. IDF Tool is used to measure progress against indicators 1.1 to 1.5 Data for all indicators will be collected during midterm and final evaluation by IDF. Data will be stored on hardcopy and softcopy in the ISO offices, PNGO offices, and in a database system in SC Field Office. This data will be utilized to help ISOs and PNGOs recognize their improvement areas and make appropriate action plans to 1.1 %(#) of ISOs and %(#) of PNGOs with “good governance” and leadership 1.2 %(#) of ISOs and %(#) of PNGOs have an effective management system 1.3 %(#) of ISOs and %(#) of PNGOs practicing participatory human resource management ISO 40% (2) PNGO 0%(0) ISO 20% (1) PNGO 0%(0) ISO 40% (2) PNGO 0%(0) MT: ISO 60% (3) PNGO 3%(1) EOP: ISO 100% (5) PNGO 89% (31) MT: ISO 60% (3) PNGO 6%(2) EOP: ISO 100% (5) PNGO 97% (34) MT: ISO 40% (2) PNGO 0%(0) EOP: ISO 100% (5) MT: ISO 40% (2) PNGO 20%(7) EOP: ISO 80% (4/5) PNGO 100% (30/30) MT ISO 60% (3) PNGO 14%(5) EOP ISO 100%(5/5) PNGO 97% (29/30) MT ISO 40% (2) PNGO 6%(2) EOP ISO 80% (4/5) Promote Good Governance and Leadership • Conduct Organizational Development Assessments in ISO • Establish Organizational Development Team in ISOs • Organize ISO Strategic Plan workshop • Develop Governance Module • Facilitate TOT for OCBOs on Governance • Training of Governance Module for ISOs • Support ISOs to conduct training to PNGOs Promote Effective Operational Management • Organize working in conflict workshop • Facilitate TOT for OCBOs for 78 Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities improve their governance structure, HRD, Monitoring System and Financial system, make the needed changes (if any) in their financial sustainability plans. These indicators will be reported in the Annual report to USAID. 1.4 %(#) of ISOs and %(#) of PNGOs that have an “effective planning monitoring and evaluation system" that enables them to monitor their achievement of their annual program targets. 1.5 %() of ISOs and %() of PNGOs that have a functioning financial management system. 1.6 % (#) of ISO and PNGOs which have established an internal system and using for measuring quality of service with beneficiaries participation ISO 40% (2) PNGO 0%(0) ISO 40% (2) PNGO 0%(0) ISO 0% (0/5) PNGO 0% (0/35) PNGO 71%(25) MT: ISO 60% (3) PNGO 9%(3) EOP: ISO 100% (5) PNGO 80% (28) MT: ISO 80% (4) PNGO 11%(4) EOP: ISO 100% (5) PNGO 77% (27) MT ISO 0% (0/5) PNGO 0% (0/35) EOP ISO 60% (3/35) PNGO 48% (17/35) PNGO 90% (27/30) MT ISO 40% (2) PNGO 11%(4) EOP ISO 100% (5/5) 100% (30/30) MT ISO 40% (2) PNGO 17%(6) EOP ISO 100% (5/5) PNGO 100% (30/30) MT ISO 0% (0/5) PNGO 0% (0/35) EOP ISO 80% (4/5) 100% (30/30) Operational Management • Training of Operational Management Module to ISOs • Support ISOs to conduct training to PNGOs Promote Participatory Human Resource System • Develop Human Resource Module • Facilitate TOT for OCBOs on Human Resource • Training of Human Resource Management to ISOs • Support ISOs to conduct training to PNGOs on HR • Promote Good Financial Management System • Develop Financial Module • Facilitate TOT for ISO Accountants on Financial Management • Support ISOs to conduct training on PNGOs on Financial Management 2. Expand the linkages among and/or between PNGOs and government and private sector institutions. Risks and Assumptions: Meeting minutes; Interviews with PNGOs’ Executive Directors and Board Members; Annually 2.1 %(#) of PNGOs whose Executive Directors or Board Members are initiating meetings with other PNGOs in the SNSN program in order to develop standards in 0%(0) MT: 23%(8) EOP: 57%(20) MT 100% (35/35) EOP 100% (30/30) Promote Resource Mobilization • Develop Resource Mobilization Module • Facilitate TOT for OCBOs on Resource Development • Training on Resource Development to ISOs 79 Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities Private Sector, Government, and NGO networks in Nepal are interested to collaborate with SNSN ISOs and PNGOs. Private sector, Government and NGO Networks that SNSN ISOs and PNGOs approach have resources for working together. (IDF; Annual Financial Report) (IDF; Annual Financial Report) ISO/NGO Profile; Cash, In-kind receipt Training certificates; Workshop recites and other reports; Annual Report Data for indicators 2.1 will be collected annually during the project's annual review process. Data for indicators 2.2 and 2.3 will be colleted during mid-term and final evaluations. Data for indicators 2.4 and 2.5 management practices. 2.2 %(#) of ISOs that are receiving not more than 40% of funds from one source. 2.3 %(#) of PNGOs that are receiving not more that 60% of funds from one source. 2.4%(#) of ISOs and %(#) of PNGOs that received mentoring or pro-bono consultancy services in management, marketing, costumer relations, etc.., or in-kind donations from the private sector to expand their the delivery of services to communities in need. 2.5 %(#) of PNGOs that have participated in at least one yearly training, workshop or learning fora hosted by a district or regional network related to providing health, education, and economic opportunity services to women, children and other disadvantaged groups in the 0%(0) 0% (0) ISO: 20%(1) PNGO: 11%(4) 0%(0) MT: 0% (0) EOP: 60% (3) MT: 3% (1) EOP: 66% (23) MT: ISO: 20%(1) PNGO: 11%(4) EOP: ISO: 80% (4) PNGO: 46% (16) MT: 0% (0) EOP: 70% (25) MT ISO 40% (2) EOP ISO 100% (5/5) MT PNGO 57% (20) EOP 70% (21/30) MT ISO 0% (0/5) PNGO 0% (0/35) EOP ISO 80% (4/5) PNGO 13 MT 0% EOP 100% (30/30) • Support ISOs to conduct training on Resource Development to ISOs Establish Linkages between SANDEEP NGOs and the FNCCI Regional Chapters • Conduct Research study • Hold sharing & strategic meeting with FNCCI at central level • Facilitate regional linkages between FNCCI and ISOs Support or facilitate SANDEEP ISO (and its PNGOs) to activate regional NFN • Hold a strategy workshop with ISOs • about meeting with NFN • Organize meeting with NFN central committee • Support ISOs to organize meeting with NFN 80 Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities will be collected annually through the PNGOs and ISOs Annual Report. Data will be stored on hardcopy and softcopy in the ISO offices, NGO offices, and in a database system in SC Field Office. This data will be utilized to help ISOs, NGOs and Networks expand their linkages to the government, other NGOs, private sector and networks. It will also allow them to monitor the tangible benefits received from these linkages. The data will be reported on an annual basis to USAID. Mid and Far Western Region 3. Enhance the program learning and sharing among and/or between NGOs, government and private sector institutions. Risks and Assumptions: NGOs, Government and Private ISO and NGO Annual Report; Proof of Publication PNGO and ISO Annual Report; 3.1 %(#)of ISOs and %(#) PNGOs producing publications on effective organizational practices that help to build service delivery capacity in health, education, and economic opportunity programs. 3.2 %(#) of ISOs and %(#) ISO: 0%(0) PNGO: 0%(0) ISO: 0%(0) MT: ISO: 40%(2) PNGO: 0%(0) EOP: ISO: 80% (4) PNGO: %50 (17) MT: ISO: 0%(0) MT ISO 0% (0/5) PNGO 0% (0/35) EOP ISO 100% (5/5) PNGO 23% (7/30) MT ISO 0% (0/5) Establish a network of SANDEEP PNGOs and ISOs • Develop R&R Guidelines • Hold Annual R &R Meeting • Support ISOs to hold semi-annual R&R with PNGOs • Support ISOs to organize learning/sharing fora Support ISOs to document & 81 Objective Measurement and Data Management Methods Indicators Baseline Targets: MTE and EOP Achievement Activities sector are interested to learn about effective organizational practices. SNSN ISOs and PNGOs have time to test and replicate effective organizational practices identified by the project within the project period. Publications themselves Data for indicator 3.1 will be collect annually. Data for indicator 3.2 will be collected during mid-term and final evaluations. Data will be stored on hardcopy and softcopy in the ISO offices, NGO offices, and in a database system in SC Field Office. This data will be utilized to let this projects’ management team know what information is being disseminated by ISOs and NGOs and if the information is relevant to share with the international community of NGOs. Data will be reported in the Annual report to USAID. PNGOs replicating effective organizational practices resulting from participating in learning fora with other ISOs and PNGOs in order to build their service delivery capacity in health, education, and economic opportunity programs. PNGO: 0%(0) PNGO: 0%(0) EOP: ISO: 80% (4) PNGO: %50 (17) PNGO 0% (0/35) EOP ISO 80% (4/5) PNGO 100% (30/30) disseminate information • Prepare guideline "what will be documented and what needs to be disseminated • Conduct small case studies to identify effective organizational practices • Support ISOs to disseminate publication Support ISOs to organize cross/exposure • Development directory of resource organizations in Nepal & South Asia • Guideline on how to conduct an effective cross/exposure visit • Support ISOs to organize exposure visit for PNGO 82 ANNEX - 5 EXPLANATION OF PMP ASSESSMENT SCORING PMP Indicators Process of Graph development according to categorization Following graphs were developed from the annual data collected by the Project on PMP indicators. The indicators are categorized as internal operations (indicators 1 to 8), external relations (indicators 9 to 14) and financial resources (indicators 15 to 19). The response of indicators was designated the following scores: Strongly Agree = 5 Agree = 4 Unsure = 3 Disagree = 2 Strongly Disagree =1 The scores were aggregated according to categories and transformed into graphs. PMP INDICATORS ANALYSIS: SCORES OBTAINED SAC Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 34 33 33 34 36 External Operation 30 21 23 26 23 20 Financial Resources 25 17 17 20 19 22 BASE Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 32 32 34 36 37 External Operation 30 28 25 23 26 29 Financial Resources 25 19 21 21 20 23 NRC-Kailali Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 29 33 31 36 38 External Operation 30 16 20 19 22 24 Financial Resources 25 22 20 21 22 22 NNSWA Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 37 34 38 38 40 External Operation 30 23 24 18 20 26 Financial Resources 25 19 20 19 22 23 TCDF Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 30 29 33 33 38 External Operation 30 14 17 25 25 26 Financial Resources 25 16 18 17 17 21 TWUC Year Full Mark 2004 2005 2006 2007 2008 83 Internal Operation 40 30 32 35 35 37 External Operation 30 25 23 27 27 27 Financial Resources 25 16 20 17 17 21 TWOF Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 31 34 31 36 36 External Operation 30 13 21 22 21 23 Financial Resources 25 16 16 16 16 16 Aggregate of all Year Full Mark 2004 2005 2006 2007 2008 Internal Operation 40 32 33 34 35 38 External Operation 30 20 22 23 23 23 Financial Resources 25 18 19 18 18 21 84 ANNEX- 6 LIST OF DOCUMENTS REVIEWED 1. ToR for Final Evaluation of SANDEEP, April, 2008 2. Proposal, Strengthen the NGO Sector in Nepal (SNSN Program) submitted to USAID, 2003 3. Detailed Implementation and Monitoring Plan (Submitted to USAID), 2004 4. Revised Detailed Implementation and Monitoring Plan, Sept 2005 5. History of SANDEEP Program (1st Oct 2003 to 31 May 2006), 2006 6. Organizational Assessment Compilation (Baseline and Mid Term), August 2006 7. Organizational Assessment report for selected ISOs and PNGOs 8. USAID Standard Performance Measurement Plan, Section III, for FY 04/05/06 9. SANDEEP Annual Report, 2004-2007 10. SANDEEP Quarterly Reports October 2007 – March 2008 11. SANDEEP Training Modules (5 nos) 12. Monitoring and Evaluation Manual for FY 2005/06, 2006 13. SANDEEP Study on Private sector 14. SANDEEP Case Studies 2006 15. SANDEEP Mid Term Evaluation Report 2006 16. SANDEEP Partnership compliance Guidelines 2008 17. AIN partnership Guidelines 2008 18. NFN Good Governance Guidelines 2008 19. SCUS strategic plan (2007-2012) 20. Annual Reports of ISOs BASE and SAC, 2007 85 ANNEX - 7 LIST OF SANDEEP ACTIVITIES Major SANDEEP Activities and Dates Major Dates SANDEEP Activities November 2003 Project Start December 2003 Situational Analysis of Mid and Far Western Regions January 2004 Project management team Strategy workshop Feb to March 2004 Selection of 5 ISO from Mid and Far Western Region of Nepal March 2004 Draft Planning Matrix for USAID March 2004 ISO orientation regarding the program April – May 2004 Baseline study and analysis May 2004 ISO DIMP workshop June – July 2004 In-service training for ISOs June – July 2004 Pre-award Surveys for ISOs July – Aug 2004 First phase of PNGO selection (16) September 2004 Study of sample NGOs by Organizational Development Center to provide information to revise NICAT September 2004 Final Detailed Implementation and Monitoring Plan (DIMP) Sept – Oct 2004 Internalization workshops with each ISO October 2004 ISO/PMT Quarterly Review and Reflection Nov 2004 Revised ISO DIMP Workshop Dec 2004 Selection and review of IDF Dec 2004 & Jan 2005 5 ISOs strategic planning workshops Dec 2004 Annual report preparation & submission for 2004 Jan 2005 Testing of appropriateness of IDF, adaptation and translation into Nepali Jan 2005 Quarterly review and reflection of the program Jan 2005 M & E training for ISOs Feb 2005 Organizational Assessment and baseline information collection Training Feb – March 2005 Pre-award surveys for first phase PNGOs March – April 2005 First phase PNGO Organizational Assessments April 2005 Eastern Region expansion April 2005 Exposure visit to India for ISO staffs to observe the organizational development practices April 2005 Review meeting on ISO progress May 2005 Organizational Assessment analysis 86 May 2005 Second phase 20 PNGO selection (36 PNGO in total) May 2005 Presentation of OCBO/As learning "projects" May 2005 Linkage and sharing workshop for ISO staff June 2005 Second phase PNGO orientation June 2005 Training on conflict to 1st phase 16 PNGOs and 5 ISO July 2005 Semi-Annual Report July-August 2005 Second phase PNGO organizational assessments August 2005 ISO organizational assessments using IDF August 2005 Governance training to ISO leaders Sept 2005 PMT prepared Planning Matrix and Output Indicators September 2005 Review and reflection with PNGOs and ISOs Sept 2005 ISO DIMP Sept 2005 PMT DIMP October 2005 ISO Annual report October 2005 PNGO DIMP workshops November 2005 Finalized output indicators November 2005 Finalized PNGO budgets & MOUs Dec 2005 Submission of Annual Report FY 05 to AID Dec 2005 TOT for OCBOs on Governance module December 2005 ISO governance module training December 2005 PNGO governance module training January 2006 ISO meeting on linkages January 2006 PMT Quarterly Meeting February 2006 TOT on Strategic Planning & Mgt (East only) February 2006 TOT on Financial Management February 2006 Review meeting of Strategic Planning process for West ISOs February 2006 Meeting on outputs indicators Feb - April 2006 PNGO Strategic Plan workshops Feb – June 2006 PNGO Financial Management Training March – April 2006 Semi-annual ISO/PNGO Review & Reflection workshops May 2006 Semi-annual PMT/ISO Review & Reflection workshop May 2006 Coaching training for OCBO/As and FOCBO May 2006 M&E training for OCBO/As and FOCBO June – July 2006 MTE OA exercises March Pre-testing of Operational Management Module 87 April-May Workshops for review of ISO strategic plan May 2007 Training to ISOs on OM module June 2007 Training on OM Module to PNGOs November 2007 Training of Trainers (TOT) for ISOs staff (phase II) on RM Module April 2007 Learning sharing workshop with the Regional NGO Federation of Nepal (NFN) in eastern region July 2007 Learning sharing workshop with the Regional NGO Federation of Nepal (NFN) in western region August 2007 Update directory of resource organizations July-August 2007 Organizational Capacity Building Research by ODC Learning and Sharing Fora July 2007 Exposure visit to meet SNV partners in Ilam, VDRC in Nawalparasi, Kalika in Kapilbastu, Siddhi Memorial Foundation and CBR in Bhaktapur, WATCH and NEWAH in Kathmandu September 2007 Pre-award surveys December 2007 Guideline preparation April 2007 SANDEEP project profile published in ODC’s “the Organization” magazine November 2007 Donors meeting July 2007 Changes in project management structure, New project manager appointed and CBPOs placed at ISOs for intensive coaching and support December 2007 HRM ToT to ISOs December 2007 HRM training to PNGOs 88 ANNEX - 8 SUMMARY OF STAKEHOLDER INTERVIEWS Evaluation Objective: 3- To evaluate net impact (net change between the baseline, midterm, and end of project) regarding the status of project interventions, capacity of NGOs to deliver development services in the areas involved and expanded linkage among NGOs, networks and public and private sector organization Respondents Response Conclusions Executive committee • Learnt how to manage organization, programmes and to work according to Mission rather than on donor requirements. Strategic planning helped to adopt appropriate methodologies based on VMGO￾like preparing baseline, facilitation, monitoring and evaluation. Understood the importance of aligning budgets with strategies • Staff and EC participation in management has increased; decisions are made through regular meetings. Staff selection process has improved. • Policies have been developed according to Constitution; they are helpful to better manage the projects. In ISOs there was system before SANDEEP, but that was not comprehensive and was not systematically implemented. In most PNGOs, there was no system at all. • Understood the responsibilities of staff and EC, role distributed according to JD, work is not stopped in the absence of an individual due to delegation of authority. Staff and EC commitment has increased. Subcommittees have been formed and delegated specific responsibilities based on policies and guidelines. EC and staff now can articulate issues, women EC members also can speak, and skills of articulation, comprehension and presentation have increased. • Increased linkages with district stakeholders and donors. Resource sharing with stakeholders has increased. Line agencies have started participating to monitor our activities. We have started taking feedback from donors and stakeholders to improve programme. (ISO). RM module helped to identify potential donors and also helped to find gaps in our strategies and plans for fundraising. Meeting with donors has taken place in a planned manner. We have been able to link PNGOs to donors and they have got new projects. (SAC, BASE) • We have been able to get skills training from GOs for CBOs, DPHO is helping to organize RH and first aid camps, Veterinary office is helping Project interventions were effective to bring changes in ISOs and PNGOs. OA helped to see the larger picture of the organization and helped to increase participation in the organization. It also helped increase internal coordination through regular meetings. The governance module helped to end dual roles and nepotism, clarify roles and responsibility between EC and staff through JD. OD team has been effective to address organizational issues. Subcommittees formed and generally active. Staff recruitment has been transparent. The module helped to have conceptual and technical skills for policy making, delegation of authority and distribution of workload as well as transparency and accountability. Policies helped to negotiate with donors. EC in PNGOs was reorganized several times to become inclusive. Module helped to have increased trust of community and stakeholders due to separation of roles. Strategic Management helped to understand the meaning and purpose of having clear VMGO. ISOs and PNGOs have clear alignment of goals, programmes, and structures and have also started aligning budgets and strategies. They have been able to identify gaps and develop new programmes. Strategic planning helped to increase partner confidence in dealing with donors, helped them to get new projects and increase 89 in animal raising, private sectors have also started to provide small support such as hand pumps, computer training to women and stationery for school children (TMM/PNGO). • Planning and review system has become systematic due to SANDEEP (daily plan, quarterly plan, annual plan, strategic plan). Filing, documentation and book keeping system improved. Store system improved. • Dual role separated, manager and leader are separate. Not all EC members were active before but now they give more time for NGO. General members are also participating in procurement subcommittees. (SAC). Two day retreat is organized annually with participation of staff, EC and life members for team building. Staff • Dual roles eliminated, sub-committees has been formed and functioning (OD, purchasing, recruitment etc). Financial systems developed (separate ledgers, reporting), physical verification system introduced, • Regular staff meeting takes place to discuss program and management issues. Log book of telephone and motorbike established. System of advances and supporting documents before making payments established. Cash transaction has been reduced. System of Petty cash introduced. Financial and administrative formats developed (advance, timesheets, and leave). Operational guidelines have been prepared. Administrative work has become fast. Established systems to run an organization effectively. • Strategic plans and programme wise action plans developed. Organization received strategic guidance from SANDEEP and helped increased trust with donors to fulfill the needs of communities. Resource directory was helpful to contact with donors. Work plan has become a mirror, and a tool to monitor progress. • Quality reports are submitted to stakeholders whereas other NGOs submit just give a 1 page report. PNGO has been able to implement programmes which other NGOs could not implement (PNGO/ RYC electrification project in Darchula). PNGOs can put their opinion confidently with donors and stakeholders (presentation skills). Staff team expanded due to expansion of programme by ISO and PNGO. coverage. It gave the partners a sense of their mission. Financial Management module helped to establish and review policy. ISOs and PNGOs are now providing timely and quality report to donors which have increased trust. Other achievement include, separation of ledgers, following procurement guidelines, use of bank checks, use of petty cash, timesheets use, use of log books, practice of physical verification and regular auditing. One of the most important achievement is the practice of reviewing and planning of both internal and external (donor) budgets. Improved financial policies provided feeling of security among staff, provided timely payment to staff and community, and ensured purchase of quality goods. This has contributed to the efficient use of resources and improvement of quality of service thereby, increasing donor trust. Resource mobilization module sensitized the ISOs and PNGOs to identify resources (not just funds) through collaboration with local stakeholders, not only through external donors. It also gave skills to identify resource gaps between goals and programmes, identify and contact donors in a planned way (not just waiting for donors as in the past). Due to the module, participation of ISOs and PNGOs in district networks (RHCC etc) has increased. Dialogue with private sector was also initiated. Some ISOs and PNGOs got new projects as a result of new contacts. EC became active in fundraising (not only staff or president as before). Operational Management module helped understand PPME, project cycle management Social audit has been highly effective for transparency, accountability and 90 • Strategic plans have provided direction to our actions; it has increased motivation to fulfill objectives. Policies have helped us to negotiate with donors. Social audit has helped us to develop programme through community and stakeholder feedback. EC and staff have realized that separation of roles is a good thing as it makes management easier (workload is divided). OCBO/FOCBO • Ended dual roles among EC members, regular meetings take place and are focused on agenda • RM team has been active to identify and tap new resources. Relationships increased with private sector, GO and INGOs. Publication and dissemination of brochures for marketing of NGO. Project monitoring with participation of stakeholders. ISOs and PNGOs have advocated at district forums to end dual roles. • Policies have been developed, systems developed for internal auditing, manuals developed for monitoring and evaluation, budget and plan prepared for internal income, expenditure are booked and reported to the EC • In the beginning PNGOs do not have any documents, now they have improved documentation (strategy, policies, and guidelines). Policies are implemented to some extent. VDC coverage, staff size and funding has increased. ODF • Policies and strategies established, financial system, petty cash introduced and implemented. • EC understood what NGO is and how to provide leadership; EC was reorganized several times to make it inclusive (CFWA). Commitment of EC increased, they started giving more time – ownership has increased. Rapport of NGO and relation with community increased. • Skills of staff increased in community facilitation and reporting CBM/CBPOs • 5 modules developed • guidelines for social audit and strategic planning developed • Partnership / compliance guidelines developed and implemented in SCUS. Supported AIN to develop partnership guidelines due to which 84 INGOs are implementing it. Supported NFN to develop NFN getting feedback from stakeholders to improve service delivery. Social audit widely replicated by other donors and NGOs. EC has started visiting community for oversight and community feedback. Some ISOs (NNSWA and SAC) developed M&E manuals (ISO). Social audit has greatly enhanced the rapport of ISOs and PNGOs – e.g. CFWA was the only NGO in Janakpur who conducted social audit. Human Resource Management module was recently introduced and action plan is just being developed. ISOs and PNGOs got CBNA skills and revised the personal policy once again. HRM remains a major capacity area in the future for ISOs and PNGOs as they still need skills to develop, mobilize and retain staff. 91 institutional and governance guidelines. • Authority delegation has taken place in ISOs and PNGOs, the decision making has become faster. No delay takes place in programme implementation due to EC. Beneficiaries District Stakeholders • EC has become active. Regularity in EC meeting and AGM. NGO has become inclusive in gender aspects. • NGO public image has increased. Accountability and transparency has increased due to social audit in villages. Progress reports are sent to stakeholders regularly. Stakeholders are invited in NGO events for sharing and feedback. • ISOs and PNGOs have started conducting social audit, which has given a positive message to other NGOs. • Dual roles in ISO have been ended, but there is doubt whether this is just a cosmetic change (NFN Kanchanpur). Donors • EC members have started visiting communities for monitoring and feedback. • Timely reporting to donors. Reporting format is excellent; these reports clearly reflect community achievements, learning and issues. Other NGOs just report about expenditure and demand next installment. (WAMP/ Surkhet) • ISO have become able to conduct social audit. It has also effectively facilitated PNGOs. However, in one PNGO there is excessive staff turnover. • PNGOs have become most responsive to our feedback, take feedback positively. (SC Japan). PMT staff • ISOs and PNGOs have become able to analyse gaps between their strategies and plans to develop new projects. • NGOs are continuously revising their policies – policies are taken as a living document • EC has been diversified. NGOs have diversified their funding portfolio that includes private sector • It seems that dual role separation will not reverse as staff are 92 empowered and systems have started functioning. Nepotism has been reduced. • Social audit has been replicated widely; SWC is thinking to make it mandatory for NGOs SCUS directors • ISOs have become more professional, they have policies and procedures • ISOs have more chances of sustainability than PNGOs • ISOs and PNGOs have become confident • Size of NGO is also a factor for sustainability Evaluation Objective: 4- To identify whether there are indications that these changes are resulting in improvements in the ISOs and PNGOs' service delivery Respondents Response Conclusions Executive committee • Interventions are taking place in a planned way, before SANDEEP we did not have plans. Now we practice bottom up planning. We have understood importance of participation within our organisation and therefore encouraged community participation to discuss programme issues (CFWA) • We have given high priority to coordinate with relevant stakeholders and share information with them • We have become transparent through social audit and have given continuous contacts with communities after end of project (continuity of support is one of the important indicators for quality of service) • Before SANDEEP we did not have capacity to effectively implement projects even if we had access to resources • We have learnt from SANDEEP to make participatory discussion in community before selecting target groups • We have become able to coordinate with DHO and provide public health services to community (Polio, contraceptives) • Some of the skills have been transferred to CBOs – book keeping, transparency, • ISOs and PNGOs are serious about identifying the target groups and focus programmes on them • Need assessment and community feedback through social audit, regular community meetings for planning • PNGOs are making planned intervention, there was no system of annual planning in PNGOs • Regular view meeting to identify programme issues and act on them to improve programmes • Stakeholder participation in 93 organizing meeting regularly • Before SANDEEP we visited beneficiary groups without clear purpose of visit. Now we have taken SANDEEP learning to the CBOs. CBO cooperatives have benefited from SANDEEP (eg they have established filing system, developed guidelines, JD and are becoming more transparent). • Detailed discussions are carried out after ISO gets demands from community in order to clarify community demand. Social audit is also very helpful to identify community demands. EC goes on monitoring visits to communities. Staff also carryout technical monitoring regularly. Stakeholders also visit for monitoring. Implementation has been efficient after SANDEEP systems. Participation of communities has increased due to inclusion provisions (for dalits, women and disadvantaged) in our policies and programme guidelines. • SANDEEP motivated for delegation, pre planning. Due to plans, budgets are effectively used and project coverage has increased as a result. Regular review has helped identify gaps and improve service delivery to communities. Beneficiary groups have started planning based on their income from community forests (FUGs) and have started to ask for matching fund from stakeholders based on their own plan. • Due to change in ISO policies, representatives of beneficiaries are participating in EC meetings. (ISO). Staff • Community trust has increased due to transparency of programme and budgets. Social audit conducted with stakeholders. • We have become totally community based, have advocated issues of community with donors • Implementation of activities on time, completion of action plans on time. • Linking community needs with related stakeholders and projects (sanitary pad) • NGO committed to best use of resources • EC is also providing oversight on quality of programme implementation, brings issues to the meetings for decision • Creating environment for beneficiaries to participate (school enrollment) • Inter sector and sectoral meeting in NGO • Difficult to see impact of SANDEEP on service delivery immediately, you need to continuously assess the impact • Community has been given monitoring skills. CBO members have increased mutual project monitoring • Timely completion of projects • Quality goods delivered to the communities due to improved financial management • More time needed to see direct impact on service delivery • Expansion of program by ISOs and PNGOs – coverage has increased. 94 trust and attend meetings regularly. Members of CBOs repay loans regularly. CBOs have started to organise meetings on agendas they decide and make decisions in a participatory way. OCBO/FOCBO • Planned action by NGO, timely delivery of projects and services. • Obtaining quotation, provision of quality goods to communities, • Time management has helped improve quality of service. • Transfer of funds from one project to another has stopped, due to which work is completed on time. • Social audit has increased community participation in planning and monitoring. VDCs have started funding NGO projects due to increased community participation. ODF CBM/CBPOs • We do not monitor improvement in service delivery of NGOs as the indicators have not been developed. We depend on donors’ feedback on NGO programme performance through the donor questionnaires on annual basis. • NGOs have started practice of community level review of projects periodically • Budget is not returned to donors due to planned work by NGOs and timely review. Beneficiaries District Stakeholders • Some of the skills have been transferred to CBO e.g. participation, transparency and accountability • Continuous follow up in communities by NGO Donors • ISO capacity to mediate disputes in communities has increased. During conflict time Maoists use to solve disputes (over water sources), now ISO solves such issues. • Waste water management is an indicator of quality service for community water schemes. • Participatory needs assessment is done by ISO, these needs are further approved by VDC assembly and forwarded to donors for funding (coordination with VDC has increased). • One of the important indicators for quality of service is whether the User Group sustains after the project is completed. Generally after completion of projects, the UC becomes inactive; some of the leaders resign or leave the community. • Other indicators for quality of service could be: achievement of project targets, problem solving capacity of NGO, and timely completion of projects. 95 PMT staff • NGO coverage has increased • Social audit has increased community participation • ISOs and PNGOs have developed specific TG focused strategies • ISOs and PNGOs make decisions on time; project implementation is done on time. SCUS directors • ISOs and PNGOs have stronger sense of their mission – moving away from project orientation • Dealing with other donors and expanding coverage • Social audit has been a good tool and indicator for improvement of quality of service Evaluation Objective: 5- To identify what effect and impact that the project has had on the other stakeholders (other NGOs, NGO networks, donors, AIN and other government organizations which were not direct beneficiaries of the project) Respondents Response Conclusions Executive committee • Many stakeholders attended out social audit and have given feedback. This has given pressure to other NGOs to be transparent. We have become role model as we don’t take salaries or other benefits from NGO. • Our contacts and relationships with other related NGOs, donors and GO has increased, we are getting more support from them on programme issues (polio eradication, school enrollment, child labour, sharing learning) • Private sector could not support us due to continuous strikes and intimidation during conflict. • Other NGOs demand coaching on modules from us on governance and policy development. Donors have started valuing good governance practices. NGOs feel that, as soon as funds come from donors, it creates conflict within organization. Therefore, they want to learn from SANDEEP how to avoid future conflict in their organization. • SANDEEP learning has been shared with other NGOs through FEDEN and NFN. ID forum was established where sharing took place on quarterly basis. But now this forum is inactive. (Kanchanpur) • Local NGO demand for support to establish systems has increased on ISOs. • SANDEEP approach seems to be more ‘ecological’, ISO and PNGO seen as role models due to social audits with participation of donors and stakeholders. • pressure on other NGOs to become transparent • District NGOs are curious to learn due to heightened image and rapport of ISOs and especially PNGOs achieved in short period of time. • District NGOs have demanded coaching on policy, systems and training on modules (governance, strategic management, social audit) 96 However, the is no uniformity among donors, there are some donors, who appreciate dual roles. Some donors like Save the Children Norway, CARE who have large programmes have shown more interest in SANDEEP learning. Other donors are not interested in OD; they want to use NGOs as an extra hand for social mobilization. These donors even do purchasing themselves. (ISO). • SANDEEP/ SCUS should continue organizing interface between ISOs and donors/AIN. This helps develop linkages with other donors. • SANDEEP has tabled the issues of dual roles, need for second line leadership and nepotism on the NGO sector. NGOs are now improving their staff hiring procedures. ISO is sharing SANDEEP learning in NGO networks. Learning was also shared with 20 CECI partners together with PNGO. Sharing also took place with SCN partners to support them for making policies and strategies. Staff • We have become able to advocate about the advantages and disadvantages of dual role with other stakeholders • Annual reports shared with VDC, NGOs, DDC and donors • Participation in networks has increased due after SANDEEP (RHCC, District Development Fund) • Other NGOs visit PNGO for a copy of organizational policy, they want to learn the filing systems. They also ask for different formats, copy of constitution and operational guidelines. • “The attitude of stakeholders towards us has changed after the success of our electrification project that gave light to 110 households” – Darchula. • We send monthly reports to DDC, now they have started to call us in meetings. However, they still feel that they are the boss; they still do not see us as partners. OCBO/FOCBO ?? ODF ?? CBM/CBPOs • Have conducted joint training to PAF, World vision and CECI partners on Governance, Strategic Planning and policy development. They have replicated these modules. • Training has also been conducted with SWC and they have highly appreciated it. There is possibility for policy change of SWC due to SANDEEP learning. Beneficiaries District • NGO has organized sharing form together with NFN, but it is still spontaneous • Sharing with NGO sector not regular due to political division of NGO sector • Private sector is skeptical about NGOs, they believe NGOs get lot of funding from donors • Diverse attitude and practice of donors regarding OD • GOs have appreciated ISOs and PNGOs but they still don’t see them as allies (partners) • District stakeholders and donors have appreciated reporting skills of ISOs and PNGOs. • Some district NGOs are improving their staff hiring procedure due to pressure created by SANDEEP • SANDEEP partners regularly share their progress reports to DDC, line agencies • Some donors are interested in sandeep module training to their partners – PAF, CECI etc • SWC highly positive about SANDEEP practice – policy implications for sector • Many AIN members are implementing common partnership policy – Task force created for facilitation of the process. • Common partnership policy in SC alliance --- sharing between 97 Stakeholders Donors PNGO has greatly enhanced their coordination and rapport with stakeholders. PMT staff • Worked with AIN for uniformity in partnership policies, AIN members have also realized that they also need good governance. AIN has formed task force on partnership policy and capacity building • PMT worked with NFN for good governance act of NGOs, dialogue with bilateral agencies has taken place to replicate SANDEEP learning (Bilateral agencies have a policy which doesn’t require separation of dual roles for grants less than USD 100,000.00) • SWC has been highly influenced by SANDEEP learning, they are willing to publish the modules jointly. • Sharing of social audit, modules have taken place within save the children alliance. Discussion is going on to create a OD cell to institutionalize learning • SANDEEP message has reached to almost all INGOs in Nepal • SANDEEP learning is shared with SCUS regional office and Headquarters, they have huge interests on the project outcome • OD has been incorporated in the new SCUS Strategic plans (partnership guidelines) partners • Dissemination of SANDEEP learning through NFN • ISOs and PNGOs have not been able to meet the demand from NGO sector. ISOs more interested in delivery projects because they see easy money coming from donors to implement projects. • Lack of sufficient inputs from Project on IR- 2 and 3 • Absence of peoples representatives in local bodies • Some donors don’t want to work in alliance – fear of loosing partner • Donors practicing wait and see in the current political climate. Evaluation Objective: 6- To identify how the conflict situation in Nepal is affecting ISOs and PNGOs, and whether the project responded appropriately to mitigate these effects. Respondents Response Conclusions Executive committee • Do not hide anything from stakeholders – social audit – helps mitigate conflict • During conflict, we locked the main door and conducted meeting inside • Clarify mutual expectation with conflicting parties – they will not abduct you. • During conflict, only those organizations suffered who were not transparent. For us, sometimes we faced difficulty in transportation. • SANDEEP hypothesis is correct and relevant. The more participation of community is insured, the more conflict is reduced. The root cause of conflict is domination of • Conflict management training helped the ISOs and PNGOs of the first phase to develop confidence to make dialogue with stakeholders and continued working in conflict situation. • ISOs and PNGOs adopted different strategies to work in 98 community and society by individuals. In order to reduce conflict, we need systems (openness and accountability). (ISO). Staff • Communities were terrified during Maoist conflict, but the programmes were not postponed. NGO hired local staff. The most effective methodology for working in conflict is to provide orientation to community on what the NGO wants to do, why, for whom and when. During Madhesh movement, we don’t challenge their call for strike (bandh) and use bicycle instead motorbike in. There is no problem for supervision at community level. • NGO solidarity and joint press release for Zone of Peace helped to reduce tensions in community • Health activities (polio, Vitamin A) are the best medium to have access in conflict situations • Project area has increased due to donor coordination – more donor trust, skills of EC and staff increased, systems established in NGO • Donors don’t return our reports and proposals., staff and board members have become articulate • Coordination with other NGOs increased due to SANDEEP • Separation and control in financial systems developed • Inter project coordination meeting in NGO • Reporting skills of staff increased • Stakeholders can get help even when key staff are away from office • During the conflict, we were successful to unite the community, due to which no one could intimidate them. OD taught us to become transparent and united. Politicians and government service holders speak sweet language, but they try to create confusion among community during conflict. • In the beginning of conflict it was very difficult to enter community, as they were terrified. The NGO workers need to take permission from Maobadi. Later we shared our programme and budgets with them and our local staff also helped to clarity expectations. After this they gave permission for NGO to work in community. However, we were cautious not to tell them that we were using US grants. • During conflict, do not select VDCs near army barracks. Keep low profile. Even don’t polish your finger nails. Make agreements with community (MOU). OCBO/FOCBO • Due to increased participation of community, the community level conflict has conflict situation: rescheduling of training, maintaining neutrality, relocation of office, delegate monitoring responsibility to community, hiring local staff, sharing fact sheets, policies and budgets with stakeholders, signing MOU with community etc. • SANDEEP emphasis on – transparency, accountability highly relevant in conflict situation. • Minimal impact on project outcomes due to project inputs to cope with conflict situation 99 decreased. However, SANDEEP so far doesn’t have impact on the root cause of national level conflict. Conflict within organization has reduced. • During conflict, we took laptops at home and worked there. Kept low profile, used rickshaw or bicycles. • In order to reduce conflict, donors need to increase funding projects in eastern Terai, otherwise it might be too late. • To effectively work in conflict situations, all donors and INGOs should practice OD principles. • SANDEEP hypothesis related to conflict mitigation was relevant during conflict, but in post conflict situation, the hypothesis needs revision. • During conflict many training were postponed to a later date due to strikes. Our field visit was also reduced. Field offices were moved to central office. However projects were not closed down. We successfully perused the following strategies: don’t tell anyone about the funding source (USAID), keep low profile (don’t use vehicle), make contacts with both sides but keep neutrality, answer the questions in short, don’t give information that is not being asked for. Sometimes, planning was done at district headquarters by inviting CBO leaders. Monitoring was done mostly by field staff, frequency of office staff visiting field was reduced. • SANDEEP prepared the ISO to be prepared for conflict by being more transparent in sharing information and hiring staff. Fact sheets and policy documents were shared with stakeholders. Communities were provided relevant information on projects. This helped mitigate the conflict. In some cases, Maobadi were driven away from village by the communities. ODF • During conflict we delegated some of our functions to local clubs and committees for selection of target groups. After local involvement in TG selection, conflict was reduced. CBM/CBPOs Beneficiaries District Stakeholders • Intimidation and extortion of government agencies is continuing in Madhesh, however, they have not targeted NGOs due to transparency. NGOs have also invited the conflict parties for negotiation. Project implementation is slow which is due to strikes. 100 Donors • To work effectively during conflict, insert a component for dalit and women empowerment in your project activities. PMT staff • Governance module came on the right time when the NGOs were facing challenges of conflict – this helped them through transparency, prepared code of conduct, some NGOs even handed over their policy papers to Maoist rebels. MOU template helped NGOs a lot. • Hiring local staff was another successful strategy to cope with conflict. • Computer backup, pen-drive support was another coping strategy. • First phase ISOs and PNGOs all received training on conflict management that was helpful • Strategies were developed for working in conflict (strategic plans) • Some NGOs organized meeting with participation of staff, EC, stakeholders with Maobadi • Centralization of structure and developed communication strategy • ID cards were issued for security • Do not name your donor (especially US grant) • Do No harm training • Sharing experiences of conflict in Review and reflection workshops SCUS directors • Empowerment of NGOs and communities in 1980s and 90s led to conflict because the marginalized wanted their rights to be fulfilled • Building civil society at grassroots level is important – that helps politicians be accountable to local people and thus helps reduction of conflict. • Strengthening NGOs is an important part of strengthening civil society. Evaluation Objective: 7- To evaluate the viability of the sustainability of capacities developed within SC/US and Partners Respondents Response Conclusions Executive • ISO approach has been effective • All ISOs/PNGOs commitment for continued 101 committee • We are hopeful to get continuous support from donor, also have submitted proposal to PAF, this might help to retain ODF and other key staff. • Performance of ODF is excellent, he conducted training to other organizations as well • OCBOs are capable to help solve issues by taking them positively • We have developed a road map for sustaining the benefits of SANDEEP. We will give continuity to conduct social audit, strategy review, policy review, JD review and organization assessment on our own. We now have resource persons who can facilitate implementation of these activities. Staff • Direct support to PNGOs might have been effective as it takes time to outreach through ISOs and also increases cost • ISO taught us modules, but did not share their learning. Inter EC idea sharing has not taken place • Not much motivation is taking place from ISO to PNGO, • OD team merged with RM committee to combine internal and external resources • Modules provide organizational basics which are always needed • EC has become capable; it can give continuity to the systems established in spite of staff turnover. • ISO approach has both positive and negative aspects. Without ISOs, PNGOs could have received more direct funding and more training and full knowledge of modules. But if trainers come from outside they might not understand the local context as ISO does. ISO understands local context and language and their facilitation is easy for us to comprehend. • OD team meets once a month and gives its recommendations to EC for decision. OD team members bring agendas for meeting and wait impatiently for discussion in meeting to address issues. This is an indication for sustainability of OD team. OCBO/FOCBO There were occasional contacts between ISO and PNGOs before SANDEEP, but now they have started submitting joint proposals have conducted joint training (PAF). application of the established OD policies, systems and processes (Organizational Development (OD) Team along with Organizational Assessment (OA) and Strategic Planning and Management Processes) • ISOs confident of retaining OCBOs and FCBOs in a suitable position in their organization despite financial constraints • PNGOs conveyed commitment to and realization of the benefit of retaining the ODFs but financial constraints make this primarily contingent on the receipt of new projects. • Sustainability of SANDEEP is supported by the internal review and implementation system initiated and established by the project. • Pool of OD based human resources have been developed to carry on the SANDEEP learning and experiences. • The OD team of ISOs and PNGOs has strong support and involvement of EC members. • NGO Federation, donors, AIN members and the Social Welfare Council (SWC) show positive attitude for further collaboration • Sustainability of SANDEEP initiatives will primarily depend on the level of change in the attitude of NGO leaders • EC members, especially new ones inducted to accommodate mandatory SANDEEP’s provisions (inclusion and avoidance of dual role) still lack OD knowledge and skills • Another area of concern is the ability and willingness in terms of availability of time, resources and commitment by EC members who have to work voluntarily without direct 102 ODF • ODF has to work hard to motivate EC, staffs are occupied with project work, ODF has to coordinate all, which is a challenging task. ODF is overwhelmed with work. I changed myself very much, now I can help and coordinate everyone. • It was challenging to transform a CBO like organization into an NGO. Due to lack of academic qualification of EC members, ODF had to work hard to make them understand the content of modules. Project period was short keeping in mind the level of EC members. • If PMT worked directly with PNGO might have been better. If ISO and PNGO work in same district, they become competitors and ISO doesn’t provide support. External ISO (advisor) might have been better (like in SNV in Ilam). ISOs just transferred bookish knowledge, it could not add value to the modules due to lack of expertise. • ODF role for supporting policy making is good. However, ODF doesn’t have resource tapping skills. There is confusion about ODF role extension due to lack of resources after SANDEEP phase-out • I have worked with PNGO as volunteer for five years before SANDEEP. I will continue working as volunteer after SANDEEP phase-out with the hope that new projects will be coming in the future. CBM/CBPOs Beneficiaries District Stakeholders Donors PMT staff financial benefit. • Contingent on availability of projects (Smaller PNGOs inadequate linkage with donors and negligible access to local resources, especially the private sector) • Smaller PNGOs not been able to adequately practice many of the policies and systems due to short time span (need time and follow up support to reinforce new practice into habits) Evaluation Objective: 8- To assess the unintended result of the project Respondents Response Conclusions Executive committee • Some donors want to dictate their “partner” NGOs. As the NGOs are empowered, donors feel they are loosing their power, because in SANDEEP, the needs of communities and the strategies of NGO to Reduced budget for fifth year affected the planned activities related to the HRM and OM modules. 103 address these needs that counts. Donors have become afraid of NGOs policy and systems. Staff • Skilled and educated members of EC have now become staff. New people have been elected in EC who have not received training on modules. OCBO/FOCBO • Administration of HRM and OM modules was rather rushed and fell short of the required level of inputs. • PNGOs can be de-motivated as they have experienced losing out to other NGOs that they consider less endowed in terms of SANDEEP’s IDF indicators. ODF • CBM/CBPOs Beneficiaries District Stakeholders New EC members, majority of whom have not been exposed to the Project’s inputs, are primarily passive contributors while the control of the previous leadership (especially in most ISOs and larger PNGOs) remains intact. Donors PMT staff The Project’s approved budget for the fifth year was significantly reduced affected the planned activities related to the HRM and OM modules. Administration of HRM and OM modules was rather rushed and fell short of the required level of inputs. Some smaller PNGOs felt assured that just being SANDEEP’s beneficiary would by itself open up linkages to donor agencies PNGOs can be de-motivated as they have experience losing out to other NGOs that they consider less endowed in terms of OD. Considerable time and effort was spent on convincing EC members and ushering attitude change. New EC members who did not get training or coaching, are passive contributors while the control of the previous leadership (especially in most ISOs and larger PNGOs) remains intact. Evaluation Objective: 9- To identify the challenges/lesson learned during the project period Respondents Response Conclusions Executive committee • PNGOs face the challenge to establish link with government and donors at central level • How to establish contacts with SCUS after phase out of SANDEEP? • Continuous practice of five modules makes organization sustainable. • We have learnt a lot, but sometimes feel whether we can give continuity to all of them as they have not been sufficiently practiced. • We need to develop staff that is capable to prepare quality grant proposals. Now we Challenges: • Develop project strategy, modules and train HR in short period of time • Less time for OM and HRM module, action plans not fully 104 hire consultant in Kathmandu. • Donors give priority to joint proposals rather than directly from PNGO • In one ISO, EC member turnover is high, now they don’t have people who have received SANDEEP training. • The notion that EC has no work and the President and PD are responsible for organization is changing. We have learnt that EC has some important functions for coordination and linkage. We have also learnt that in order to improve linkages with stakeholders, just meeting them is not enough, we need to give them reports and documents. • Partner selection criteria of many donors are not transparent. Personal contacts and political standing of NGO play a decisive role in getting projects from donors. Some donors have regretted their decision later for selecting wrong partners based on political influence. • Before SANDEEP, there was general feeling among NGOs that they should keep information about their projects, organizational policies and donors in secret. Now, there is a feeling that we should not hide such information and share it openly. • There are some NGOs who work on good governance issues, but they have practice of dual roles. • After 4 years of practice, governance and finance modules have been much practiced. OM and HRM modules were recently introduced and we have not got time to practice them fully. We feel that OM is a module that will bring results slowly. • Same module is not suitable for ISOs and PNGOs as they have different size and coverage. Facilitating and making action plan on modules was a challenge in small organizations. Even ISOs have not been able to fulfill some indicators like single chart of account for all donors as donors use different software and formats. (ISO). • We have learnt that implementing specific programmes in specific locations as per community needs improves quality service. • Cosmetic changes are seen in some ISOs. If the leaders of ISOs have not realized the need for change from within, these changes will not last long. (ISO). • We failed in mobilizing resources from private sector. SANDEEP input was shallow in this regard. Private sector organizations outright reject our request for collaboration. Private sector is not strong in hilly districts. Staff • SANDEEP needs to develop a module in proposal development and provide training implemented • Some PNGOs could not practice their new systems and policies due to lack of projects • Private sector skeptical to work with NGOs • ISOs not always being role models for PNGOs • Budget cut from USAID for last year of project • Measuring quality of service • Donor coordination in Kathmandu level Lessons: • ISO and module approach is the right way to go if you have large coverage and diverse partners • Time commitment is important • One size don’t fit all • Constant follow-up, coaching on action plans. • Continuous orientation of staff and EC on policies to reinforce change • It is difficult to change those organizations in which the organizational culture and hierarchy has already been set. • Leadership attitude is decisive in OD process. • For quality of service, specialization is needed. 105 and follow-up. A module in social mobilization will be helpful. PNGOs need skills in case study and M&E. Charts of accounts needs to be taught to PNGOs. If SANDEEP provided delivery project, might have been easier to show its impact. Until PNGOs get new projects from other donors, SANDEEP needs to provide some form of support however small • If the needs of community are analyzed in depth and with their participation, the impact of project will be better. • Once you install system in your organization, it will be difficult in the beginning, but later it will be easier to manage the organization later. • Challenges in mobilizing communities due to diverse policies and approaches of NGOs and donors • There is a need for constant orientation of staff and EC on policy provisions so that policies are internalized in full. • Staff turnover is a major challenge in ISOs and PNGOs. • Only some selected staff get training of SANDEEP all the time. • In future we need skills on English language, Proposal development skills, and facilitation skills. SANDEEP needs to develop modules on these themes. • Though dual role is separated, still the staffs are dominant over the EC. • Resource mobilization is the biggest challenge for NGOs in remote locations. The community here is very backward, they don’t trust development workers, and they think NGOs work for themselves not for community. • “OD is like taking weeds out of the fields. Once you take out some weeds, other will grow later. It is a continuous process”. • Delegation of responsibilities is more important than ending dual roles. • ISOs learn the modules (TOT) for 9 days; PNGOs are given training for 3 days only. PNGOs need more time to practice and institutionalize the learning of SANDEEP modules OCBO/FOCBO • PNGOs have yet to implement some of the modules like OM, RM. They have developed policies, but needs follow-up for some time to see results. • Dual role has not been eliminated in ISO constitution. ISO is continually postponing GA (SCDC) in order to avoid such change (e.g. SCDC). • OCBOs are given other office work which the leaders are expected to perform therefore, OCBO is overstretched. Sometimes the OCBO has to perform the 106 function of CBPO to sensitize the EC of ISO. • ISOs could not become role models for PNGOs (dual roles). CBPOs and PMT could not give enough efforts to motivate ISO leadership for change in dual roles in the constitution (e.g. SCDC). • Some PNGOs are far ahead than others, so the time needed for coaching is different. Most PNGOs are lagging behind in IR 2-3. • RM module is difficult to implement, difficulty in diversifying resources, difficulty in meeting the high demand of communities. PNGO is covering just one village in each VDC due to lack of resources. • It is highly challenging to adjust modules in diverse organizations. If there will be no second phase of SANDEEP, how will be our feedback be implemented? • It is difficult to change those organizations in which the organizational culture has already been set. Some of the ISOs provided negative examples, which discouraged PNGOs to separate dual roles in the beginning. • Managing staff time for sharing of modules was difficult. Governance module took bulk of the time, whereas other modules were conducted in haste (OM, HRM). Time was short for planning and action for later modules. There was not enough time for IR-II and IR-III activities. • The staff of PNGOs have not received salaries as mentioned in policies. Policies need to be tailored to PNGOs. PNGOs are facing difficulty in getting professional experts demanded by IDF. SANDEEP input to PNGOs to work with private sector was shallow. There is no indicator to measure SANDEEP hypothesis. • Finance module was the easiest to implement as it was more technical and provided visible output to PNGOs. Governance module was highly appreciated during training but later it became difficult to actually separate roles • PNGOs cannot identify learning and best practices of their work. They need external experts to help facilitate in this important work. • In PNGOs a single person has to participate in various subcommittees, where as in ISOs different people are engaged in different subcommittees. Hence number of subcommittees in PNGOs should be less. • Some ISOs have not incorporated dual role separation clauses in their constitution. OM is also a very challenging module as it demands bottom up planning, monitoring and evaluation. But for this, cooperation of donors is needed. Donors have their own perception of community needs which may not match with the real needs of 107 communities. ODF • How does PMT monitor quality of ISO-PNGO relationships? What are the indicators? • If there are no projects, how can PNGO practice SANDEEP learning? Those organizations who have few and small projects, the practice of IDF is slow and it becomes a theoretical exercise. • Functioning of OD team depends on quality of human resources. OD team approach might be effective in those organizations, where there is much workload. Most members of PNGO are peasants, the educated ones are out of the villages, they cannot contribute. • In some PNGOs, the educational qualification of EC is very low, they cannot understand management concepts. • ISOs could not add much value. It just became a medium for conveying issues and feedback between PMT and PNGOs. Therefore, in future, PMT needs to work directly with PNGOs. • Subcommittees were reduced from 7 to 3 in PNGO. CBM/CBPOs • Outreach structure (PMT-ISO-PNGO) is too lengthy, layered; ISOs had to provide training on modules before they digested the issues themselves. Not enough time for ISOs to practice the modules before they train PNGOs. • Same modules applied to all types and sizes of NGOs. If an NGO has 3 staff, does it need HRD policy? • Senior management team is not feasible in small NGOs. Some NGOs formed 10 subcommittees initially, which became dysfunctional. Later they were merged into three subcommittees. • Due to lack of indicators, project goal is difficult to measure. • Sustaining changes in ISOs and PNGOs is a challenge. The changes have not been institutionalized yet, not transformed into habits. More practice is needed especially PNGOs. Beneficiaries District Stakeholders • Individuals have contributed to the organization voluntarily and they have control over the organization, which is natural. As staff get salaries, EC members also want to become staff. This has been the practice in NGOs so far, which cannot be changed in short period of time. No one can work as volunteer for extended period 108 of time. Therefore dual role is a compulsion. Now due to AIN policies, NGOs are looking for retired government officials to join their EC. (NFN, Dhanusha). • SANDEEP policy on dual role separation has been very harsh; it could have been more flexible and slow process. SANDEEP needs to appreciate the NGO context first. • ISOs and PNGOs could not market SANDEEP properly. Many stakeholders understand that the project is just about ending dual roles. SANDEEP could not come out of the ISO and PNGO circles. There was no mechanism for participation of stakeholders in SANDEEP. SANDEEP achievements, issues and learning were not properly publicized or disseminated. NFN has its own good governance network, SANDEEP partners could have used this forum for sharing, but no one took initiatives. In the beginning, PMT was very positive to work with N/FN, but later ISOs and PNGOs did not give much importance to such collaboration. (NFN Kanchanpur) • Though dual role has been ended in paper, it seems that the staffs are dominant in SANDEEP partners. Donors • Donors cannot get information in the absence of key staff. Sharing and coordination within NGO needs to be improved. • NGOs need specialization on certain sectors, define their Mission more clearly and make it focused (is NGO for social mobilization or should it be involved in technical aspects as well, hire engineers?) PMT staff • Some ISOs have not incorporated dual role separation in their constitution. • Some INGOs are still resisting implementing AIN partnership policies, they think that it is over ambitious. Some INGOs don’t have focus on capacity building of partners; they also don’t have adequate knowledge on the concept of OD. They think that capacity building is a non issue for them. Some INGOs face difficulty to apply partnership guidelines with their existing partners due to partner pressure • Change in attitude and behavior of leadership is important. One shot support is not enough for success of OD. Lots of time and coordination is needed. Now we have tools and methodologies and systematic experience for sharing the learning. It took time (1-1½ years) to internalize the concept of SANDEEP by ISOs and PNGOs • SANDEEP had to work on reduced budget during the last year and some of the staff 109 have to be out placed. • Difficulty to bring all the donors of ISOs and PNGOs (15 donors/INGOs) for coordination at national level. There is a challenge to bring uniformity in donors and sectors. Challenge to sensitize donors and INGOs • There is still a challenge to bring change in NGO culture • The partnership of NGO with private sector has not yet achieved. SCUS directors • OD builds up slowly, length of commitment is important • Follow up support is important in OD • You cannot use same module for diverse organizations – one size don’t fit all Evaluation Objective: 10- To identify specific OD issues and needs of PNGOs in the health, education and livelihood sectors and make appropriate recommendations for replication and improvement of prevailing OD practices applied by SANDEEP as well as other donor agencies for effective service delivery by their respective partner organizations working in these sectors. Respondents Response Conclusions Executive committee • Social entrepreneurship – we get contraceptives for Rs 15/- and sell to community for Rs. 20/-. We have published health bulletin in local language which has helped increase awareness in public health and RH. Community has demanded income generation activities as we motivated them to send their children to school and they need more income • In future staffs need skills in follow-up and monitoring. SANDEEP should help develop skills of PNGOs in documentation, HR skills so that we don’t need to hire external consultants. Programme development, linkage skills development should be given priority. We could not update our website. Staff • In order to increase quality of service, NGOs should be provided with integrated programmes a single project (eg drinking water) cannot address the community needs effectively. • PNGOs need TOT on Modules, proposal writing. OCBO/FOCBO • Even after phase-out, there should be mechanism for follow-up of SANDEEP. Continue support to the ISOs and the PNGOs in terms of information, follow-up and linkage after the project phase-out. Provide support for biannual review and reflection sessions and reporting. Provide follow up and monitoring of the Road Map. Provide linkage support to ISOs and PNGOs with SC Alliance and AIN members. Provision of adequate annual budget for the implementation of these activities through the respective ISOs. SANDDEP OD Modules should be published and made available for 110 PNGOs have yet to get used to in applying policies and systems. • Other INGOs need to sincerely apply partnership guidelines like SCUS. • SANDEEP modules are good for new NGOs. But for existing ISOs and PNGOs, service delivery and OD project should go hand in hand. There is a challenge to monitor progress in improvement of quality of service. Therefore, same donor needs to support service delivery project as well to see the link between OD and service delivery. ODF CBM/CBPOs Beneficiaries District Stakeholders • Donors have not given priority to NGO capacity building so far. Providing training is not enough, physical and equipment support should also be provided by donors. (SIMI) Donors PMT staff • Best practices developed by SANDEEP: Review and reflection, Coaching and follow-up, SANDEEP approach (module-training-coaching-follow-up), OA, Social audit, Partnership policy/ Compliance guidelines • May be in future, we could develop a module how to link OD with service delivery • There might also be a need for Technical Modules (HIV, Education etc) distribution at subsidized rates through the NGO Federation and/or other appropriate channels. . Support SANDEEP initiatives with actual project implementation and closely monitor service delivery through specific criteria and indicators. Supplement the SANDEEP Module with sections on social mobilization and proposal development which fit in OM and RM modules respectively. … Evaluation Objective: 11- To recommend specific advocacy measures targeting potential donor agencies who can support SANDEEP’s second phase design and implementation Respondents Response Conclusions Executive committee • It is good that projects like PAF, CECI have given priority to select SANDEEP partners. Their experience with ISOs and PNGOs could be marketed with other INGOs. • NGOs are labeled as “dollar cultivation”. Donors need to support NGO capacity building to eliminate this myth. Donors need to understand that due to lack of NGO capacity, the programmes have not reached the target groups. Therefore, donors need to invest in capacity building together with programme funding. Just giving funds to NGOs will not help improve quality of service. Develop and implement a Second Phase SANDEEP project, in collaboration with the SC Alliance, AIN and the SWC, as there is more to be done in this field. Provide extensive dissemination of SANDEEP learning and experience targeting donors, NGO networks, 111 Staff OCBO/FOCBO • AIN partnership guidelines should be honestly implemented. • Some INGOs even don’t have indicators to measure impact of their work. They need to learn from SANDEEP. ODF • SANDEEP needs to advocate INGOs and donors on the need of policies and systems in their partner NGOs. • It is difficult to work with private sector at the moment in Madhesh due to ongoing conflict. Some donors are interested in OD (CARE) but others are not (WE). Many INGOs don’t give priority to separation of dual role, availability of policies/ systems and lack of nepotism in NGO partners CBM/CBPOs Beneficiaries District Stakeholders • SANDEEP experiences should be shared and marketed through INGOs. Donors PMT staff • Incorporate SANDEEP learning in AIN strategic plans • Lobby the AIN – bilateral negotiations • Sensitize NFN to demand 10% overhead from donors for OD • Continuous dialogue with SWC to incorporate SANDEEP learning in their policies representatives of relevant government agencies, private sector and OD experts. Advocacy strategy for policy change in INGOs and bilateral and multilateral development in favor of support to OD processes in civil society organization and funding specific OD focused projects to be formulated and implemented. SCUS regional offices to be activated and take up ownership to carry forward the SANDEEP initiative and coordinate with donors at the regional level. Consult and lobby potential donors (such as CARE, CECI, PAF, other AIN members demonstrating strong OD support, DANIDA, SC Alliance) for full or partial course on SANDEEP to train their partners for a period ranging from one to two years. 112 ANNEX 9: List of ISOs/PNGOs SN ISO Name District Region Caste/Ethnicit y 1 Srijana Community Development Centre (SCDC) Siraha ER Newar 2 Social Awareness Center (SAC) Surkhet MWR Mixed 3 Backward Society Education (BASE) Dang MWR Tharu 4 Nepal Red Cross Society (NRCS)-Kailali Kailali FWR Mixed 5 Nepal National Social Welfare Association (NNSWA) Kanchanpur FWR Dalit No. Name of ISO Name of NGO District Caste/ Ethnic Remarks 6 SCDC Shiva Shakti Youth Club (SSYC) Siraha Mixed/Dalit 7 SCDC Nepal Rishikul Saday Kalyan Samaj (NRSKS) Siraha Dalit 8 SCDC Janachetana Dalit Sangam (JDS) Saptari Dalit 9 SCDC Sripurraj Samudayik Bikah Kendra (SCDC) Saptari Tharu 10 SCDC Community Family Welfare Awareness (CFWA) Dhanusha Mixed 11 SAC Suriya Samajik Sewa Sang (4S) Jumla Mixed 12 SAC Samadik Swasta Tatha Bikas (CHD) Jumla Mixed 13 SAC Himali Ekriti Bikas (HEB) Jumla Dalit 14 SAC Samajik Sewa Kendra (SOSEC) Dailekh Mixed 15 SAC Gramin Mahila Utthan Kendra (RWRC) Dailekh Mixed Terminated 16 SAC Mahila Jagaran Munch (MJM) Surkhet Mixed Terminated 17 SAC Suppressed Development Centre (SDC) Surkhet Dalit 18 BASE Kamaiya Pratha Unmulan Samaj (KPUS) Kailali Tharu 19 BASE Nepal Rastriya Dalit Samaj Kalyan Sanstha (NNDSWO) Kailali Dalit 20 BASE Gramin Mahila Utthan Kendra (GMUK) Dang Tharu 21 BASE Gramin Mahila Bikas Sanstha (GMBS) Dang Tharu Terminated 22 BASE UNESCO Banke Muslim 23 BASE Tharu Samudaya Bikas Manch (TCDF) Bardiya Tharu 24 BASE Nepal Rastriya Dalit Janjati Mahila Utthan Sangh (NNDEWUA) Bardiya Dalit Terminated 25 BASE Tharu Mahila Utthan Kendra (TWUC) Bardiya Tharu 26 BASE Dalit Bikas Samaj (DBSA) Salyan Dalit 27 BASE Samatamulak Samajko Srijana (CES) Salyan Dalit 28 NRCS Nepal Red Cross Society (NRCS)-Achham Achham Brahmin/Chh etri 29 NRCS Utpidit Janajati Tatha Mahila Sasetan Kendra (OCWAC) Achham Brahmin/Chh etri 113 30 NRCS Nepal Red Cross Society (NRCS)-Doti Doti Brahmin/Chh etri 31 NRCS Nepal Red Cross Society (NRCS)-Dadeldhura Dadeldhura Brahmin/Chh etri 32 NRCS Samudaya Bikas Kendra (CDC) Doti Brahmin/Chh etri 33 NNSWA Rajakot Yuba Club (RYC) Darchula Bhotiya 34 NNSWA Markande Bikash Samaj (MBS) Darchula Bhotiya 35 NNSWA Gramin Bikas Yabam Batabaran Byabasthapan Samaj (GBYBBS) Baitadi Mixed 36 NNSWA Basuling Jana Jagriti Samaj (BJJS) Baitadi Mixed Terminated 37 NNSWA Mahila Mukti Samaj (WDS) Dadeldhura Mixed 38 NNSWA Ekikrit Bikas Samaj (IDeS) Dadeldhura Mixed 39 NNSWA Nepal Paryabaran Tatha Shikha Bikas Samaj (NEEDS) Kanchanpur Brahmin/Chh etri 40 NNSWA Tharu Mahila Manch (TWOF) Kanchanpur Tharu 114 ANNEX - 10 Final Evaluation Sample ISOs and PNGOs, OA and Final Evaluation Dates Name of ISO Location Led by Donor of monitoring program Evaluation date OA date NNSWA Kanchanpu r Dalit and Tharu SC/US 25-26 May 13 – 16 May SAC Surkhet Mixed SC/N 7-8 June 3-6 June BASE Dang Tharu World Education 6 – 9 June 6-9 June The consultants will interact with other two ISOs (SCDC and NRCS) for one day though they are not in sample. SCDC 17 May NRCS 1-2 June CFWA supported by Srijana Janakpur Terai (Madhesi) World education 15-16 May 12-14 May RYC (Supported by NNSWA) Darchula Sauka (Ethnic) PAF 22- 23 May 12-14 April TMM (Supported by NNSWA) Kanchanpur Tharu woman SC/US 27- 28 May 22-24 April NRCS Dadekdhura (Supported by NRCS Kailali) Dadeldhura Mix NRCS center 29-30 May 4-6 May UNESCO Banke (Supported by BASE) Banke Muslim World Education 12-13 June 2-4 May SDC (supported by SAC) Surkhet Dalit CECI 8-9 June 15-17 April KPUS (supported by BASE) Kailali Tharu LWF 12-13 June 15-17 April Criteria for selecting sample: (Geography, leadership, size of budget and staff, program coverage, Sector, sex, caste and ethnicity. ISOs 1) BASE – Large membership based with expanded structure in six districts and lead by Tharu. Working as an ISO before SANDEEP, working with more than 15 donors and partners, big size of staff and budget. Leading in regions, implementing different programs. Supported by other donors for organizational development. A separate study done on BASE governance structure. 2) NNSWA – Midium size organization lead by Dalit. Growing in region with district structures. Working with diversified donors and focusing on three issues. 3) SAC- relatively small in staffing and budget among five ISOs. Lead by youth mix caste groups. Working as umbrella partner of SC/N (supporting other local NGOs). Working in districts that were highly affected by conflict. 115 4) PNGOs: a) RYC (Rajakoth Yuba Club) Darchula: Lead by Sauka (ethcic) group, growing in district. Remote hill based NGOs. b) TMM (Tharu Mahila Manch) Led by Tharu woman and Terai based NGO. Implementing income generation and education programs c) KPUS (Kamaiya Pratha Unmulan Samaj) Led by freed Kamaiya Tharu and working for freed Kamaiya. Small organization implementing income generation program. d) NRCS (Nepal Red Cross Society) Dadeldhura: Lead by mixed group, hill based, entertaining multiple donors, centrally guided, and growing and youth leadership. e) UNESCO Banke: Lead by Muslim and working for Muslim women, growing and dynamic leadership. Medium size organization in Terai. f) SDC (Social Development Center) Surkhet: Hill and lead by Dalit. Small organization implementing health programs. g) CFWA (Community Family Welfare Association) Dhanusha, Lead by Madhesi group, in east. Working in conflict.