Mid-Term Evaluation of Vietnam Forests and Deltas Program January 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by David Callihan (Team Leader), Hoang Tran Vu, Trinh Dinh Hoang, Nguyen Thu Que, Chang Le and Management Systems International. Mid-Term Evaluation of Vietnam Forests and Deltas Program Contracted under AID-440-C-14-00003 USAID/Vietnam Evaluation, Monitoring and Survey Services (VEMSS) Cover photo: Forested area of Thanh Hoa Province, Vietnam, D. Callihan. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation iii TABLE OF CONTENTS Acknowledgement ........................................................................................................... vii Acronyms and Abbreviations ........................................................................................ viii Executive Summary .......................................................................................................... x I. Introduction .................................................................................................................... 1 A. The USAID Vietnam Forests and Deltas Program ........................................................... 1 B. Geographic Focus ................................................................................................................. 2 C. Evaluation Purpose and Methodology ................................................................................ 2 Evaluation Purpose ................................................................................................................................................... 2 Objectives ................................................................................................................................................................... 2 Methodology .............................................................................................................................................................. 3 II. Country Context Vietnam’s Climate Change Challenge ......................................... 4 A. Introduction .......................................................................................................................... 4 B. Policy Context ....................................................................................................................... 5 III. VFD Performance Sustainable Landscapes (SL) ...................................................... 6 A. Overall Progress toward the Achievement of High-level Indicators ............................... 7 Sustainable Landscapes: Summary of Indicator Targets and Actual Achievement to Q3, 2015 ............ 7 B. Payment for Forest Environmental Services ..................................................................... 8 VFD Activity ............................................................................................................................................................... 8 Findings/Stakeholder Views .................................................................................................................................. 10 Conclusions .............................................................................................................................................................. 12 C. Reducing Emissions from Deforestation and Forest Degradation (REDD+) ................ 13 VFD Activity ............................................................................................................................................................. 13 Findings/Stakeholder View .................................................................................................................................... 15 Conclusions .............................................................................................................................................................. 15 D. Development of Green Growth Action Plan in Thanh Hoa ........................................... 15 VFD Activity ............................................................................................................................................................. 16 Findings/Stakeholder Views .................................................................................................................................. 16 Conclusion ............................................................................................................................................................... 16 E. Revision of Land Use Plans in Thanh Hoa and Nghe An ................................................. 17 VFD Activity ............................................................................................................................................................. 17 Findings/Stakeholder Views .................................................................................................................................. 17 Conclusion ............................................................................................................................................................... 18 F. Strategic Action Plan for Thanh Hoa Bamboo Development ........................................ 18 VFD Activity ............................................................................................................................................................. 18 Findings/Stakeholder Views .................................................................................................................................. 18 Conclusion ............................................................................................................................................................... 19 G. Water Accounting in Nghe An ......................................................................................... 19 VFD Activity ............................................................................................................................................................. 19 Conclusions .............................................................................................................................................................. 20 H. Nature Reserves ................................................................................................................. 20 VFD Activity ............................................................................................................................................................. 20 Findings/Stakeholder Views .................................................................................................................................. 20 Conclusions .............................................................................................................................................................. 21 I. Forest Management Support – Others .............................................................................. 21 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation iv IV. VFD Performance: Adaptation ................................................................................22 A. VFD Activities ..................................................................................................................... 23 1. Implementing VCA/ CBDRA/ CBDRM.............................................................................................. 23 2. DRM school-based activities.................................................................................................................... 27 3. Program for Enhancement of Emergency Response (PEER).................................................30 4. Supporting Early Warning System (EWS)...................................................................................... 32 5. Other Assistance .......................................................................................................................................... 33 B. Overall Conclusions ............................................................................................................ 34 V. VFD Performance Cross-CuttingWork in Livelihoods ..........................................35 A. Findings ................................................................................................................................ 36 1. VFD Activity ........................................................................................................................................................ 36 2. Livelihood Training Support and Application of Knowledge ................................................................... 38 3. Analysis and Scaling............................................................................................................................................ 39 4. Livelihood Performance and Stakeholder Observations .......................................................................... 40 B. Conclusions .......................................................................................................................... 42 Limitations ................................................................................................................................................................ 42 Overall Conclusion ................................................................................................................................................ 42 VI. VFD Performance Crosscutting themes ................................................................. 43 A. NGO Capacity Building ...................................................................................................... 43 1. VFD Activity ........................................................................................................................................................ 43 2. Findings/Observations ....................................................................................................................................... 43 3. Conclusions ......................................................................................................................................................... 45 B. Support to Build Climate Change Awareness and Political Commitment among GVN Officials ..................................................................................................................................... 46 1. VFD Activity ........................................................................................................................................................ 46 2. Engagement with Ho Chi Minh Academy of Politics and Public Administration ................................ 47 3. Conclusions ......................................................................................................................................................... 47 C. Gender Integration ............................................................................................................. 47 VFD Activity ............................................................................................................................................................. 47 Conclusions .............................................................................................................................................................. 49 VII. VFD Performance Planning, Coordination and Monitoring and Evaluation .....49 A. VFD’s Management Structure........................................................................................... 50 B. Project Planning and Coordination ................................................................................... 51 Work Plan Development ...................................................................................................................................... 51 GVN Comments on VFD’s Workplan Process.............................................................................................. 53 Conclusion ............................................................................................................................................................... 54 C. VFD Monitoring and Evaluation System .......................................................................... 55 VFD Performance Monitoring and Evaluation Plan (PMP) ............................................................................ 55 Performance Monitoring Plan (PMP) ................................................................................................................. 55 Conclusion ............................................................................................................................................................... 57 D. Delayed Project Start-Up .................................................................................................. 57 VIII. Summary Conclusions and Recommendations ...................................................58 A. Sustainable Landscape ....................................................................................................... 58 PFES ........................................................................................................................................................................... 58 Water Accounting in Nghe An ........................................................................................................................... 60 Nature Reserves ..................................................................................................................................................... 60 B. Adaptation ........................................................................................................................... 60 C. Livelihoods ........................................................................................................................... 61 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation v D. Crosscutting Themes ......................................................................................................... 62 Capacity Building for Local NGOs ..................................................................................................................... 62 Raising Awareness and Building Political Commitment ................................................................................. 62 Gender Integration ................................................................................................................................................. 62 E. Planning and Coordination ................................................................................................. 63 Workplan.................................................................................................................................................................. 63 Monitoring and Evaluation .................................................................................................................................... 63 Delays in Project Start-Up.................................................................................................................................... 64 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation vi LISTS OF FIGURES AND TABLES TABLE 1. PFES FUNDS DISTRIBUTIONS IN THANH HOA..................................................................................................................... 9 TABLE 2. PFES PAYMENTS FROM VFD’S WORK IN THANH HOA ...................................................................................................11 TABLE 3. AWARENESS OF PFES AMONG VFD PFES TRAINING PARTICIPANTS: BA THUOC DISTRICT .....................................12 TABLE 4: VFD-ASSISTED SITES FOR DISASTER RISK REDUCTION AND CLIMATE CHANGE ADAPTATION IN NAM DINH AND LONG AN (BY JUNE 2015) ..........................................................................................................................................................22 TABLE 5. VCA/ CBDRA/ CBDRM ACTIVITIES ..................................................................................................................................24 TABLE 6. VFD-ASSISTED CBDRA/CBDRM TRAINING AND MEETING SUPPORTED (BY END OF JUNE 2015) .......................24 TABLE 7. EVALUATION OF TRAINING SUPPORT BY STAKEHOLDERS ...............................................................................................25 TABLE 8. SCHOOL-BASED ACTIVITIES (BY JUNE 2015) ......................................................................................................................28 TABLE 9. PEER/CADRE ACTIVITIES .....................................................................................................................................................30 TABLE 10. CADRE TRAINING EFFECTIVENESS....................................................................................................................................31 TABLE 11SUMMARY OF VFD-ASSISTED ACTIVITIES’ PERFORMANCE – ADAPTATION COMPONENT: .........................................34 TABLE 12: INDICATOR 4.3.2 ..................................................................................................................................................................36 TABLE 13: LIVELIHOOD TRAINING BY ACTIVITY................................................................................................................................. 38 TABLE 14: USEFULNESS OF LIVELIHOOD TRAINING ...........................................................................................................................39 TABLE 15. HAVE YOU APPLIED/IMPLEMENTED THE MODEL FROM THE TRAINING? ......................................................................39 TABLE 16. SUMMARY OF VFD’S ORGANIZATIONAL DEVELOPMENT ACTIVITIES FOR FOUR CORE LOCAL NGOS...................44 TABLE 17. LIST OF ACTIVITIES TO STRENGTHEN CAPACITY FOR LOCAL NGOS PROPOSED IN WORKPLAN 2015 ................45 TABLE 18. TRAINING COURSES EXCLUSIVELY ON GENDER..............................................................................................................49 TABLE 19. WORKPLAN ACTIVITIES BY COMPONENT AND REGION...............................................................................................52 FIGURE 1. USEFULNESS OF CBDRM MEETING AND ASSESSMENT IN THE COMMUNITY ................................................................... 25 FIGURE 2. DISTRIBUTION OF PRE- AND POST-TRAINING TEST SCORES ........................................................................................... 29 FIGURE 3. RESPONSE OF HOUSEHOLDS TO THE STATEMENT: “CLIMATE CHANGE AFFECTS WOMEN AND MEN DIFFERENTLY."........................................................................................... 47 FIGURE 4. RESPONSE OF HOUSEHOLDS TO THE QUESTION: “DO YOU THINK CURRENT ACTIVITIES RELATED TO CBDRM/CADRE IN YOUR COMMUNITY HELP WOMEN RESPOND EFFECTIVELY TO CLIMATE CHANGE AND/OR NATURAL DISASTER IF IT OCCURS?”.............................................................. 48 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation vii ACKNOWLEDGEMENT This mid-term Vietnam Forests and Deltas (VFD) performance evaluation is the consolidated work of the evaluation team commissioned by USAID/Vietnam. However, it would not have been possible without the kind support and help of many individuals and organizations. The evaluation team would like to express our sincere thanks to the many Vietnam Government officials and agencies from central, provincial, district and commune levels who arranged their time to meet and share information with us. Particularly, the fruitful information and opinions shared by officials from responsible departments and bureaus under Ministry of Agriculture and Rural Development provided the team with many insights on VFD’s activities and accomplishments. Our special appreciation goes to VFD Project Management Units at central and provincial levels for their generous support in making necessary local arrangements to facilitate the team’s field trips. We would also want to extend our gratitude to the household representatives who agreed to participate in our interviews. Additionally, we thank the management and field staff from Winrock International, Netherlands Development Organization, American Red Cross, Vietnam Red Cross, and Sustainable Rural Development for their cooperation in providing documentation and information about their activities, as well as their assistance in arranging contact with relevant stakeholders. Finally, yet importantly, our appreciation goes to the USAID Development Program Office and Office of Environment and Social Development for their support in meeting and assembling information, particularly Randolph Flay, Ha Thi Nguyen, Terhi Majanen, and Khuong Chinh Tran. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation viii ACRONYMS AND ABBREVIATIONS ADB Asian Development Bank AEC Agricultural Extension Center AOR Agreement Officer’s Representative ARC American Red Cross BSM Benefit Sharing Mechanism CADRE Community Action for Disaster Response CAP Commune Action Plans CBDRM (A) Community-Based Disaster Risk Management (CBDR-Assessment) CC Climate Change CCA Climate Change Adaptation CCAP Climate Change Action Plans CCM Climate Change Mitigation CCWG Climate Change NGO Working Group CERDA Center for Research and Development in Upland Areas (National NGO) CFSC Committee for Flood and Storm Control COP Chief of Party CPC Commune People’s Committee CPMU Central Project Management Unit CSA Climate Smart Agriculture CSO Civil Society Organization CSR Corporate Social Responsibility DARD Department of Agriculture and Rural Development DCOP Deputy Chief of Party DoNRE Department of Natural Resources and Environment DoTI Department of Trade and Industry DPC District People’s Committee DPI Department of Planning and Investment DPP Disaster Preparedness Plan DPD Detailed Project Document DPO Detailed Project Outline DRR Disaster Risk Reduction ERPD Emissions Reduction Project Document (FCPF program of World Bank) ERPIN Emission Reductions Program Idea Note (FCPF program of World Bank) ERT Emergency Response Team EWS Early Warning System FCPF Forest Carbon Partnership Fund World Bank) FFS Farmer Field Schools FIPI Forest Inventory and Planning Institute FPDF Forest Protection and Development Fund FSC Forest Stewardship Council FY Fiscal Year GG Green Growth GHG Greenhouse Gas GIS Geographic Information Systems GVN Government of Vietnam HCMA Ho Chi Minh Academy IEC Information, Education and Communication IMHEN Institute of Meteorology, Hydrology and Environment (of MONRE) INBAR International Network for Bamboo and Rattan (headquartered in China) IR Intermediate Result ISPONRE Institute of Policy on Natural Resources and the Environment (within MONRE) KOICA Korean Overseas International Cooperation Agency LED Low-Emission Development LEDP Low-Emission Development Plans / Provincial REDD + Action Plans LOP Life of Project M&E Monitoring and Evaluation MARD Ministry of Agriculture and Rural Development MCD Center for Marine Conservation and Development (National NGO) MOET Ministry of Education and Training MONRE Ministry of Natural Resources and Environment Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation ix MOIC Ministry of Information and Communication MPI Ministry of Planning and Investment MRV Monitoring, Reporting and Verification MSI Management Systems International NAEC National Agricultural Extension Center NAMA Nationally Appropriate Mitigation Action NFI National Forest Inventory NGO Non-Governmental Organization N-HMS National Hydro-Meteorological Service NTFP Non-Timber Forest Product OCA Organizational Capacity Assessment OCAT Organizational Capacity Assessment Tool ODA Official Development Assistance PANNATURE People and Nature Reconciliation (National NGO) PC People’s Committee PCFSC Provincial Committee for Flood and Storm Control PEER Program for Enhancement of Emergency Response PES Payment for Ecosystem Services PFES Payment for Forest Ecosystem Services PHNR Pu Hoat Nature Reserve PIN Project Identification Note (of FCPF) PMEP Performance Monitoring and Evaluation Plan PMU Provincial Management Unit PPC People’s Provincial Committee PPMU Provincial Project Management Unit PRAP Provincial REDD+ Action Plan REDD+ Reducing Emissions from Deforestation and Forest Degradation SALT Sloping Agriculture Land Technique SEDP Socioeconomic Development Plans SFM Sustainable Forest Management SNV Netherlands Development Organization (International NGO) SRD Center for Sustainable Rural Development (National NGO) SRI System of Rice Intensification TERA Trilogy Emergency Response Application ToT Training of Trainers UNDP United Nations Development Program UNFCCC United Nation’s Framework Convention for Climate Change UNREDD United Nation’s Reduction of Emissions from Deforestation and Forest Degradation USAID United States Agency for International Development USAID LEAD USAID Low Emissions for Asian Development Program (USAID) USAISD LEAF USAID Lowering Emissions in Asia’s Forest Project (USAID) USAID/OFDA USAID/Office of U.S. Foreign Disaster Assistance USG United States Government VCA Vulnerability and Capacity Assessment VFD Vietnam Forests and Deltas VKCET Vietnam-Korea Center for Environmental Research and Training VMESS Vietnam Monitoring, Evaluation and Survey Services VNFF Vietnam Forestry Fund (MARD) VNFOREST Vietnam Administration of Forests (MARD) VNRC Vietnam Red Cross VRO Vietnam REDD Office (MARD) WA Water Accounting WATSAN Water and Sanitation WI Winrock International WRD Water Resources Directorate (MARD) Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation x EXECUTIVE SUMMARY The United States Agency for International Development (USAID) awarded the Vietnam Forests and Deltas (VFD) program to Winrock International (Winrock) as a cooperative agreement on September 25, 2012. The cooperative agreement began in late 2012 and runs through 2017. Total approved life-of-project funding is $26.5 million. Winrock implements VFD through sub-awards with the American Red Cross (ARC), the Center for Sustainable Rural Development (SRD), SNV–Netherlands Development Organization (SNV) and the Vietnam Red Cross (VNRC). VFD’s goal is to accelerate Vietnam’s transition to climate-resilient, low-emission sustainable development. The program assists the Government of Vietnam (GVN) to implement recently enacted national policies and legislation on climate change adaptation and green growth at the provincial and local levels. VFD focuses on adaptation activity in two coastal delta provinces and on sustainable landscapes/mitigation activity in two upland forest provinces. VFD has three main areas of activity: 1) sustainable landscapes; 2) climate change adaptation; and 3) coordination and national policy support, which was recently added at the GVN’s request. VFD also features crosscutting themes on livelihood support, gender integration and institutional capacity building. Evaluation Purpose The purpose of the evaluation is to assess progress toward the achievement of VFD’s objectives and to recommend management adjustments. The objectives of the evaluation include the following: Assess Progress toward the Achievement of Objectives: Analyze the degree to which the program is making progress toward its goal, which is “to accelerate Vietnam’s transition to climate-resilient, low-emission sustainable development.” The assessment includes a review of key intermediate results and sub-results, including VFD’s ability to leverage non-USAID sources of funds. Crosscutting Themes of Gender and Capacity Development: Assess the effectiveness of the program’s gender awareness and capacity-building actions. A five-person independent team from Management Systems International (MSI) conducted the evaluation under the USAID/Vietnam Monitoring, Evaluation and Survey Services Project. Program Context: In agreement with USAID, the GVN established VFD as an overseas development activity (ODA) project, whereas Winrock’s agreement with USAID is a cooperative agreement. One implication of the use of dual contractual modalities was that GVN was required to set up a Central Project Management Unit (CPMU) to oversee and co-manage the project. This unit did not become operational until January 2014, which means that VFD’s implementation period has only been about 18 months. This is a short implementation timeframe and makes it difficult to draw firm conclusions on several aspects of VFD’s work, including progress in greenhouse gas reduction and support for water accounting. VFD Performance: Findings Sustainable Landscapes Component VFD has achieved notable progress in the sustainable landscapes component, including: • VFD has provided extensive support for Vietnam’s Payment for Forest Ecosystem Services (PFES) program. At the central level, VFD supported revision of PFES Decree 90 (PFES enabling legislation) and supported the development of Decree 40, which tightens regulations for ecosystem users who are non-compliant in their payments. Decree 40 will help to increase overall PFES Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation xi revenue. In Thanh Hoa and Nghe An, VFD helped to accelerate completion of PFES documents and revenue distribution to forest owners. • VFD has worked with the World Bank’s Forest Carbon Partnership Fund (FCPF) to develop the Northern Central Emission Reduction Program Idea Note (ER-PIN). The approval of the PIN resulted in the release of $650,000 for the development of Vietnam’s Reducing Emissions from Deforestation and Forest Degradation (REDD+) program. FCPF expressed appreciation for VFD’s support and acknowledged this as VFD’s most significant REDD+ contribution. In addition, VFD is supporting the development of two feasibility studies that will contribute to the development of REDD+ action plans in Thanh Hoa and Nghe An. • Together with USAID/LEAD program, VFD has assisted in the development of the Thanh Hoa Green Grow Action Plan (GGAP). This has included assistance for the development of a provincial bamboo sector strategy. Provincial GVN agencies found the assistance provided by VFD to have been effective and helpful. • VFD has provided modest support in Nghe An to pilot water accounting activities and this is a potentially helpful start to establishing a water accounting system. At this point, however, the GVN’s commitment to adopting the system is unclear, and recently the Asian Development Bank (ADB) has begun implementation of a large water sector support activity (valued at around $45 million), which makes the importance of VFD’s continued support in this area unclear. On the other hand, some VFD activities have not met government expectations, as noted below. • VFD provided consultants to assist with revising land use plans in two provinces to include a greater emphasis on climate change considerations. With limited assistance, and insufficient engagement with local GVN partners, GVN counterpart agencies characterized VFD assistance as ineffective. VFD has no further plans for support in this area. • VFD's support to the improved management of natural reserves has focused on Pu Hoat Nature Reserve (PHNR). This assistance included study tours, staff training, partial forest demarcation, and several consultant studies. PHNR staff indicated the assistance provided was helpful, but said the support was inadequate to addressing their larger management needs. The nature reserve’s staff, overall, have been dissatisfied with VFD’s engagement and planning process and are unclear as to what future assistance will be provided, if any. VFD indicated to the evaluation team that some of the activities for which PHNR requested assistance were inappropriate for VFD to support, for example the construction of facilities and the purchase of large amounts of equipment. As of now, the evaluation is unable to conclude that VFD has made a meaningful contribution to achieving its objective to "slow, stop and reverse emissions from deforestation and degradation." However, this is a long-term goal and VFD has only been implementing activities for about 18 months. VFD reports 18,575 hectares of biophysical significance or natural resource showing improved conditions and 32,158 households having received incentives/rewards through sustainable natural resource management. • VFD assistance has been useful in supporting PFES implementation, particularly in Thanh Hoa and Nghe Anh, however, the revenue distributed was relatively small. There is no evidence showing that PFES- revenue helps to prevent deforestation and forest degradation. • The Thanh Hoa GGAP is under development, but has not yet received final approval. VFD’s support has been instrumental in advancing the development of the provincial bamboo strategy; however, key private sector partners still need to conduct detailed feasibility studies before they can make a firm commitment to the strategy’s implementation. The Bamboo strategy has a high level of potential income generation benefit. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation xii • There is no clear evidence that rural households have received direct economic rewards as results of VFD assistance in natural resource management. VFD’s work has resulted in some increased income from PFES, but the amount of funds distributed was relatively small and, because of this, distributions were made to villages rather than individual households. The amounts distributed in Nghe An in 2014 were as follows: an average of $157 distributed to each of 136 communities in Ba Thuoc District; and an average of $389 distributed to each of a 124 communities in Quan Hoa District (communities include dozens of households). VFD’s work did lead to an acceleration in PFES payments, and in strengthening revenue generation. Other VFD activities have not directly generated household income – perhaps with the exception of a few dozen households engaged in livelihood activities, but this is not conclusive. VFD may help to increase future household income, but the degree to which this is likely to happen is unclear. Adaptation Component VFD has made significant contributions to helping local communities increase the resilience of people, places and livelihoods in delta areas. Results were achieved through implementing community-based disaster risk assessments, school-based disaster preparedness activities, enhancement of emergency response (PEER) organizations, and assessing and upgrading effective early warning systems (EWS). VFD plans to implement these activities in 60 communes in Nam Dinh and Long An. All of VFD’s activities in this component directly support the GVN’s natural disaster risk management program (Prime Minister’s program 1002). VFD has been successful in building local system capacity in disaster risk management. VFD training courses have provided community members with practical knowledge and skills, which they were able to apply in conducting disaster risk assessments, and in planning for improved disaster preparedness and response. VFD has funded small projects to address immediate disaster risk management needs (preparedness, response and mitigation). Local community leaders are highly appreciative of the support provided by VFD – especially given the limited local funds available to support such activities. There remain challenges in the process of integrating disaster preparedness and climate change adaptation into local social and economic development plans. Currently, there are no government guidelines for the process, and local authorities do not have the funds to implement many of the recommendations made by the assessments. VFD’s central-level work has not progressed. Several outcomes, including development of an enabling decree under the natural disaster prevention and control law, and using an SMS text system for the EWS, have not been achieved. Livelihood Component The evaluation team visited five VFD livelihood models under both the sustainable landscapes and adaptation components. VFD demonstration models are designed to increase income from agricultural activity and reduce emissions, and include smart rice cultivation, maize growing along contour lines, and native chicken raising using earthworms. • VFD implements its livelihood demonstration models on a very small scale. Demonstration models typically involve a few dozen households and a few hectares of land. • VFD livelihood models have achieved mixed results. Some models seem to have increased productivity (smart rice in Long An and potatoes in Nam Dinh); however, some models have not been successful (the maize model in Nghe An). • VFD’s models do not include detailed planning and scaling plans. There is no field staff to support the implementation of the models, value chain analysis has not been undertaken, and the models do not meet government criteria for adoption and thus are not likely to receive government support Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation xiii once VFD ends. The rice model and the potato model may be exceptions, as they seem to benefit from government commitment – although detailed plans for these activities were not available. Government partners expressed a number of concerns regarding VFD’s livelihood models: • The government has certain criteria for the approval of livelihood models, which include potential impact and coverage, ability to support the system, and proven effectiveness. VFD has not followed the process, or supported GVN models. This makes it difficult for the AEC to support these models. • Financial support for “models” is limited and too low. There is too much training and not enough implementation support. • Government partners spend a lot of time on field visits, workshops, surveys and training, but there is no impact. VFD’s activities are too small. Management and Coordination VFD operates under two parallel management structures: one is under USAID regulations with Winrock as prime contractor, and the other is under GVN regulations to manage VFD as an ODA project with the Ministry and Agriculture and Rural Development’s CPMU as the official project co-manager. The use of two parallel management structures has led to a lack of clarity between Winrock and the CPMU on VFD’s strategic priorities, implementation mechanisms, and on planning and coordination. The CPMU and three of the four Provincial Project Management Units (PPMUs) expressed significant dissatisfaction with VFD’s management and coordination process. The fourth PPMU expressed indifference about VFD due to the small size of activities supported. VFD’s 2015 work plan contains a large number of activities (at least 208). The average budget per activity was $13,060. Lack of a strategic focus to achieve specific outcomes, too many activities, an unclear budget allocation process, and lack of engagement from local government agencies were the key concerns of government stakeholders regarding VFD’s work. Several issues with VFD’s monitoring and evaluation system reduce the system’s usefulness for assessing progress. These include: 1) the absence of logical links between IRs, sub- IR and proposed indicators; 2) the quality of M&E data for a number of indicators is questionable and not verifiable; and 3) the system does not serve as a results-based management system that provides information for program decision-making. Principal: Conclusions Transition to a low emission economy will require many years of focused, sustained effort, and Vietnam is in the early stages of this process. VFD is helping to build a foundation for the reduction of emissions in Vietnam, but no actual emissions reduction can yet be attributed to VFD’s work. Nevertheless, VFD’s support for PFES, and for the development of Vietnam’s REDD+ program, is helping to create the systems and capacities that Vietnam will require to reduce emissions. VFD’s performance has been somewhat mixed. In the key areas of sustainable landscapes and adaptation, work has been effective — sometimes highly so. Some of VFD’s achievements are particularly impressive considering the program has been implementing activities for only 18 months. In the areas of planning and coordination, and in its livelihood activities, the evaluation teams concludes that adjustments are required. The evaluation’s principal conclusions are as follows: VFD’s forest sector work has been effective to highly effective. VFD has made important contributions to Vietnam’s PFES program, including for national policy development and for operational improvements in Thanh Hoa and Nghe An provinces. This work has resulted in a new policy that addresses payment noncompliance by forest environmental service users, and on-going revisions to the main PFES policy (Decree 99). Operational work has helped transform PFES payment distributions in Nghe An province from Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation xiv a troubled program to a system that operates effectively and with a high degree of transparency. VFD also helped to re-verify forest owners in two districts in Thanh Hoa, which enabled forest owners to receive funds more quickly. Vietnam’s PFES program has made progress in collecting forest environmental service fees from water users (nearly $70 million was collected last year by the GVN), and redistributing the funds to forest owners. However, there is not a monitoring system in place to provide information on whether PFES has resulted in improved forest management. VFD is in the early stages of helping to develop such a system. VFD’s adaptation and disaster risk reduction work, which is led by the American Red Cross (ARC), has been useful and of high quality. This work has supported community-level disaster preparedness planning and the review and strengthening of EWSs. Accomplishments have included disaster risk assessments completed for 32 communes; approvals to upgrade disaster early warning systems in 10 communes; 19 safe-school assessments; and provision of 18 equipment sets to communal emergency response teams. VFD’s livelihoods models may succeed as pilot projects in that the households directly involved may financially benefit, but it is too early to reach this conclusion since the implementation timeframe has been short. These small-scale pilot initiatives, however, do not have clear plans for sustaining the activities beyond the life of VFD. The evaluation concludes that VFD’s livelihood activities are not likely to achieve significant impact by the end of the project, nor are most of these activities likely to receive continued support beyond the life of VFD. VFD’s work with climate smart rice in Long An may be an exception, as this work benefits from a close working partnership with the provincial AEC, but a better definition of the activity is needed. VFD’s work lacks a sufficient strategic focus. The work plan contains too many activities to enable consistently effective planning and coordination. Most of the activities in VFD’s most recent work plan are small and are not prioritized toward the achievement of specific outcomes. This makes impact less likely than would be the case under a more focused strategy, and makes support and monitoring challenging. It also means that it is often difficult to attribute the success of GVN programs to VFD assistance, as VFD activities are often small and within areas supported by multiple donors – or in areas that receive significant GVN financing. The Ministry of Agriculture and Rural Development’s (MARD) CPMU finds VFD’s planning system confusing and stated that they have not had an opportunity to provide meaningful input into the program’s strategic priorities and work plan. Consequently, a number of VFD’s activities do not align with GVN expectations. The funding of VFD’s work plan is an unclear process, which makes government planning and coordination challenging. There are differences in opinion between VFD and the CPMU as to the work plan’s focus, the heavy reliance on short-term training, which can often be general, and on the selection and approval of consultants. The CPMU has difficulty monitoring the activities of VFD’s sub-awardees. The adaptation work of the Red Cross is an exception, as the VNRC is integrated into the PPMUs, and their programs are closely integrated with government structures and processes. • Related to the issue of strategic focus, VFD’s monitoring and reporting system is confusing and does not provide a clear sense as to the achievement of key outcomes. VFD’s performance reporting contains too much reporting (45 pages of data tables) and does not give enough emphasis to analysis of the program’s major emphases, for example for PFES. • VFD uses a version of USAID’s Organizational Capacity Assessment process to measure NGO capacity strengthening. Data from this process indicates that VFD has successfully strengthened NGO capacity for six NGOs – for example, by supporting the development of stronger governance structures. • VFD’s success in strengthening the capacity of its main GVN counterparts is less clear. The data in VFD’s monitoring system is unhelpful in assessing these efforts as “organizations strengthened” are Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation xv sometimes double counted, and an organization – sometimes a large organization – may be considered strengthened even after limited training support. Overall, it would appear that the Forest Protection and Development Fund has been strengthened as a result of VFD support, but that the Department of Environment and Natural Resources, the AEC and the Department of Planning and Investment (DPI) have not been meaningfully strengthened, although all have gained new knowledge and skills from their participation in various training workshops. • VFD integrates gender concerns into all project activities. For example, VFD activities have reached out to women in PFES districts to ensure they have knowledge of PFES and opportunities to participate in how benefit funds are used. VFD has involved the Women’s Union in various support activities and training, but the effort appears a bit ad hoc in that their follow-up role is not clear, and there is not a defined plan for additional support. Principal Recommendations To increase effectiveness, VFD should build on its success and concentrate its activities to achieve greater impact. This will require a refocus of VFD’s strategy on the achievement of fewer priority outcomes. VFD’s impact will likely increase if it takes the following actions: places less emphasis on general widely offered short-term training; conducts consultancy assignments only when there is a clear counterpart need and substantial counterpart engagement; and, if it can provide deeper and longer-term technical support to a few select programs and institutions. For example, VFD could expand successful PFES work to a greater number of districts, or more comprehensively address the Pu Hoat Nature Reserve’s management needs. As part of a strategy to achieve specific outcomes, VFD could ensure that when providing training it covers all workforce members who may benefit, and that they are provided the tools they need to apply the skills they have acquired. VFD’s priority outcomes and supporting activities should be identified according to potential emissions reductions, and in consideration of the role of other donors. VFD should increase its support for forestry sector programs, especially for PFES, and for disaster risk reduction. VFD should drop or curtail activities that are not likely to achieve measurable impact within the life of the project. The evaluation recommends that VFD stop introducing new small-scale livelihood models. Effort in this sector should focus on reviewing GVN priorities and exploring opportunities to support existing promising initiatives. Existing and planned VFD livelihood activities should continue only if they specify significant expected impact, and contain post-VFD support plans. VFD has only about 18 months of implementation before it must begin initiating a phase-out process; this makes it unlikely that the introduction of new livelihood activities can be effective and sustained. VFD’s work with climate smart rice in Long An appears to enjoy government support and should be considered for expanded support. USAID should play a support role to help refocus VFD’s priorities and develop a more collaborative GVN￾VFD planning and coordination system. VFD’s institutional strengthening efforts with government partners will benefit from clearer institutional development objectives and more in-depth engagement. VFD supports a wide array of institutions and programs, and achievements in several key areas have been impressive. The program has an opportunity to use the experience gained to date to sharpen its focus within its areas of success, and further improve the program’s effectiveness. Detailed findings, conclusions and recommendations are contained in the body of this report. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 1 I. INTRODUCTION A. The USAID Vietnam Forests and Deltas Program The United States Agency for International Development (USAID) awarded the Vietnam Forests and Deltas (VFD) program to Winrock International (Winrock) as a cooperative agreement on Sept. 25, 2012. The cooperative agreement, which began in late 2012, is scheduled to run through 2017. Total approved life-of￾project (LOP) funding is $26.5 million. The goal of VFD is to help accelerate Vietnam’s transition to climate-resilient, low-emission sustainable development. The program’s aim is to assist the Government of Vietnam (GVN) in implementing recently enacted national policies and legislation on climate change adaptation and green growth at the provincial and local levels. The focus is on both lowland delta areas and upland forested watersheds. Winrock implements the program in direct partnership with Vietnam’s Ministry of Agriculture and Rural Development (MARD) and through sub-awards with the American Red Cross (ARC), the Center for Sustainable Rural Development (SRD), SNV–Netherlands Development Organization (SNV) and the Vietnam Red Cross (VNRC). VFD has three main activity sets: 1) sustainable landscapes; 2) climate change adaptation; and 3) coordination and national policy support. Sustainable Landscapes The Sustainable Landscapes component supports the adoption of land use practices and improved sustainable forest management to slow, stop and reverse emissions from deforestation and forest degradation, as well as promoting landscape activities that can leverage financial opportunities to mitigate climate change. Specific objectives of this component are to (1) establish effective governance and capacity; (2) build a sustainable financial base for improved land use; and (3) take actions that demonstrate net emission reductions from forests. Emphases under the component include the following: green growth planning; support to Vietnam’s Payment for Forest Environmental Services (PFES) program; and support for feasibility studies for Vietnam’s Reduction of Emissions from Deforestation and Forest Degradation (REDD+) program in Thanh Hoa and Nghe An Provinces, which include emissions estimates for both forestry and agricultural emissions. . The project also has a livelihoods component, which promotes activities to increase household income while reducing emissions. Adaptation This component focuses on ensuring that government authorities and local leaders develop an improved understanding of climate risk and possess the tools, capacity and motivation to address the risk. A significant number of commune-level climate vulnerability assessments have been conducted under this component and these are intended to be followed up with small-scale disaster risk reduction/climate change adaptation projects. VFD also works to build awareness of climate change vulnerability among key decision-makers. 1 These are initial results originally proposed by Winrock that represent the project’s objectives and areas of focus. Key Proposed VFD Results1 • 2 provincial-level integrated land use plans designed and implemented • 2 million tons of CO2e emissions reduced or sequestered • $5 million leveraged from public and private sources for REDD+ • 60 climate vulnerability assessments conducted • 16 institutions with improved capacity to address climate change issues • 200,000 person-hours completed in climate change training • 40% increase in household income generated from climate-resilient livelihood activities Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 2 Activities under this component are designed to increase climate change resilience and reduce disaster risk to vulnerable communities. Coordination and National Policy Development This set of activities supports and ties together the Sustainable Landscapes and Adaptation components. GVN recently requested support in this area, and Winrock will emphasize it in the third-year work plan. All aspects of VFD feature institutional capacity building and an emphasis on gender and social inclusion. B. Geographic Focus The program’s geographic focus areas and associated principal activities are presented below. • Hanoi: Policy coordination with the national government, and particularly with the MARD. VFD’s main project office is in Hanoi. • Thanh Hoa Province: Project support to Vietnam’s Green Growth Action Plan at the local level; land use planning with climate change integration; improved landscape management, particularly in forested watersheds; development of sustainable financing systems for improved land use; community-based natural resource management; climate-smart agricultural support; and bamboo sector development with engagement of the private sector. • Nghe An Province: Participatory planning, including CC integrated socioeconomic development plans (SEDP); land use planning with climate change integration; improved landscape management, particularly in forested watersheds; development of sustainable financing systems for improved land use; community-based natural resource management; development of a natural resources capital accounting system, with an emphasis on carbon and water; and climate-smart agricultural support. • Nam Dinh Province (Red River Delta): Reducing vulnerability to climate change and conducting disaster risk reduction activities; and climate-smart agriculture (rice and potatoes). • Long An Province (Mekong Delta): Reducing vulnerability to climate change, conducting disaster risk reduction, climate-smart agriculture (rice) and animal husbandry. C. Evaluation Purpose and Methodology Evaluation Purpose The purpose of the evaluation is to review the Winrock VFD program to assess progress on its objectives and to recommend management adjustments. The key audience for the lessons produced by this evaluation includes USAID, the GVN and Winrock International. Objectives The objectives of the evaluation focus on the following areas: • Assess progress toward the achievement of objectives: Analyze the degree to which the program is progressing toward its goal, “to accelerate Vietnam’s transition to climate-resilient, low-emission sustainable development.” The assessment will include a review of the degree to which key intermediate results and sub-results are being achieved, including VFD’s ability to leverage non￾USAID sources of funds. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 3 • Crosscutting themes of gender and capacity development: Assess the actions and effectiveness of the program’s gender and capacity-building work. • Review the existing performance monitoring system: Determine if the program’s monitoring system effectively measures implementation progress and provides a useful assessment of its short and long￾term objectives. • Review of start-up and implementation: Analyze the reasons for the program’s start-up delays and determine any lessons relevant to VFD’s ongoing management and the management of future USAID/Vietnam activities. Methodology The evaluation team, which included one international and three national experts, performed the following tasks from mid-July until the end of August 2015: Desk Study Materials containing VFD-related quantitative and qualitative data were collected in collaboration with USAID and implementing partners. The evaluation team received 2,000 documents related to VFD. Reviewed materials included: 1) VFD project documents (project proposal, 2015 work plan); 2) project quarterly and annual reports; 3) technical reports (studies, assessments, operational research); 3) the performance monitoring plan; 4) VFD’s training database; and 5) other relevant materials Field Visit and Stakeholder Interviews In addition to stakeholder meetings and interviews in Hanoi, the evaluation team visited project sites all four provinces (Nam Dinh, Thanh Hoa, Nghe An and Long An). The field visit plan was developed in consultations with USAID Vietnam, the VFD office and the Central Project Management Unit (CPMU). The evaluation team met with and interviewed 23 VFD staff (seven from Winrock, five from SNV, four from SRD, four from ARC and three from VNRC) in 16 meetings; 62 government stakeholders (at all central / CPMU/MARD, Provincial Project Management Units (PPMUs)/Department of Agriculture and Rural Development (DARD), district and commune levels); and 12 beneficiaries (farmers). The team also met with USAID technical experts (including the VFD agreement officer’s representative, or AOR). The detailed data collection plan is in Annex 3 and the stakeholder list is in Annex 4. Beneficiary Survey The evaluation team surveyed 382 VFD training recipients in four provinces, selecting participants randomly from VFD-provided trainee lists. Trained interviewers used questionnaires developed by the evaluation team (attached in Annex 6) in selected communes. Data Analysis and Report Writing Evaluation team members at VEMSS in Hanoi performed data synthesis and analysis and developed this report. During the process, the team scheduled additional stakeholder meetings for verification and more data collection. The team presented its preliminary findings to USAID, MARD/CPMU and VFD in September 2015. Limitations and Challenges VFD is a complicated program with three big components and hundreds of activities. In the absence of a good results-based framework (discussed in subsequent chapters), the team needed time to understand the Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 4 program specifics. The VFD office provided 2,000 documents to the evaluation team. Because of time limitations, the team was not able to review all of the documents. II. COUNTRY CONTEXTVIETNAM’S CLIMATE CHANGE CHALLENGE A. Introduction Vietnam is one of five countries that are most vulnerable to the adverse effects of climate change.2 The nation lies within the tropical cyclone belt and contains more than 2,000 miles (greater than 3,200 km) of coastline, much of which rests just above sea level. The Government of Vietnam (GVN) increasingly views climate change as one of its central development challenges. Vietnam’s large population centers and key agricultural sectors are exposed to rising sea levels and increased storm risks in low-lying deltas and on its long coastline. On average, six to eight typhoons or tropical storms strike Vietnam annually. Increasing temperatures and worsening droughts and floods threaten food security, livelihoods and lives; these factors pose significant long-term challenges to Vietnam’s sustainability and development goals. The country’s two major river basins, the Mekong and Red River, are at risk from the impacts of climate change. The livelihoods of farmers and fisheries are particularly vulnerable. Shifting climate conditions like heavy rains, floods and prolonged drought already create significant stress in the system and can trigger natural disasters. A 2011 report on the modeling of climate change scenarios by the Ministry of Natural Resources and Environment (MoNRE) indicates that a sea-level rise of approximately 1 meter by the year 2100 is a plausible scenario for a high-emissions case; this would inundate approximately 39 percent of the Mekong Delta. The Red River Delta is part of the country’s second-largest river system after the Mekong, but is even more densely populated and highly developed. Much of the delta is low-lying, and susceptible to rising sea levels and floods. Dikes and canals provide flood control that is critical to the culture and economy, particularly rice production, fishing and aquaculture. The United Nations Development Program (UNDP) identified mountainous rural populations as vulnerable to long-term climate change with extreme weather, increased drought, flash floods and landslides threatening their already fragile lifestyles. Addressing long-term climate change risks through adaptation and disaster risk reduction measures will be central to Vietnam continuing its progress toward industrialization, building an expanding middle class and increasing climate resilience in rural communities. As Vietnam develops, its greenhouse gas emissions from industry, energy production, transportation and agricultural are increasing. Vietnam’s forests remain under pressure, and deforestation and degradation are a further source of emissions. Although currently a relatively low emitter, under business-as-usual development pathways Vietnam’s projected future emissions are significant. Improved forest management and restoration programs offer opportunities to sequester carbon and leverage funding to support management and livelihoods development. A cleaner development path offers opportunities for economic growth and improved standards of living due to a healthier environment and maintenance of vital ecosystem services. It also contributes to global reductions in greenhouse gases. 2 Dasgupta, S., Laplante, B., Meisner, C., Wheeler, D. and Yan, J. (2007). “The Impact of Sea Level Rise on Developing Countries: A Comparative Analysis.” World Bank Policy Research Working Paper 4136. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 5 B. Policy Context The GVN has enacted strong policies on forest management. It recently announced the completion of the “5 Million Hectares” reforestation plan. Vietnam has also implemented two innovative approaches to conserving and restoring forests. After the initial success of its pilot, the government instituted a national PFES policy to implement the program in 13 provinces. Currently, 37 provinces implement the PFES policy and have provincial forest development and protection funds. Under this scheme, downstream hydropower producers, municipal water users and ecotourism operators pay for upstream forest protection and restoration. Vietnam is also a leader in promoting the Reducing Emissions from Deforestation and Forest Degradation and enhanced carbon stocks (REDD+) program, with pilot efforts beginning in a number of provinces. The Government of Vietnam has been proactive in advancing disaster risk reduction and climate change policy. The “National Strategy for Natural Disaster Prevention, Response and Mitigation to 2020” was adopted in 2007. The 2008 “National Target Plan for Climate Change” focused largely on adaptation. It prioritized water, agriculture, marine and coastal systems as most sensitive, and in particular prioritized the need for action in the Mekong and Red River deltas. In December 2011, the prime minister ratified the “National Climate Change Strategy”; in October 2012, a National Action Plan on Climate Change reaffirmed the importance of adaptation and gave added emphasis to reducing greenhouse gas emissions as part of a green growth strategy. The prime minister approved the framework of Vietnam’s “National Green Growth Strategy” in September 2012. Vietnam will need significant financial and capacity-building support to implement these national policies, notably at the sub-national or provincial levels. An overview of the country’s main policies follows. The National Target Program to Respond to Climate Change (NTP-RCC): In 2008, the prime minister approved the NTP-RCC to run until 2015 with a budget of $93.5 million. The Support Program to Respond to Climate Change (SP-RCC) was set up to mobilize and coordinate resources from international donors. The SP-RCC is based on a policy matrix of actions across sectors, including energy, transportation, construction, forestry, agriculture, waste, water, coastal management, natural risk disaster management and health care. Vietnam’s Green Growth Strategy (VGGS): This strategy aims to accelerate the process of economic restructuring and use natural resources efficiently, reduce greenhouse gas emissions through research and application of modern technologies, develop infrastructure to improve the entire efficiency of the economy, cope with climate change, contribute to poverty reduction and drive economic growth in a sustainable manner. “The ultimate aim of a low-carbon development strategy is to catalyze concrete actions that support development, but with less emissions than without intervention.” 3 The National Strategy for Natural Disaster Prevention, Response and Mitigation: Approved by decree on Nov. 16, 2007, the National Strategy for Natural Disaster Prevention, Response and Mitigation to 2020 is Vietnam’s primary disaster risk management framework. This framework focuses on water-related disasters and has a budget of $13 billion. Around $1 billion is for structural measures, such as reservoirs, dams, dikes and safe harbors, and includes more than $2 billion for non-structural measures. National Reducing Emissions from Deforestation and Forest Degradation Plus Program (REDD+): In 2011, the Ministry of Agriculture and Rural Development (MARD) set up the Vietnam REDD+ office to implement a national REDD+ program, which Vietnam adopted in 2012. The goal of Vietnam’s REDD+ program is to contribute to the successful implementation of the National Climate Change Strategy and the sustainable development poverty reduction goals.It is estimated that REDD+ could generate $80 million to $100 million in annual revenue for Vietnam once program requirements are met. 3 Van Tilberg et al., 2011. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 6 Decree 99: This decree mandated the nationwide implementation of Payments for Forest Environmental Services (PFES) beginning Jan. 1, 2011. Vietnam is the first country in Asia to initiate a nationwide PFES scheme. The basic idea of payments for environmental services is to create incentives for individuals and communities to protect environmental resources by paying the costs incurred in managing and providing those services. III. VFD PERFORMANCE SUSTAINABLE LANDSCAPES (SL) Evaluation question 2: To what degree is the SL component achieving results within its key result areas? Evaluation question 4: Is VFD contributing to the development of a sustainable financial base for improved land use? (PFES and REDD+) The SL component’s main objective is to slow, stop and reverse emissions from deforestation and degradation of forests and other landscapes. Specific objectives of this component are (1) establish an effective governance and capacity; (2) create a sustainable financial base for improved land use; and (3) implement actions that demonstrate net emission reductions from forests. This project component’s work is focused on forested areas of Thanh Hoa and Nghe An provinces. Activity in Thanh Hoa and Nghe An will address local issues, and the experience is intended to help inform national policy. Many of the activities in this component cut across VFD’s intermediate results (IRs). Due to this, the presentation and analysis of this component is structured according to VFD’s main areas of activity: • Support to Vietnam’s Payments for Forest environmental Services (PFES) program; • Support for the implementation of the REDD+ provincial action plans in two provinces; • Development of Thanh Hoa’s Green Growth Action Plan; • Integrating climate change considerations into land use plans in Thanh Hoa and Nghe An; • Development of a pilot water accounting system in Nghe An; • Development of Thanh Hoa Bamboo Sector (public-private partnership approach); and • Support to nature reserves. FD’s common technical approach is to use staff and external consultants as follows: • Build capacity (through training, study tours, workshops) of partners at different levels; • Carry out studies, assessments and field work with or without participation from local stakeholders; and • Organize workshops for validating or disseminating research and consultancy results. VFD has developed themes to guide its SL work. In Nghe An VFD follows a natural capital accounting approach, which features water accounting as one of its principal tools, and in Thanh Hoa a green growth approach is used, and it is in this province that VFD has provided support for the develop of a provincial Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 7 GGAP.4 These themes feature various activities, including REDD+, improved forest management, and capturing finance to support green growth activities. The themes do not equate to strategies, as they do not contain specific objectives or measures of success. As such, the evaluation team is not able to comment on the effectiveness of the concept. The themes could perhaps offer a way to focus future resource allocations and results reporting, but this is not currently the case. A. Overall Progress toward the Achievement of High-level Indicators Sustainable Landscapes: Summary of Indicator Targets and Actual Achievement to Q3, 2015 The table below lists the major targets for VFD’s SL component. Since the total number for FY2015 is not yet available, the actual achievements for Q3, FY2015 is calculated as the sum of implementing/implemented activities in three quarters in FY2015 (activities in the planning process are not included). Note: Green = >110% of target, Blue = within 90-110% of target, Red = <90% of target No. Performance Indicators Y3 Target Actual - to Q3 FY2015 (Partially or Completed) 3.0.2 Number of hectares of biophysical significance and/or natural resources showing improved biophysical conditions as a results of USG assistance 20,000 40,067 3.1.2 Number of hectares of biological significance and/ or natural resources under improved natural resource management as a result of USG assistance 40,000 125,084 3.2.2 Number of households receiving benefits / incentives and economic rewards through sustainable natural resource management as a result of USG assistance 2,500 0 3.3.2 Quantity of GHG emissions, measured in metric tons of CO2 equivalent, reduced or sequestered as a result if USG assistance 295,000 0 Note: The GHG emissions reduction target appears to be entirely unmet and is displayed as zero because emissions reporting requires complex analytic work that is only undertaken and reported annually, and because programs to reduce emissions may require a long time to show results. VFD has reported partially or fully completed accomplishments for two of its targets: 1) 3.1.2 – the number of hectares of biological significance and/ or natural resources under improved natural resource management as a result of USG assistance; and 2) Number of hectares of biological significance and/ or natural resources under improved natural resource management as a result of USG assistance. However, the evaluation team notes some concerns with VFD’s performance reporting. For example, the number of hectares of biophysical significance and/or natural reserve showing improved biophysical condition is measured by counting areas where VFD provides assistance, regardless of the intensity of its 4 VFD Publication, Climate Change Challenges, Opportunities and the Way Forward, March 2014. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 8 activity. The accomplishments for these indicators are mainly from Pu Hoat Nature Reserve. VFD has supported a number of training events and study tours for the NR, as well as provided assistance for the development of a biodiversity monitoring system. It is unclear if the reserve has experienced actual improvements in it condition, or if the training has resulted in improved forest protection. Each main SL component is discussed in the sections that follow. B. Payment for Forest Environmental Services PFES is a financing mechanism for forest protection and development. The concept is that users of forest environmental services (hydropower plants, water suppliers, ecotourism companies, industries) pay to use the services, and then forest owners and managers receive funds under agreements that they will protect and manage the forests. Most commonly, the funds are collected from water users and then distributed to forest owners by the Forest Protection and Development Fund (FPDF). Vietnam began its PFES program in 2008 in Lam Dong and Son La provinces. After a three-year pilot phase, nationwide implementation of PFES started in 2011 through GVN’s Decree 99/2010/ND-CP (Decree 99). PFES is still relatively new in Vietnam, and particularly so in many provinces. The fund faces a number of challenges that require further attention. These include: • Significant differences in the amounts of income earned by individual provinces; • Slow progress in reviewing and identifying forest owners; • A lack of guidance on provincial administrative structures; • No connection between PFES and activities in Vietnam’s Forest Protection and Development Plan to 2020, although the programs share similar goals; • A lack of guidance on user services for several sectors, including industry and aquaculture; • Lack of a forest protection and management monitoring mechanism; • Lack of a good mechanism to mobilize NGO participation; and • Lack of guidance on direct payments from users to suppliers. VFD Activity VFD has been providing national-level policy support for PFES and implementation support in Thanh Hoa and Nghe An provinces. PFES has been one of VFD’s most-supported activities. Support has included a wide range of training, study tours, policy reviews and creating a community of practice for the FPDFs to share experiences. VFD’s most significant support to PFES follows, but this is not a comprehensive list of VFD’s support. To date, VFD has trained 795 people on PFES-related topics. National Policy Support • VFD hired consultants for a review of three years of PFES implementation, titled Three Years of Implementation of Policy on Payment for Forest Environmental Services in Viet Nam, 2011-2014. The study is detailed and contains a set of specific recommendations to improve PFES, including in payments, administration and user fee collection. As a result of the review’s recommendations, VFD supported a revision to Decree 99. The revision to the policy was expected to be complete by September 2015. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 9 • VFD sponsored three regional community-of-practice events for 36 provincial FPDFs to share experiences. Vietnam Forestry Fund (VNFF) and FPDF managers in Nghe An and Thanh Hoa expressed great appreciation for these events and hope that additional events will be supported. • The issue of payment noncompliance by water users came up in the three-year review and community-of-practice events. VFD subsequently undertook a study on the issue, which resulted in Decree 40/2015/ND-CP (approved in April 2015), a national policy outcome generated by VFD’s analytic work. The level of fines in the decree are less than the amounts recommended by the study; however, the deputy director of VNFF said he appreciated VFD’s support and was pleased with the issuance of Decree 40. • Together with Asian Development Bank (ADB) consultants, VFD is supporting VNFF in an assessment of monitoring and evaluation (M&E) systems for similar programs in other countries. Thanh Hoa Assistance VFD provides support to two of seven PFES districts in Thanh Hoa Province. • VFD has provided the PPMU with satellite images and a grant to conduct a forest status review and determine forest ownership in 36 communes of Quan Hoa and Ba Thuoc districts. The results have been approved and PFES payment distribution to forest owners is ongoing. Together with the Thanh Hoa Forest Fund, VFD’s efforts contributed to the first distribution of PFES money to households and village communities in these districts. Table 1 shows PFES funds distributions in Thanh Hoa districts where VFD works. Table 1. PFES Funds Distributions in Thanh Hoa Disposition Quantity Quan Hoa District Ba Thuoc District Thuong Xuan District PFES forest area 83,474 (ha) 36,930 (ha) Information has not been provided. This district was not supported by VFD (info requested). Total PFES revenue distributed VND 1,077,816,000 (approximately $48,329) VND 476,840,000 (approximately $21,382) Average payment, ha/year Max (2014) VND 7,061 ha/yr (approximately $0.32) VND 7.061 ha/yr (approximately $0.32) Min (2013) VND 5,851 ha/yr (approximately $0.26) VND 5,851 ha/yr (approximately $0.26) # of HHs receiving No distribution to HHs No distribution to HHs # of HH groups receiving No distribution to HH groups No distribution to HH groups # of communities 124 136 # of commune PCs 07 State/government forest owners 04 02 (Source: Data provided by Thanh Hoa Forest Protection and Development Fund) FPDF rangers received training and necessary mapping equipment (iPads, map contents, etc.). Twenty people from the Department of Forestry, four protected areas and the Thuong Xuan District FPDF received training by the provincial FPDF. • In 2013, there was a study tour to Son La, Lai Chau, with the participation of Thanh Hoa FPDF officials and various members of provincial and district People’s Committees and Commune PCs (CPCs). Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 10 • Three trainings took place for Thanh Hoa FPDF, the steering committee, district-level working groups and commune staff on PFES policies and reviewing forest owners using satellite images. This technology later helped re-map/re-verify forest owners in Quan Hoa and Ba Thuoc districts. • VFD conducted various training courses, including a training of trainers (ToT) on PFES policies for Forest Fund outreach staff; communication skills for district agricultural and forestry staff; and behavior change communication for women at the provincial, district and commune levels. Two staff of the FPDF will be trained to use PFES monitoring software at the Forestry University of Vietnam. • The project also supported the VNFF director to join a study tour to Dien Bien (JICA’s Project site) to learn about technology for forest quality monitoring. Nghe An Assistance VFD provides support to three of six PFES districts in Nghe An. • VFD supported a review of three-year PFES implementation in Tuong Duong and Ky Son to find ways to implement these services more effectively. Nghe An FPDF indicated a high level of appreciation for the review and workshop. Following up on the issue of non-transparent PFES fund distribution, VFD provided a grant to PPMU to develop provincial PFES implementation guidelines. Consultation workshops and meetings have been organized by the FPDF and the document is expected to be finalized soon. • VFD provided satellite images and a grant to the Nghe An PPMU to conduct a forest status review and owner identification for 17,000 hectares in Nam Pong Watershed area. This work is in process. • VFD supports development of a system for village-level PFES funds utilization in Que Phong, Ky Son and Tuong Duong. The system includes regulations, guidelines and a template accounting book. Village communities and households that receive PFES funds will use this system. • VFD conducted community awareness on sustainable forest management, climate change and PFES as a basis for a community outreach strategy to increase the understanding of PFES. • On the topic of PFES and sustainable forest management, VFD conducted ToTs for 32 provincial, district and commune staff on PFES communication, as well as communication skills training for 34 women from provinces, districts and communes. It also developed outreach training materials for three districts: Que Phong, Ky Son and Tuong Duong. Findings/Stakeholder Views Nationally VNFF indicated a close working relationship and good cooperation with VFD, saying that VFD’s support, especially for PFES policy review and reform, has been “effective.” The three-year review led to several follow-up actions, which have included development of the policy on user payment noncompliance. The main proposed changes in the revision of Decree 99 are: 1) an increased rate of payment for service users; 2) a revenue increase from 10 to 15 percent, which can be used for FPDF administration; and 3) an adjustment to the amount of the funds that PFES revenue beneficiaries receive. Thanh Hoa Progress in registering forest owners in the two districts receiving VFD’s support is greater than in the other districts, according to the FPDF. VFD supported GPS-based mapping methods for forest owner identification and concentrated its work in two districts. This accelerated the completion of PFES documents and helped forest owners receive funds more quickly. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 11 VFD’s support increased the number of communities that received PFES money. Before VFD’s involvement in supporting PFES distributions in Thanh Hoa, only Thuong Xuan District had distributed funds. Following VFD’s work, two additional districts are now distributing payments: Quan Hoa and Ba Thuoc. Table 2 shows the forest coverage for the two new districts covered by PFES and the amount of payments distributed. Table 2. PFES Payments from VFD’s Work in Thanh Hoa Disposition Quantity Quan Hoa District Ba Thuoc Dist. PFES forest area (hectares) 83,474 36,930 Total PFES revenue distributed VND 1,077,816 (approximately $48.33) VND 476,840,000 (approximately $21,382) Payment rate/ha/yr (average): 2013 VND 5,851 (approximately $0.26) VND 5,851 (approximately $0.26) 2014 VND 7,061 (approximately $0.32) VND 7,061 (approximately $0.32) # of HHs receiving payments No HH distribution No HH distribution # of communities receiving payments 124 136 # of commune PCs receiving payments 7 State/government forest owners receiving payments 4 2 (Source: Data provided by Thanh Hoa Forest Protection and Development Fund) Because the amount of money paid per hectare (ha) and per forest owner in Thanh Hoa is small (an average of VND 7,000/ha/year), the hamlet agreed to receive a community-level payment instead of providing payments to individual households. Village leaders manage this money and use it for common goals and community benefits. One community, for example, used the money to create a revolving loan/distribution fund, and another has used the funds to purchase fire management equipment, e.g., chainsaws. Quan Hoa District: The District People’s Committee (DPC) indicated that it was pleased with the support from VFD, which focused on helping review and identify forest owners and accelerate the disbursement of payments. PFES funds were distributed in 124 village communities in late July 2015. It is unknown when payment distribution would have been possible without VFD support — possibly after the ongoing provincial forest inventory is completed. Hoi Xuan Commune, Quan Hoa District VFD’s support helped officials identify forest owners and accelerate the PFES disbursement in six villages, covering about 750 households. At the DPC meeting, the FPDF ranger characterized VFD’s support for PFES in his district as “highly effective.” Nghe An Forest Fund The GIS training to enable rangers to conduct GPS forest owner mapping, in combination with the use of satellite imagery, was helpful; however, it involved only two or three staff members/rangers. Mapping training was high quality; the training improved the skills of those who participated, but few staff have received the training. It is not clear if additional training will be provided. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 12 VFD hired a consultant to conduct a three-year review of national PFES implementation. The information has led to discussions on increasing the flexibility of the PFES payments within watersheds, as now a great deal of disparity exists between households. For example, the difference between the highest payment (VND 604,690/ha/year in Binh Dinh) and the lowest (VND 825/ha/year in Dien Bien) is VND 603,865. A number of workshops and meetings have followed the completion of the report. The Nghe An FPDF Director told the evaluation team, “We are working toward a refinement of how PFES payments are calculated. This has been VFD’s most important contribution to date in terms of supporting PFES in Nghe An.” VFD is also providing support for community workshops to encourage communities to bundle payments, which community leaders and FPDF officials have considered helpful. Nghe An Women’s Union The Women’s Union engaged with VFD on the following activities: (1) gender training; (2) implementing PFES communication trainings in three districts (Tuong Duong, Que Phong and Ky Son) for women from the districts and communes; and (3) providing comments on drafts of PFES communication flip posters. The union said it had participated only as an invited stakeholder and was unsure if it would continue involvement in the program. PFES Knowledge A household (HH) survey in three communes — Luong Noi, Thiet Ong, Thiet Ke in Ba Thuoc District — measured PFES awareness.5 The results are in Table 3. The data shows high general knowledge on PFES, but calls for further education to help its audience understand the details. The team also interviewed 10 people from Dien Trung commune who attended livelihood activities in the district; nine of them did not know about PFES and one had heard of the program, but was not familiar with its details. Conclusions The support VFD has provided to PFES has been relevant and effective. VFD has provided extensive support to PFES at the national level, as well as operational support in Nghe An and Thanh Hoa. A positive working relationship exists between VFD and VNFF. VNFF’s director characterized VFD’s work as highly effective. In part, the positive working relationship has resulted from a clear alignment of priorities between VFD and VNFF. In its two focus provinces, VFD works in five out of 13 PFES districts, which indicates there is an opportunity for VFD to extend some of its already successful efforts to other areas. 5 Sixty-five people were randomly selected from VFD’s list of PFES meeting participants. Forty were interviewed and 25 could not be located. Table 3. Awareness of PFES among VFD PFES Training Participants: Ba Thuoc District Level of awareness % of Respondents (#) Aware of PFES 97.5% (39) Fully understand PFES 35% (14) Familiar with PFES, but not sure of the details 62.5% (25) Do not know about PFES 2.5% (1) (Source: Data from VFD Mid-term Evaluation Household Survey, 2015) Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 13 VFD’s accomplishments have included a policy review of three years of PFES implementation, which has led to a national policy on payment noncompliance. Work is underway to update the PFES policy (Decree 99). VNFF and the provincial fund managers who participated in a PFES community of practice to share experience between provinces valued VFD’s work in creating the forum. In Thanh Hoa Province, VFD has provided effective training to the FPDF for forest demarcation and forest owner identification in Thanh Hoa. This work has helped to complete forest owner identification and enable PFES payments in two districts that had not previously distributed PFES funds. In Nghe An province, VFD conducted a three-year review of PFES that identified issues relating to non￾transparent payment distribution, low awareness of PFES and the “misuse” of PFES funds. In response to these issues, VFD is supporting development of: (1) provincial PFES guidelines; (2) a framework for village regulations on using PFES money, which includes an accounting book template to record PFES expenditures; (3) behavior change and communication materials; and (4) mapping and forest owner identification in Nam Pong Basin using GIS and satellite imagery. VFD’s main PFES partners are not clear about the priorities or schedule for future support (as per stakeholder interviews). These partners include the FPDFs in Thanh Hoa and Nghe An, and Pu Hoat Nature Reserve staff. While VFD’s support has been targeted and helpful, it has at times been light in terms of the depth of its engagement to achieve particular objectives. For example, VFD provides skills training to provincial FPDF staff, but may train only a few of the total number of rangers who could benefit from training. As another example, GPS training will be conducted, but GPS units are not provided so there is a limited opportunity for the training to be applied. C. Reducing Emissions from Deforestation and Forest Degradation (REDD+) REDD+ is an effort to create financial value for the carbon stored in forests, offering incentives for developing countries to reduce emissions from forested lands and invest in low-carbon paths to sustainable development. Vietnam has taken important early steps to develop a REDD+ program and to integrate REDD+ into its national forest sector and green growth strategies. VFD has provided support for these initiatives. VFD’s support has focused on working with the World Bank’s Forest Carbon Partnership Fund (FCPF) to develop the Northern Central Emission Reduction Program Idea Note (ER-PIN) and on supporting the development of two feasibility studies that will contribute to the development of REDD+ action plans in Thanh Hoa and Nghe An. VFD Activity National and Regional-Level Support VFD reports that it has provided support for the development of the Northern Central Emission Reduction Program Idea Note (ER-PIN), which FCPF has approved. This has led to the release of $650,000 for further development of Vietnam’s REDD+ program. VFD, with other REDD+ projects and Vietnam’s REDD+ Office (VRO), established the FCPF working group and is supporting an Emission Reduction Program Document (ER-PD) for six Northern Central provinces. Two of these provinces include VFD’s focus provinces. VFD will work to help these provinces meet REDD+ requirements. Specific VFD support for REDD+ has also included the following, as per VFD reporting:6 6 Based on VFD’s Quarter 3 FY2014 quarterly report. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 14 • VFD finalized two feasibility studies for REDD+ in Thanh Hoa and Nghe An provinces that include Tier 1 emissions estimates for both forestry and agricultural emissions. These emission estimates will be further refined through the Forest and Land Cover Assessment study, which has not yet begun. • During the second quarter of FY 2014, VFD provided both funding and technical support to the Vietnam REDD+ Office to develop the ER-PIN for submission to the World Bank’s Forest Carbon Partnership Fund. In June, the World Bank approved this ER-PIN. • The GVN plans to develop provincial REDD+ steering Committees, but these groups are not yet established. • A Thanh Hoa REDD+ study tour was conducted to Lam Dong. • A field assessment and a consultation workshop on deforestation and forest degradation was conducted. • VFD produced a review of forest and forestry land changes at the provincial level, which includes forest carbon stock assessment and provincial forest mapping. (Technical studies completed include: historical forest cover and land use change assessment; assessment of carbon stocks using NFI 1-4 data; and review of three provincial forest types: protection, production and special-use.) • Overview REDD+ trainings for staff at all levels and district PCs in Thanh Hoa and Nghe An. In Nghe An REDD+ support has included the following: • Consultants conducted an assessment on deforestation and forest degradation, which included field visits to Ky Son, Tuong Duong and Quy Hop District; and two consultation workshops, with the participation of district PCs, FPDs, the forest protection management board, Pu Hoat Nature Reserve, Pu Huong Nature Reserve and Pu Mat Nature Reserve. • Fieldwork led to the development of forest cover and agricultural maps from 2000–2014 in Con Cuong, Tuong Duong, Ky Son and Que Phong. The REDD+ office organized a consultation workshop to get comments and ideas from related departments and agencies. • Winrock contracted the Forest Resource and Environment Center (FREC) in October 2014 to conduct the following for provincial REDD+ action planning: o Development of forest cover maps for Thanh Hoa and Nghe An in 2000, 2005 and 2014 (completed July 2015). o Development of maps of land use/vegetation (agricultural and forestland) status for Thanh Hoa and Nghe An in 2000, 2005 and 2014 (was scheduled for completion by June 2015). o Assessment of changes in forest and agricultural land in the periods of 2000–2005, 2005–2010 and 2010–2014 (was scheduled for completion by August 2015). o Carbon accounting and development of reference emission levels (REL) and reference levels (RL) for the periods cited in the previous bullet (completed in early August 2015). o Provided trainings on REDD+/REL for forest managers, remote sensing and GIS (August 2015) for technical staff. o Fieldwork took place in cooperation with provincial forest inventory and planning entities (within the provincial DARD). Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 15 Findings/Stakeholder View There was little information among provincial stakeholders regarding VFD’s support for REDD+. The evaluation was able to speak with staff from the FCPF and from Vietnam’s REDD+ Office. The following are their viewpoints on VFD’s assistance. Comments from the FCPF FCPF highly values the support of VFD in exchanging information (on forest and land use, GIS data, etc.). This contributes significantly to the setting of baselines for REDD+ in Thanh Hoa and Nghe An provinces, reduces time and avoids duplication of effort. FCPF views the VFD program as an important long-term REDD+ implementation partner for the design and success of the emission reductions program for the region; cooperation is expected to lead to Vietnam eventually receiving significant funds from the Carbon Fund (Communication with FCPF REDD Readiness Project). Comments from the Vietnam REDD+ Office Overall, the cooperation between VRO and VFD is limited. No joint activities have been implemented. Comments on specific activities follow: · VFD has coordinated with FCPF-REDD+ to support the development of REDD+ in Nghe An and Thanh Hoa. These are two of the six provinces where the GVN plans to develop REDD+ programs. Once the provincial plans are approved, VFD has committed to providing support for plan implementation. Specific areas of assistance have not yet been identified. · The coordination of VFD with other [REDD+ support] projects and agencies is weak. This is due to complicated procedures and unclear decision-making. · VFD has not provided support for the development of the framework for MRV, as it is being developed and fully supported by FCPF- REDD+. The reference database for the period of 2000- 2010 is also being finalized with support from FCPF- REDD+, not from VFD. Conclusions VFD worked with the FCPF to develop the Northern Central Emission Reduction Program Idea Note (ER￾PIN), which FCPF has approved. The approval of the PIN resulted in the release of $650,000 for the development of Vietnam’s REDD+ program. FCPF expressed appreciation for VFD’s support. This has been VFD’s most significant REDD+ contribution to date. In addition, VFD is supporting the development of two feasibility studies that will contribute to the development of REDD+ action plans in Thanh Hoa and Nghe An. Overall, VFD’s assistance to support REDD+ has been helpful. D. Development of Green Growth Action Plan in Thanh Hoa Vietnam’s Green Growth (GG) Strategy 2011–2020, with projections extending to 2050, supports low￾emission development, enrichment of natural capital, emissions reduction and increased greenhouse gas sequestration. The objectives of the National Green Growth Action Plan 2014–2020 include: (1) establishing local-level green growth regimes and action plans; (2) reducing emissions and promoting clean and renewable energies; (3) green production; and (4) sustainable consumption at the provincial level. The Department of Planning and Investment (DPI) is responsible for producing the GG Action Plan for Thanh Hoa, which covers all development sectors. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 16 The GVN selected the Thanh Hoa Province to pilot the green growth planning process; VFD has provided input into the plan’s forestry and agricultural sections, and separate consultancies by the USAID Bangkok LEAF project provided input into the plan’s sections on energy, industry and waste management. A draft green growth plan was recently submitted for DPI review, although it requires further work before it can be finalized. VFD Activity VFD organized a training for the GG task force on greenhouse gas (GHG) accounting. VFD’s GHG accounting training was three days in duration, and the subjects were complex. Besides training, VFD supported five members of the GG Working Group to participate in a low-emission and GG workshop in Indonesia and supported DARD to organize a workshop on green growth in agriculture in Thanh Hoa. VFD also conducted a GHG accounting study on deforestation, forest degradation and agriculture production for DARD. The first draft of the Thanh Hoa GG Action Plan included the results of the accounting input. Other VFD assistance included hiring consultants to conduct assessments on green production in related provincial enterprises and departments; green lifestyle in the province; and animal husbandry and emissions. To support Thanh Hoa’s GGAP development, the USAID LEAF project provided training on energy planning tools, assessment of renewable energy and marginal cost abatement analysis. Findings/Stakeholder Views The Thanh Hoa DPI reported that training participants gained basic knowledge on GHG accounting, establishment of baseline emissions levels, renewable energy and the marginal abatement cost curve. Following the training, the GG Working Group planned to oversee the GHG emissions inventory process supported by VFD, but did not have sufficient experience or knowledge; they have continued to rely on VFD consultants to provide inputs for the plan. Almost all work related to the development of the GG Action Plan lacked participation from working group members; VFD consultants did the work independently, according to comments from DPI and the Department of Natural Resources and Environment (DoNRE). This included a lack of coordination in developing consultant terms of reference and in selecting the consultants. These institutions reported that research results were not satisfactory due to data errors that resulted from failing to consult with local DoNRE staff. Some results of the VFD consultants’ research did not meet DoNRE’s expectations and reportedly did not always reflect local realities. The work supported by VFD did contribute to a first draft of the GG Action Plan. The Thanh Hoa DPI commented: “It was written by VFD’s consultants and looks like a research paper rather than an action plan; and the plan was somewhat of low quality.” Conclusion VFD’s training support helped local stakeholders increase their knowledge of GG planning. However, local institutions have not gained enough skills to develop and implement a green growth action plan autonomously. A first draft of the Thanh Hoa Provincial GGAP is nearly complete. However, the assistance provided by VFD did not satisfy provincial government stakeholders interviewed by the evaluation team, who cited insufficient local engagement and consultation, data errors and unsatisfactory results. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 17 E. Revision of Land Use Plans in Thanh Hoa and Nghe An Vietnam’s provinces are rewriting their land use plans to include climate change considerations. VFD has provided support to integrate climate change considerations into the Thanh Hoa and Nghe An provincial land use plans. VFD Activity A VFD consultant conducted an assessment on district and provincial land use plans in Thanh Hoa and Nghe An. The consultant produced a set of recommendations to revise the plans. Subsequently, in each province, VFD organized a workshop to discuss and validate the findings. After collecting feedback from workshop participants, the consultants revised the report. The report is available in VFD’s archives and has been sent to Thanh Hoa PPMU and Thanh Hoa DoNRE; in Nghe An, the PPMU has yet to receive the report. VFD reported that an agreement on next steps is lacking. The provinces have since employed a company to further develop the Land Use Plan 2016–2020 and are not requesting further VFD support. Findings/Stakeholder Views DoNRE staff in Thanh Hoa noted the following: • VFD hired a consultant to help integrate information into the land use plan; however, the role of the consultant was somewhat unclear. DoNRE had expected the consultant to write the plan, but the consultant’s actual role seems to have been to visit Thanh Hoa to collect information and reports. He did not conduct field visits to collect updated information and DoNRE indicated he did not spend enough time in the field. DoNRE was unhappy with the assistance and indicated that some of the recommendations made are not practical. DoNRE did not receive training in integrating climate change concerns into land use plans; the agency received only the assistance of a consultant. • Thanh Hoa DoNRE reported that work is ongoing, but is of "low quality" and not progressing well. The agency assessed VFD's land use planning support as "somewhat effective." DoNRE staff in Nghe An noted the following: • VFD hired a consultant to review the land use plan and make recommendations for integrating climate change concerns into the plan. However, DoNRE expected the consultant to produce an updated report, but the consultant interpreted the job, per the TOR, as just making recommendations for updating the plan. • DoNRE stated it was not involved in drafting or reviewing the TOR and reported having “zero voice” in the selection of the consultant. Much work remains to be done. DoNRE did not have any knowledge of plans by VFD to provide follow-up support. The department said the climate change integration consultancy required a lot of work on the part of department staff, but the result was not useful. They suggested that if VFD supports additional consultancies DoNRE staff should be more directly involved (e.g., working with the consultant), as they have better knowledge of the local issues and direct involvement is the only way to improve their skills and knowledge. DoNRE staff assessed VFD’s effort as “low quality” and “somewhat ineffective.” One of VFD key proposed results included two provincial level integrated land use plans designed and implemented.” These plans are being developed by DONRE staff in Thanh Hoa and Nghe An, but have not used VFD’s inputs. In both provinces, DONRE staff indicated they do not desire further VFD support. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 18 Conclusion VFD provided some initial support for the integration of climate change concerns into land use planning. This seemed to be help the provinces gain a somewhat better understanding of how to revise their land use plans. However, VFD’s support did not meet the provinces’ expectations in terms of its focus or output; both provinces reported that the assistance was of low quality. This dissatisfaction relates to the process of identification and implementation of VFD’s support, for which local stakeholders were not sufficiently engaged — from TOR development to activity implementation. VFD’s assistance under this activity was not successful in building DoNRE’s capacity. F. Strategic Action Plan for Thanh Hoa Bamboo Development Thanh Hoa province has the largest area of planted Luong bamboo in Vietnam (70,000 ha).The province is pursuing the development of a bamboo sector strategy, which is a planned public-private partnership to increase production, add high-end processing and help growers improve their production techniques. DOFA is leading this effort, but it also involves DPI, the Department of Trade and Industry (DoTI) and DARD. VFD Activity VFD has helped create the bamboo sector strategy development plan by hiring consultants from the International Bamboo and Rattan Association (INBAR). The Department of Foreign Affairs (DOFA) led the strategy’s development, which included participation from DARD, DPI, DoTI, VFD and two private sector companies. The plan is under review; once finalized, it can serve as a basis for determining further VFD assistance. This concept pre-dates VFD’s engagement by several years, but interest by the Lam Son Sugar Company (Lasuco) in building a large, higher-end processing plant is a somewhat new addition to the concept. VFD has contributed to the development of two documents: (1) the Strategic Action Plan for Thanh Hoa Bamboo Development and (2) Guidelines for Investment and Corporate Responsibility of the Bamboo Sector in Thanh Hoa Province. Other VFD assistance has included: • Support for a Thanh Hoa delegation to attend a global bamboo event in Ho Chi Minh City, which included a marketing room for Thanh Hoa’s bamboo products. The Tre Xu Thanh Company revealed that participation in the exhibition led Southern clients to order charcoal. • Support for 10 bamboo enterprises to promote their products at a national exhibition in Hanoi. • Sponsorship of a study tour for a Thanh Hoa delegation to China; including an experience-sharing event after the study tour, a training on finance sources for bamboo and a workshop on product promotion with the Quan Hoa Bamboo Enterprise Group. VFD has also coordinated with Lasuco to develop a possible investment in a bamboo eco-park and a potentially large bamboo processing facility. A bamboo processor in Quan Hoa District, Tre Xu Thanh Company, has also been identified for support in providing low-emission charcoal production. Findings/Stakeholder Views DOFA took the lead on the development of the provincial bamboo strategy and characterized VFD’s assistance as “highly effective.” Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 19 The bamboo strategy concept is in an early stage of development and the strategy does not yet include a detailed work plan or coordination mechanism. As conceived, the activity will benefit substantially from a large processing facility investment by Lasuco. The company is considering proposed two projects: • Bamboo park (estimated investment fund: $3 million) • Strand wooden bamboo production (estimated fund: $20 million) Lasuco indicated it is too early to tell if they will invest and the decision will depend on many factors, including government support and incentives, a feasibility analysis and approval from their operating board. Lasuco’s investment decision is likely several years away, due to the complicated process. The project does have a potential high pay-off. It focuses on increasing income for some of the province’s lowest-income households and could benefit up to 1 million people if fully realized. DOFA estimated that the average bamboo-producing household’s income could increase many-fold — perhaps as much as 500–700 percent. Conclusion VFD’s assistance has been helpful, but the concept is in an early stage. The activity offers a high potential payoff, but next steps are not well defined. The government is in the process of providing greater clarity. G. Water Accounting in Nghe An Proper water resource management is critical to Vietnam’s future growth. VFD plans to pilot approaches in ecosystem services measurement and natural capital and water accounting in Nghe An. The province has a growing beverage, sugar and dairy industry, so proper water management will be necessary to ensure sustainable economic growth. VFD Activity VFD has undertaken several pilot water accounting activities, including: • An initial study on water accounting in the Ca River Basin that identifies problems, objectives and challenges for the basin’s water management. • A number of related trainings, including: 1) introduction of the water accounting framework and accounting techniques; and 2) access to open-source satellite data and its usage for the accounting. • Development of a white paper that introduces water accounting methods and identifies water issues in Ca River Basin and the need for water accounting. VFD held a related workshop in July 2015. • Support for the establishment of a working group that includes staff from the DARD Department of Irrigation and the Department of Water Resource, Hydrometeorology, Sea and Islands within the DoNRE’s Integrated Water Resources Program. • Strengthening water accounting capacity via trainings and workshops for the Ca River Basin Management Council, which the DANIDA Project helped support. VFD’s future assistance plans include training on remote sensing and accounting processes; coordination with the new ADB project on water accounting, which covers the Ca River Basin and 15 others in Vietnam; completing data layers for the Ca River Basin on surface and ground water; and development of future implementation plans. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 20 Conclusions VFD has provided initial modest assistance to date, but this activity is in a very early stage of development. H. Nature Reserves VFD is supporting improved management in four protected areas. The evaluation was able to visit two protected areas, Pu Hu Nature Reserve in Thanh Hoa and Pu Hoat Nature Reserve in Nghe An. These are relatively large nature reserves (Pu Hoat is 85,000 ha; Pu Hu is 23,000 ha). Information on the status of these activities follows; it comes primarily from Pu Hoat, where the VFD program has focused. VFD Activity VFD supported seven trainings in the Pu Hu Nature Reserve on communication, PFES, seed management, (Decree #160), proposal writing, community development, forest patrol and using tablets for data collection. In Pu Hoat Nature Reserve (PHNR), VFD has provided the following assistance. • Supported PHNR’s study tour to the Phong Nha-Ke Bang National Park in Quang Binh and the city of Hue. • Supported PHNR to provide training and raise awareness on forest fire control and forest protection to 13 villages in five communes. The training was considered effective and the nature reserve said it would benefit from additional support in this area. • Supported PHNR to demarcate forests in two communes. This work is ongoing, with participation by commune authorities and villagers. Managing forest uses and threats requires mapping and demarcation and a lot of work remains to be completed. • Signed a contract with Vinh University, which has helped PHNR develop a biodiversity monitoring system for use by its rangers. The work has been of high quality and both Vinh University and the PHNR are appreciative of VFD’s support. • Supported a biodiversity survey training by Vinh University that was successful; the PHNR manager recommended additional similar training. He commented: “Following the biodiversity training from Vinh University, our staff can take action to define management needs and actions. Previously we looked for illegal activity but, after the training, we have incorporated biodiversity monitoring into our patrolling work. We now also bring cameras on patrols to take pictures of new species. Our awareness of biodiversity has increased a lot.” • A biodiversity survey with Vinh University’s professionals and students is ongoing, to finish by the end of 2015. Biodiversity support from the university has enabled staff to conduct monitoring (transect lines and species identification). Rangers now identify and monitor select keystone species (flora and fauna) and report threats and incursions. • PanNature is carrying out a co-management feasibility study for the protected area, which is ongoing. The reserve has had a number of discussions with VFD for a range of support, but PHNR staff said they have no idea if additional support is forthcoming. Findings/Stakeholder Views Pu Hoat staff observed, “Twenty-two staff went on a study tour to other parks and nature reserves within Vietnam to study management models and activities. Staff were previously not familiar with forest protection and fire suppression practices, but we learned a lot on the study tour. Our patrolling system has improved Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 21 and we are now much better at preventing illegal exploitation.” While the training the PHNR received was useful, they thought it involved too few staff and did not meet their overall management needs. As such, they assessed VFD’s support as “somewhat effective” in strengthening staff capacity to manage the nature reserve. They characterized planning with VFD as “one-way,” with little opportunity for input or effective joint planning. The reserve’s management team expressed some frustration about interactions with VFD. “We are asked our needs by VFD and were requested to submit a proposal to the PPMU on developing a nursery/seedling garden, which we submitted in December 2013. This was over 19 months ago (seedling garden) and we do not have any word as to if or when it will be approved, although VFD assured us a year ago that they would approve the activity proposal. There has been some back and forth communications with VFD on the proposal’s ideas and budgets, but so far no approval. The proposal process is completely unclear. There are no guidelines for proposals, no formats, no sense of the level of budget that might be available and no clear criteria for approval. Under normal projects, we submit proposals to the PPMU and there are clear guidelines and processes, but not for VFD.” Regarding consultant studies/assessments: “Consultants come and do their work, collect data and leave without ever sending us their reports. A lot of consultants come to do studies or training, but the quality is mixed. Many just report to their organizations and we do not have a chance to be involved or to review the reports. There is a lack of coordination and engagement. A PFES study was conducted in the area but we were not informed as to the results.” “It would be better to involve our staff more in the studies undertaken. We know the area better and we will learn more if directly involved. We need more on-the-job training and less theory. [PHNR] was not involved in reviewing consultants’ TORs. We do not have input into the TORs; we do not have any approval over the consultants used; we do not have an opportunity to review or comment on the reports that are produced.” “VFD’s work planning process is very slow and unclear, and lags our own process so we cannot count on their support, or coordinate, when we make plans. We are very confused about VFD’s working procedures. They are not clear.” Conclusions VFD’s protected area work focuses on improving forest protection and management. For Pu Hoat, VFD has provided a fair amount of support to nature reserves, and especially to PHNR. Recipients have considered this support helpful and effective (training, study tours, field mapping, demarcation, community awareness￾raising), but somewhat inadequate considering their overall needs. The training for Pu Hoat by Vinh University was critical to the establishment of a biodiversity mapping and monitoring system, which is now operational. On the other hand, management staff viewed the support as somewhat piecemeal, noting that it included “some on this and some on that, some in here and some in there, with considerable time delays between activities. VFD’s work has not made significant or considerable contribution to the management of the nature reserve.” I. Forest Management Support – Others VFD supports a number of other forest management activities. The level of support for these activities has been mostly small. The evaluation team did not have time to review these activities, but other activities include support for participatory forestland allocation (to clarify legal status/tenure) and forest restoration and seed and management. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 22 IV. VFD PERFORMANCE: ADAPTATION Evaluation Question: To what degree is the project achieving results within its key result areas under IR 4 (“Increased resilience of people, places and livelihoods in delta areas”)? Evaluation Question: Are project activities, e.g., training, climate vulnerability and land use assessments, and organizational development support resulting in the achievement of the higher-level results? The objective of the component on climate-change adaptation (IR 4) is to increase the resilience of people, places and livelihoods in delta areas through assistance for adaptation and disaster risk management in support of the National Strategy for Natural Disaster Prevention, Response and Mitigation to 2020.7 The American Red Cross (ARC) and the Vietnam Red Cross (VNRC) implement the majority of activities under this component. The approach is to build local capacity in adaptation and disaster risk reduction planning, and to assist communities to take actions to reduce the impacts of natural disasters like floods and storms. VFD is committed to providing support in 60 communes in Nam Dinh and Long An (see Table 5). The assistance includes: 1) Capacity building on community-based disaster risk management (CBDRM) with community-based disaster risk assessment (CBDRA) as an entry point, and the development of community disaster preparedness and climate change adaptation plans; 2) Assistance to implement school-based disaster risk management activities; 3) Assistance to implement Program Enhancement of Emergency Responses (PEER); and 4) Assessment and upgrading of early warning systems (EWS). Other activities include review and upgrade of commune evacuation shelters with improved water and sanitation (WATSAN), and identification of low-cost solutions for households’ safe drinking water. VFD’s assistance includes not just technical assistance, but also financial support. Each commune can receive grants up to $23,000, which may include up to $11,000 for a microclimate change mitigation project; $6,000 for upgrading EWS; $5,000 for emergency response team equipment; and $1,000 to support a safe school model. At the provincial level, VFD supports capacity building for provincial hydro-meteorological forecast stations in both Nam Dinh and Long An. In Long An, VFD has provided additional assistance to DoNRE to upgrade the provincial climate change adaptation action plan, and to DARD for the administration of water resource management by upgrading the M&E system and website of the Committee for Flood and Storm Control (CFSC). At the national level, VFD supports policy formulation activities, including development of an enabling decree under the law on natural disaster prevention and control (with MARD) and the application of mobile short messaging service (SMS, or text messaging) for a disaster warning system with the Ministry of Information and Communication (MoIC). VFD support for DRR and adaptation is presented in the following table. Table 4: VFD-Assisted Sites for Disaster Risk Reduction and Climate Change Adaptation in Nam Dinh and Long An (by June 2015) 7 Detail Project Document, March 2014, page 14. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 23 Province District Commune Nam Dinh (3 districts) Provincial level Hydro-meteorological forecast station Nghia Hung (6 communes) Nghia Thang, Nghia Hai, Nghia Binh, Nghia Phuc, Nam Dien, Rang Dong Hai Hau (6 communes) Hai Hoa, Hai Phuc, Hai Dong, Hai Loc, Hai Chinh, Thinh Long Giao Thuy (7 communes) Giao Hai, Giao Long, Quat Lam, Giao Phong, Bach Long, Giao Xuan, Giao Thinh Long An (2 districts) Provincial level DARD (Water Resource Management), MONRE, Hydro￾meteorological forecast station. Can Duoc (10 communes) Long Huu Dong, Long Huu Tay, Phuoc Dong, Tan Chanh, Phuoc Tuy, Tan An, Tan Lan, My Le, Long Hoa, Phuoc Van Tan Thanh (9 communes) Nhon Hoa, Kien Binh, Bac Hoa, Hau Thanh Dong, Tan Thanh, Tan Ninh, Nhon Hoa Lap, Tan Lap, Tan Hoa A. VFD Activities 1. Implementing VCA/ CBDRA/ CBDRM Vulnerability and capacity assessment (VCA) and community-based disaster risk assessment (CBDRA) and management (CBDRM) take place the commune level. The American and Vietnamese Red Cross organizations (A/VNRC) use the organization’s existing system, from the central to provincial, district and commune levels, to facilitate the process. The central VNRC, together with ARC, oversees the process and quality of implementing assessments at project sites. VNRC provincial staff have participated a ToT course on VCA and are capable of providing training to district staff. Since VFD began, the VCA approach was replaced by the CBDRA, which is a part of the national CBDRM program. To supplement existing local capacity, VNRC provided additional training on CBDRA/CBDRM for VFD’s VNRC staff in Nam Dinh and Long An. The switch from VCA to CBDRA/CBDRM resulted in greater involvement of the local community, as technical (representatives from local authority and agencies) and community (village leaders, government leaders) groups jointly conduct the assessment. The final CBDRM report includes a CBDRA report and the action plan on disaster prevention and control. Normally, it takes five days to complete a VCA and 10 days to complete the CBDRA/CBRDM (five days for training and five days field assessment). Regardless of which approach is used, the assessment process generally follows these steps: 1) Preparation and planning: Engagement of local authorities and community members; 2) Data collection with various methods, including secondary data collection, group discussions, household interviews and others; 3) Data/information analysis and verification; 4) Development of an assessment report and action plan; and 5) Implementation of the approved action plan. To some extent, and where possible, VFD has facilitated the process of linking CBDRA with commune socio-economic development plan (SEDP) Table 5 shows progress to date. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 24 Table 5. VCA/ CBDRA/ CBDRM Activities Activities Achievement Target (LOP) # Training course on VCA/CBDRM for commune technical group 30 NA # Commune technical and community groups established for CBDRA 18 Tech group 18 Com group 60 # VCA/CBDRA completed (reports finalized and approved) 32 60 # Disaster preparedness and climate change adaptation plan developed 32 60 # Action plan integrated into SEDP at commune level 0 60 # Small grant to implement DRR intervention 12 60 (Source: VFD training database) VFD provides assistance to this activity through the Vietnam Red Cross system, which exists at all levels of government (central, provincial, district and commune). Specific assistance includes: • Building a system for disaster risk management. VFD assistance established and provided capacity building for the technical working group at the commune level. • Technical support included provision of ToT courses for VNRC staff, training on VCA/CBDRA for technical and community teams and technical monitoring and mentoring during field implementation. VNRC staff assisted with final reports and the action plan. At both the commune and district levels, ARC and VNRC are helping to develop the guidelines on integration of the CBDRM action plan into the SEDP. • Financial assistance to cover implementation costs have included the fee for the district VNRC to facilitate the process in the communes. In addition, after completion of the assessment, and based on the assessment findings, each commune may apply for a small grant to implement climate-smart micro-DRR interventions to address high-priority needs (up to $11,000). Capacity-Building Assistance and Application of Knowledge A large number of people have received training on VCA/CBDRA/CBDRM in Nam Dinh and Long An, as presented in the table below. Except for the training for technical and community groups, which was several days, the training courses and meetings lasted just four hours or less. These courses focused on the introduction and collection of information on community disaster risks, or discussions to develop the plan for community responses to natural disaster. Table 6. VFD-Assisted CBDRA/CBDRM Training and Meeting Supported (By End of June 2015) Training/Meeting Nam Dinh (2 districts, 14 communes) Long An (2 districts, 15 communes) # Event # Participant # Event # Participant Training on CBDRA/CBDRM for technical and community groups 5 105 6 150 Community meeting on VCA/CBDRA/CBDRM 58 3,112 140 8,618 (Source: VFD training database) Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 25 During field visits, the evaluation team interviewed 10 people who were involved in the CBDRA/CBDRM process at all levels. All 10 valued the effectiveness and usefulness of the VFD-assisted training on CBDRA/CBDRM. Respondents reported that the training has equipped them with useful knowledge and skills, which improved their capacity significantly. Observations from provincial, district and commune VNRC leaders and staff included: “DRR and disaster preparedness are our big focus. The knowledge and skills we got from VFD assisted￾training courses help to strengthen our capacity in this area.” “We have learned how to build an action plan on natural disaster responses.” “We have become more and more skillful in conducting CBDRA/CBDRM.” The CBDRA/CBDRM training was followed by the implementation of a participatory assessment at the commune (as presented in the table that follows), and trainees had the opportunity to practice what they learned. Field experience showed that they found the training useful and practical. Table 7. Evaluation of Training Support by Stakeholders Training effectiveness Highly effective Somewhat effective Somewhat ineffective Ineffective Total 4 6 0 0 10 Capacity improvement Improved my work Somewhat improved Somewhat not improved Not improved Total 10 0 0 0 10 (Source: Interviews with participants in CBDRA/CBDRM process at all levels) VFD supported a series of CBDRA/CBDRM community meetings, mostly for awareness raising and the collection of community knowledge on the risk of natural disaster and climate change-related vulnerability. Of 121 community members surveyed, 115 people (95 percent) thought that VFD-assisted training and meetings were useful. Among 73 people who participated in the assessment, 59 (80 percent) applied what they learned from the training in field practice, indicating its usefulness and effectiveness. Figure 1. Usefulness of CBDRM Meeting and Assessment in the Community Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 26 (Source: Data from VFD Mid-Term Evaluation Household Survey, 2015) Findings/Stakeholder Views As shown in Table 6, after two years of full implementation, VFD has achieved more than 50 percent of the target for the life of the project (32 of 60 communes have completed VCA/CBDRA, of which 12 were CBDRAs). A plan exists to update the action plans in the communes where VCAs were previously completed. Only counting CBDRA, the project is a little behind its target of achievement, but this is an unfair comparison due to the change in requirements from VCA to CBDRA. Among 60 assessments planned, 15 are at SL project sites (in Thanh Hoa and Nghe An). None of these has been conducted. ARC staff noted that they are not aware of this plan, which has changed from the initial target of 60 communes in Nam Dinh and Long An. VFD and the GVN should clarify this target, since it was a commitment under the original terms of the project. Stakeholders noted that CBDRA/CBDRM complies with and closely supports the government’s program (Program 1002). VFD technical and financial assistance are required, as the GVN does not have enough human and financial resources to carry out CBDRA/CBDRM activities on its own. Without VFD assistance, the activity might be conducted, but MARD staff told the evaluation team that the quality of the assessments and plans would not be as high. Stakeholders, especially at commune level, expressed appreciation for the CBDRA/CBDRM assistance. Particularly: • Among 73 community members who participated in CBDRA/CBDRM activities, 65 (89 percent) considered it a useful activity in their community (see Figure 1). • Commune leaders noted that CBDRA/CBDRM provided useful and systematically synthesized information for disaster preparedness and climate change adaptation. • VNRC staff noted that before CBDRA, VCA involved only facilitators from outside the community, while involvement by local authorities, mass organizations and especially the community was limited. CBDRA/CBDRM is a process that involves the local authorities, government agencies, mass organizations and other community representatives. This approach helps to improve their capacity, roles and responsibilities. Being involved as a leader in the assessment and planning helps local authorities understand their own community’s existing vulnerabilities to natural disasters and climate change, which did not previously receive much attention. There is a concern about monitoring and evaluation of this activity. VFD’s M&E plan mainly focuses on general indicators, such as number of assessment completed, the number of people trained or the number of administrative units implementing targeted actions that are resilience to climate change. There is lack of data and information on the outcomes of assistance, for example, the potential effectiveness of DRR small grants. An M&E plan to monitor the implementation of disaster preparedness and climate change adaptation plan does not exist. Status of Disaster Preparedness and Climate Change Adaptation Plan Integration into SEDP None of the approved action plan recommendations has been integrated into SEDPs because: 1) No specific government guidelines exist for integration at commune level, and it is a complex process that requires the involvement of multiple government agencies. VFD/VNRC is working with DARDs, DPI, and others to finalize the guidelines. 2) The action plans were developed and finalized outside the SEDP development cycle. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 27 3) Concerns exist about the feasibility of implementing the action plans given the limited local resources (and the competition for resources among other priorities). 4) USAID/VFD additional future assistance is unclear, due to factors like restrictions on infrastructure support or lack of funding. Twelve small grants have been approved for DRR/CCA community intervention. It is still too early to see the effectiveness of those small-scale projects. However, the evaluation team found that local communities are highly appreciative of this assistance, especially given the limited commune resources to support such activities. They expect these projects will improve communities’ preparedness to respond to natural disasters. Delays have occurred in the approval of grants for small-scale DRR intervention, as the required proposal procedures are complicated and time-consuming. The proposals have to go through multiple approval layers: PPMU, provincial Red Cross (RC), central RC and ARC. Reimbursement procedures also took time and contributed to delays in implementing the activities. The likelihood of implementing the recommended CBDRA action plans without external financial/technical assistance is slim. Several stakeholders at the provincial, district and commune levels expressed concerns about these unmet needs. In addition, the possibility to scale up this activity to non-VFD sites is questionable. None of the stakeholders were clear about how this activity could be broadened to cover other communities (i.e., they provided no financial and technical assistance plans). In Long An, staff from DARD’s Water Resources Directorate (WRD) provided training courses on CBDRA/CBDRM because of the lack of human resources at the provincial Red Cross. WRD found challenges in completing the process for provinces given their busy schedule of CBDRA/CBDRM work with communes over the past year. The Red Cross and the WRD shared their concern for the situation, but are constrained from addressing it due to limited personnel. This suggests the need for building a provincial network of technical providers and a coordinating mechanism to allow required personnel to be effectively mobilized. Conclusions The assistance from VFD to implement CBDRA/CBDRM was found to be effective, relevant and beneficial to the communities the program serves. Observations include the following: • It supports the government’s program by identifying and building the capacity of local systems in response to natural disaster and climate change. • Through participatory processes, communities have developed their own disaster preparedness and climate change adaptation plans. The plans were of high quality and they provide a framework for community action. • VFD follow-on assistance through small projects, after the development of action plans, partially meets community demands. 2. DRM school-based activities Primary and secondary students are among the groups most vulnerable to natural disasters. The Ministry of Education and Training (MOET) issued Decision #4068/QD-BGDDT Aug. 9, 2011, on the action plan of implementation, the National Strategy for Natural Disaster Prevention, Response and Mitigation to 2020. Key focuses of the MOET program include: • Develop evaluation criteria for schools/classes for prevention and disaster reduction in specific areas and in the country (2011 - 2012). Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 28 • Research and design model schools/ classes to meet the requirements of the prevention and mitigation of natural disasters in specific locales. • Financial resources: Ensure contingency funding for the prevention and mitigation of natural disasters in MoET administrative capitals for management and emergency units (from 1 percent to 3 percent). VFD has provided assistance, through ARC and VNRC, to implement school-based programs for DRR and climate change adaptation (CCA) in three districts of Nam Dinh and two districts of Long An. The programs include ToTs for schoolteachers, DRR/CCA training for students, DRR/CCA day events, and establishment of disaster risk management boards with training on the safe school framework. In addition, VFD assisted with conducting safe-school assessments, which resulted in safe school plans. Table 8. School-Based Activities (by June 2015) Activities Nam Dinh Long An Quantity # Participant Quantity # Participant # Schools planned for assistance 32 NA 41 NA # Training course on safe school model 3 61 2 54 # TOT on DRR at schools 5 116 2 58 # Safe-school assessment 10 124 9 243 # Training course on DRR/CCA for 4th- and 5th-grade students 67 2,030 77 2,368 # DRR Day school event 8 3,280 7 2,099 # Safe school plans developed 10 9 (Source: VFD training database) Assistance from VFD, mainly through ARC and VNRC, includes: • Financial support to cover training, meeting or other event-associated costs; and • Technical support directly from Red Cross staff or VFD-hired consultants. Capacity-Building Assistance and Application of Knowledge VNRC has updated previously used training materials on DRR in primary schools, which were developed by the German and International Red Cross. The school training materials include the booklet “Disaster and Climate Change Reduction for Primary School Students.” Ten sections of the booklet cover an introduction of natural disasters, human and natural disasters and risks, early warning systems and commune evacuation plans. Evaluators met with a primary school teacher in Nam Dinh who attended a two-day training course on DRR. She observed, “The training method was very creative. It helped increase attendants’ participation” and “the contents of the training were useful, practical and crucial for young students.” The training report noted follow-up activities that included development of school-specific teaching or safe￾school assessment plans, human resource allocation and implementing schedules. The plans were submitted to VNRC, and VNRC staff conducted field visits for mentoring and coaching. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 29 Percentage of students Figure 2. Distribution of Pre- and Post-Training Test Scores Number of questions answered correctly (Source: VFD training database) Figure 2 shows an analysis of existing data on students’ knowledge about climate change before and after they received training in Nam Dinh and Long An. Primary school students answered 10 questions about natural disasters, their consequences and students’ skills to react. The analysis results suggest significant improvements in students’ knowledge and skills. Specifically, before the training, only 23 percent of students responded to eight or more questions correctly. After the training, correct responses rose to 87 percent. Findings/Stakeholder Views No indicator target was set for this activity during the program design phase. VFD, through discussion with DoETs, planned to provide assistance to 73 schools by the end of VFD. The activities were planned for 19 primary schools in 18 communes during FY 2015. VNRC worked with DoETs in Long An and Nam Dinh to select participant schools. In FY 2014, 11 schools were selected (unplanned as per CA). In FY 2015, the work plan aimed to select 38 schools; by the end of June 19 (51 percent) were selected. The overall approach used is sound: capacity building for schoolteachers and managers to: 1) provide DRM knowledge and skills to students; 2) perform assessment and develop their own safe school plans. An official from PPMU in Nam Dinh noted, “Unlike other projects which were previously implemented in Nam Dinh, VFD does not only provide knowledge, but also helps build safe schools. DoET has requested this and VFD provides invaluable assistance to implement it.” While the evaluators received positive feedback on the training (ToT and safe school model), some delays occurred in providing other support to schools. In one school, where a safe-school assessment was completed, the Disaster Risk Management Board has submitted an assistance request for installing a new water filter system. However, VFD has not provided any feedback or answers. The evaluation team found no evidence that suggests efforts to institutionalize the ToT training, teaching plan or the booklet. Clarity about plans is lacking in terms of whether student training will be replicated in other communes in the coming years. Conclusions VFD assistance has made a significant contribution to implement the MOET climate change program in 18 communes of the VFD project site. Stakeholders said the assistance was effective, noting: • The program had clear objectives. 2 2 4 9 32 27 17 5 1 1 3 10 26 50 11 0 20 40 60 80 100 2 3 4 5 6 7 8 9 10 Pre- test score Post- test score Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 30 • Effective capacity building and implementation strategies included participatory training for teachers and managers, followed by specific plans to implement activities. • It provided potential benefits to more than 3,000 primary school students by increasing their knowledge and skills to respond to natural disasters. VFD has an opportunity to provide further assistance to strengthen the capacity of local systems, such as supporting DoETs to develop strategic long-term plans to implement MOET's Decision 4068 and Prime Minister's Decision 1002 in Nam Dinh and Long An. Institutionalization of training materials and approaches should benefit the community in the long term. 3. Program for Enhancement of Emergency Response (PEER) The Program for Enhancement of Emergency Response (PEER) with Community Action for Disaster Response (CADRE) curriculum has been implemented with the formation of emergency response teams (ERTs) at the commune level. Each ERT consists of 24 members from the local authority (people’s committee), uniform forces, health workers, the youth union and other community groups. VFD/ARC/VNRC assists the communes to: • Select members and form the team (specific selection criteria developed by VNRC); • Provide training on disaster preparedness and response skills; • Perform needs assessments and provide equipment to the team (up to $5,000 per team); and • Prepare and conduct commune disaster simulation drills. • VFD activity to support PEERs is presented in the table below. (Source: ARC and VNRC reports) Capacity-Building Assistance and Application of Knowledge VFD provided 30 PEER/CADRE training courses to existing ERTs. The four-day training curriculum covered a wide range of topics, including community-based practice and exercises. All ERT members in the two communes the evaluation team met with agreed that the training was useful with a reasonable integration of theory and practice. Compared to trainings some had previously attended, they said the training provided by VFD was “a different quality” and “should be applied widely.” Among 182 people surveyed in Long An and Nam Dinh, 123 (68 percent) had heard about CADRE, and 85 (47 percent) had attended CADRE training. Table 10 demonstrates the effectiveness of the training, as rated by training participants. Table 9. PEER/CADRE Activities Activity Status Quantity Actual Performance Planned Performance (Life of Project) ERTs established Ongoing 30 60 PEER/CADRE training courses Ongoing 30 60 Needs assessment of equipment completed Ongoing 36 60 Equipment sets provided Ongoing 18 60 Simulation drills/practice Ongoing 1 60 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 31 Table 10. CADRE Training Effectiveness Ranking Nam Dinh % (N) Long An % (N) Total % (N) The effectiveness of the training Normal 4.2 (2) 8.1 (3) 5.9 (5) Useful 14.6 (7) 29.7 (11) 21.2 (18) Very useful 81.3 (39) 62.2 (23) 72.9 (62) Possibility to apply trained knowledge and skill Very likely 100 (48) 97.3 (36) 98.8 (84) Will consider 0 2.7 (1) 1.2 (1) (Source: Data from VFD Mid-term Evaluation Household Survey, 2015) Findings/Stakeholder Views Overall, implementation progress is progressing as follows: • 30 of 60 of ERTs have been established and trained with the CADRE curriculum. • 36 of 60 equipment needs assessments have been completed. • 18 of 60 groups have received equipment. • Only one simulation drill/practice in Nam Dinh province has been organized, while other communes are still waiting for approval and implementation. Despite some delays in implementation (e.g., providing equipment for ERTs or organizing simulation/drills), this is an activity that has a practical significance in raising community preparedness. Community leaders (government and party leaders) highly appreciated the assistance from VFD. Stakeholders highly value the provision of equipment for ERTs based on the local needs assessment. The evaluation team found only some minor issues with the provided equipment; for example, a flashlight requires a type of battery that is not available at the commune level. Under Prime Minister's decision 1002, disaster simulation exercises need to be conducted at least once a year to improve the capacity of ERTs and communities. ARC/VNRC is working to develop simulation scenarios. Only one simulation exercise was conducted due to slow planning and approvals. It is unclear how VFD develops simulation scenarios, but VNRC is known for its experience in this area and is expected to make a significant contribution. The ERTs were built from the task force teams, which are normally under commune CFSCs and mobilized for storm and flood responses. VFD-supported ERTs were placed under management and direction of CPCs. The team has not found sound rationale for “upgrading” the task force teams to ERTs since they: 1) have the same functions; 2) have almost the same team composition, in which youth are core members; and 3) either directly (for ERT) or indirectly (the task force) are under management of CPCs. (The commune leader also leads the commune CFSC.) Because VFD is providing financial support for ERTs, one team leader raised the issue of the funding source after VFD. The CFSC task force operation is currently funded by the state budget. Conclusions VFD’s assistance for PEER/CADRE activities is practical and effective, in that: • It meets local demand for preparing the community to be ready to respond to natural disaster. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 32 • The training was of high quality and appreciated. • Follow-up assistance was provided. 4. Supporting Early Warning System (EWS) ARC and VNRC have done significant work to build early earning warning systems for natural disasters at the community and commune levels. VFD, through ARC and VNRC, is continuing to support communes to upgrade local EWSs. At the central level, ARC and VNRC are initiating discussions to establish a EWS via mobile phone text messages with the MoIC. VFD has assisted the process with EWS assessments in 18 communes. These led to recommendations and plans to upgrade local EWSs to ensure their effectiveness. During the process, VFD also supported communes to develop of hazard maps to provide people access to the information, understand about the hazard and minimize the risks when a disaster occurs. VFD assistance includes the training for assessment teams with members from provincial and district VNRC, and technical assistance during the process. Participating communes were provided the opportunity to develop proposals for upgrading local EWSs using recommendations from the assessment. Each proposal grant is up to $6,000, which is provided by VFD. Local communities are expected to contribute labor cost, or others costs if the proposed budget was more than $6,000. To date, VFD has approved 10 proposals, mainly for providing equipment for local radio networks. District radio station staff, along with related agencies, reviewed and approved all proposals to upgrade local radio networks. Capacity-Building Assistance and Application of Knowledge The capacity building on EWS mostly focused on the VNRC system at the provincial and district levels. The community groups have been involved in providing information and, to some degree, helping in the development of recommendations. Stakeholders also noted that the project has helped strengthen community capacity to maintain the EWS properly. Findings/Stakeholder Views By the end of quarter 3, FY 2015, EWS assessments have been conducted at 18 communes, involving 180 people (10 in each commune), out of 60 communes previously identified. Among those, 10 proposals for upgrading current EWSs have been approved but not implemented. Potential reasons for slow implementation include slow approval for EWS proposals. VNRC noted that it could take up to three months, but locally involved people reported that approval for some proposals took seven months. Currently, no EWS project has been completed. Procurement bidding is being conducted at the central level. Therefore, it is still too early to evaluate the effectiveness of this assistance from VFD. However, since it will partially address local system issues identified through the assessment, expectations are high that it will benefit local communities by enabling effective responses to natural disasters. At the central level, discussions are on hold between VNRC and MoIC on setting up EWS via mobile text, or SMS. ARC/VNRC has developed a concept note to promote the use of SMS for disaster response and preparedness. The concept note has been circulated to different departments within MARD and MoIC. However, little progress has been achieved because of personnel change at MoIC, or due to the low interest of the ministries. Conclusion VFD’s assistance to improve the EWS in Nam Dinh and Long An is appropriate and effective, because: Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 33 • It involves local communities in the process, supports a multi-hazard approach and increases awareness and understanding of warnings, risks and responses. • It utilizes readily available local technologies, such as local radio networks, which can improve early warning. Drills and simulations with local government leaders and agencies are expected to link early warning systems to the implementation of disaster response plans. The activity is behind its implementation schedule, especially for small grants to upgrade EWSs. The initiation of EWS using SMS technology seems to require agreements beyond the discussions between VNRC and MoIC. It may require involvement of multiple ministries. 5. Other Assistance VFD has provided assistance on other activities in Nam Dinh and Nghe An. The activities VFD supports includes: • Capacity building for provincial hydro-meteorological forest stations, • Supporting Long An to revises and upgrade the provincial climate change adaptation plan, • Set Up an online M&E database and website for the Long An Provincial Floods and Storm Control Program • Development and demonstration of climate-resilient approaches for individual houses and communal shelters with improved physical structures and WATSAN • Contributing to the development of an enabling decree under the Law on Natural Disaster Prevention and Control VFD's assistance includes need assessments, capacity training, and development of follow-up plans. Most of the above activities and VFD's assistance are in early stage of implementation. Findings/Stakeholder Views In overall, local partners highly appreciated assistance from VFD: • Key informants from Long An DoNRE noted that VFD's assistance helped to significantly improve the quality of the provincial climate change adaptation plan. The PCP and related provincial departments considered the old plan as not comprehensive and practical, and therefore has not been implemented effectively. VFD has provided technical assistance, including training, mentoring and coaching, to the DoNRE taskforce. Initial reviews by DoNRE and other provincial leaders suggested that the upgraded plan is of high quality. An interviewed DoNRE leader noted that close collaboration between the DoNRE taskforce and capable VFD technical staff (consultants) resulted in a high quality plan. The training provided to DoNRE staff was also highly effective and useful. One DoNRE leader was confident that PCP would soon approve and implement the plan. • With technical and financial support from VFD, the Long An DARD has developed an online M&E software package and website for its provincial floods and storm control program. The new M&E software package helps district staff (13 districts, one town and the province’s capital city) better manage and report program data, and therefore, data quality is improved significantly. The website is very useful in providing correct information in timely manner (including flood and storm warning) -- not only for flood and storm control staff, but also for the community. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 34 There was not much information available on VFD assistance and its contribution to the development of an enabling decree under the Law on Natural Disaster Prevention and Control. ARC perceived that the assistance was modest, e.g., providing comments and inputs to the draft decree. Other listed activities are in the planning process, and it is too early to see outcomes and assess effectiveness of VFD assistance. However, interviewed stakeholders expressed their concern that it always takes a long time to get VFD assistance plans finalized and approved. Government partners are sometimes unclear about the approval status for their proposal of assistance. B. Overall Conclusions Table 11 summarizes the progress of activities and assesses their effectiveness. Overall, most activities were effective and of high quality, potentially and significantly improving community capacity. Table 11: Summary of VFD-Assisted Activities’ Performance – Adaptation Component: Assistance Progress Effectiveness (Highly Effective, Effective, Low Effectiveness, Unclear) Long An DoNRE to revise climate change adaptation plan Ongoing Highly Effective Long An DARD online M&E system for Flood and Storm Control program Completed Effective Long An DARD to upgrade CFSC website Completed Effective CBDRA/CBDRM capacity building 30/60 Effective Integration of the action plan into SEDP 0/60 Unclear Micro mitigation project 12/60 Highly Effective School- based activities 19/73 Highly Effective Community awareness raising Planned No activity Support National Hydro- meteorological forecast 2 training for province staff Unclear Commune Early Warning System 18/60 (10 EWS upgrade proposal approved) Effective Strengthen local level response (PEER/CADRE) 30/60 (18 equipment sets provided) Effective Demonstration of disaster- resistant house 0/(no target) No activity Disaster evacuation shelters 0/12 completed Unclear Water sanitation Planning Unclear (limited activity) Support development of Enabling Decree under the law on natural disaster prevention and control Completed Low Development of EWS using mobile SMS text system Ongoing (slow) Low Effectiveness The keys for success in programing and implementing activities include: • Clear objectives and outcomes (although some specific objectives were missing in the project document and work plan). • A clear process to achieve outcomes: Building a commune’s capacity in response to natural disaster yielded obvious examples, such as knowledge and skills training and for follow-up assistance provided for specific activities that were designed with significant community involvement. • Most importantly, activities are aligned with government programs (i.e., Program 1002). In other words, assistance pushes local communities to implement government programs. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 35 The program has experienced delays in implementing small-scale projects (mitigation and EWS upgrade) and some other activities (e.g., communal shelters or water sanitation), mainly due to the late start-up of the project and the complexity of a time-consuming approval process. ARC and VNRC have encountered some challenges in providing financial assistance to implement micro projects at the commune level. Under USAID and ARC regulations, VFD cannot directly transfer funds to the CPC, although the local authority prefers to manage the funds. The identified solution seems to be effective, which is that no VFD funds were deposited into the CPC’s bank account, however the A/VNRC and the local PCP mutually agreed on activity items. The contract to provide equipment or other assistance was signed between Red Cross, PCs and contractors. The ARC made payments directly to the contractor. ARC, VNRC are serving as direct support providers, and focusing activity in VFD previously selected sites. This requires an extensive labor commitment from VNRC staff at all levels. VNRC has undertaken the activity with limited human and financial resources. V. VFD PERFORMANCE CROSS-CUTTINGWORK IN LIVELIHOODS Evaluation Question: To what degree are households benefitting from VFD activity in terms of increasing income and increasing their livelihood resilience to climate change? Do key activities appear sustainable and scalable? Note: The main ways that VFD is working to increase household income is through their livelihood and PFES activities. This section covers the project’s livelihood activities. VFD promotes a number of livelihood “models”, which are technologies to help households increase income. All VFD models also contribute to the achievement of climate change objectives, for example, by reducing emissions. Most of VFD’s models focus on agricultural production. VFD’s livelihood work cuts across the adaptation and sustainable landscapes components. For example, climate-smart rice that requires less water is an adaptation strategy, whereas promoting bamboo production is a sustainable landscape activity intended to reduce pressure on harvesting timber from forests. While promotion strategies vary somewhat among models, VFD’s basic approach is to design and test a model with the intention to scale it through farmer-to-farmer mechanisms, with support from the AECs. In some cases, public-private partnerships may also be part of the process. VFD uses a nine-step scaling process, more or less as follows: • Conduct surveys and consult with communities to identify community-appropriate livelihood activities; • Develop a technology package (generally focused on production techniques); • Select sites for the pilot activities; • Adjust the model during and following initial application; • Select participants based on criteria (usually interest and ability to participate, e.g., can provide labor, land or input contribution for the activity); • Conduct a ToT process, which includes AEC extension agents and some local farmers; • Provide training on the technology to farmers; Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 36 • Implement the model in one or more villages using local labor and land; • Establish a public-private partnership (in some but not all cases); • Develop communication strategy and materials; and • Build a plan for scaling. VFD staff have conducted training of project participants (at the commune and village levels). Training has often, but not always, included participation by AEC personnel. A. Findings 1. VFD Activity Table 12 shows the status of ongoing and planned livelihood activities and includes data taken from the VFD background document prepared for the mid-term evaluation (through June 30, 2015). Table 12: Indicator 4.3.2 Number of households showing improved income generated from climate-resilient livelihood activities as a result of USG assistance Livelihood Activity Status Number of Households Actual Performance Planned Performance (Life of Project) Bamboo Forest Model Planning 50 250 Shellac Model8 Planning 30 ? Cinnamon Model Planning 30 250 Community-based Seed Nurseries Model Planning 50 50 Improved Cow Grazing Planning 27 50 Grass brooms Planning 60 100 Straw bailing and mushrooms Planning 90 ? Agricultural residues and composting Planning 100 100 Sustainable potato production Planning 150 150 Climate Sustainable Shrimp Planning 3 ? Pu Hoat Non-timber Forest Products Planning 0 50 Sloping Agricultural Land Techniques (SALT) – Maize application Ongoing 19 250 Cook stove Ongoing 300 250 Climate Smart Rice Model (Thanh Hoa & Long An) Ongoing 65 100 Climate Smart Rice Model (Long An) Ongoing 300 ? Native Chicken Model (with earthworms) Ongoing 123 100 Low-emission corn (Long An) Ongoing 90 ? Note: Additional livelihood activities are mentioned in other documents, and the Livelihood Impact Assessment in the Uplands of Thanh Hoa and Nghe An Provinces lists a total of 25 recommended models. Models in the VFD assessment report include (in addition to the above) beekeeping, duck raising, women’s handicrafts, pig raising, goat raising, cow and buffalo fattening and home gardens. The evaluation team was able to visit a few of VFD’s livelihood sites. A description of these activities and site visit observations are presented below. 8 Shellac is a resin secreted by the female lac bug and used to make liquid shellac. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 37 Potato Model (Yen Dong Commune, Nam Dinh) The potato model is a 2.5 ha production demonstration plot farmed by 25 families under a production model earlier promoted by GIZ, the German Development Agency. Under this model, VFD has provided training in production practices and purchased seed potatoes and fertilizer. To date, there has been one harvest, which was successful. The harvested potatoes will be used for next year’s seed stock. The PPMU’s vice chair stated that the productivity of the potato model is higher than that of traditional practices in the area. VFD consultants directly conducted the training. DARD’s AEC has not been closely involved (per site interviews); however, VFD records indicate that training included ToT support to AEC. Some confusion under the livelihood models exists regarding the involvement of the AEC. All livelihood models generally involve AEC personnel in the trainings; however, when questioned, AEC staff often say the plans for their future involvement is unclear, or that no longer-term agreement on their support role exists. SRD plans to conduct a potato value-chain analysis, but this study has not begun. No formal scaling plan exists for the model, nor have any marketing strategies been undertaken. VFD has three focal districts in Nam Dinh, and this model is not located in a focal district. Smart Rice (Long An, Thanh Hoa, Nam Dinh). The evaluation team visited climate-smart rice model areas in two provinces, Thuong Xuan District, Thanh Hoa, and Can Duoc District, Long An. The models operate somewhat differently, as noted below. Low-Emissions Rice Can Duoc District, Long An promotes the use of a low-emission rice intensification model that reduces water, fertilizer and emissions. The AEC has promoted the model since prior to the start of VFD. In the first crop, the system of rice intensification (alternative wet and dry) was introduced and demonstrated in Long An with good results (a greater yield than the common rice grown in the area, according to an AEC report), but no upscaling potential or opportunities have been pursued. Due to the difficulties of the model and an unsuitable production mode and practice of the local farmers, VFD decided to go with other models that large numbers of farmers can more easily apply and upscale. The current demonstration being supported by VFD is a 1.2 ha paddy farmed by two households. The rice requires lower amounts of fertilizer and seed compared to traditional rice grown in the area, which reduced input costs. However, as per an AEC technical report on the activity, due to increased labor requirements the profitability of the rice was lower than traditional practices (when labor is factored into the calculation). In the current model, the AEC has provided training support, seed and fertilizer. To support this model, VFD helped to organize a common interest group that meets monthly and helps spread the model through farmer-to-farmer messaging. The group has 150 members, about half of whom have applied the model and new rice variety and production techniques — meaning as many as 75 to 80 households now use the model. A plan exists to expand the model to additional farmers, but it is a bit unclear and relies mainly on farmer-to-farmer expansion. VFD provided funding for the model to enable the AEC to expand its promotion and outreach efforts. This will be done through training for AEC district staff and farmers. The AEC provides support to this approach, as it is an ongoing priority program. Smart Rice Thuong Xuan District in Thanh Hoa is supporting tests of three new seed varieties to determine which are most appropriate for the area. Under this approach, VFD rented land from local farmers to use as a test plot, and SRD has been training farmers on an improved rice growing process. This model has been in operation for two years and has produced three harvests. The AEC in Nam Dinh stated that they have been running this program since around 2000 and adapted it for coastal areas in 2012. They also stated that the rice model is not receiving VFD support; however, VFD Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 38 records show that the program has provided training of trainers to support this activity. The AEC did say VFD has provided a few trainings in the area, but said these have mainly focused on raising climate change awareness. Asked about future VFD plans to support the model, Nam Dinh AEC staff said they were not aware of any planned further actions. As noted in the coordination section, VFD sub-awardees that provide assistance do not always seek approval from or coordinate with the local PPMUs. Because of this, some confusion exists among government provincial staff as to what VFD provides and its plans for future support. Sloping Agricultural Land Technology/Maize (Phuc Van Commune, Que Phong District, Nghe An) This model is being promoted in Que Phong, Nghe An (Tien Phong Commune). The size of the maize demonstration plot is two hectares and eleven households are participating. The maize is a new variety designed to offer greater resilience to heat and low rain. VFD provided training for the project and paid local laborers to dig contour lines and plant the maize. AEC provided the seed and fertilizer. The households involved volunteered to participate, with the selection criteria that they express interest and provide labor, which VFD paid. VFD’s project costs were about $1,300 for two plantings. The participating households received training from VFD consultants, with participation from the district AEC. Two crops of maize were planted about a month apart in mid-2015. Both plantings died due to low rainfall in the area (non-VFD maize in the area also mostly died). The participants said they hope to plant the crops again next season. The commune’s AEC agent was not aware of any follow-up plans by VFD. VFD has also promoted a corn model in Long An, which involves the participation of a private sector company, Ecofarm. Under this model, the harvest was successful and showed improved yield productivity; however, due to depressed maize prices, the input costs exceeded income from sales. The activity with Ecofarm began prior to VFD’s involvement; the AEC supports the model. Chicken and Earthworm Model (Nghe An) VFD has begun work on a native chicken/earthworm model in Que Phong, Nghe An. To date, around 20 households have received training (two of three planned trainings have been conducted). The plan is that the families will buy their own poultry and chicken feed once the training is complete. The evaluation team was told when visiting the site that no project chickens could be seen because they had not yet been purchased; however, the team learned from VFD in Hanoi that the families already have chickens, but the new elements are raising them and the veterinary practices. The team could not find a summary of the project that clarifies the process and future plans. Without a clear understanding of how the project is supposed to work it is difficult to determine prospects for it to be sustained or broadened (as the support process and roles are not clear). The expected level of impact – on climate change and on income – is unclear. While it is true that VFD’s models were developed through an extensive community consultation process, it is not clear that models were selected based on their potential or relative impact (relative to cost, ability to decrease emissions, or relative to the benefits of other potential activities). AEC staff said they have been involved in a number of trainings related to the VFD chicken model, perhaps as many as five. However, they also said they do not understand their role on the project, and are not able or planning to provide any follow-up support to the activity. The project is in a somewhat remote location. 2. Livelihood Training Support and Application of Knowledge VFD reports that it trained 2,800 people in livelihood training. The activity breakdown is in Table 13. Table 13: Livelihood Training by Activity Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 39 Livelihood topic Number Trained Low-emission corn (Long An) 170 Sustainable potato production 379 Climate Smart Rice Model (Thanh Hoa & Long An) 463 Native Chicken Model (with earthworms) N/A Sloping Agricultural Land Techniques (SALT)/Maize 248 Cookstoves 218 Other topics 1,322 All livelihood trainings, workshops, study tours 2,800 (Source: VFD database on training courses) Participants surveyed as part of this evaluation reported high satisfaction with the training and strongly indicated that they either have applied or intend to apply the skills and techniques they learned. The tables that follow provide information on the survey questions related to acquiring and applying new livelihood skills and knowledge. Tables 14 and 15 include data from the evaluation’s household survey on VFD training conducted to support livelihoods. Table 14: Usefulness of Livelihood Training Question Model Useful Highly Useful Total Useful & Highly Useful No. of Responses To what degree was the training useful in helping you to acquire new skills? Rice 29.3% (17) 69% (40) 98.3% (57) 58 Potato 21.4% (6) 75% (21) 96.4% (27) 28 SALT/Maize - - - - Chicken/earthworm 30.7% (16) 63.5% (33) 94.2% (49) 52 Total 28.3% (39) 68.1% (94) 96.4% (133) 138 (Source: Data from VFD Mid-term Evaluation Household Survey, 2015) Table 15: Have You Applied/Implemented the Model from the Training? Question Model Not at All Partially Fully No. of Responses Have you applied the skills learned in the training? Rice 0 20.7% (12) 79.3% (46) 58 Potato 3.6% (1) 39.3% (11) 57.1% (16) 28 SALT/Maize - Not included in survey Chicken/earthworm 25% (13) 25% (13) 50% (26) 52 Total 10.1% (14) 26.1% (36) 63.7% (88) 138 (Source: Data from VFD Mid-term Evaluation Household Survey, 2015) 3. Analysis and Scaling VFD has developed a basic nine-step scaling process. This process, however, focuses mostly on production, with some attention to spreading the models. The process for spreading the models involves communications Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 40 and awareness support (and materials), promoting common interest groups, farmer-to-farmer networks, and some coordination with the AECs, such as providing ToT workshops. In general, interaction and support with the AEC has been uneven and challenging. A number of issues exist concerning VFD’s ability to pay AEC staff to conduct training (covering their transportation costs and paying weekend salaries). In addition, the MARD has criteria for adopting and promoting new technologies. These include proof-of-concept data, and MARD has stated that VFD’s models are being implemented on a scale that is too small for the success to be validated. Thus, they are not able to generate the level of data that is required for the government to adopt and promote the models. As such, the government commitment to adopt VFD models and integrate them into their regular work appears uncertain. However, some models, such as the rice in Long An, seem to have AEC support. VFD has a general scaling strategy, but does not provide an analysis of the factors that are critical to the adoption by the AEC, or the increased capacity of government required to make this happen. Only one of the models — Long An rice — has produced a scaling plan. VFD plans to undertake value chain analyses for the various models, and a few of these are scheduled to begin soon. The VFD document “Livelihood Impact Assessment in the Uplands of Thanh Hoa and Nghe An Provinces” does not examine the priorities of the AEC in regard to agricultural models and priorities, and it does not examine the AEC’s role, capacity and needs in regard to scaling VFD models or existing AEC models. 4. Livelihood Performance and Stakeholder Observations VFD is supporting, or planning to support, at least 16 livelihood activities designed to increase household income, according to the current performance monitoring report. Of these 16 activities, VFD reports that six are being implemented and generating income benefits for the households involved. The six activities in active implementation are: • Climate smart rice models in Thanh Hoa and Long An. The project in Thanh Hoa is testing alternative rice varieties, and in Long An results indicate improved yields, but higher labor input so the cost-effectiveness is inconclusive. • The SALT/maize model in Nghe An failed in its first two plantings and has generated income from maize for participants. The maize model in Long An was successful in increasing yields, but the cost of the inputs apparently exceeded the income from sales. • The chicken model in Nghe An has provided training on the chicken/earthworm model, but it is unclear how much families are benefitting – including in comparison to common existing chicken raising practices. Only two of three trainings have been completed, although some confusion exists about whether or not families are supposed to buy chickens following the training, or if they are to apply the techniques to raising chickens they already own. • The potato model in Nam Dinh has produced one successful harvest from its 2.5 ha, but the potatoes are being saved as seed for next year and thus participants have not earned any income. • Under the cook stove model, about 100 families have received free cook stoves. This activity is not generating any income. The model could be reducing the labor time required to collect firewood, but this is unclear. The team visited two families that had received stoves and one was not using the model because it required too much splitting of wood and the other because they cook with propane – but this is an inadequate sample upon which to base conclusions. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 41 Only one of the active livelihood projects that the evaluation team visited (rice in Long An), and which VFD’s performance reporting lists as generating income, appears to be doing so. However, some question exists about whether that model is economically viable, due to its higher labor input costs. As reflected in the VFD PMP table listing active and proposed livelihood activities, most of the proposed models involve very few participants and are conducted at a very small scale. For example, the chicken model in Nghe An has a target to directly benefit 100 participants, but this is the impact expected by the activity’s scale process. Different sources reported different numbers for the initial demo/training, ranging from 11 to 22 households. The evaluation team was not able to obtain clear data on the income these projects generated, or on their economic viability, although we know these models are in a relatively early stage of implementation. Clear summary reporting on the models — such as documents that provide details on the approach, the participants, activity to date, and data on results or income — is lacking. Stakeholder Observations on VFD’s Livelihood Activities The following notes come from government stakeholder interviews concerning the performance of VFD livelihood activities. • CPMU: The models introduced by VFD to date are not successful, and they are too small. If VFD continues with its current approach on livelihood support by the end of the project there will be virtually no impact. • CPMU: The government has certain criteria for the approval of livelihood models, which include potential impact and coverage, ability to support the system, and proven effectiveness. VFD has not followed the process, or supported GVN models. This makes it difficult for the AEC to support these models. • PPMU: Financial support for “models” is limited and too low. There is too much training and not enough implementation support. • PPMU: The activities VFD conducts are done on a very small scale. There is no impact from VFD’s work. • PPMU: The project is very busy. We spend a lot of time on field visits, workshops, surveys and training, but there is no impact. VFD activities are too small. • AEC: We do not have any ownership or involvement in the chicken-raising model activity, and we do not believe the project is having any impact. • PPMU: VFD’s livelihood models are not proven, and they are different from the models being promoted by the government. VFD’s models seem spontaneous and they cannot be scaled without government support, but there is no significant support. From VFD’s perspective, many of the models have government support or are trying to improve on existing practices. It is possible that the apparent difference in perception between the government and VFD regarding whether or not the models have government support has to do with the way the models are implemented and their small scale. For example, while government may favor supporting the types of projects VFD implements, they may not feel the scale and process will lead to significant impact. This may account for the differences in opinion the evaluation team heard from various stakeholders as to the GVN’s support for VFD livelihood activities. As noted, the GVN has voiced general support for several of VFD’s models — including climate smart rice in Long An, and perhaps for potatoes in Nam Dinh. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 42 B. Conclusions Limitations It was difficult for the evaluation team to assess the livelihood component, and we expect various stakeholders will come to different conclusions about some activities. Some confusion or differing understanding exists between VFD, local participants, PPMUs and local AEC staff regarding how the models are intended to be supported, future activity plans and to what degree they are AEC models or are new VFD￾developed models. The evaluation team was unable to clarify these details because of the differing viewpoints, and because plans that describe the livelihood models seem to be lacking. The GVN’s support for VFD’s models may not be black and white. The GVN may in principle support several of VFD’s livelihood sectors, or commodity support efforts, but may not support the approach. The dissatisfaction with the models may be due to VFD’s implementation process and the scale of implementation. The AEC, as a large agency, would prefer to directly implement the activities, and thus would prefer to have direct financial support instead of technical assistance. This appears to be a significant issue, as does the issue of the small scale of the livelihood activities. Overall Conclusion Many or most of VFD’s livelihood models may be successful as demonstration models, but the short implementation time makes this unclear. Significant impact from the VFD livelihood models is unlikely by the completion of the project. This conclusion is based on the following: VFD’s models are being implemented on an extremely small scale; they have experienced mixed performance; AEC’s support for the models is lacking, or unclear; and the models are not supported by value chain analysis or realistic scaling plans. Conclusions on particular aspects of VFD’s livelihood component follow. • Training: VFD has provided training to more than 2,800 people on its livelihood models. The participants report high satisfaction with the training (98 percent of participants rated the training as useful or highly useful) and a high percentage (64 percent) either are implementing or plan to implement the models. • Alignment with GVN programs and priorities: A clear definition of the AEC’s role in supporting VFD’s agricultural activities is lacking. The scale of the models promoted by VFD does not enable collection of data on the model’s success that is adequate to meet government requirements. Thus, GVN/AEC does not appear to have plans to adopt and support VFD’s models – except for those it already supports, such as rice in Long An and potatoes in Nam Dinh. • VFD signed a memorandum of understanding with the national AEC on general cooperation to develop climate smart agriculture (CSA - January 2015). However, the document is vague; it does not list specific activities and has no implementation plan, budget or guidance on available resources. • From VFD’s perspective, alignment of livelihood models with GVN programs is not always an explicit objective, as VFD’s intent sometimes involves promoting new and more effective production techniques (which also address climate change issues). In this case, VFD’s approaches may be technically sound and advantageous; however, plans to sustain the efforts require further clarification as does an analysis of likely effectiveness. • Implementation Status: VFD has six livelihood models in an active state of implementation. Some projects are showing some success, for example high yield rice, but participants have not begun Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 43 earning higher income under any of the models. Current and planned projects may earn significant income for direct participants, but it is too soon to tell. To date, six of VFD’s planned 16 (or 25) livelihood models have begun implementation. The process to introduce and test these models is time-consuming and labor intensive (with all VFD support being provided from Hanoi), and it is unlikely that VFD’s remaining timeframe is sufficient to allow new models to be implemented, validated and sustainably adopted. • Scaling and sustainability: VFD’s models do not include documented or credible scaling plans, or clear plans to support the projects once VFD ends. VI.VFD PERFORMANCE CROSSCUTTING THEMES This section addresses VFD’s support for the following crosscutting issues: NGO institutional development; building political awareness for climate change action; and gender integration. A. NGO Capacity Building 1. VFD Activity VFD’s support to Vietnam NGOs is in two main areas: focused organizational strengthening for six main NGOs, and NGO network strengthening targeted to 35 members of the Vietnamese NGO and Climate Change Network. Target organizational development support has included training and conducting organizational capacity assessments (OCA), followed by coaching and mentoring. VFD provided support to help the NGOs conduct OCAs, a self-assessment process to review organizational capacity on seven key aspects.9 The results reveal each organization’s strengths and weaknesses and are used for self-improvement. Selected organizations received follow-up coaching and mentoring. The Center for Sustainable Rural Development (SRD), VFD’s sub-awardee and a beneficiary of organizational development support, coordinates VFD’s assistance to strengthen the local NGO network. VFD organized training on climate change literacy, PFES and REDD+, and skills training advocacy, communication, proposal writing and planning. 2. Findings/Observations Practical training focused on organizational development has been conducted since year 2 of the project. Courses for six local NGOs covered topics that included organizational development, proposal writing, monitoring and evaluation, and financial and administrative management. • The perceived effectiveness of the knowledge and skills provided to participants was generally positive. For example, a Center for Marine Conservation and Development (MCD) staff member reported that the two-day training course on proposal writing provided hands-on skills and knowledge that he found applicable and useful to his work. All NGOs expressed appreciation for the training on financial management. • All four NGOs interviewed said the training on financial management, which focused on compliance with USAID requirements, was “very useful.” Three of these four NGOs were implementing 9 Seven aspects assessed in a VFD OCA include: (i) governance; (ii) organizational management; (iii) program management; (iv) project performance management; (v) administration; (vi) human resource management; and (vii) financial management. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 44 activities for USAID, including for VFD’s work. One NGO indicated the instruction on the USAID financial procedure was “of great help to our work with [the] USAID project.” Through the close follow-up and instruction in implementing activities, VFD has significantly contributed to MCD’s organizational improvement. As MCD leaders and staff noted “Awarding [the] sub-grant [and follow-up guidance] is the most effective way to enhance our organizational capacity.” The OCA proved to be effective in helping local NGOs review their organizational health: • First-round OCAs took place from July–November 2013 for four NGOs: SRD, MCD, the Center for Research and Development in Upland Areas (CERDA) and People and Nature Reconciliation (Pan Nature). • After the OCAs, SRD and MCD were selected for follow-up coaching to strengthen organizational capacity on financial management, governance (i.e., the Board of Trustee establishment roadmap and succession plan) and program strategy. Both NGOs expressed high appreciation for the VFD’s guidance on financial management, especially concerning USAID’s financial regulations and requirements. MCD also highlighted the effectiveness of VFD’s Chief of Party in providing mentoring on strategic planning for organizational development. • Two years after the first OCA, all four NGOs interviewed have reviewed their organizational operations and reported significant improvements in their capacity. Second-round OCAs are tentatively planned for these four NGOs in 2015. So far, SRD has completed the first draft on results of the second-round OCA and will soon submit the results to VFD. Table 16 summarizes VFD’s organizational development activities. Table 16. Summary of VFD’s Organizational Development Activities for Four Core Local NGOs Support Type Topics SRD MCD Pan Nature CERDA OCA √ √ √ √ Coaching Board of Trustee and CEO recruitment √ √ Financial/Administrative management √ √ Strategic planning/Programing √ Training Proposal writing √ √ √ Fundraising √ Organizational governance development √ √ √ Financial management (in compliance with USAID requirements) √ √ √ √ (Source: Interview with VFD staff and beneficiaries) Regarding capacity building for the local NGO network, support has been limited to training events for members of the Vietnamese NGO and Climate Change Network. Some courses on knowledge and skills — for example, climate change literacy, proposal writing, planning, project cycle management and M&E — have been organized, providing knowledge and skills for 35 NGOs in the network. For example, Table 17 lists the activities to strengthen the capacity of local NGOs proposed in Workplan 2015: Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 45 Table 17. List of Activities to Strengthen Capacity for Local NGOs Proposed in Workplan 2015 No. Activity Component Location 1. NGO Network Meetings SL TH, NA 2. Strengthen the capacity of local NGOs AD ND 2.a Conduct a training on proposal writing skills for executive committee of VNGO-CC working groups in the Red River Delta, Thanh Hoa, Nghe An and the South AD TH 2.b Conduct a training on climate change and climate change adaptation for the VNGO￾CC working group in the Red River Delta AD ND 2.c Conduct a training on climate change advocacy AD NA 2.d Conduct a training on PFES and REDD for members of the VNGO-CC in Thanh Hoa and Nghe An working groups AD NA 2.e Conduct a training on climate change and climate change adaptation for the VNGO￾CC working group in the South AD HCM (Source: VFD Workplan 2015) The evaluation team conducted interviews with five of 35 benefitting organizations. • Generally, organizations found the training useful in providing new skills and knowledge. To some extent, participants applied what they gained in their work. However, the informants said their organizations were invited to join only one or two training events or workshops, but VFD provided neither follow-up assistance nor information about next steps. The exception was one respondent in Thanh Hoa who said he received follow-up coaching on proposal writing after the training course. Through the bulletin “Climate Change in the News” on the Vietnamese NGO and Climate Change Network website, the NGOs in the network have information about upcoming events; however, they do not have budget for travel or per diem to participate. • As seen in work plan activities, VFD has provided more training for the working groups of CC NGOs in Nghe An and Thanh Hoa as compared to other regions. One participant in the training on proposal writing in Thanh Hoa in early 2015 expressed his high opinion about the course, which he said was “very useful” in writing proposals to donors. Yet the general plan as well as outcome of the capacity-building activities remains unclear. The information on the overall plan and objectives of support to the working groups in these two provinces were not specified in work plans or progress reports, nor communicated to beneficiaries. Without a clear strategy or plan, the objective to “strengthen the capacity of local NGOs” through capacity building is too general. In addition, it is unclear about the method or tools to evaluate the level to which the capacity of these NGOs has improved as the result of VFD’s support. 3. Conclusions • The organizational development activities have been effective, particularly for improving skills regarding USAID financial management requirements. The benefitting NGOs hope that follow￾up activities can be accelerated. • A number of training activities have taken place for local NGOs in the network, providing knowledge and skills on climate change and organizational operation in general. VFD has not yet demonstrated a clear plan for continued support to the VNGO & CC. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 46 B. Support to Build Climate Change Awareness and Political Commitment among GVN Officials 1. VFD Activity VFD has implemented many activities on capacity building for GVN officials throughout the program. Their outcome is mixed. • The training courses cover a wide range of topics, such as climate change, green growth, PFES, REDD+, proposal writing, project life cycle management and project M&E. • According to the latest background document prepared for the mid-term evaluation10 , regarding Indicator 3.3.1, VFD is currently supporting 26 institutions to improve capacity to address climate change issues; support for 18 of them have been partially completed. Most of these institutions are GVN agencies at different levels from national to commune, such as VNFF, Forest Protection Department, DPI, Women’s’ Union, etc. Support has included training, technical assistance, development of SEDPs or action plans, and involvement in the cook stove livelihood model. However, the level to which the capacity of these institutions has been improved remains unclear. It is probably too soon to measure the outcome because none of the support for above-mentioned institutions has been completed. VFD confirmed that they have not conducted reporting and evaluation on the improvement of institutions assisted (step 5 in six-step procedure to improve capacity for an institution as demonstrated in Performance Indicators Reference Sheet for Indicator 3.1.111). • GVN personnel feedback on the training are mixed. For some participants, VFD training was practical and useful; they used the skills and knowledge gained to develop internal training courses for their organizations. Other participants said their knowledge and skills increased a small amount after the training since they had already received training on or were familiar with the subject, or that the training content did not have much relevance to their work. In other cases, the training was helpful, but only a few agency staff members participated; or, after the training, they were not able to apply the techniques and knowledge due to a lack of equipment. • Participants said some training topics were interesting, but the training was too introductory — not detailed or comprehensive enough to enable staff to be able to apply or develop the topic further. The recipients’ most common feedback was that the training lacks follow-up support and VFD did not inform them of the next steps. VFD conducted a pre-training assessment of GVN officials’ knowledge and attitudes, and a second-round survey is planned for Q4, 2015, to evaluate the effectiveness of the training. [Note: During the period when the evaluation was being conducted VFD was undertaking a survey to gauge the effectiveness of its training of government officials on climate change awareness; however, the results of the survey were not available in time for use in this evaluation. The evaluation team did speak with a number of officials who had participated in these trainings and indications were that the usefulness of the training was mixed. For example, officials from HCMA indicated their interest in providing this type of training to government leaders, but said the materials would need to be redesigned as they appeared to be taken from courses offered in other countries and were not always relevant to the situation in Vietnam. Vietnam has made a strong policy commit to climate change. Any future climate change training should be undertaken based on specific objectives and potential application, and should be designed to build on existing knowledge as the issue receives a high level of attention in Vietnam.] 10 Vietnam Forests and Deltas Background Document in Preparation for the Mid-term Evaluation, 2015, p.20 11 Vietnam Forests and Deltas Performance Monitoring and Evaluation, 2015, p.24 Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 47 2. Engagement with Ho Chi Minh Academy of Politics and Public Administration In 2015, a workshop with the Ho Chi Minh Academy (HCMA) of Politics and Public Administration reviewed the capacity building efforts for GVN officials in 2014 and discussed future cooperation between VFD and HCMA. VFD and HCMA signed a memorandum of understanding in June 2015, focusing on Enhancing Leadership Capacity in Response to Climate Change in Vietnam. This serves as the basis for the development of an on-the-job training program that aims to provide practical exercises and orient GVN agency leaders on mainstreaming climate change and green growth considerations into planning and decision￾making. The two parties are moving toward a one-year contract on training material development for this purpose. So far, HCMA has completed a TOR, an outline for the structure of the materials, and a plan for training implementation. The training material will be tentatively completed by June 2016, with one set for ToT (for lecturers and researchers of HCMA) and one set of materials for training central- and provincial￾level officials. 3. Conclusions • In general, GVN officials found the large number of training courses and workshops useful and applicable to different extents. The training covers a wide range of topics, yet its content was not specific enough to practically support the officers’ work. Participant selection is another concern; the training content and the level of detail provided should be based on the needs of the participants, rather than a general training for all relevant departments and agencies. • Given the role and mandate of HCMA, transferring the climate change knowledge and awareness building function to the institution is an effective way to build a sustained training capability in this area – and to phase out VFD’s role in providing general CC awareness. C. Gender Integration VFD Activity VFD’s crosscutting gender theme aims to benefit two main groups: community members and GVN officials. Under this evaluation’s SOW, the evaluation team conducted a survey with 382 households receiving training in all four VFD provinces. The adjacent figures show respondents’ opinions on some aspects of gender issues regarding climate change and VFD’s activities. VFD household beneficiaries have a relatively high awareness of the difference of climate change effects to women and men. Figure 3 shows that 69 percent of respondents agreed or strongly agreed that the impacts of climate change on male and female members of the community are not the same; 22 percent had a neutral opinion, i.e., no clear agreement or disagreement; and the other 9 percent did not agree. Although the term “climate change impact” may have a broad interpretation, these figures show that project beneficiaries are aware of gender differences in climate change effects. This indicates that people are open to the perception that males Figure 3. Response of Households to the Statement: “Climate change affects women and men differently.” (Source: Data from VFD Mid-Term Evaluation Household Survey, 2015) 0% 10% 20% 30% 40% 50% 60% Strongly disagree Disagree Neutral Agree Strongly agree 1% 8% 22% 52% 17% Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 48 and females should be approached and supported differently to prepare, adapt and respond to unusual climate fluctuations. In the household survey for this mid-term evaluation, we included a question to assess whether gender concerns had been mainstreamed into the CBDRM/CADRE activities at the community level. The large majority (94 percent) of 180 survey respondents in these two provinces responded positively (see Figure 4). Women are generally considered more vulnerable to disasters compared to men (IFRC, 2010, and UNDP, 2013). Therefore, it is useful to prioritize support to help female members of the community better respond to adverse climatic conditions. Evidence is insufficient to say that VFD’s effort in mainstreaming gender issues in activities related to CBDRM/CADRE has been successful, but the survey results do suggest that people have a high level of awareness of gender issues related to climate change programs. Following the requirement of the USAID ADS, the gender and social inclusion analysis was completed in Thanh Hoa and Nghe An from early stage of the program. Findings of this analysis can serve as the basis for gender and social inclusion consideration in planning and implementation of program activities in the surveyed areas. As the analysis revealed, gender equity and gender integration have not been realized and implemented by GVN officers in relevant departments. The responsibility for gender integration is not clear and no concrete practices are identified on gender integration in forestland allocation, protection, development and PFES. One reason was the lack of comprehensive training on gender for these agencies’ officers. Acknowledging this, VFD has provided considerable capacity building to this target group. The next section of this report discusses this further. The assessment also pointed out women’s inferior role and increased barriers compared to men’s in many aspects, such as access to information and resources on forest management, forestland ownership and in decision-making regarding the use of forestry resources. Within the scope of VFD activities, VFD made recommendations to improve the policy implementation and policy enforcement, aiming to promote gender equality and social equity in forest management and PFES. Specifically, VFD is developing PFES community practice guidelines, which promote the entitlement of women as equal beneficiaries, i.e., in a household, a woman’s right to receive and manage the payment is equal to her husband’s. Advocacy for this regulation is in progress. VFD developed a gender checklist based on the findings of the Gender and Social Inclusion Assessment. In actual implementation of the activities, new gender checklists are tailored to target groups and topics. The purpose is to ensure that VFD’s services consider gender in engaging both women and men, as well as treating other social groups equitably. VFD ensures that gender concerns are integrated in all activities and specified work plans in that: • Gender integration in climate change issues is important in training courses and facilitation sessions for staff of GVN partner agencies at the provincial and commune levels to raise awareness and integrate the gender issues into planning and decision-making. Figure 4. Response of Households to the Question: “Do you think current activities related to CBDRM/CADRE in your community help women respond effectively to climate change and/or natural disaster if it occurs?” (Source: Data from VFD Mid-Term Evaluation Household Survey, 2015) 94% 6% 0% 20% 40% 60% 80% 100% Yes No Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 49 • VFD has worked with the Long An Provincial Women’s Union on the development of a pilot set of communication sessions on gender integration in climate change issues. This activity is in process. A member of this union in Can Duoc District said the group is still waiting for input from VFD on technical content and further discussion about next steps. • Three intensive training courses on gender mainstreaming took place in years 2 and 3 of the program, as shown in Table 18. Table 18: Training Courses Exclusively on Gender Name of training course/workshop Time Location Duration (hours) Participants Training on gender integration and facilitation skill Jan.-Feb. 2015 Thanh Hoa 24 29 (7F, 22M) Gender mainstreaming in climate change adaptation Oct. 2014 24 26 (12F, 14M) Gender mainstreaming in climate change response March 2015 Nam Dinh 24 35 (19F, 16M) (Source: VFD database on training courses). Note: F: female; M: male VFD also cooperates with international NGOs and the Climate Change Working Group (CCWG) to promote gender mainstreaming in climate change issues. One important output is the recent workshop “Integrating Gender in Climate Change Adaptation Design” that VFD organized in cooperation with CCWG, Care International and Southern Voice in July 2015. This was regarded as a good opportunity to share experiences, develop an initial map of gender-related climate change adaptation interventions, and identify opportunities for joint advocacy efforts. VFD and CCWG have signed a memorandum of agreement for joint efforts in addressing climate change, including gender mainstreaming training. Conclusions • VFD integrates gender concerns into all project activities, but it is still early in the process to realize significant effects. Though training outputs are clear, a more focused assessment will be needed to evaluate the outcomes of training on gender integration. VFD could conduct this. • Engaging local women’s unions on raising awareness of gender in climate change issues is effective. However, most progress is from training provided to the unions. Follow-up cooperation is still slow. • Solutions to gender issues vary by activity and locality, which means flexibility is required. Nevertheless, the current efforts are more or less ad-hoc. Common approaches could be useful as a basis for solutions to gender issues throughout the program and to ensure that those solutions are aimed toward specific gender objectives. VII.VFD PERFORMANCE PLANNING, COORDINATION AND MONITORING AND EVALUATION Evaluation Questions: Are VFD’s activities sufficiently focused and concentrated to achieve meaningful results within its programmatic and geographic areas? Is VFD effectively supporting climate change activity coordination and the development? … This also includes the effectiveness of coordination between the VFD project and the MARD CPMU, and between central activities and policy and implementation in the provinces. Monitoring System: Determine if the activity’s monitoring system effectively measures implementation progress and provides a useful assessment of its near- and long-term objectives. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 50 This section covers an expanded version of the evaluation question by including analysis on VFD planning, management and implementation, including how VFD interacts with the GVN. Strategy, management and coordination were raised by the GVN as issues of concern as the evaluation proceeded. A. VFD’s Management Structure In agreement with USAID, the GVN established VFD as an overseas development activity (ODA) project; however, the agreement between Winrock and USAID is a cooperative agreement. Each management mechanism has a different set of rules and expectations. Winrock is responsible for following USAID regulations and the GVN project owner — the management board of VNFF — must report to the MARD, which ensures compliance with GVN ODA regulations. The basic structure of the project follows: • Project Steering Committee (PSC): The PSC’s role is to provide overall guidance and review to make sure the project achieves its objectives and meets government compliance requirements. The PSC is chaired by a representative from MARD and the PSC’s members include representatives from MPI, MOF and MONRE, representatives from the four provincial PPMUs; representatives from USAID, Winrock, the Management Board of Forestry Projects (MBFP) the Water Resources Directorate, VNFOREST and relevant departments of MARD, such as ICD, the departments of Planning, Finance, and Science, Technology and Environment. The evaluation team was told that the PSC has never met. • Central Project Management Unit (CPMU): The CPMU provides day-to-day management of the project on behalf of the government and in coordination with Winrock. Two project co-directors lead it: one from Winrock and the other from the MBFP. Other members of CPMU include VFD technical coordinators representing the sustainable landscapes, adaptation and national policy components; one national coordinator; and a chief accountant. • Provincial Project Management Committees (PPMUs): The Provincial People’s Committees established PPMUs in each of the four provinces of project operation; they operate under DARD. The PPMUs act as a coordination unit to support the project and coordinate with local agencies. VFD has provided modest grants to the CPMU and PPMUs, which total $100,000 over the project’s first three years. VFD operates the following structure: • Central VFD Office/Hanoi: VFD’s main team is in Hanoi, and includes leaders for the project’s components, plus various managerial, accounting and technical staff. VFD has around 27 staff in its central office. • VFD Field Staff in the Provinces: The staffing configuration varies somewhat from province to province. All provinces employ a VFD accountant and a project coordinator. Currently, VFD does not have staff in locations other than Hanoi and the capital cities of the four provinces of operation, and staff in the regions focus on accounting and coordination. This leaves VFD without any technical field staff to support implementation even though most activities are located in areas that are distant from the provincial capitals. The following chart shows the VFD co￾management structure. It reflects VFD’s view that the project operates under two parallel management structures, with the PSC acting as a coordination mechanism between them. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 51 B. Project Planning and Coordination The project has held a number of meetings about the planning and coordination process. Issues have included participation in planning and setting objectives; budget support allocation; how fieldwork is monitored; the role of subcontractors; and hiring consultants. VFD staff indicated to the evaluation team that the dual management structure has introduced many additional steps for review, approval and implementation of VFD activities and constitutes a major challenge for implementation and in gaining substantive country buy-in. It requires the COP and DCOP to achieve consensus with a wide range of individuals, organizations and government agencies. To reach consensus on the planning and coordination process, VFD worked with the CPMU over the past year to develop a project implementation manual (PIM). A draft PIM exists, but VFD has not submitted the document to the CPMU for formal approval. Nevertheless, both VFD and the CPMU have stated their intention to use the existing document as guidance for managing VFD. The PIM contains sections on the cooperation mechanism, planning, monitoring and evaluation, finance, and administration and procurement. VFD’s basic planning mechanism is an annual work plan. The work plan development process and an analysis of work plan activities follows. Work Plan Development As of this evaluation, both USAID and the CPMU had developed and approved only one VFD work plan (FY2015). The FY2016 work plan was under development during the evaluation. The work planning process is based on consultation with NGO subcontractors and PPMU staff to identify priority activities. Priority activities are listed by “key themes.” Currently, the VFD is working on its FY16 work plan, and the CPMU is conducting an independent analysis of FY2016 priorities (independent from VFD’s work planning process). Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 52 VFD’s work plan does not appear to be based on a hierarchy of objectives, but does have groups of key activities and sub-activities. Based on a review of the work planning process, the evaluation team was unable to determine how activities are prioritized or how budgets are allocated. Table 19 provides an overview of activities in the FY15 work plan, sorted by region and component. Table 19. Workplan Activities by Component and Region Activity Nam Dinh Long An Thanh Hoa Nghe An Central Others Total Adaptation Number 41 44 1 2 3 1 92 Average Value ($) 17,997 15,736 7,981 4,507 19,000 6,761 16,424 Sustainable Landscape Number 0 0 75 71 5 - 151 Average Value ($) 0 0 13,307 11,394 5,400 0 12,146 Policy Support Number 5 5 1 2 42 1 56 Average Value ($) 1,200 0 0 0 19,605 13,000 15,043 Total Number 46 49 77 75 50 2 299 Average Value ($) 16,171 14,130 13,065 10,907 18,148 9,880 B14,005 Activities with no budget 6 4 21 24 14 0 69 Based on a review of the FY2015 Workplan, and the data in Table 19, we note the following: • The VFD Workplan FY2015 has 299 activities (counted by the activity and the location), but each requires implementation plans that provide details on consultants, cooperation, participants and objectives. This counts each activity in each province as a separate work plan item. • 69 of the 299 activities have no budget number indicated. • The average budget per activity is $14,005 (activities with no budget included). [Note: As per VFD, there are 208 activities, not 299 (220 in the CPMU work plan because activities are broken down into sub-activities). When this evaluation was undertaken 38 activities were completed (18%) and 126 were ongoing (61%). 44 activities (21%) were not started or cancelled due largely to lack of GVN commitment. 24 activities had no budget (not 101) as there was agreement with the CPMU that budgets would be provided if the GVN showed commitment.] In reviewing the proposed FY2015 work plan against the most recent VFD quarterly report (Q3, FY2015), we found 276 activities reported, accounting for 77 percent of the activities proposed in the FY2015 Workplan (299 activities). VFD noted that 79 percent of activities are being implemented. Yet after reviewing the FY 2015 Performance Progress Report, the evaluation team was unable to verify this proportion due to a number of reasons, including: • Narrative description of project implementation does not provide sufficient information or data to clarify the status of all activities. In Annex 2 (Quarter 3 Progress in the Year 3 VFD Workplan), the status of activities is not always clearly stated, although a number of activities were noted as ongoing or completed. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 53 • It is difficult to compare the M&E indicators, work plan and performance progress reports due to the mismatch between the indicators monitored and the set of activities. In the tables reporting USAID indicators (Annex 1 of the VFD performance progress reports), data for each indicator is reported with status (e.g., “How many laws or policies as a result of USG assistance are in planning, ongoing, partially completed or completed?”). Yet these are the aggregate of several work plan activity results and could not be used to determine the status of the activities. GVN Comments on VFD’s Workplan Process The following are summaries of GVN stakeholder comments on VFD’s planning process: • The topic of climate change is broad and VFD has not developed a strategic focus. • Policy support and general planning: Winrock and partners’ strategy is not focused or designed to achieve impact. Activities are mainly focused on raising awareness, e.g., PFES and capacity building. • The action/work plan is too ambitious with too many activities. • At an annual planning meeting, 78 activities were identified, but they were not categorized or prioritized. The level of budget available to support them is not known. The project is difficult to monitor. • VFD has focused on developing a large wide-ranging set of activities and expects the PPMU to coordinate the logistics. • Work planning: Winrock and sub-awardees develop the work plan for the province — and then PPMU is asked to compile the documents with no substantive stakeholder involvement in developing project activities. • Activities do not have budgets, so GVN counterparts do not know what can be done. How can we plan for or support implementation? • Many activities have no deliverables. On the other hand, the CPMU acknowledged that these problems occurred partly due to the passivity of PPMUs, largely relying on implementing partners and NGOs to conduct local activities. VFD noted some activities do not yet have budget allocation because some PPMU proposals were not appropriate or relevant to VFD climate change objectives. Therefore, the VFD Office and CPMU jointly decided to leave them in the work plan without a budget until the GVN could show a clear commitment to that activity, along with links to the VFD climate change objectives. Another coordination issue is the development and approval of activity TORs, and the hiring of consultants. Information in the SL chapter indicates that partner GVN agencies do not always receive technical assistance TORs for review and approval, and the involved technical agencies do not always vet and approve consultants. This is not always the case, but it was mentioned as an issue in at least three provinces and by several different agencies – and may be particularly the case regarding the work of VFD’s subs. The CPMU and PPMUs expressed dissatisfaction with this aspect of VFD’s operation and indicated they believed it lead to consultancies and studies being conducted that were not particularly useful. The project’s start was delayed, and the team has received only one approved joint work plan. Winrock faced a great deal of pressure to get the project moving after the lengthy delay of the GVN establishing a CPMU, and this was part of the reason for developing such an extensive list of activities. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 54 VFD Studies. VFD has conducted a large number of operational research, assessment and evaluation reports. Although some of these activities were of high quality and provided useful data to guide programing, the practical value of a number of these studies is not well understood. • The PPMU in Thanh Hoa noted that it is hard to identify provincial audiences for 13 of VFD’s assessments and studies in the province. When the evaluation team visited, the draft reports were at the PPMU office and had not been distributed. • VFD supports the development of livelihood models as a significant part of its overall activity. However, the evaluation team did not find any well-documented report showing the potential effectiveness of the models (or plans to test and validate the approaches). Some models appear to not have any associated reports (e.g., the maize and potato models). Some livelihood model reports were found, but they mainly dealt with outputs rather than outcomes or the efficacy of the approaches (smart rice models). Without sound design and careful testing plans, it is not possible to verify the effectiveness and feasibility of livelihood models, or to make a case that the activities can or should be successfully deployed at scale. • The CPMU/PPMUs also noted that VFD M&E data is rarely used to supplement the government’s M&E system. Some data has been used as a reference only. Conclusion Stakeholders expressed concerns that work plan lacks a strategic focus and has too many activities, most of which are small and not prioritized to the achievement of specific outcomes (i.e., no obvious causality between activities and outcomes). This makes impact less likely and activity support and monitoring difficult. The PPMUs feel they have not had an opportunity to have meaningful input into the work plan. VFD asked them to provide their suggestions for inclusion in the work plan, so they reviewed and approved it; however, VFD’s focus areas are not clear to the GVN so the work plan is viewed as an activity list (with several hundred activities). There are not clear budgets associated with work plan activities when they are developed (as per comments by the PPMUs). A number of work plan activities suggested by PPMUs are not implemented, which means that even when GVN suggestions are included in the approved work plan there is no assurance of funding for these items. There is a lack of criteria and transparency in how resources are allocated to support work plan items and which items receive funding. This process makes it difficult for the government to align its efforts and resources with VFD’s activities, or to plan accordingly. Once the work plan is approved, VFD allocates funds for implementation. At this point, VFD’s partners develop plans and begin implementation. The PPMUs are often unaware of the sub partner plans, and the NGOs (e.g., SRD, SNV) report back to Winrock instead of to the PPMUs. This marginalizes PPMU involvement. The Red Cross organizations are an exception as they are PPMU members. VFD coordinators are assigned to assist each PPMU, and they update the PPMUs on upcoming activity, but the PPMUs are not aware of the detailed plans and budgets of the NGO sub partners, and often only know of implementation activities on very short notice. GVN technical agencies do not always have an opportunity to review and approve TORs or consultants for work that involves their agencies. VFD undertakes many consultant studies; however, there is not always a clear demand or audience for the work undertaken. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 55 C. VFD Monitoring and Evaluation System VFD Performance Monitoring and Evaluation Plan (PMP) VFD has developed an activity monitoring and evaluation plan, which has been revised and amended three times. USAID approved the most recent version in June 2015. The plan covers the general data collection plan, results framework, summary of indicator targets, and performance indicators reference sheets. Other specific tools, such as the VFD governance outcome scorecard, are also included. In the PMP, 18 indicators are identified to provide data measuring two intermediate results (IR), six sub-IRs and activities. Capacity-building and gender-related indicators are also included. Reporting Structure and Data Flow VFD has developed a system of data collection, processing and reporting for program activities, which results in a monthly report shared with the CMPU and the quarterly report submitted to USAID. Provincial VFD staff directly collect M&E data monthly, mostly from VFD-supported training courses. The diagram to the right depicts data flows in the M&E system. Sub-awardees do not collect additional M&E data. VFD shares its aggregated data with sub-awardees after processing at VFD office. Performance Monitoring Plan (PMP) The VFD PMP does not include a program logical framework, a presentation of a theory of change or assumptions used to design the program. • The M&E plan contains a results framework, but several key result statements are not linked to USAID assistance (attributable, direct or USAID-specific). • It is difficult to understand the logical links between IRs, sub-IRs, results and proposed indicators. Examples of this are key intended results and indicators under sub-IR 3.1 (GVN establishes effective governance systems and capacity) and intended results and indicators under sub-IR 4.3 (: Local governments and communities take actions to build resilience to long-term climate changes). • As worded, indicators for the IR level are not well connected or linked to the IR. For example, one indicator (“Number of stakeholders [individuals] with increased capacity to adapt to the impacts of climate change as a result of USG assistance”) measuring IR 4 (“Increase Resilience of People, Places and Livelihoods in Delta Areas”) is defined as an outcome indicator in PMP, but it is unclear if the indicator helps to demonstrate IR 4 itself, as it is not a direct measure of the intended result. The PMP does not mention a data quality assurance plan. M&E System Performance A large part of the M&E system is a project implementation/work plan tracking system, as opposed to a system to report results. The system contains too much data, much of which focuses on activity and output data (even defined as “outcome”), which means it is not useful for determining project outcomes and progress toward impact. The most recent VFD performance report contains 46 pages of indicator reporting, but no reports that summarize strategy or progress in the activity’s major program areas, such as livelihood activities and PFES. CPMU (monthly) USAID (Quarterly) VFD Office- Processing (data compiling by project components), and report generation PPMU- Review and Approve VFD province- based staff- Data collection Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 56 The complexity of the system places a heavy data collection burden on VFD provincial staff, who are responsible for collecting data, in addition to coordinating, implementing and monitoring activities. Therefore, they are able to spend only limited time on M&E activities. The VFD Hanoi Office includes only one M&E manager. With more than 200 activities planned for FY2015, the level of personnel assigned for M&E appears to be insufficient. The M&E system contains a significant amount of data that is questionable in terms of accuracy, or that stretches the concepts of plausible association and attribution. For example: • The number of hectares of biophysical significance and/or natural reserve showing improved biophysical condition is measured by counting areas where VFD provides assistance, regardless of the scale of activity. For example, if VFD provides training at a nature reserve then the area of the nature reserve appears to be counted under the indicator. • VFD reports having trained 29,833 people. However, this number mixes a variety of activities, including general and short awareness-building workshops. As such, it is not particularly useful for understanding the level of skills development that has occurred, or what purpose the training serves. (It is also difficult to imagine how training for 29,000 participants in 18 months could have been subject to careful targeting or follow-up processes – and the evaluation team heard comments from several agencies that participant targeting and advance notification of training was poor.) • The evaluation team’s household survey could not find +/- 15 percent of training participants — as they were not known to leaders in the communes that the survey teams visited. This appears to indicate a data quality issue. • An issue of over-counting and reporting exists on the indicator “Number of institutions with improved capacity to address climate change as a result of USG assistance.” Measurement of improved capacity has not been conducted for government partners, and specific institutional strengthening objectives are lacking. A number of GVN institutions are counted as having achieved increased capacity when even few individuals from those institutions had received a training course from VFD, or an institution had worked with a VFD consultant. For example, provincial DoNRE offices count as four institutions having been improved. Two of the “improvements” relate to the production of consultant reports on land use planning, but DoNRE did not participate in the development of the reports and said they are of low quality. It is difficult to make a case that DoNRE has been institutionally strengthened by VFD support. Although some of the trainings may have be useful, or interesting, DoNRE staff has not been able to apply the skills covered in the courses. Similar examples in VFD’s reporting relate to improved institutional performance. • Under the livelihood models, VFD’s reporting regarding households earning improved income appears to contain some misrepresentations. The reporting indicates that 300 households have benefitted from the cook stoves activity and 150 households have benefitted from the potato activity, but this does not appear to be the case. • The reporting indicates a $4 million financial leverage benefit from Lasuco’s involvement in the bamboo strategy, but Lasuco’s investment decision has not been made and may be a year or two away. This investment has not occurred, but is listed as complete in the quarterly report. VFD stated that the MOU with Lasuco was an indication of the funds leveraged, but the MOU is general and had not been signed by Lasuco. • Under “Increased number of Vietnamese NGOs with increased capacity or technical assistance,” about 30 local NGOs were counted as strengthened even though they attended only a few trainings on topics like proposal writing and general climate change awareness. It is difficult to imagine that most of the NGOs have significantly increased their capacity because of one or two workshops. One Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 57 of the NGOs we met with said the training was useful, but that they had received no other support from VFD, and did not believe they would have any role in the project. Conclusion VFD’s M&E system is not effective as a results-based management system, and does not provide useful information for monitoring program outcomes. The system is not designed to provide data to measure how activities contribute to higher-level achievements (to some degree this is caused by the lack of strategic focus within VFD’s work plan). Other issues include a potential lack of quality control in the collection of training data, and probably an insufficient level of personnel to manage the system, given the large number of activities undertaken by VFD. D. Delayed Project Start-Up Several factors caused the late start-up of VFD, including the following: • There was no formal agreement between the U.S. Government and GVN on VFD. The GVN requires formal agreement before it can authorize implementation, and has specific rules for how ODA projects are managed. These rules include financial tracking and reporting requirements and the need to set up a central project management unit to co-manage the project. • USAID designed VFD as an NGO program, but it was approved as an ODA program. USAID did not intend for VFD to be implemented as an ODA program and therefore did not involve government ministries/departments in a detailed way in the project’s design phase. Therefore, involved government agencies did not understand the program’s details and took a long time to review and approve it. These approvals were required before the request to approve the project could be sent to the Prime Minister’s Office. • The project document (approved by USAID in the cooperative agreement to Winrock) did not follow GVN requirements. It took time to rewrite and prepare official documents for GVN approval. • Most ODA projects take longer than 14 months to receive approval. This should be considered for future projects. This makes it imperative for USAID to coordinate more closely with the GVN on project design, and to allow for a longer planning and approval process. Also, when new projects are designed there should be an early decision reached with the GVN as to whether or not the project will operate as an official ODA project as ODA projects have defined requirements in terms of the approval process and management requirements. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 58 VIII. SUMMARY CONCLUSIONS AND RECOMMENDATIONS This section contains a summary of conclusions and recommendations. Each chapter of the report that addresses VFD’s performance contains a set of more detailed conclusions than those that appear in this section. The evaluation’s principal conclusions/recommendations are as follows: • Forestry sector support work has ranged from effective to highly effective. VFD has made important contributions to Vietnam’s PFES program, including for national policy development and review, and for operational improvements in Thanh Hoa and Nghe An provinces. This work should be intensified and continued. • VFD’s adaptation work, which has been led by the Red Cross, has been useful and of high quality. This work has supported community-level disaster preparedness planning and the review and strengthening of early warning systems. Work in this area should continue. • VFD’s work in the livelihoods sector has met with mixed results and, for the most part, initiatives do not appear to be sustainable or scalable. Activity implementation occurs on a small scale. Existing successful work in this area should be consolidated and continued, but the program should curtail the introduction of new small-scale initiatives. A closer working relationship with the AEC should be developed, perhaps focused on a few key commodities, including for climate-smart rice. • VFD’s work is programmatically scattered and lacks a strategic focus. VFD’s performance will benefit from better-defined programs in fewer sectors; better alignment of activities against explicit longer-term outcomes; deeper and more thorough engagement within select high-priority program areas; and more collaborative planning based on GVN priorities and needs. Specific conclusions and recommendations follow. A. Sustainable Landscape PFES Conclusion VFD’s support for PFES has been relevant and effective. VFD’s has provided extensive support to PFES at the national level and in Nghe An and Thanh Hoa provinces. VFD has a positive working relationship with VNFF, whose director has characterized VFD’s work as highly effective. Recommendations VFD should continue to provide support for provincial operations and expand work into additional districts to accelerate forest owner identification and increase payment distribution for additional service providers. It would be helpful if VFD works with provincial FPDFs to develop a plan and schedule for multi-year support. Support should be comprehensive to allow completion of required work as soon as possible. This would include ensuring training for all necessary staff, and provision of necessary equipment, with follow-up assistance as needed. Another priority should be to continue and increase support for the development of a PFES forest protection monitoring system. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 59 VDF should continue work on the revision of Decree 99 in coordination with VNFF. Reduction of Emissions from Deforestation and Forest Degradation (REDD+) Conclusion VFD worked with the FCPF to develop the Northern Central ER-PIN, which FCPF has approved. This approval triggered the release of $650,000 for the development of Vietnam’s REDD+ program. FCPF expressed appreciation for VFD’s support. This has been VFD’s most significant REDD+ contribution to date. In addition, VFD is supporting the development of two feasibility studies that will contribute to the development of REDD+ action plans in Thanh Hoa and Nghe An. Overall, VFD’s assistance to support REDD+ has been helpful and VFD should continue to support the development of REDD+ provincial action plans. However, VFD is not likely to become one of the principal supporters of REDD+ in Vietnam so it should carefully define its assistance in recognition of the role of other donors. Development of Green Growth Action Plan in Thanh Hoa Conclusion VFD’s training support was useful in helping local stakeholders increase their knowledge in the area of green growth planning. However, VFD’s assistance did not satisfy local stakeholders due to insufficient local engagement and consultation and unsatisfactory consultancy results. Recommendations VFD should continue to coordinate with the DPI to enable the GGAP’s completion and closely coordinate assistance with the agencies’ needs. For example, TORs should be shared or jointly developed, and consultant selection should involve the local institutions. Once the plan is completed, investment-ready proposals will need to be developed, and VFD could provide assistance for this. Revision of Land Use Plans in Thanh Hoa and Nghe An Conclusion VFD’s support did not meet the expectations of the provinces in terms of its focus or output, and in both Nghe An and Thanh Hoa provinces the assistance was considered to be of low quality. VFD’s assistance under this activity was not successful in significantly building DoNRE’s capacity. Recommendation No further action. Strategic Action Plan for Thanh Hoa Bamboo Development Conclusion VFD’s assistance has been helpful, although the Bamboo Strategy concept is in an early stage of development. The activity offers a high potential pay-off, but its next steps are not well defined. The GVN, under the lead of DOFA, is undertaking a planning process to provide greater clarity. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 60 Recommendation Once the bamboo strategy/action plan is complete and approved, VFD should coordinate with DOFA to determine if it can play a useful support role. Water Accounting in Nghe An Conclusions/Recommendations VFD has provided initial modest assistance to date, but this activity is in an early stage of development. The team was able to meet with only one person to discuss this activity. As such, we do not have any conclusions or recommendations for this program area. Nature Reserves Conclusions VFD’s focus in the protected areas has been to improve forest protection and management. The assistance VFD has provided to Pu Hoat Nature Reserve was of high quality and useful. Activities have included sponsoring study tours to improve protected area management, and helping the nature reserve establish a biodiversity mapping and monitoring system, which is now operational. However, key staff viewed the support as somewhat piecemeal -- “some on this and some on that; some in here and some in there with considerable time delays between activities. VFD’s work has not made significant or considerable contributions to the management of the nature reserve.” Recommendation Boundary demarcation was completed in two of nine communes for Pu Hoat. As a useful next step, VFD could provide assistance to complete additional required demarcation mapping. VFD should undertake longer-term planning with nature reserve managers to identify objectives that can be achieved within the life of VFD. B. Adaptation Conclusion Work has focused on community-level disaster vulnerability planning, assessing and upgrading early warning systems and supporting micro-mitigation projects of up to $11,000 each. Under VFD guidance, the VNRC and ARC have implemented most of this work, and it has been high quality and effective. Recommendations VFD should continue providing assistance to improve the capacity of local communities for natural disaster preparedness and response. VFD should consider extending assistance to cover additional communes in supported districts, particularly where the assistance is highly needed, but is not in their current plans. While activities have been implemented in all communes in two selected districts in Long An, a number of communes in Nam Dinh, considered high-risk areas, are not currently included in VFD’s plan. The evaluation team recommends that VFD work with its partners to simplify the approval process of small grants, for example by reducing the number of approval requirements. VFD should have a mechanism in place to ensure better communication with communes regarding micro-mitigation projects to reduce implementation delays. VFD and its partners (ARC and VNRC) should review the progress of using SMS text messaging for the early warning system. If only minimal outcomes are feasible, the team recommends ceasing this activity. If it is Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 61 determined to be worthwhile and feasible to continue, the roles and responsibility, degree of involvement and specific expected outcomes should be better defined. VFD should consider providing assistance to benefit additional communities — beyond current sites. This would include helping provinces develop action plans to implement GVN climate change programs, such as Program 1002 or MOET’s decision 4067 by developing an appropriate network of technical assistance providers (including the RC, DARD, and MoNRE). Institutionalization of training, assessment methods and materials would help increase the sustainability of VFD assistance over the longer term. VFD should help provinces to develop an M&E plan for the implementation of Program 1002 in both focus coastal provinces to better document and disseminate lessons and accomplishments. C. Livelihoods Caveat It was difficult for the evaluation team to assess the livelihood component as VFD, local participants, PPMUs and local AEC staff have differing levels of understanding on VFD’s models, future activity plans and if the models are to be AEC models or newly developed VFD models. The evaluation team was unable to clarify many of these details because of the differing viewpoints and because plans that describe the livelihood models seem to be lacking or are not sufficiently detailed. Conclusion It is unlikely that VFD’s livelihood models will achieve significant impact by the completion of the project, although some of the demonstration models may succeed as demos. This conclusion is based on the following: 1) VFD’s models are being implemented on an extremely small scale; 2) the models have experienced mixed performance to date; 3) AEC commitment to support most of VFD’s models is lacking (at least as currently implemented); and 4) the models are not supported by value chain analysis or realistic scaling plans. VFD does not have clear livelihood strategies or clear government support, and it is late in the project cycle to introduce new activities. VFD implements a wide variety of livelihood activities and runs a single small-scale demonstration for each activity. This spreads VFD’s efforts thin and means that each individual livelihood activity involves a very small number of direct beneficiaries. The models are time consuming and costly to implement (as technical support is provided from Hanoi-based staff). It also means that there is not a sufficient level of scaling support to spread the activity beyond its particular demonstration area, which is usually a single commune. Recommendations VFD should consider stopping or substantially scaling back the introduction of new, small-scale, untested livelihood models. These projects take time to start, and the project has only about 24 months of operation remaining. The likelihood of sustaining new models post-VFD appears to be low. If existing models are continued, clear analysis and plans should be available to support these decisions. VFD may have greater success in this area by selecting just a few livelihood models to support, and by intensifying the level of support each receives. Demonstrating the models in multiple locations and focusing on approaches that can be supported following the project’s completion could achieve this. If VFD decides to pursue a collaborative partnership with the AEC, it should develop specific objectives for the memorandum of understanding it signed with the AEC in January 2015, as well as a work plan and budget. Regarding collaboration with the AEC, it is recommended that VFD focus on a few (one or two) high-priority livelihood areas, such as climate-smart rice. This should begin with an assessment of the AEC’s needs, and by building on efforts in Long An and Nam Dinh. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 62 D. Crosscutting Themes Capacity Building for Local NGOs Conclusions The organizational development activities targeted to VFD’s six focus NGOs have been effective, particularly for improving skills related to USAID financial management requirements. Because of VFD assistance, at least one NGO has strengthened its governance structure. Quite a number of training activities for local NGOs in the climate change network have taken place; however, VFD has not demonstrated a clear plan for continued support to these NGOs. It is not apparent that VFD’s support has improved these NGOs’ capacity. Recommendation Institutional strengthening support for the six focus NGOs should continue. VFD should disengage from further support to strengthen VNGO&CC capacity. A clear support plan does not exist, nor is there a clear role for the NGOs in VFD’s primary areas of operation. Raising Awareness and Building Political Commitment Conclusions The large number of training courses and workshops for GVN officials were useful for building general climate change knowledge. However, the objective of participant selection focused on providing broad coverage for a large number of people, as opposed to aligning content to participants’ specific needs and knowledge. Cooperation with the HCMA to hand over climate change training for political and administrative personnel would appear to be a useful way to focus and institutionalize this program. Recommendations Given HCMA’s mandate to provide training for GVN officials, VFD should transfer this program to the HCMA. The next step is for VFD to develop a clear cooperation plan with HCMA. Gender Integration Conclusions VFD integrates gender concerns into all project activities. Though training outputs are clear, a more focused assessment will be needed to evaluate the outcomes of training on gender integration (which VFD could conduct itself). Engaging local women’s’ unions on gender awareness related to climate change issues is an effective approach. However, the progress has been limited to training of unions’ members themselves, as further rollout of the program is limited or unclear. Recommendation The collaboration with women’s unions and other NGOs to implement gender-related interventions will benefit from a clearer approach. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 63 The development of an overall VFD gender strategy would be useful so the project can pursue a consistent approach to address gender issues in accordance with the USAID Gender Equity and Female Empowerment Policy. E. Planning and Coordination Workplan Conclusions The work plan lacks a sufficient strategic focus, as there are too many activities, most of which are quite small and not prioritized to achieve specific outcomes. This makes the achievement of impact less likely, and makes support and monitoring difficult. Issues mentioned by multiple government partners was VFD’s lack of focus and concentration, and its provision of only “lite” or partial assistance toward the achievement of outcomes, or organizational goals. VFD frequently provided training for a particular program or issue that involved just a few participants, and no follow-up. VFD’s activities engage with a wide array of stakeholders, but often stop short of providing the level of assistance that is required to fully accomplish a program goal, or to enable an organization to achieve a particular objective. Because of this, and because there are often several donors working in areas of VFD assistance, it is difficult to attribute GVN program accomplishments to VFD assistance. The PPMUs do not feel they have had an opportunity to make a meaningful contribution to VFD’s work plan. There is a lack of criteria and transparency in how work plan items are prioritized and funded. This process makes it challenging for the GVN to align its efforts and resources with VFD’s activities, or to plan accordingly. GVN technical agencies do not always have an opportunity to review and approve TORs or consultants for work that involves their agencies. This sometimes leads to a misalignment between government needs and the structure of the technical assistance provided. VFD support will be more effective if it is developed in closer coordination with GVN agencies, and if there is greater involvement of GVN staff in the technical assignments undertaken. Recommendations VFD needs to develop a strategy for the remainder of the project that is focused on a limited and well￾defined set of outcomes. The work planning process should be structured to ensure more collaborative planning between the GVN and VFD. To increase GVN capacity, deeper and more focused technical assistance is needed, along with less reliance on short-term training. USAID should play an active role in working with the CPMU to refocus VFD’s objectives. Monitoring and Evaluation Conclusions VFD’s M&E system is not effective as a results-based management system, and does not provide useful information for monitoring program outcomes. The system is not designed to provide data to measure how activities contribute to higher-level achievements, although to some degree this is caused by the lack of strategic focus within VFD’s work plan. VFD’s monitoring reports contain a number of instances of incorrect data. Other issues include a potential lack of quality control in training data collection, and probably an insufficient level of personnel support to manage the system, given VFD’s large number of activities. Vietnam Evaluation, Monitoring and Survey Services | Vietnam Forests and Deltas Program Mid-Term Evaluation 64 Recommendations VFD should develop a reporting system focused on measuring outcome achievement. However, a revision to the M&E system should be completed only after VFD’s strategy and work-planning process is complete. USAID/Vietnam’s standard IP activity M&E template and guideline should be used to develop a more comprehensive and effective M&E system and plan that includes measurements of outcomes, effectiveness/impacts and a data quality assurance procedure. Attention should be given to the M&E system’s data quality procedures. USAID should consider conducting a data quality analysis of key VFD data, including for training, once the M&E system is revised. Delays in Project Start-Up Recommendation USAID needs to allow for a longer timeframe for the GVN to approve new projects (reviews and approvals typically take 14 months). It is also important to work closely with the GVN on the design of new projects so that there is agreement on project objectives and management structures.