THE END-OF-PROJECT EVALUATION REPORT ACCESS FOR ALL (AFA) – DRD (finalized) by SNPO, July 2021 Table of Contents 1. PROJECT BACKGROUND 2. ECONOMIC, POLITICAL AND SOCIAL CONTEXT WHEN IMPLEMENTING THE PROJECT 3. OBJECTIVES OF THE END-OF-PROJECT EVALUATION 4. METHODS OF EVALUATION 5. FINDINGS 6. CONCLUSION 7. RECOMMENDATIONS Tables Table 1. Role of consultants and members of groups participating in performance evaluation .... 8 Table 2. Levels of achievement of target indicators .................................................................... 10 Table 3. Percentage of PWDs participating in the survey confirming changes to the PWDs community .................................................................................................................................. 10 Table 4. Percentage of PWDs participating in the survey confirming changes to themselves ..... 11 Table 5. Percentage of surveyees acknowledging levels of changes to their awareness of PWDs’ rights........................................................................................................................................... 11 Table 6. Percentage of surveyees naming PWDs’ rights under the Law on PWDs....................... 12 Table 7. Percentage of surveyees talking about opportunities to have their awareness of PWDs’ rights enhanced........................................................................................................................... 13 Table 8. Percentage of members talking about benefits of participating in PWDs’ clubs and groups......................................................................................................................................... 14 Table 9. Percentage of surveyees agreeing on degrees of improvement in access to public services........................................................................................................................................ 17 Table 10. Number of learners receiving training in social work with PWDs................................. 18 Table 11. Percentage of surveyees talking about the degrees of response of the project to PWDs’ needs........................................................................................................................................... 20 Table 12. Sources of support outside the project......................................................................... 21 Table 13. Percentage of surveyees assessing the integration of elements into the project......... 22 Table 14. Percentage of surveyees expressing satisfaction with the project ............................... 23 1. Project Background From 2007 up to now, Vietnam has passed many documents related to disability issues, most important of which is the Law on Disability issued by National Assembly of Vietnam in 2010. Since 2011 the disability movement has changed in a positive way. However, policy implementation has been hampered due to a lack of enforcement of the law. In addition to the barriers of behaviour and attitude of persons with disabilities (PWDs), the biggest obstacles to equality of opportunity are physical environment and the low self-confidence of people with disabilities. These barriers prevent people with disabilities from integrating into the community. In 2015, the Accessibility for Inclusion Project (AIP) is being conducted by DRD. The project does not only to increase awareness of disability rights but also to improve physical accessibility for people with disabilities in Tay Ninh and Binh Dinh. By the end of 2017, DRD will have established two monitoring teams for physical accessibility compliance; improved 35 accessible public sites for people with disabilities; trained and communicated with nearly 1,000 people on disability rights and creating a barrier-free environment. AIP is laying the groundwork for improving disability rights advocacy in Tay Ninh and Binh Dinh, but barriers to inclusion for PWDs remain. Public awareness of disability rights is still low and PWDs do not have a strong enough voice or profile to take on advocacy at a national level. Opportunities for PWDs to come together and share experiences, for example through self-help groups, are still limited. These opportunities are needed in order that groups of PWDs support each other to raise their expectations and aspirations for the future. Recent policies to support PWDs are not yet being implemented due to a lack of enforcement of the law and the lack of any system for monitoring implementation of these policies. An effective monitoring system must include PWDs in the assessment of policy implementation success. Therefore, following the AIP project, DRD implemented the project "Access For All” (AFA). The AFA project was implemented from August 1, 2017 to July 31, 2020, and an extension to July 31, 2021. The overall goal of the project is to promote the equality of opportunity for persons with disabilities by building capacity and improving physical accessibility in Tay Ninh, Dong Nai and Binh Dinh provinces of Vietnam. To implement this goal, the project will pursue the following objectives: 1. Increased awareness of disability rights among persons with disabilities and the wider public; 2. PWD self-help organizations are established and strengthened at commune/district levels; 3. Building capacity to promote accessibility right of people with disabilities in Tay Ninh, Dong Nai and Binh Dinh; 4. Improved capacity of DRD staff and local partners on project management and community mobilization. The project will target at least 7,740 beneficiaries, including 2,160 direct beneficiaries (PWDs, disability leaders), and 5,580 indirect beneficiaries (Service providers, including government officials, private sector, family members of persons with disabilities, local authorities, lecturers, social workers, social work students, journalists, governmental agencies and non-profit organizations). The implementing organization and local partners: - The implementing organization: Disability Research and Capacity Development (DRD) - Main partners: DOLISA in Tay Ninh, Dong Nai and Binh Dinh - Other stakeholders: local departments, universities, hospitals, centers for social work practice 2. Economic, political and social context when implementing the project General context There is strong consensus between the disability community and the Government of Vietnam that people with disabilities should have equal opportunities to contribute to their country. The National Coordinating Council on Disability (NCCD) was established in 2001 by the Ministry of Labour, Invalid, and Social Affairs (MOLISA), who took responsibility for coordinating, promoting and monitoring disability policies, and implementing the 2012-2020 National Action Plan to support persons with disabilities. From 2010 to 2013, the government passed the Law on Disability and issued regulations to implement some areas of the law. In November 2014, Vietnam ratified the Convention on the Rights of Persons with Disabilities (CRPD). On October 9th, 2015, the Prime Minister reinforced the role of NCCD to comply with provisions of the CRPD. This Committee is tasked with assisting the Prime Minister to steer the coordination between ministries, sectors and localities in addressing issues related to persons with disabilities. The government is gradually adopting a rights-based model of disability and supporting the activities of the self-help groups of PWDs. The project plan is consistent with Vietnam's National Action Plan 2012-2020 - Supporting People with Disabilities. In addition, the target beneficiaries, objectives and activities of the project are also consistent with the annual plan of the local partners. The project operates in three priority provinces of the United States Agency for International Development (USAID) including Binh Dinh, Dong Nai and Tay Ninh, which are heavily sprayed with Agent Orange (dioxin). Factors affecting the approach, implementation schedule, and the project’s objectives • At the end of 2019 and early 2020, the Covid-19 pandemic appeared and spread rapidly, not only seriously affecting the health and life of people but also having a negative impact on the economic development and social activities. In that difficult and challenging context, many project activities were put on hold under the social distancing directive of the Prime Minister. This has affected the progress of project activities. Many activities (seminars, trainings, skills training, events, etc.) have moved to online form. Under the agreement and acceptance of the United States Agency for International Development (USAID), the project has extended the completion schedule to July 31, 2020. Even so, the project's goals are still guaranteed. • Global climate change and natural disasters (droughts, floods) caused damage to many countries and localities. However, the impact of natural disasters in Tay Ninh, Dong Nai and Binh Dinh is not significant on the approach, implementation progress and objectives of the project. • Policies and institutions of Tay Ninh, Dong Nai and Binh Dinh provinces have not changed significantly. The change in personnel in the organization of local partners (Tay Ninh and Dong Nai Departments of Labour, Invalids and Social Affairs) only had a small impact, not seriously affecting the approach, implementation progress and objectives of the project. 3. Objectives of the end-of-project evaluation The overall goals of this consulting work are to evaluate achievements of the project made after three years of implementation (and a one-year extended part) and to put forth conclusions and recommendations based on findings. Specifically, the performance evaluation is to record and analyze • Changes generated from project implementation in beneficiary groups and • Approaches made by the project to realize these changes. Questions of performance evaluation and their details as follows: 1) What targeted indicators and achievement levels has the project made? 2) What changes has the project made? In terms of o Awareness of PWDs's rights o Establishment and development of PWDs's clubs and self-help groups o Capacity of service providers of public works and public services o Capacity of DRD staff members and project partners 3) What are the relevant points during project management and implementation? In terms of o Relevancy, effectiveness and efficiency o Involvement of PWDs and the community o Gender mainstreaming 4) What are the lessons learned from the project? 5) What are the next steps after the project? The DRD leaders and consulting team have expected that by participating in the evaluation process, DRD staff members and local staffs would gain their own experience in self-evaluating their work and working efficiency through interactions and experience sharing with beneficiary groups and the consulting team. This is also how DRD the capacity of performance evaluation for project staffs and local staffs through the “hand-on learning” approach. 4. Methods of evaluation 4.1. Combine qualitative and quantitative evaluation Performance evaluation combines qualitative and quantitative evaluation. Qualitative information is collected through desk research, in-depth interviews, and group discussions. Quantitative information is collected through a structured questionnaire. Secondary information The consulting team has researched the following documents: • AFA’s for phase 1 and phase 2 • Project’s MEL plan • Project pre-evaluation assessment in Binh Dinh and Tay Ninh provinces • 6-month and 1-year project reports • Milestone summary reports, from No. 1 to No. 51 Qualitative information from in-depth interviews and group discussions 4 semi-structured questionnaires are developed to collect qualitative information from • Local partners • Union officials and teachers • PWDs participating in the project • Families of PWDs and community In-depth interviews and group discussions were conducted in different ways to better suit the context of the Covid-19 pandemic in the project areas. The consulting team conducted in-depth interviews and group discussions with local staffs on Zoom and Zalo. The local staffs, after some training, knew how to collect information through direct interviews with beneficiaries and community members. Quantitative information from the structured questionnaire A structured questionnaire was developed to collect quantitative information from PWDs beneficiaries. Local staffs and volunteers take charge of collecting the information. Quantitative information was collected via Kobo Toolbox software program or via hard copies (printed on paper) for actual convenience. The hard copies were made online and converted into the forms available on Kobo Toolbox accordingly. DRD officials and staff members were in charge of information storage and on-site supervision. 4.2. Source and sample size of information and sampling The source of qualitative information was selected purposefully, based on the relevance of project participants, beneficiaries and implementation processes through management related to planning, financing, training, etc. Information was collected by the consulting team from officials involved in project management and implementation, PWDs beneficiaries, families of PWDs and community members. 40 participants joined in-depth interviews and group discussions. The sample size for quantitative information was calculated on the assumption that obtained statistical results have an error of 5% and a confidence level of 95%, the rate p = q = 0.5, and applied to an identified population of 2,160 PWDs beneficiaries. The sample size is calculated using the one-sample proportion evaluation formula. These samples are distributed to the 3 provinces participating in the project, according to the principle of probability proportional to the size of the beneficiary population. After data cleaning, the number of quantitative samples used for performance evaluation is n = 324, distributed among Binh Dinh (n1 = 98), Dong Nai (n2 = 62) and Tay Ninh (n3 = 164). PWDs were interviewed in the most convenient way possible, with the common criterion of benefiting from the project, regardless of any particular benefit. However, the quantitative survey included questions to differentiate beneficiary groups. As a result, a total of 324 PWDs participating in the quantitative survey and 40 members related to the project in in-depth interviews and group discussions. 4.3. Group evaluation The performance evaluation was made in a participatory manner. Participants in performance evaluation consist of: independent consultants, DRD staff members, local staffs and community volunteers. Table 1. Role of consultants and performance evaluation group members Consultants Project staff members Preparation  Design the performance evaluation  Map out plans  Develop evaluation tools  Set up KoBoToolbox  Develop reference documents  Provide information  Comment evaluation tools  Prepare field works Information collection  Conduct online and offline in￾depth interviews and group discussions  Support information collection  Select participants in performance evaluation  Conduct offline in-depth interviews and group discussion  Collect quantitative information Information analysis  Conduct qualitative and quantitative analysis  Check quantitative sheets (hard copies)  Convert quantitative sheets (hard copies) into tables on KoBoToolbox Report making  Identify initial findings  Outline the evaluation report  Finalize the evaluation report  Discuss initial findings  Compile project overview and background  Comments the evaluation report 4.4. Implementation process and limitations Due to social restrictions to prevent the spread of Covid-19, field activities for evaluation were changed. The consulting team conducted in-depth interviews and group discussions via phone calls using software programs instead of direct. Volunteers received online support in information collection instead of direct support. Periodical meetings were organized during fieldwork between project staff members, beneficiary groups and consulting team to enable project staffs to self-estimate their own workload and efficiency. The consulting team, at the same time, promoted online discussions and enhanced roles of project staff members during the evaluation process. The evaluation group was unable to approach initial survey data so it was impossible to incorporate initial data into the performance evaluation report for comparative analysis. 5. Findings 5.1. Levels of achievement of target indicators The project achieved almost all the target indicators. The vital indicators of the project have been achieved at very high levels, i.e., numbers of PWDs beneficiaries and their involvement, and roles of PWDs’ organizations in local programs and projects. Here are some typical examples: • The number of trained service providers (individuals) serving vulnerable groups reached 2,142, compared to the target number of 1,850, # up to 115.7%. • The number of audited public places/buildings toward enhancing increasing PWDs’ right of access reached 490, compared to the target number of 210, # up to 233.3 %. • The number of PWDs’ organizations participating in monitoring and evaluating local PWDs support programs reached 19, compared to the target of number 14, # up to 135.7%. • The number of PWDsreceiving services directly provided by USAID-funded programs reached 2,729; against the target number of 2,160, # up to 126.3%. The indicators above exceeded set targets thanks to the fact that the project was implemented with interactive interventions and accompaniment. As capacities of service provision for PWDs groups are enhanced (indicator IR2.3 PM2), the number of PWDs’ organizations participating in programs went up (indicator DMP 9), and access to public services improved (indicator DMP 16 and indicator DRD 2). Consequently, there was an increase in the number of PWDs receiving the services (indicator DO2PM6). Moreover, thanks to the attraction of the project, when training courses or community activities were held, there were more PWDs participants than the invitees. This also resulted in an increase in the number of PWDs receiving training in gender-based violence, PWDs’ rights in the number of PWDs receiving services directly from the project. Two indicators reached lower levels than the set targets. It’s not because that project interventions are beyond reach, but due to the practical conditions at a certain extent. Besides, the targets had been set with high expectation that unintentionally pushed the expected numbers to impractical level. Specifically, the project set to have 16 legal documents, policies or procedures compiled, amended and/or issued to support the social inclusion of vulnerable groups. Actually, nine policies (indicator IR 2.3. PM1) are enough to enhance the social inclusion. It’s unnecessary to change policies (which are currently effective) to reach the set target (13 policies); this even will backfire. Similarly, there are no more than 66 organizations under State management giving support to vulnerable groups (indicator DO2. PM7). Table 2. Levels of achievement of target indicators Indicator Unit Target numbers Actual numbers DO2PM6: Number of PWDs receiving the services directly provided by USAID-funded programs Person 2,160 2,729 DMPG2: Number of PWDs receiving training in gender￾based violence Person 380 544 DMP 16: Number of audited public places/buildings Building 210 490 DMP 9: Number of PWDs’ organizations participating in monitoring and evaluation of local PWDs support programs Organization 14 19 IR 2.3. PM1: Number of legal documents, policies or procedures compiled, amended and/or issued to support the social inclusion of vulnerable groups Legal document, policy, procedure 13 16 IR2.3 PM2: Number of trained service providers (individuals) serving vulnerable groups Organization 1,850 2,142 DO2. PM7: Number of organizations under State management and local PWDs’ organizations dedicated to vulnerable groups having capacity enhanced Organization 100 66 DRD2: Number of organizations/unions with contribution to improvement of accessibility Association/ Union 9 25 5.2. Changes made thanks to the project Generally, the project has helped the PWDs community to raise awareness of PWDs’ rights, supported PWDsto set up groups to realize the rights and improve their role and status in society. In addition, the project has promoted adjustments related to building public works and strengthened capacities of service provision staffs. PWDs’ access to public services has been enhanced alike, as the project title reveals, “Access for All” (AFA). Table 3. Percentage of PWDs participating in the survey confirming changes to the PWDs community Binh Dinh Dong Nai Tay Ninh Total Awareness of PWDs and community about PWDs’ 77.6 54.8 84.8 76.9 rights enhanced PWDs’ clubs and self-help groups established and helped with operation 2.0 16.1 84.8 46.6 PWDs helped to use public works 37.8 19.4 48.8 39.8 Capacity of officials and social workers enhanced so that they can work with PWDs more effectively 36.7 30.6 41.5 38.0 No considerable changes made 5.1 3.2 1.2 2.8 No changes seen 7.1 12.9 1.2 5.2 From PWDs’ perspectives, the project has helped improve their quality of life. Specifically, PWDs become more self-confident, self-reliant and even can help other people in the same light. The role and status of PWDs in the family and community are strengthened. Changes acknowledged by those from Tay Ninh province are clearer than those of the two other provinces. Table 4. Percentage of PWDs participating in the survey confirming changes to themselves Binh Dinh Dong Nai Tay Ninh Total Quality of life improved 30.6 32.3 68.3 50.0 More self-confident 71.4 58.1 91.5 79.0 More self-reliant 23.5 51.6 59.8 47.2 Able to help other PWDs 15.3 35.5 54.3 38.9 Role and status in the family enhanced 15.3 11.3 31.7 22.8 Role and status in the community, society strengthened 14.3 8.1 18.9 15.4 No considerable changes made 8.2 3.2 3.0 4.6 No changes seen 8.2 1.6 0.6 3.1 5.2.1. Changes of awareness of PWDs themselves, family and community about PWDs’ rights Changes of PWDs themselves The project has helped PWDs to raise awareness of their own rights. Most of the participants (# 75%) in performance evaluation process admitted that the project has significantly improved their awareness of PWDs’ rights. Table 5. Percentage of surveyees acknowledging levels of changes to their awareness of PWDs’ rights Binh Dinh Dong Nai Tay Ninh Total Significantly improved 12.2 53.2 43.3 35.8 Considerably improved 56.1 19.4 36.6 39.2 Just improved 17.3 16.1 12.8 14.8 Partly improved 13.3 11.3 7.3 9.9 No improvement seen 1.0 0.0 0.0 0.3 In-depth interviews, group discussions and quantitative data show that PWDs’ awareness of their own rights has been enhanced, thereby making them more self-confident. This is thanks to their participation in groups and clubs dedicated to people like them, thanks to the sharing from members of such groups and clubs, and thanks to communication activities of the project. o “Earlier, I didn’t pay much attention to laws or policies dedicated to PWDs. I’ve become more self-confident and happier since my joining the club (…). It’s great that I’m now very self-confident thanks to receiving training in issues related to PWDs…” Opinion expressed at a group discussion of a PWDs’ club in Tay Ninh o “Before joining the club, my husband just stayed at home, so he was afraid of communicating with other people, including our relatives. But since he participated in the club set up by the project, he has become more communicative (…). I think the project has made many changes and helped PWDs better understand their own rights and policies and laws dedicated to them.” An interview with a disabled person’s wife in Tay Ninh o “We have had better understanding of PWDs’ rights and been more confident in joining groups to get sharing from those who are in the same situation, thereby creating power through solidarity and gradually changing the way the community views PWDs.” Opinion expressed at a group discussion of a PWDs’ club in Binh Dinh Most of the PWDs participating in the survey can list their common rights under the Law on PWDs. The most-mentioned rights are the eligibility for monthly allowance, the eligibility for health insurance coverage and the eligibility for tuition fee exemption and reduction when PWDs study at general or vocational schools. About one third of PWDs participating in the quantitative survey mention PWDs’ rights related to supporting and enabling PWDs to get employment at business households and enterprises. Table 6. Percentage of surveyees naming PWDs’ rights under the Law on PWDs Binh Dinh Dong Nai Tay Ninh Total Monthly allowance 96.9 91.9 95.7 95.4 Health insurance coverage 98.0 96.8 85.4 91.4 Tuition fee exemption and reduction 26.5 32.3 40.2 34.6 Free vocational training 40.8 30.6 37.2 37.0 Discount tickets for culture, sports, entertainment services… 25.5 27.4 32.9 29.6 Persons with severe disabilities (PWSD) are exempt from service fees when participating in traffic 26.5 30.6 32.9 30.6 PWDs or households generating jobs for PWDs are eligible for loans with preferential interest rates, receive training in production, business… 28.6 30.6 48.8 39.2 Production and business establishments where PWDs account for 30% of the workforce receive funding to improve working conditions to better suit PWDs and are eligible for corporate income tax exemption… 8.2 21.0 29.3 21.3 Other rights according to legal provisions 18.4 48.4 25.0 27.5 Generally speaking, about two thirds of PWDs have their awareness and self-confidence raised thanks to activities of the project. Regarding the participating provinces, PWDs in Tay Ninh have their awareness better enhanced thanks to their greater participation in groups, clubs and communication activities of the project than that of those in Binh Dinh and Dong Nai. Table 7. Percentage of surveyees talking about opportunities to have their awareness of PWDs’ rights enhanced Binh Dinh Dong Nai Tay Ninh Total Through participating in communication activities of the project 39.8 37.1 90.9 65.1 Through participating in PWDs’ clubs, groups set up by the project 10.2 54.8 98.8 63.6 Through other PWDs 37.8 43.5 28.7 34.3 Through dear ones 64.3 30.6 20.1 35.5 Through books, newspapers, radio and TV shows 72.4 59.7 34.8 50.9 Changes to families and community The project has raised awareness of families and the community about PWDs’ rights and capacities, thereby creating favorable conditions and contributing to improving the role and status of PWDs in the family and community. Here are some remarkable statements from in￾depth interviews and group discussions in the project areas. o “As the awareness of families and the society has been much improved in recent times, PWDs have had more chances to live a normal life (…). People’s view on PWDs has also been changed positively. And PWDs’ families have always endorsed support for PWDs to stay self-reliant and remove barriers.” o “Earlier, I didn’t support PWDs’ social inclusion because I thought they would get hurt easily by others due to their lack of understanding of their own rights and policies dedicated to them. But when seeing changes made since they joined activities of the PWDs’ club, I’ve changed my mind. I see that they are now more self-confident and able to protect and take care of themselves. So, I feel more reassured now.” o “As far as I know, the public now limits the use of the term “handicapped” and uses the term “disabled” instead to refer to people with disabilities. They have also given more enthusiastic support to PWDs. They have no longer discriminated against PWDs, either because they have had better understanding of PWDs’ rights.” o “Among the changes, we have seen an important change in terms of awareness of the community about PWDs.” o “The community has had a more positive view on PWDs. Earlier, people thought that PWDs could do nothing. So, they gave help by giving cash or money to PWDs. But now some enterprises and production establishments have enabled PWDs to do the jobs that suit PWDs’ conditions and physical health. Therefore, PWDs can enjoy fully the rights and obligations of a citizen.” 5.2.2. Establishment and changes of PWDs’ clubs and self-help groups Number of clubs, groups and their membership The project has helped to establish 12 PWDs’ clubs and groups (compared to the target number of 6), with 281 official members. According to DRD staff members, the clubs that were set up when the project was in the final stage have not yet been equipped fully with capacities related to group leadership planning, event planning, crowdsourcing, legal policies, policy advocacy, gender and gender-based violence prevention. Therefore, the clubs still have confusions while implementing their activities. 8 PWDs’ clubs established at the final stage of the project have no chance to take part in awareness enhancement and capacity improvement activities. Among the PWDs participating in the quantitative survey, 97% are members of clubs and self￾help groups. The figure is lower in Dong Nai (48.4%) and Binh Dinh (11.2%). Capacity of heads and members of PWDs’ clubs and self-help groups Members of PWDs’ clubs and self-help groups received trainings in the Law on PWDs and relevant policies, management skills, teamwork skills. They are supported to connect with legal services, vocational training and rehabilitation, etc. Table 8. Percentage of members talking about benefits of participating in PWDs’ clubs and groups Binh Dinh Dong Nai Tay Ninh Total Trained in legal provisions and policies dedicated to PWDs 63.6 76.7 91.2 87.5 Trained in management skills, teamwork skills… 27.3 36.7 73.0 65.0 Familiarized with special subjects during monthly meetings 9.1 13.3 58.5 49.0 Provided with consultancy and supported in terms of legal services, vocational training, and rehabilitation… 72.7 43.3 33.3 37.0 Acquainted with people sharing the same situations 27.3 60.0 57.9 56.5 Enabled to promote mutual support 63.6 66.7 61.0 62.0 During in-depth interviews and group discussions, members listed changes made to themselves thanks to their participation in the PWDs’ clubs and self-help groups. For example: o Become more self-confident at work after well trained as members of the club executive board in terms of management and supervision of monthly meeting agenda o Have club management skills significantly improved compared to the beginning o Acquire skills to help PWDs feel more confident and encourage them to better integrate into social life o Feel more optimistic and happier when having a place to come to share thoughts and expectations o Have an equal opportunity to access State agencies and local support policies for PWDs o Attract more PWDs to join so that they can raise their voice about issues facing PWDs and become more confident in life Employment and payment for members The project has helped PWDs to access many different sources of funding to invest in livelihood development. A total of VND820 million was mobilized to support livelihood development and emergency cases. In addition, thanks to the project, PWDs have become more confident in life, getting fair access to State livelihood support services, such as vocational training, technical training and bank loans. o “Earlier, they had to struggle very hard to earn their living. Thanks to the financial support of the club and DRD, life of club members being PWDs has become easier and more stable.” Opinion expressed at a group discussion of a PWDs’ club in Tay Ninh o “PWDs like me are always at a disadvantage when they want to borrow just a small sum of money, VND1-2 million, for example. Many people often think that PWDs don’t have any source of income to pay the loan. In that situation, I didn’t know what to do rather than look for help from others. Since my joining the club, I have become more confident, although life is still hard. Specifically, I came to relevant agencies to learn more about preferential loans. Instead of keeping silent as before, now I am confident enough to raise my voice to protect myself when asked what I could do to pay the loan.” Part of an in-depth interview with a disabled person in Binh Dinh Enhancing roles of PWDs, clubs and self-help groups in the community and the society PWDs’ clubs and self-help groups have played well the role of a connector among members so that they can meet, share things and promote mutual support. Through such gatherings, PWDs have affirmed their role in community activities and society. o “Thanks to joining the club, PWDs are enabled to exchange and share experience and help each other to become happier and more confident in life.” o “Since the establishment of the club, we have received both spiritual and material support and encouragement from State agencies at different levels, local authorities and benefactors and improved knowledge about PWDs’ interests and obligations, and especially support policies for PWDs of the local authorities.” o “I have had a chance to survey PWDs’ expectations about the building of public works. I felt a bit afraid in the beginning and didn’t know what to do first. But after getting involved in this work for a while, I have realized how meaningful it is. I have also had a chance to raise my voice and get my petitions noticed. Although such petitions are not yet to be realized, it is still important because I can raise my voice about the needs of PWDs like me and how to make it easier for us to travel. I do hope that my petitions will be realized soon.” o “The policy advocacy training course held by DRD has helped club members to have better understanding of PWDs’ interests and know how to get their expectations and petitions heard.” o “Positive changes, such as ramps for PWDs built, and better access to public services for PWDs, have enabled PWDs to get involved in social activities, find jobs, learn new knowledge and take care of themselves. Therefore, they can unleash their inherent abilities and are no longer burdens for the family and society.” Opinions expressed at group discussions of PWDs’ clubs in Tay Ninh and Binh Dinh 5.2.3. Changes of awareness and capacities of service providers toward increasing PWDs’ accessibility to public works and public services Changes of awareness and attitude of local authorities and public service providers The project has helped improve local officials’ understanding of PWDs’ rights and change their attitude and practice toward providing public services with greater care, respect, and convenience for PWDs. o “Local authorities at all levels have paid attention to PWDs’ social inclusion needs by renovating ramps for greater convenience so that PWDs can get better access to such public works as culture houses, buildings of the social security sector and State management agencies, etc. So, PWDs’ right of access has been better ensured.” Opinion expressed at a group discussion of a PWDs’ club in Binh Dinh o “Changes have really been made. Local authorities have given more enthusiastic support to PWDs. Attitude to them has also been changed positively.” Opinion expressed at a group discussion of a PWDs’ club in Tay Ninh o “Local authorities have paid greater attention to us than before and enabled us to perform administrative procedures more quickly. They have also taken better care of our life. So, it can be said that this project has made visible changes here in our locality.” Opinion expressed at a group discussion of a PWDs’ club in Tay Ninh Quantitative data showed that the project has helped PWDs with better access to public services. This positive trend is seen more clearly in Tay Ninh. Table 9. Percentage of surveyees agreeing on degrees of improvement in access to public services Binh Dinh Dong Nai Tay Ninh Total Much improved 6.1 24.2 41.5 27.5 Improved 35.7 12.9 30.5 28.7 Normal 39.8 51.6 17.7 30.9 Partly improved 14.3 11.3 10.4 11.7 Unimproved 4.1 0.0 0.0 1.2 Public works are made more accessible, supervised and renovated Improvement of access to public works, supervision and renovation of such works are the highlights of the project, making it different from other projects and programs. Not only PWDs beneficiaries but also providers of such services highly appreciate this intervention. o “Many changes have been made. For example, civil employees at State agencies like People’s Committee have treated PWDs more kindly; and ramps have been built at public works to facilitate the movement of wheelchair users.” o “I highly appreciate the survey on access to public works in the precincts of State agencies, which serves as the foundation to send petitions on improvement of access to such works for PWDs. This is a meaningful activity, creating favorable conditions for PWDs to access basic services at administrative agencies. Importantly, working with DRD staff members and surveyees being PWDs, now I better understand the necessity to build ramps for PWDs.” In-depth interviews with project staff members in Binh Dinh D Map – A helpful mobile app for PWDs About one third (32.1%) of PWDs participating in the survey have got to know or heard about the access map of DMap mobile app. The number of people knowing about DMap in Tay Ninh is higher than that in Dong Nai and Binh Dinh. Almost half of those knowing about DMap (44.2%) have installed the app on their mobile devices. Most of those have installed the app give positive comments on the app, with the percentage of people saying they are satisfied and very satisfied with the app being 47.8% and 28.3%, respectively. Those who have not installed the app yet are those who have not had a mobile phone yet. o “We have heard a lot about DMap and its benefits. But even some executive board members do not have smartphone. And many PWDs do not have one, either. So, we cannot use DMap app.” Opinion expressed at a group discussion of a PWDs’ club in Tay Ninh 5.2.4. Capacity improvement among DRD staff members and project management partners Capacity improvement among DRD project staffs in-charge and project provinces DRD has applied the rights-based approach of the Convention on the Rights of PWDs and the asset-based community development (ABCD) approach of the social work sector to support activities for PWDs. Specifically, interventions of the project turn toward raising awareness of related parties about PWDs’ rights, awareness of individuals and PWDs’ club members about rights. In addition, DRD has mobilized the involvement of related parties in giving support to PWDs, especially mobilizing nearly VND1.4 billion worth of support to comprehensively meet PWDs’ needs. The capacity of the provincial Coordination Board and in-charge groups in the provinces is enhanced through training courses and the implementation process of the project. o “We are enabled to join training courses on crowdsourcing and community-based survey, evaluation and supervision activities which are really helpful for our work.” o “Now we can give good advice to senior leaders on PWDs’ real needs and sources of support for PWDsin terms of vocational training, healthcare, support programs, etc. More importantly, we know that it’s more effective to attract the involvement of many related parties.” In-depth interviews with representatives of DOLISA Capacity improvement among local staffs working with PWDs 1,654 staff members at organizations and bodies supporting PWDs and social work students were trained in social works with PWDs. Table 10. Number of trainees in social work with PWDs Participants in training courses on social works with PWDs Target number Actual number Lecturers, social workers 200 209 Healthcare workers 480 488 Staff members at social facilities 480 474 Social work students 480 483 Total 1,640 1,654 Building groups of core trainers in social work with PWDs has reduced the dependence on DRD staff members, contributing to extending training capacity to more related parties. Specifically, DRD has collaborated with the Society for Nonprofit Organizations (SNPO) in the South to offer a Training of Trainers (TOT) courses on social works with PWDs to 20 members. The core trainers in turn have given training to social workers. Reports on the training course showed that learners were provided with further knowledge about disabilities, support models for PWDs, crowdsourcing to support PWDs and social works with PWDs. Most of trainees said that after those courses, they were able at least show other people what they have learned. More importantly, they could other people understand the knowledge and do what needed; and they could become trainers. The trainees grasped what barriers facing PWDs are, how to communicate effectively with PWDs, how to remove the barriers, make rational adjustments and support PWDs based on the community. Trainees know how to apply learned knowledge to work and life, from different perspectives. Some remarkable applications include changing old ways of working by employing new ways of thinking and giving support to specific cases. In addition, the learners can share experience with their peers at training workshops or give advice to senior leaders. o “The training course helps me realize that we have still followed old models that are no longer suitable. I also realize that disabilities do not make people dependent, but barriers from society do. Such barriers make PWDs become helpless in other people’s eyes. When we do not understand them, there’s no way for us to know what they really need. Doing social work is not doing something that you think is right. This course makes me understand that we should do what that can help PWDs become more confident, remove barriers for their social inclusion.” Sharing of a staff member at Ben Tre psychiatric hospital after completing the training course o “I myself have started changing my ways of thinking and doing. And I have shared learned knowledge with my colleagues at Long An psychiatric hospital. The training course is really helpful. My colleagues and I have had better understanding of PWDs’ problems and needs.” Sharing of a staff member at Long An psychiatric hospital after completing the training course 5.3. Relevancy in project management and implementation 5.3.1. Succession The Access For All (AFA) project can be seen as a successor of the Accessibility for Integration Project (AIP). AIP has laid the foundation for enhancing awareness of PWDs’ rights and promoting the community’s participation in supervising the accessibility of public works. Many problems in the implementation of AIP remain, such as barriers to PWDs’ social inclusion and poor awareness of the community and PWDs themselves about PWDs’ rights. Moreover, there are few PWDs’ groups and clubs with limited capacity. PWDs’ voice and image are not strong enough to advocate issues at provincial and national levels. Support policies for PWDs have not been implemented fully yet; and there is a lack of an effective system to supervise the implementation of these policies. AFA has promoted activities of AIP in terms of form and scope of implementation. The project’s scope is expanded to other wards of the districts, other districts of the provinces, and another province of Dong Nai. AFA has also aimed at solving problems that are more systematic. AFA is an upgrade from push for integration to push for equal opportunities. 5.3.2. Relevancy of interventions The interventions and ways of implementation of the project have met PWDs’ real needs. It’s thanks to DRD’s success in involving PWDs and PWDs’ clubs and self-help groups in building the project. In the implementation process, the project conducted a survey to identify the needs of beneficiary groups. DRD has not simply implemented the project according to set targets but also paid comprehensive attention to local PWDs’ needs. Specifically, in the process, DRD has realized that PWDs lack means of transport, funding doing cultivation and animal husbandry or small-scale business. That’s why DRD has raised nearly VND1.4 billion to give wheelchairs, three wheelers, artificial limbs, loans and Covid-19 relief grants to PWDs. Nevertheless, due to limited resources, the project’s interventions just meet PWDs’ needs exactly, not fully. Table 11. Percentage of surveyees talking about the degrees of response of the project to PWDs’ needs Binh Dinh Dong Nai Tay Ninh Total An absolute majority of the needs met 0.0 9.7 26.2 15.1 A majority of the needs met 13.3 9.7 36.6 24.4 Half of the needs met 27.6 12.9 18.3 20.1 A small part of the needs met 58.2 67.7 18.3 39.8 No needs met 1.0 0.0 0.6 0.6 5.3.3. Involvement of local partners in project management processes The project has attracted the active involvement of local partners in project management processes, in which officials from the Labor, Invalids and Social Affairs sector and People’s Committee at all levels play the key role. Total capital and cost estimate of the project are provided through agreements and during the process of extending project activities to related parties in the province, especially the provincial People’s Committee. The provincial Department of Labor, Invalids and Social Affairs (DOLISA) issues and sends specific documents and plans to district Office of Labor, Invalids and Social Affairs (OOLISA) and relevant agencies of communes in the scope of the project. Based on issued documents and plans, the involved parties will build specific action plans and collaborate with People’s Committee at district and commune/ward/townlet levels for implementation. DOLISA assigns in-charge staff to join DRD staff members in tracking progress and reporting results on a scheduled basis or an ad hoc basis if required. The results are fully updated in the project MEAL plan and project progress reports. 5.3.4. Mobilizing resources outside the project Apart from resources from sponsors, DRD, DOLISA and OOLISA of the provinces and districts participating in the project have mobilized many other resources to support PWDs. Supports are diverse, ranging from support for specific cases (artificial limbs, wheelchairs, etc.) to long-term (vocational training, loans, etc.) and emergency support (Covid-19 relief). Table 12. Sources of support outside the project Forms of support Number of beneficiaries Value (VND) Grants for livelihood development 46 550,000,000 Covid-19 emergency relief grants 271 270,000,000 Artificial arm 1 23,000,000 Artificial leg - 447,000,000 Wheelchair 10 50,500,000 Three-wheeler 1 15,000,000 Total 1,355,500,000 Here are some opinions expressed at group discussions in Tay Ninh, Binh Dinh and Dong Nai: o “DRD has connected different sides to give support to PWDs. Specifically, thanks to benefactors connected by DRD, members of our club have capital for production and cultivation activities, creating greater changes.” o “The project has helped mobilize resources to provide PWDs with necessary equipment like wheelchair, three wheeler, walker, hearing aids, artificial limbs, etc.” o “Qualified PWDs are helped to apply for loans from the Vietnam Bank for Social Policies, provincial PWDs funds, etc. PWDs are also enabled to take part in suitable vocational training courses.” 5.3.5. Integrating elements into a rights-based development and intervention project PWDs have taken an active part and played important role in implementation of the project. They have joined the project as individuals, through their clubs and self-help groups. PWDs’ involvement is well reflected through in-depth interviews and group discussions, for example: • Trained in legal provisions and policies dedicated to PWDs, life skills, communication skills, teamwork skills, skills for social inclusion, etc. • Enabled to join DRD staff members in the survey on the accessibility to public works at People’s Committee, schools to get suitable designs for PWDs. • An exchange event for PWDs entitled “One World for All” held to help PWDs become more sociable and have a chance to join joyful activities like wheelchair racing, art performances, entertainment, and sports activities, etc. • A survey on PWDs’ families in difficult circumstances conducted to create favorable conditions for them to do business. • Trained in life skills, communication skills, teamwork skills, etc. to make PWDs more self￾confident and self-reliant in life for better social inclusion. • Enabled to take part in exchange activities of clubs in Da Nang to learn their experience in operating the clubs and supporting club members. • Enabled to join the medical assessment club dedicated to PWDs of the commune and monthly meetings of the club to find ways to help PWDs to approach and enjoy support policies of the State. • Enabled to join the supervision team in charge of making feedback reports in an effort to make public works more accessible for PWDs. Table 13. Percentage of surveyees assessing the integration of elements into the project Very good Good Acceptable Not good Bad/unfulfilled Involvement of PWDs in the project 31.5 42.0 17.9 1.5 0.3 Rights-based support for PWDs 27.2 46.3 19.8 1.9 0.3 Resource mobilization to meet PWDs’ needs (wheelchair, funding, scholarship, etc.) 23.1 38.0 21.9 8.3 0.3 Great attention to women and gender equality 22.2 48.1 13.9 3.7 0.0 Transparent implementation of project activities 29.3 44.4 8.3 3.1 0.3 Interventions of the project with no impact on environment natural and ecosystem 29.0 45.4 7.7 3.1 0.0 5.4. Satisfaction with the project Most of the PWDs participating in performance evaluation process are satisfied and very satisfied with the project. However, about a quarter of surveyees said that they were neither satisfied nor dissatisfied with the project, depending on specific cases. Table 14. Percentage of surveyees expressing satisfaction with the project Binh Dinh Dong Nai Tay Ninh Total Very satisfied 9.2 8.1 49.4 29.3 Satisfied 53.1 48.4 31.7 41.4 Normal, it depends 35.7 37.1 14.0 25.0 Dissatisfied 2.0 1.6 0.6 1.2 Very dissatisfied 0.0 4.8 4.3 3.1 5.5. Lessons learned and future directions Some project activities should be boosted, especially the activities requiring collaboration of organizations and bodies concerned. After participatory survey with many stakeholders concerned, building of ramps for PWDs is suggested to implement the sooner the better. Implementation and expansion of early intervention and inclusive education programs for children with disabilities, including children with mental delay, autism, speech delay, etc., should continue. Direction and coordination for implementation of project activities were notsynchronous yet and resources not properly used to build works accessible for PWDs. Organizations and bodies concerned should give advice to provincial People’s Committees so as to timely issue plans and projects. They should take the initiative in implementing assigned functions and tasks. In addition, collaboration among organizations, unions, NGOs in giving support to PWDs is strongly needed. The needs of people with disabilities are diverse, including health care, rehabilitation; assistive devices (wheelchairs, prosthetics); education, vocational training, employment, loans, skills improvement, social participation, advocacy and policy monitoring, therefore, with its resource mobilization strengths, DRD should continue promote community-based resource mobilization methods with the participation of all stakeholders to be able to comprehensively support the needs of people with disabilities. Specifically, it is recommended to find out the needs of clubs with disabilities, then analyze the roles and resources of stakeholders and develop a plan for cooperation, connection and mobilization of resources to support people with disabilities. It’s necessary to expand club membership. Each commune should have a PWDs’ club with promotion activities about laws, employment and policy approach for PWDs. Besides, PWDs have chances to share their feelings and expectations. The more PWDs’ clubs are set up, the better PWDs’ voices will be heard and the greater their impacts on policies are, contributing to better ensuring PWDs’ rights and interests. PWDs’ clubs and groups have demand for direct exchange with other localities to learn more practical models so as to expand club membership. 6. Conclusion This is a project in collaboration with many local departments and spread across 3 provinces. The advantage of the project is the realization of the very goal that Vietnam has been pursuing since 2007. Therefore, the project is strongly supported and synergized by the local mass organizations in the overall purpose. The aim of the project is to promote equal opportunity for PWDs by building capacity and improving access to public works in Tay Ninh, Dong Nai and Binh Dinh provinces. Thanks to the connection between the goals of the government, and the leaders of the provinces in the project area and the donors, the project has created a common voice to bring activities to people with disabilities in the area. The project also calls for the cooperation of leaders of departments and agencies in localities, an effective component in the implementation of relevant policies and programs, and in disseminating and disseminating information widely to people with disabilities. and community. After 3 years of efforts, the project raises awareness not only for PWDs but also the community thanks to effective communication, training and consulting activities of project staff, volunteers of the PWD Club as well as good coordination between departments and organizations. And it is noteworthy that the project contributes to the creation of a legal entity "raising awareness of the law on people with disabilities" which has been achieved not only for the people with disabilities themselves but also for the community to understand about the rights of people with disabilities through the Law on Disabilities. training and communication in the project area. The community has a positive view of PWDs. Many PWDs actively participate in clubs, self-help groups and social activities. The project has helped create jobs for many PWDs who need to find jobs through counseling, vocational training support, job placement or self-implementation of farming and livestock models. As a result, many PWDs can find suitable jobs, stabilize their lives and avoid the risk of being dependent on family or social assistance. 7. Recommendations The role of the PWD Club as a bridge organization for DRD with the locality, so DRD should consider the club as a strategic partner to jointly deploy activities towards the goal that these clubs will move towards establishing a local disability association that can operate activities autonomously. It is necessary to continue developing synchronous disability clubs in the provinces and equipping them with capacity so that the clubs themselves will provide support services to club members, such as peer counseling, independent living skills, etc. setting up, managing personal finances, consulting career guidance, introducing jobs, consulting regimes and policies. On the local side a commitment to maintain activities as an official organization for the locality, to maintain periodic activities such as visits, legal communication, integration and vocational training, job…. Otherwise, the sustainability of the results generated (especially counseling activities, periodic activities to learn about the needs of people with disabilities, etc.) will be very low in the future. Because the positive actions of the PWD Club are currently only recorded as initial changes, if not maintained, it will lead to PWDs and the community that may not be sustainable. The locality should have a Committee to Support People with Disabilities under the Provincial Committee and have a leader of the Committee as its Head to effectively coordinate the parties involved in supporting PWDs. In fact, it is proved that localities have boards for hunger eradication and poverty reduction, boards for the advancement of women.... Both have the leadership of the Provincial Committee as the head of the department and the relevant Departments/Departments as permanent members, the coordination with the Departments and Sectors is very favorable. For vocational training/job placement models, DRD may consider negotiating with the following prudent, but necessary, approach: (1) More active in communication with vocational schools to the school's team assesses the actual needs of people with disabilities to organize them to suit their needs; (2) In case the school is not ready, it is recommended to conduct a survey through the local Departments/PWDs Club for PWDs who need to find evidence of the appropriateness of the contents and methods as the basis for the school. to convince schools to have appropriate programs. DRD should consider deploying a model to support livestock/cultivation through revolving credit capital so that the elderly/disabled PWDs can have production conditions to improve living conditions live for themselves PWDs to integrate more strongly into life.