Ianoi, Viet N 'am Decemb :, Displaced Children a nd Orphans Fund L PROJECT NAME: EXPANSION OF COMMUNITY SUPPORT 1 FOR CHILDREN WITH DJSABI LITIES - A MIDTERM EVALUATION REPORT PROJECT GRANT NUMBER: 896-98-002 PROJECT FUNDING AGENCIES: UNITED STATES AGENCY ON INTERNATIONAL DEVELOPIIIENT, DISPLACED CHILDREN AND ORPFIANS FUND (IXOF) PROJECT EVALUATORS International Evaluator: Susan M. Ryan, P11.D. Viet Nam Evaluators: TS. Nguyen Tuyet Nga ThS. Nguyen Thi Nhu Mai Bs. Nguyen Thi Quyen ThS. Nguyen Thi Iloang Yen TS. Kieu Oilc Thanh CN. Le Thi Duc TS. Vu Quoc Chung TS. Nguyen Cong Khan11 Ths. Nguyen Lang Binh Bs. Nguyen Thi T11uy Iluong TS. Tran Bich ITanh Bs. Trinh Quang Dung Project Director: Bs. Nguyen Thi Minh Thuy, CRS Project Manager: ThS Le Van Tac, NIES-MOET Project Assistmt: Pham Minh Muc Date: December 4,2000 Acknowledgements To the children, families, teachers and community members of Luong Son, Yen Khanh, and Yen Hung district who are buildiiig inclusive classrooms and schools in Viet Nan?. Executive summary In 1995, Catholic Relief Services funded the Viet Nam Center on Special Education to pilot a program to include children who experience disabilities in preschools. The "Expansion of Community Support for Children with Disabilities" expanded in 1998 with funding from the United States Agency on International Development under the Displaced Children and Orphan fund and implemented by the Hanoi Center on Special Education, National Institute of Educational Science and the Catholic Relief Services. It went from an initial one province pilot project to a three province project in Northern Viet Nam. The three provinces, Luong Son-IJoa Binh; Yen Khanh-Ninh Binh; and Yen Hung-Quang Ninh were chosen because of variations in geographic, socio-economic, education, and other characteristics including: (a) assessed need; (b) capacity of the district education office; and (c) potential for collaboration anlong communities and organizations in the district. Two thousand six hundred and six children with disabilities across the three provinces benefited from this project. The project budget of $1,170,152 and $4 19,959 included contributions of $152,899 from the Catholic Relief Services and $287,500 from local agencies (e.g., National Institute of Education Studies and the school sys tems .) A midterm evaluation was conducted during November 2,000. This report describes the methods, participants, findings, recommendations and implications of that evaluation. The project has many areas of success. Results indicate the following project strengths: (a) impact on the lives of children with disabilities; (b) increased community awareness of issues related to children with disabilities; (c) improvements in the quality of teaching in schools; (d) strong community infi-astn~cture; (e) age appropriate and natural proportions placement of children with disabilities in classrooms; (f) strong support to and for families; and (g) strong collaborative partnerships. Areas of continued improvement include: (a) expanded and in-depth training for teachers; (b) implementation of academic/individualized educational services for children with disabilities; and (c) identification of the levels of supports children with disabilities need to be successful in inclusive classrooms. A discussioil is included in this report that highlights findings of the evaluation that go beyond the goals and objectives of the project; yet, are relevant to the continued overall success and ultimate sustainability of the project. Inlplications and recommendations are offered that will enhance the current project and provide guidance for project expansion throughout Viet Nam. Description of Project The "Expansion of Community Support for Children with Disabilities" project funded by the United States Agency on International Development under the Displaced Children and Orphans Fund (DCOF)"~~~ implemented by the Hanoi Center for Special Education, National Institute of Educational Science (CSEINIES) and Catholic Relief Services (CRS) expanded in 1998. It went from an initial pilot to a three province project in Northern Viet Nam. The three provinces, Luong Son-Noa Binh; Yen Khanh￾Ninh Binh; and Yen Hung-Quang- Ninli were chosen because of variations in geographic, socio-economic, education, and other characteristics including: (a) assessed need; (b) capacity of the district education office; and (c) potential for collaboration among communities and organizations in the district (as cited in RFA #492-98-005). Two thousand six hundred and six children with disabilities across the three provinces were identified. The project budget of $l,l7O,l52 + 4 19,959 included contributions of $152,899 from Catholic Relief Services and $287,500 from local agencies (e.g., National Institute of Education Studies and the school systems). Background of Project (as cited in RFA #492-98-005) Children in Viet Nam, with mental retardation, physical disabilities, vision and hearing impairments, deaf blindness, learning disabilities, autism, and multiple disabilities, and disabilities resulting from agent orange syndrome, have had limited access to schooling in the past. Traditionally, children with disabilities have been cared for by their families, who often viewed them as burdens to society or sources of shame and pity. The grant application states that "due to limited capacity and distance from the family and community, separate special education programs and classes have not succeeded in educating the vast majority of children with disabilities in Viet Nam, keeping them on the fringes of the society in which they will live and compete for resources. These children have depended on others for their health, welfare and education; all children have been made vulnerable by rapid social change and economic growth" (p. 2). Project collaborators are encouraged by the government of Viet Nani' s (e.g., CSEINIES) positive trend and priority provision af "education for all". Viet Nam is proud of its position as first Asian signatory to the United Nations Convention on the Rights of the Child, which states that children with disabilities have a right to a fidl and decent life (Article 23) and that "every child has a fundan~ental right to education" (Article 28). In fact, national legislation in Viet Nam has been adjusted favorably in line with the provisions of the UN Convention. The right to education for children who experience disabilities was expanded in the 1994 statement of the UNESCO World conference on special education needs in Salamanca: Education for children with special education needs should be provided within the regular e.ducation system, which has the best potential to combat discriminatory attitudes, create welcoming communities, and build an inclusive society. Inclusive schwbls are the most effective at building solidarity between children with special needs and their peers. Countries with few or no special schools should establish inclusive - not special - schools. i The provisions of the UN Convention on the Rights of the Child and UNESCO 1994 World Conference statement on Special Education form the bases of the current project on successfi~lly building the capacity of teachers and con~munity members to support children with disabilities in regular education classrooms in the three identified provinces in Viet Nam. Specifically, the purpose of the current project is: to ensure inclusion of children with disabilities and their families in their communities by increasing awareness of their needs, training educators to meet their needs in general education classrooms, and strengthening community support. The objectives and activities delineated in the project grant application included the following: Project Objectives and Activities *Increase awareness of and support to children with disabilities in Viet Nan1 by documenting and disseminating experiences, methodology and training curricula for use in preschools and primary schools; *Expand inclusive education into three districts of northern Viet Nam by increasing awareness of families and community members about the abilities and needs of children with disabilities and by providing teacher training; *Establish comnlunity based support for children with disabilities. Main Activities of Project *Develop training materials and textbooks on inclusive education; "Organize training courses on teaching children with disabilities for inservice teachers of preschools and primary schools and preservice teachers of teacher training colleges; *Increase awareness of disabilities in general and the unique needs of children who experience disabilities; "Strengthen community support Evaluation Description and Design Evalrlatior~ Purpose The purpose of this midterm evaluation is to evaluate the overall impact of the project, identify limitations, and determine implications for continued project activities. The following section describes the evaluation plan, goals and objectives as stated in - -the initial grant application. Evaluation Plan The evaluation plan was designed by the project staff of the Expansion of Community Support for Children with Disabilities (ECSCD). The evaluation plan outlines: (a) evaluation objectives; (b) expected outcomes; (c) activities of the project; (d) indicators; and (e) interviewee. Table I below further delineates the evaluation plan. Table 1 Evaluation Plan (rehabilitation group) 0 bjectives Increase awareness of community towards clldren .-A .- with disabiliges Vie Expected Improve awareness of local authorities, teachers, families and community workers toward CWDs ability and needs Activities of the project 2J Develop training materials ands books on E for in- and pre-service teachers at pre￾primary-school levels. 3 Conduct training courses on teaching CWDs for in-service teachers of pre-and primary schools and pre-service teachers of Teacher Training Colleges (TTC) 3 Hold seminars for local authorities and steering committees 5J Advocacy on local mass media a Training courses for community workers, parents.. . 2 Organize Inclusive Mid-- autumn Camp Indicators Number of books and materials developed Quality of training materials Changes in regulations, policies, action plans of the local authorities to support incIusive education Teaching skill of key teachers of districts and schools Involvement of families, and social organizations as well to facilitate disabled children' s attending school Interviewee 2. District/comm une project steering committee members 3. Headmasterlte acher 4. Community support team members 5. FarniliesICW Ds Objectives IE for ems in 3 districts Expected The CWDs go to school and be equal members in their classes Activities of the 1. The questionnaires to identifl the needs and abilities of CWDs 2. Survey of CWDs 3. Facilitate disabled children to go to School of Education Support disabled children to go to the inclusive classes Indicators 1) Number of CWDs who go to school before the project and at the present 2) Number of CWDs who have not gone to School of Education Teachers' skills in identifying and assessing the needs and abilities of CWDs 3) Teaching method of teachers in teaching different children with different types of disabilities 4) Educational plans for CWDs 5) The involvement of parents of CWDs, children without disabilities, rehabilitation workers, individuals, social organizations in supporting disabled children to go to school 6) The participation and interest of CWDs in the school/class activities 7) CWDs' performance at school Interviewee I) Districu'com mune Steering Committee 2) Headmaster s of primary schools pre- - SG~OO~S 3) Primary and preschool teachers 4) Community support team 0 b jectives Cornmuniry support for children with disabilities Expetted outcome 1) Set UP districtlcommune steering committees 2) Set up District Recourse Centers to provide consultancy and monitoring the activities to support disabled children 3) Training courses for districtlcommune health workers about community based rehabilitation 4) Set up community support teams in communes Activities of the 1) District Steering Committees (12 persons) 2) Commune Steering Committees (6 persons) 3) Set UP community support teams which include a commune health care worker, a women" s union staff, a preschool teacher and a volunteer Indicators 1) Size of a community support teador a rehabilitation team 2) Select volunteers 3) Training: durationlcontent 4) Functiodduty 5) Activities 6) Financing 7) Communication with individuals or social bodies 8) Manage team activities 9) Sustainability of the team when the Project finishes Interviewee Community Support Team, preschool teachers, the commune health workers, a women' s union staff and volunteers Evaluation Goals *To describe the impact of the project at the agency, provincial, district and community levels; *To assess the appropriateness of the project objectives, implementation strategies and organizational structure; *To describe results and identify recommendations for continued project implementation. Evaluation Specific Objectives: (Note: The international evaluator has altered the wording of the following objectives to improve overall readability). "Determine appropriateness of basic course training materials and the impact of the training on in-service teachers; "Determine appropriateness and impact of the thematic training courses; "Determine effectiveness of the structure of the program and materials used for preservice teachers; *Determine appropriateness of teaching methodologies and skills of CSE staff; "Evaluate teaching skills and knowledge base of the teachers iniplementing inclusive education; *Determine benefit of the project on participants, especially children who experience disabilities; "Identify advantages and disadvantages of project implementation on the capacity of the local con~munity member's ability to carry out the project; *Evaluate public awareness campaigns; "Evaluate impact of the project on (a) children without disabilities and their families; and (b) the community; *Assess project sustainability; "Determine recommendations for future implementation, Methods The evaluation design is based on a collaborative, responsive and emerging philosophy. Evaluation team members included professors from national and local universities in Viet Nam; three medical doctors from pediatric hospitals and clinics; ministers of education and training (MOET) employees and one international consultant. Multiple methods and sources across multiple settings and participants were used in this evaluation. Methods included review of project records and materials, direct observation of inclusive classrooms; surveys/questionnaires, interviews and case studies, This section describes each of these methods. Review of Project Training Materials A comprehensive review of project training materials was undertaken by local members of the evaluation team. Evaluation members conducted on site observation of both inservice and preservice trainings. Specifically, evaluators met with and observed trainings at three target districts: Preschool Department of Quang Ninh, Provincial Training Teacher College; Preschool Department of Hoa Binh, Provincial Training Teacher College; Yen Iiung District Health Center of Ninh Binh Province; and Two Commune Health Stations in Yen Hung and two in Luong Son. Three reports, developed by the local evaluation team members are contained in the appendix of this evaluation report. These reports delineate: (a) description of the materials reviewed; (b) evaluation of the content and format of the training materials; and (c) recommendations and suggestions for future trainings and training materials. Approximately seventeen direct observations of children with disabilities in regular education classes srccurred across the three pravinces and tan cotnrnunes during the weeklong evaluation. Information gathered during each observation generally included: (a) description of clnssroom including seating and room arrangements; (b) description of the teaching observed (lecture, small group, hands on materials); (c) description of learners' experience (Are differences accommodated? Are students responding to what teachers want them to learn or creating and exploring their own ideas about what to learn?); (d) description of interaction (How are students relating to one another? How do teachers and other adults interact with students?); (e) specifics related to student(s) with disability (Does the focus students' activity, learning, grouping and interactions - match or not match what' s happening for the other students?); and (f) description of classroom lesson. A copy of the observation guide used by the evaluation teams is included in the appendix of this report. Three hundred and thirty two survey questionnaires were disseminated to (a) authority members; (b) commune lenders; (c) teachers; snd (d) parents of cl~ildren with and without disabilities across the three province and ten district communes. Surveylquestionnaire items were written by the project staff and local Vietnamese evaluation team members prior to the evaluation. Survey/Questionnaire of Commune Authorities. The following is a sample of survey items: (a) Do you want to support children with disabilities?; (b) Are the project steering committees at the commune level qualified?; (c) Describe the collaboration between the steering committee and the project officials; (d) Describe the challenges occurring in the project; and (e) Describe the successes of the project. Szrrvey/Questionnnire of the Teachers in the I~zclzisive Cfnssrooms. Fourteen items were included on this questionnaire. The following is a sample of the items contained on this survey: (a) What kind of inclusive training have you received'?; (b) tlow many children with and without disabilities are in your classroom?; (c) Describe the children' s disabilities; and (d) In what ways do you adapt your lessons to meet the needs of the children who experience disabilities? , , Stirvey/Qtiestionrzaire of the Parents of Children with Disabilities. Ten items comprised this questionnaire. The following is a sample of the items: (a) Describe the progress that your child has made since attending the inclusive classroom; (b) What are your hopeslexpectations for your child?; (c) Describe the contact that you have with the teacl~er(s); and (e) Describe the supports that your child receives. S~irvej~/Que,~tionnaire of Parents of Children willtotrt Disnbiliiies. Four items were on this questionnaire. The following is a sample of the items contained: (a) Do you think that children with disabilities should go to school?; and (b) What do you want your child to learn? Stlrve})/Qziestionnni~-e for Corwnurte Leaders. Five items appeared on this questionnaire. The following is a sample of the items: (a) Describe the educational program for preschool and prinlary school children with disabilities in your commune; (b) Describe the type of support that the conlmune provides to inclusive teachers; ancl (c) Describe the major challenges that your commune faces in implementing inclusive practices. Szirvey/Questionjzaire for Cornrnune Wonzeri ' s Union Members. Eight items made up this questionnaire. The following is a sample of the items: (a) Describe the difficulties that you have experienced in implementing inclusive practices; (b) Describe the contribution that the women' s union has made to this project and to the children with disabilities and their teachers; and (c) What types of training have you provided for women in your community? A copy of all the surveys/questionnaires used in this evaluation is included in the appendix of this report. Interviews Approximately 46 interviews were conducted across the three provinces. Twenty-eight parents of children who experienced disabilities, nine parents of children without disabilities, over 20 teachers, and nine head masters were inkrvicwed. Evaluation team members were responsible for home visits in each province and in most communes. Interviews were tape recorded with parent permission and general themes emerging from the interviews were summarized. Case studies Home visits were conducted in at least one child' s home in each commune. A full description of the child's home life including the identified strengths and needs were elicited from parents during the visit. In most cases a videotape of the home visit was conducted. Settings and Participants Each evaluation team visited one of three project sites. The Quang Ninh evaluation team members were Mrs. Binh (Management), Mr. Chung (Primary Education), Mrs. Mai (Preschool Education), and Dr. Dung (Health). The Ninh Bin11 evaluation team members were Mr. Khanh (Management), Mrs. Nga (Primary), Mrs. Quyen (Preschool Education) and Dr. Huong (Health). The Hoa Binh evaluation team members were Mrs. Yen (Management), Mr. Van (Primary Education), Mrs. Duc (Preschool Education) and Dr. Manh (Health). The international evaluator/evaluation coordinator (Dr. Susan Ryan) visited all three sites. All three provinces are in northeast Viet Nnm; they are dominated by the Red River Basin and the sea. The fertile northeast is described as the "cradle of Vietnamese civilization". This section provides a brief description of each province, the school, and the demographics, as provided to project evaluators by the commune steering committee chairperson. Quang Ninh Province The Yen Hung District with a population of 130,000 covers 32,909 acres. Eighteqn preschools in this district enroll 3,377 children between the ages of three and six. Twenty two primary schools enroll 17,054 children. According to the steering committee members, 879 children in the entire Yen Hung District experience disabilities. Of these, 439 children with disabilities attend regular education classrooms. Approximately 440 children who experience disabilities do not attend school. There were three communes in the Yen FTung District participated in the inclusion program and were visited by the evaluation team. Cor~znzinze #I: Yen Giang. Yen Giang has a population of 4,123 people. There is one preschool with 105 children and one primary school with 350 children enrolled. The commune steering committee reported 48 children with disabilities were in their commune. The evaluation team observed in two classrooms. The first was a preschool classroorn of 35 four and five year olds. The second was the 3B primary classroom of 42 cl~ildren. Con~nz~r~ze #2: Liei~Yi. Lien Vi has a population of 8,589 people. 'There is one preschool with 270 children. There is one primary school with 623 children. Commune steering committee members report that there are 64 children with disabilities. Of these 64 children, 53 of them attend school. Eleven children with disabilities in Lien Vi commune do not attend school. The evaluation team observed two classrooms. The primary 3A class of 40 children was observed, and a preschool class or live-year-olds was observed. C'onz~nrrne #3: Qzrnng Yen. Quang Yen has a population of 8,711 people. There is one preschool with 260 children and one primary school with 766 children. The steering committee reported that there are 66 children with disabilities, of which 61 attend school. The evaluation team observed two classrooms. The primary 1A class of 40 children was observed and the preschool class for four-year-olds was observed. Twelve interviews were conducted in the Quang Ninh province. Ten parents of children who experience disabilities, nine parents of non disabled children and three head masters were interviewed. There were five types of surveys/questionnaires disseminated in Quang Ning province. Thirty two authorities including the community steering committee members, 16 commune leaders, 110 teachers, 30 children who experience disabilities, 30 no11 disabled students, 30 parents of children with disabilities and 39 parents of nor1 disabled children completed questionnaires/surveys. Ninh Binh Provillce The population of Yen Khanh District is 137,386. The District has 21 preschools and 23 primary schools. The steering committee menhers report 1040 children experience disabilities in the entire Yen Khanh District. Of these, 790 children with disabilities attend regular education classrooms. Approximately 102 children who experience disabilities who do not attend school. Three communes in the Yen Khanh District that participated in the Inclusion Program and were visited by the evaluation team. Conzmuize #I: Khnh NItnc. Klianh Nhac has a population of 11,485 people across 1 1,115 acres. There is one preschool with 585 children and two primaly schools with 1575 children enrolled. The Khanh Nhac District steering committee reported that there were 60 children with disabilities in their district and 50 of these children attended school. Conzrmrlze #2 Klzanh PIZZL Khanh Phu has a population of 6,500 people across 553 acres. One preschool enrolls 345 children and one primary school enrolls 797 children. The steering committee reported that 48 of the 52 children with disabilities attend school. Conzrnzrtze #3: Khaizlz An. Khanh An has a population of 6,282 people across 720 acres. There is one preschool with 320 children and one primary school with 700 children. The steering committee of this commune reported that 66 of the 71 children with disabilities in the commune attend school. The evaluation team conducted 16 interviews in Yen Khanh district. Thirteen parents of children with disabilities and 3 head masters were interviewed. Hoa Binh Province The population of Luong son District is 76,170. There are 576 children with disabilities in this district. Four hundred ninety six children with disabilities attend school in regular education classrooms and 47 children do not attend school. There are 14 preschools and 22 primary schools in Luong Son District. Cornnzzrr~e #I: Hung Son. There are 51 children who experience disabilities in the Hung Son commune. The evaluation team observed two classes. Class 2A1 was observed. There were 40 children in this class; one child experiences a disability. Class SB was observed. There were 42 children in the class; two children experience disabilities. Corninme #2: Nhan Trach. Nhuan Trach has a population of 4,078 people. There is one preschool and one primary school in this commune. The steering committee reports 3 lchildren with disabilities attend regular education classrooms. The evaluation team conducted observations in class 4B. One child out of the 39 children in the class experienced a disability. Co?mttrr~e #3: Yen Binh. The Yen Binh commune has a population of 5,165. There are three schools, one preschool and two prilnary schools. The steering committee reported 31 children with disabilities with 27 of these children attending school in regular education classrooms. The evaluation team conducted an observation in class 5A. There were 42 children, and one child experiences a disability. Commme #: Hop MOIL The evaluation team conducted an observation in class 2A. Of 39 children in this class two experienced disabilities. Eight interviews took place with parents of children who experience disabilities and three head masters. Additional information on each province and commune can be found in Table 2. Table 2 Province Commune Demographics - Luong Son district District Yen Binh commune I Total uouulation 1 1 I 76.170 I 5.165 l Total area I 360.5km2 I No of pre-schools 14 101 No of pre-school students 3001 1 331 No of primary schools 22 / 02 No of primary school 9519 1 698 students Total number of CWD 576 (543 up to 32 1 0/2000) No of children went to 354 @re-school: 22 (pre-school: 0; school before project started 7; primary school: primary school: 347) 22) No of children go to school 466 (pre-school: 1 27 (pre-school: 3; - since project 3 1, primary primary school: school: 435) 1 24) - No of children mobilized in 466 (1 12: newly 1 27(5: newly 1999 mobilized) / mobilized) 460 (47: newly 27 No of children mobilized in mobilized) 2000 19 @re-school: 0; 17 @re-school: 0; primary school: primary school: 1T) 19) 3 1 @re-school: 3; 21 (pre-school: 2; primary school: primary school: 19) 28) 3 1 2 l(4: newly mobilized) 3 1 23 (2: newly mobilized) Yen Khanh district Total population - - Total area No of pre-schools No of pre-school students No of primary schools No of primary school students Total number of CWD No of children went to school before project started No of children go to school since project No of children mobilized in 1999 No of children mobilized in 2000 - District Khanh An Khanh Phu Khanh Nhac commune commune c~mmune 137.386 6.282 6.500 1 1.485 13,778,2 ha 7.200 km2 6.900 km2 12 hi2 2 1 0 1 0 1 0 1 6.643 320 345 5 85 23 01 0 1 02 17.517 700 797 1575 1040 7 1 52 60 352 (pre-school: 15 ( pre-school: 07 (pre- school: 1 1 @re-school: - - 32; primary 4; primary 2; primary 2; primary school: 320) school: 11) school: 5 ) school: 9 ) 550 66 48 50 Total population Total. area No of pre-schools No of pre-school students No of primary schools No of primary school students Total number of CWD No of children went to school before project started No of children go to school since project No of children mobilized No of children mobilized Yen Hung district District 879 primary school: 260 school: 68; primary school: 410) Lien Vi commune 8.589 3.075 ha 64 07 (pre- school: 02; primary school: 05) 62 (pre-school: 15: primary school: 47) Yen Giang Quang Yen commune 4.123 8.71 1 I 02 @re-school: 1 28 (pre- school: 01; primary 04; primary school: 01) school: 24) 17 (pre- 40 @re-school: school: 06; 05; primary primary school: 35) school: 11) 16 28 DATA COLLECTION Data was collected over one week from November 13-November 17, 2000. Five sources of data were used in this evaluation: (a) surveylquestionnaires, (b) horne visits, (c) group in-depth interviews, (d) individual interviews, and (e) evaluator field notes. The appendix includes the interviewlsurveys. Follow-up evaluation meetings were held when fbrther clarification or probing was required. The evaluators collected data through ongoing field notes, during observations, and throughout the period of data collection and analysis. Data collected across the three provinces rcsulted in approximately 17 hours of classroom observations, 10 hours of audio taped interviews, and over 500 pages of survey/questio~lnaires. DATA ANALYSIS Data Analysis Although data analysis was a continuous process, it was divided into three stages. The first stage focused on individual evaluation teams sumniarizing and analyzing surveys, interviews, observations and case studies conducted in their individually assigned provinces. The second stage of intensive analysis occurred when the three evaluation teams met with the evaluation coordinator and project staff. At this stage, all data were re-analyzed and findings were combined or collapsed into the following categories: (a) results of survey questionnaires; (b) results of inclusive classroom observations; (c) results of interviews with head masters, teachers, parents and others; and (d) home visits notes and interviews. During the final stage of data analysis, the evaluation coordinator reviewed and summarized major findings and themes. Multiple sources of data were used to maximize the validity and credibility of this study, including member checks, evaluator debrietings, use of evaluators' notes, and use of an insider perspectives (e.g., local evaluators) (Bogdan & Biklen, 1982; Lincoln & Cuba, 1985; Stainback & Stainback, 1989). RESULTS Overall Statement The Expansion of Community Support f~r Children with Disabilities (ECSCB) Project, funded by the USAID and implemented by the CRS and NIESICSE has acl~ievecl impressive results in a short time. Initiatives to improve educational services for children who experience disabilities in the three identified provinces it1 Viet Nntu hnvc the support of several collaborative partners. Community steering committees, teachersi and parents are demonstrating in specific ways how the lives of young children with disabilities are being improved. The following are a list of encouraging -and positive features of this project. Project Strengthens Improving the Lives of Children with Disabilities. Preschoolers and primary school children in the three provinces benefiting from this project are becoming full members of their classrooms and communities. In many cases, across the three provinces, young children in preschool and primary school classrooms are attending the schools and classrooms they would attend if they did not have disabilities. Children in classrooms observed by the evaluation teams experienced disabilities including hearing impairments, mental retardation, and physical disabilities. In all cases, the other students and classroom teacher demonstrated acceptance and support of the child experiencing a disability. For example, Tam in Ninh Binh comes to school each day with a group of his friends in the commune. He lives several kilometers from the school. His friends help him by pushing his wheelchair down the dirt road to the school. Without these friends, Tam would not be able to go to school. In another commune in Quang Ninh, Na, a six-year-old child who experiences a hearing impairment has begun to learn sign language and is able to express her needs. Her mother reports she has friends who often visit her at home, In Hoa Binh, the evaluation team was given a poem written by a young girl, Van, who experiences a severe physical disability. She expressed her dream of a day when she would have the chance of going to school. She "did not dare to express" this dream because her family lived very far @om school. School personnel and community members worked together to create a room at the school where she could live. Without the commune steering committee's leadership, and the support of the schoolteachers and her peers, Van would not be able to attend school. These are just examples of the many ways children with disabilities are supported by this project and the mobilization of the community steering committees. Community Awareness. Each province documented the specific ways in which public awareness of the issues of disability have been improved. Community members surveyed and interviewed indicated that caring for and supporting all children, including children with disabilities, is clearly the responsibility and desire of the community. One commune leader describes it as "humanitarian" and said his community practiced "charity for all"; he admired the value of inclusive education and felt these values were consistent with the Vietnamese way of life. It was impressive to see teachers' enthusiasm and visible caring for all children. Commune leaders are facilitating public awareness campaigns in the local papers and local radio stations. Most communes are working through their Women' s Unions and Farmers' Unions to raise funds for schooling children with disabilities. In Quang Ninh a videotape has been produced to describe the benefits of inclusive education. Development of Training Materials and Teacher Training. Local evaluation team members reviewed and evaluated the training modules and materials used by the project staff. Three reports by local evaluation team members can be found in the appendix to this report. These reports, written originally in Vietnamese have been translated into English. In addition, an extensive evaluation of a week long inservice training seminar was conducted by one of the local evaluators. This report can be obtained via Dr. Thuy, project director. The development of materials by local and international consultants is a worthwhile endeavor. Locally referenced and relevant training materials will build the capacity of Vietnamese teachers and community members. This project has endeavored to develop, implement and disseminate significant materials in the area of inclusive education. Improvement of the Quality of Teaching in the Schools. Participants in this project at both district and commune levels describe how training seminars and workshops have improved the general quality of the teaching in their schools. By participating in inclusive education training, community leaders and teachers describe that: (1) Interaction among teachers and their students have increased; (2) Students are exploring new ideas via cooperative learning; (3) Students and teachers attitudes and feelings about children with disabilities have changed; (4) The belief that all children should be valued members of a classroom has increased. One community leader in Hoa Binh shared that teachers in his schools have demonstrated new skills that inclusion was happening not just in " name only" . The project has been successful in beginning to bring about fbndamental reform. Inzastructure. The project has established a strong infjrastructure at the province, commune, and project levels. Community steering committees in all three provinces are active and involved. Community leaders eagerly embrace their " responsibility" and express their "desire" to support all children in their districts including those children with disabilities. In each district commune, leaders described specific ways in which they were actively supporting children with disabilities and their families. The types of support provided by commune committees have been: (a) Securing adaptive equipment including wheelchairs and walkers; (b) Providing tuition assistance so families can send their children to school; (c) Providing a room in the school for a young girl who lived too far from the school to walk; and (d) Organizing a circle of friends for a young boy who uses a wheelchair to get to and from school each day. Project staff should be commended for acknowledging and fostering this important and critical element of effecting change in communities and schools. Building strong infrastructures requires time, persistence, and focus. This project has built the infrastructure critical for demonstrating sustainable change. This infrastructure, a strength of the program, should continue. Age Appropriate Placement and Natural Proportions. In general, children with disabilities were in age appropriate classrooms. In other words, children were in classrooms with their age peers. Yet, in several classrooms visited, the child with a disability was significantly older than histher peers. If possible, children should be in classrooms they would be placed in if they did not experience disabilities. In most cases this placement would be within a two to three year range of age peers. In all cases, the ratio of children who experienced disabilities to children who were developing normally was in agreement with the concept of natural proportions. In other words, to the greatest extent possible the percent of children who experience disabilities in a given class should be no higher than the overall percentage of children who experience disabilities in the general population of the school or community. The schools visited in this project have complied with the principle of natural proportions. In fact, a strength of this project was adherence to the underlying principles of inclusive education in the placement of children into classrooms. Supporting Families. Several examples, shared by parents who were interviewed by the evaluation team, reflect the impact of the project on their lives and their child's. For example, after observing the preschool class in Lien Vi, Quong Ninh Province, the evaluation team met with a mother of a preschooler who experiences cerebral palsy. She shared that her son, Bon, wakes up very morning excited to go to school. She tells how he is now able to sit longer and pay attention to both his lessons and his peers. When asked about her expectations and dreams for her son, she shared that she wanted her son to learn to "read and write". She is going through a difficult time in her life, she has three other children, and says the benefit for her son from this project has made her life happier. An elder in one village is a grandfather of a primary school boy, Thang, who experiences a disability. This grandfather laughed with joy when he told the evaluation team his grandson comes home and "relives the lessons every day". He is proud of his community and the efforts they are making to improve the lives of children. He relates how medication given to his grandson when he was a boy caused a physical disability. He explained to team members how his grandson is now learning to better use his arms and hands. Collaboration. The project continues to demonstrate strong recognition of the need to develop and maintain collaborative partnerships. hovative programs, such as the ECSCD project, are faced with enormous challenges related to collaboration. Yet, ECSCD recognizes their project success depends on strong partnerships. The collaborative relationships which the project has developed include, but are not limited to: (a) NIES/CSE; (b) health care providers; (c) between and within CRS and other NGOs; (d) teacher training programs in Hanoi and within the province; and (d) between and within project participants. In fact, this current evaluation design is an example of the tremendous effort to collaborate across departments with The Ministry of Education and Training (MOET) and. NIES. The relationships developed as a result of this evaluation provide a foundation for future collaborative projects. Areas of Improvement Children With Disabilities Physical, Social and Academic Inclusion. Children who experience disabilities are "children fwst". Faced with the " task" of teaching and supporting children with disabilities citizens of many communities and districts across the world choose to isolate and provide only limited opportunities for children with disabilities to "visit" classrooms of their non-disabled peers. Fortunately, schools and communities involved in this project reflect the belief that children with disabilities can and should receive their education along side their peers. Activities of this project are reflective of best practice internationally in inclusive education. This project admirably defines inclusion as ensuring that a child with disabilities is supported in the regular education classroom and in the community where hdshe lives. Evaluators were impressed to see no cases of isolated special education classrooms. During the observation, evaluators observed preschool and primary school children physically and socially included in the classroom. Children were fidl members of the classroom. Often a student was seated in the fiont of the room, closer to the teacher; yet, near other peers. Evaluators observed the teacher encouraging interactions with peers, using the peers as supports for the child with disabilities, and providing individual verbal and physical guidance to the child with a disability. A high priority of this project is to move toward academic inclusion. Teachers will need to understand more fully how to move from the social and physical inclusion of students to how to ensure that children with disabilities are in fact learning something while they are in school. One specific way to achieve academic inclusion is to ensure that education is individzlally appropriate. It will be important for project leaders to identify ways in which the specific and individual needs of each child who experiences a disability will be fostered. For example, not all children are able to learn at the same rate, with the same materials, and with the same methods. Teachers will need to learn how to assess the needs and abilities of their students who experience disabilities. Next, teachers will need to learn specific methods and interventions for increasing the learning and abilities of each child. All children have individual needs; it is critical for teachers in this project to understand how to mekt diverse needs of students who learn differently and who experience disabilities. Across the three provinces in this project, evaluators were repeatedly informed by teachers that they were not fully confident in assessing the learning abilities and in supporting the learning of the students with disabilities. Furthermore, they felt unprepared to adapt instruction and methods to meet students' individual needs. Assessment and the Educational Program Evaluation teams observed that children with disabilities were mis- or under￾identified. In other words, commune leaders reported children by general disability category (e.g., severe, moderate, mild), yet, when asked to identify how these categories were derived they offered no consistent method used. All evaluators in general, and the physicians evaluating the health and medical aspects of this project specifically, articulated a serious concern over the misidentification and inconsistent/incorrect classification system used by local teachers and community leaders. When children are identified with a disability, the identification needs to be accurate and reliable. More importantly, the purpose of the assessment or diagnosis of a disability needs to be linked to improving the life of the child and the services Iie/she receives. An alternative method of identification, which would serve the child more productively would be the identification of the types and levels of supports needed. In fact, in the United States, the AAMR Ad Hoc Committee on Ternlinology and Classification recommended a radical departure from the traditional system of classification of children with mental retardation based on I. Q. scores, to a system that would classify children according to the level of support they need to function as competently as possible. The following depicts these levels of support. Intermittent: Supports on an "as needed basis". Limited: An intensity of supports characterized by consistency over time and time-limited but not of an intermittent nature, may require fewer staff members and less cost than more intense levels of support. Extensive: Supports characterized by regular involvement (e.g., daily) in at least some environments (such as work or home) and not time-limited. Pervasive: Supports characterized by their constancy, high intensity, provided across environments; potential life-sustaining nature. Pervasive supports typically involve more staff members and intrusiveness than do extensive or time-limited supports ( Hallahan & Kauffman, p. 124). As this project moves toward supporting academic and educational needs of children who experience disabilities, teachers must be trained in more fully identifying the types of adaptations and the levels of supports each child needs to develop competence. Other Constraints and Challenges Faced by the Project The following section delineates findings from the interviews, observations, questionnaires and home visits conducted during the course of this evaluation. Several of the findings go beyond the goals and objectives of this project; therefore, they are not the explicit responsibility of the inclusion project. Yet, the findings, relevant to the continued overall success and ultimate sustainability of the project, are included in this section. Teachers. Participants in this project shared the following concerns: I) Teaching is hard work. "Teaching normal children is difficult", teachers told the evaluators. They teach 40-60 children. One teacher shared "the burden becomes harder when we have children with disabilities", it "requires more energy", "more patience", and we "work longer hours". 2) Teachers ' Need for Additional Resources. Teachers interviewed described the lack of teaching materials and equipment. This lack of resources exists in the scl~ools in general,,and is a greater challenge to them as they struggle to implement inclusive education and cooperative learning methods without material resources. In every .meeting held with the community steering committee, the eval~~ators were told of this dilemma. The community was poor, and the teachers lack basic teaching aids and materials. 3) Teachers' Needs for Advanced Knowledge and Skills. Teaching in Viet Nam has only recently moved towards a less traditional model. Teachers are increasingly being trained in such promising practices as multiple intelligence, cooperative learning, creative problem solving, Blooms Taxonomy and other innovative teaching methods. Evaluators observed teachers struggling to implement innovative practices. In most cases this was being done without supervision and adequate mentoring. In terms of children with disabilities, teachers expressed their concern over their own lack of expertise and knowledge. For example, some teachers had only rudin~etitary signing skills, yet they were attempting to communicate with children who experienced deafness. In other situations, teachers were unfamiliar with how to adapt a traditional curriculum to meet the needs of a child with mental retardation or a specific leamind disability. - - Teachers reported that training they received on inclusive practices was general and introductory in nature. Many teachers expressed interest and the need to obtain fiwther training. Specifically they expressed the need for training to be less theoretical and more practical. 4) Teachers' Desires for Incentives. Teachers interviewed by the evaluation teams overwhelmingly expressed their fn~stration with the lack of incentives. Most teachers voiced a desire to receive an increase'in salary. They expressed their desire to continue teaching children with disabilities; however, they were taxed by th:: enormous effort required as well as lack of support, resources and incentives. 5) Teachers' Needsfor Supervision and Mentoring. Evaluation team members observed teachers struggling to implement cooperative learning and other innovative practices. Most teachers had received training in seminars and workshops regarding general strategies for facilitating inclusion and specifics related to building cooperative learning lessons. During observations it was evident teachers were implementing new strategies. Teachers would benefit from increased opportunity to receive coaching, supervision and mentoring as they explore new practices. I 6) Teachers' Need for Increased Knowledge Regarding the Unique Strengths and Needs of the Children with Disabilities In Their Classrooms. Teachers often had little knowledge related to the needs and abilities of the child who experiences disabilities in their classrooms. Even though they aren't expected to be specialists in every disability, they would benefit from increased awareness of basic needs of a child in his/her class. Teachers want to effectively facilitate greater participation and learning. Findings described regarding teachers are both related and tangential to this project. Two items of particular relevance to the project are: (1) the need for teachers to receive training they can directly apply in their classrooms. In other words, the teachers are providing feedback to the project staff that they need training that is less theoretical and more practical; and (2) the need for increase in knowledge related to how to teach the child with a disability in the typical classroom. This second finding speaks to the importance of supporting the teachers in this project during their teaching. This topic will be further described under technical assistance. Collaboration Policy Statement on Inclusion. Throughout the weeklong evaluation, issues related to the need for an increase in collaboration between MOET and the inclusion project surfaced. In fact, a review of documents of this project since 1995 revealed similar issues, Specifically, project participants at all level enthusiastically requested that MOET establish a policy on inclusive education. Colnmunity leaders recognized the importance of such a policy; teachers shared the concern for being asked to do more with less; and project staff and evaluators perceived that in order for inclusive education to be provided throughout Viet Nam, a national policy statement from MOET is essential. I Technical Assistance and Technical Support Teachers in the project informed evaluation team members of the need for special training and assistance in the classroom. For example, teachers who work with children who experience language and communication delays would benefit from suggestions by a communication specialist or a speech and language pathologist. There are preschool and primary school children who have significant physical disabilities who would benefit from assistive technology or adaptive equipment. Specialists in Viet Nam could serve as consultants tosthis project and provide that type of technical assistance to teachers. It will be important that technical assistants share the philosophy of building inclusive communities. Mobilizntion of Existing Resources. Several commune leaders expressed the need for greater coordination between health and education. In addition, projects with similar goals, objectives and activities (e.g., other NGOs) could share resources and personnel, Existing resources could be enhanced resulting in stronger services for all children Community Economics I The communities participating in this project all reported being poor economically. Some commune leaders described their communities as econon~ically depressed. They shared that in some cases the inclusion project tapped an already desperately poor community. When community members left the field to do work associated with the project, they explained it resulted in less time in the field and less farming income. Implications and Recommendations 1. Provide Support, Incentives, and Training to Teachers Teachers hold the keys for learning for all children. It is a tribute to the dedicated teachers in this project that they have strong attitudes and willingness to open the doors of their classrooms and welcome all children. In every commune visited, committed teachers overwhelming told evaluators about their concerns and needs. The continued success of this project will depend largely on how the concerns of the teachers are addressed. Some suggestions for addressing these concerns are listed below: a) histen and acknowledge. At a basic level, it will be important for project staff to let the teachers know that they are appreciated and their concerns have been heard. - - -Recognizing and validating the issues they raised during this evaluation is a necessary step. b) Aslc teachers to problem solve solutions. Meet with teachers in groups or individually to address their concerns. Collaborate with them to develop solutions that are workable and reasonable. Praise their contributions to the collaborative effort. c) Enhance the Level of Inservice Training. The project has provided inservice training to teachers resulting in improved teaching abilities and introducing them to concepts such as cooperative learning and community building. Currently teachers are voicing the need for training to advance their teaching in (1) cooperative learning; and (2) meeting educational needs of students who experience disabilities. Teachers have requested training that they can immediately and directly integrate into their classrooms. Future training might achieve this outcome by focusing on case studies; facilitating leaders can draw on examples of the actual children in this project and the real life scenarios teachers' face. d) Provide technical assistance. It is not sufficient ,to provide seminars and workshops to improve teachers' skills and knowledge. The teachers in this project voiced the need for specific recommendations and suggestions on how to work effectively with the child(ren) in their classrooms. They complained that some training in seminars they attended was too general, with little application to real life situations. Although Viet Nam may not have a surplus of physical and occupational therapists, speech and language pathologists or behavior specialists there is a tremendous wealth of expertise. In fact, several team evaluators have expertise in curriculum, assessment, and behavioral supports for children with disabilities. It will be important for the project to mobilize existing expertise and to provide teachers with information and technical assistance on an itinerant basis. e) Consider creative use of project funds. Fund outstanding teachers to share " promising practices" with other teachers at local and national conferences (have project staff and national consultants co-present with teachers). f) Review all reports conducted by the local evaluators related to project training nzaterials and insewice trainings. Prior to the November evaluation members of the evaluation team were enlisted to review all project training materials. The appendix contains copies of the findings of these reviews. In addition, Dr. Thuy, Project Director hired a local physician to observe and evaluate a week long training on inclusive education. Project staff and CRS staff would be well advised to review these extensive reports. Each report contains a description of the training materials (and the actual training observed), strengths of the materials, and suggestions for f~~rther training material development. 2. Establish a Reliable and Accurate Data System of Identit'ication and Monitoring Evaluators on this project make the following recommendations based on the observation that children who experience disabilities in this project were (1) not accurately identified; and (2) not correctly assessed. 1) conduct child find and local screening activities to identify the children in the communes who experience disabilities who are not attending school; 2) develop a home based early intervention program to provide support to families who have infants and toddlers who experience disabilities; 3) correctly assess children's areas of strengths and offer information to assist teachers in improving educational outcomes for students who experience disabilities; 4) accurately maintain a count of children who experience disabilities and the gains that are made as a result of this project. I 3. Collaborate With Other Agencies to Provide Tech~iical Assistatlce to Teachers. All communities evaluated indicated the need for resources. Specifically teachers and steering committee members indicated the need for adaptive equipment such as wheelchairs, hearing aides, and supportive chairs. The grant proposal indicates that specific materials would not be funded. Instead, local steering committees and communes would be required to mobilize funds to secure the needed materials. This has occurred on a limited basis in most schools. It would appear that the three provinces and the specific schools, community steering committees, head masters, and teachers would benefit from a greater understanding of how to make that happen. In the interim, children with disabilities, their parents and teachers, are suffering. Develop a Technical Assistance/lzterdisciplina~ Team There is a need for a technical assistance branch to this project. In c~khorution with other NGO's, and MOET, this project could initiate a proposal to develop an Interdisciplinary Team that would provide technical assistance to local teachers and communities on the needs of children with disabilities. Within the existing sewice delivery system there are specialists in hearing, vision, communication and behavior. Specialists from these areas could serve as itinerant specialists and provide hands on consultation and technical assistance to teachers. The development of a technical assistance/interdisciplinary team would strengthen the current inclusion project and ensure that the theory and principles of inclusion and services for children with disabilities was in fact occurring. Recommendations For Sustainability of Inclusive Education in Viet Nam Improve Curriculum of All Children As the project seeks to support children in regular education classrooms who experience disabilities, it would be wise to support the improvement of educational services for all children. The national curriculum and standardized texts do not fully provide for the educational needs of all students. In particular, the evaluation team did not see evidence of a readinglliteracy program in effect in the provinces. Research demonstrates the importance of a school reading program. A serious review of the current curriculum for all children should be undertaken. What are the goals of education for all children in the provinces? In what ways are the teaching materids, teaching activities, and the learner outcomes matched to these goals? In a few of the classes observed by the evaluation team, all students were required to learn the same material at the same time, in the same way. In fact, in several lessons observed students were required to simply respond in a rote fashion to teachers' questions. Project participants in a few situations did, however, demonstrate understanding of the concepts of respect for individual differences in learning styles as well as the importance of nurturing creative problem solving and critical thinking skills. There is emerging evidence that the concept of fostering reflective thinking, abstract concepts, and analysis and synthesis of knowledge is being encouraged by the project teachers. Teachers in the project would benefit fiom continued practice and mentoring to master this new teaching approach. During one observation, an evaluator demonstrated this teaching approach to the classroom teacher. In the fbture, project staff could offer their expertise in MOETfNIES and the national/local university training. More importantly teachers could be encouraged to team teach, or form teacher groups in an effort to share positive examples of their own teaching. The concept of teachers helping teachers is a powerfbl tool to foster growth, confidence, and a sense of belonging among teachers. Continue to Enhance the Current Project and Develop Plans for Expansion of this Project Across Viet Nam The project has made significant and impressive results. All participants speak of the benefit of this project to children with and without disabilities, to parents of children with a. without disabilities, to teachers, and to the community. The evaluation documents the many strengths of this project. The dedicated teachers, parents, children and authorities associated with this project should know countries worldwide are struggling with siniilar issues and concerns. Yet, few programs across countries begin with the premise that all children attend their home schools. It is the strong and clear vision of this program that guides inclusive practices in Viet Nam. Ovenvhel~ningly evaluators heard that participants and others associated with this project strongly recommended that it continue. In fact, a dcsirc to expand tllc pro.ject was voiced by project staff. Resolving some of the issues itlentiliecl in this evaluation will ensure that the project is "positiot~ed" eff'ectivcly, strategically, politically, and pragmatically to succeed tliroughout Viet Nani.