Education Development Center, Inc. dot-EDU Honduras Interactive Radio Instruction for Kindergarten Project Summative Evaluation Final Report Submitted by Education Development Center, Inc Contributing Authors: Yvette Sanchez, Project Director EDC Judith Evan, Consultant ECD Pathways July 1 2005 Education Development Center, Inc. Table of Contents Background 3 Design of Summative Evaluation 5 Results 9 ECERS 9 Child Development 11 Parent Interview 15 Teacher/Educator Interview 23 Community Interview 37 Conclusions 41 July 2 2005 Education Development Center, Inc. In 2004 the Juego y Aprendo project was implemented in Honduras in three geographic areas. A total of 53 centers were created for the purpose of implementing a preschool program through the Interactive Radio Instruction (IRI) approach. Since Juego y Aprendo is to serve as an alternative learning system for the country of Honduras as it attempts to meet its own educational goals of increasing access to preschool education, it was determined that a summative evaluation be undertaken. The outcomes of the evaluation were seen as important in order for the government and other local and international NGO’s to decide how best to expand the project and take it to scale. The purpose of this report is to provide a summary of the evaluation undertaken to assess the impact of the project on children, families, teachers and the community. Included in the report is a description of the sample, the instruments used and their purpose, as well as a description of the methodology and timelines for data collection. Baseline data were collected in April, 2004, and the comparative data collected in November 2004. The results obtained on each of the instruments is summarized and conclusions drawn about the short-term impact of the Juego y Aprendo project. It is important to note that the data collection points spanned a 7-month period. The results of the evaluation show gains for children and changes in the knowledge and skills of adults who support children’s development (parents and teachers) that suggest that over time there will continue to be positive outcomes for children from the Juego y Aprendo approach. BACKGROUND As part of its overall poverty reduction strategy, the Government of Honduras is seeking to accelerate progress toward meeting its Education for All (EFA) goals. To date, Honduras has yielded positive results in certain areas of EFA. Growth rates in school enrollment, reduced illiteracy rates and an increase in the average schooling a Honduran child completes are indicators of progress. But Honduras is still behind in achieving its main goal: the universal completion of primary education by 2015. Therefore, the Education Development Center, Inc. (EDC) and the Honduran Fundación para la Educación Ricardo Ernesto Maduro Andreu (FEREMA), have focused its support, with funding from the United States Agency for International Development (USAID), on the development of an alternative learning system of preschool kindergarten education that will provide the cornerstone for a national program that eventually will reach children not served by the current system – estimated to be 125,000 annually – as well as children already enrolled in the existing preschool system. The project represents more than a pilot project to develop new curriculum: it is a significant investment toward the creation of a project that can be brought to scale as quickly as public and private commitments permit. EDC has provided the overall mechanism and the technical assistance to accomplish the set of goals described above. FEREMA has provided the local resources and personnel to see that in-country activities are achieved. Both institutions worked with the Honduran Ministry of Education to ensure that the design of the package and follow-up activities were in line with the July 3 2005 Education Development Center, Inc. national curriculum for preschool education. The project titled Juego y Aprendo is an interactive learning package providing 108 forty-five minute preschool kindergarten lessons. The multi-channel learning system includes print materials for students, a guide for teacher/educator-led activities, posters, games, parent support education materials, and on-the-job teacher/educator professional development materials related to each lesson. The IRI project includes teaching tips and explanations for the teacher/educator development that is expanded upon throughout the print materials. Consequently, the multi-channel learning system addresses both student learning and teacher/educator development objectives. Through the Juego y Aprendo project centers were purposely established so that children and the community educator would have a place to meet. Depending on the resources of the community, preschool centers were established in a variety of places such as a formal classroom in the local school building, local community center or church and even at the home of the community educator, with lessons taking place in the open air. The Juego y Aprendo project was designed to serve a “dual audience” of children four to five years old and community educators from the community with no formal training and a minimum of a sixth grade education. The intent of the project, along with training and materials support, was to provide educators with information on how environments impact children’s learning and behavior. The methodology was purposely designed so that teachers would use the IRI Juego y Aprendo audio program only three days per week and as the year progressed and their skills increased it was anticipated that teachers would be able to support children’s learning through teacher-planned activities. A calendar was created showing teachers on what date the programs were to be used. The calendar took into account holidays and other closures The IRI audio programs included information for the community educator on how to set up a “learning center” in the classroom and during the audio programs the community educators were provided with specific direct messages telling them “why” a certain activity was being performed, ‘how” the activity would be realized, “what” skills and information children were learning, as well as examples of words and phrases for encouraging children, and “when” they should realize the activity, as well as think of new activities that will support children in developing certain skills and abilities. Through these direct messages community educators were able to revise the center environment using local materials and resources so that it would become a conducive learning environment for the children and the educators. The community educators received a one week training before the center opened, which included basic child development, some strategies for working with children and engaging parents and on how to use the CD player and how to use the program Juego y Aprendo. Community educators were also provided with quarterly trainings throughout the project year, each about a week long, that focused not only on the audio programs content but also provided additional information on children’s behavior and temperament, classroom management, the use of recyclable materials for creative July 4 2005 Education Development Center, Inc. activities, problem solving and parent involvement. The trainings coincided with the educators receiving the next set of CD’s. As part of the project, a summative evaluation was undertaken. The evaluation was conducted in order to establish if by using an alternative method of education there was some change in the knowledge that community educators, parents and community had about appropriate activities for young children, and if the children themselves benefited from the project. However, due to the short duration of the project (12 months), which included development and implementation of the intervention and the evaluation, the timeline for the full project was too short to be able to identify distinct differences in developmental outcomes for the children participating in the Juego y Aprendo project. Nonetheless, as will be seen in the discussion that follows, there were some interesting outcomes. DESIGN OF SUMMATIVE EVALUATION In order to develop, administer and analyze the results in an effective way, EDC and FEREMA agreed to hire an evaluation expert whose responsibilities included: • Serve as an advisor to guide the development, administration and analysis of the summative evaluation for the dot-EDU Honduras activity. • Work with the EDC education specialist to develop the actual summative evaluation design and the accompanying instruments to be administered locally. • Work with the EDC education specialist and the local evaluation team to analyze the data and interpret it, including making reference to international research on the relationship of particular activities/behaviors/environments that lead to learning gains or other improvements in social profile of participating children. • Summarize and/or analyze the findings of the local evaluation team and their significance. EDC and FEREMA agreed to subcontract with Christian Children International Honduras (CCF-H), a non-governmental organization that has experience in Early Childhood Education and evaluation, to conduct face-to-face interviews and gather information on the communities included in the project. One other advantage is that the evaluators from CCF-H were familiar with the communities in which the evaluation took place. EDC provided CCF-H with assistance in organizing and scheduling observation visits. CCF￾H’s duties included: • To complete the evaluation instruments • To collect the data during the baseline and comparative periods • To tabulate the data collected and deliver it to EDC for the analysis and use in reports. Once data were collected, CCF-H was responsible for tabulating the results of the Child Development Scale which they created. They provided EDC and FEREMA with the originals of all information collected, as well as a summary that provides a description of July 5 2005 Education Development Center, Inc. the actual process for implementation of the tools. EDC was responsible for tabulating the remainder of the data collected, as well as writing the report on the final analysis. The sample The control and experimental samples were chosen from three communities where the IRI was piloted and the initial 53 centers established. The communities were in three geographic areas: Comayagua, Francisco Morazan, and La Paz. The selection criteria for the centers were developed by EDC and FEREMA and included: 1. Accessibility 2. Distance from main office (Tegucigalpa) 3. Geographic characteristics – Urban vs. Rural 4. No existing formal preschool centers in the community (i.e., no Community Centers for Preschool Education (Centro Comunitario de Educación Prebásica –CCEPREBS) 5. Children who have never received any type of formal education At the present time preschool education in Honduras is formal and often only available to those who can afford it whether private or state sponsored. As noted above, within the Juego and Aprendo project, preschool centers were created in the community to reach a population not currently being served. Those who attended the Juego y Aprendo program constituted the experimental groups and the teachers received training specific to the project. The control group consisted of children attending a preschool already in existence in the community where neither the teachers nor the children were exposed to the IRI program. Thus the centers presented in the control group were only similar to the experimental group in terms of geographic location. Table 1 provides a summary of the sample, by community, experimental and control groups, and the urban/rural dimension. It also provides the number of centers, teachers, children, parents, and community members involved in the summative evaluation process, as well as the instruments completed with each group. Interviews with the parents, teachers, and community members were only performed with the experimental group, while the ECERS and child development tool was completed in both the control and experimental groups. Table 1 Summary of the Sample, Pre and Post, by Community, Experimental and Control Groups July 6 2005 Education Development Center, Inc. Comayagua RURAL URBAN CONT PRE CONT POST EXPER PRE EXPER POST CONT PRE CONT POST EXPER PRE EXPER POST TOOL CENTERS 2 2 3 3 1 1 3 3 ECERS CHILDREN 20 14 20 20 20 10 27 27 CCF-H CHILD DEVEL. PARENTS 10 12 13 12 INTERVIEW TEACHERS 3 3 3 3 INTERVIEW COMMUNITY 4 3 7 0 FOCUS GROUP Francisco Morazan RURAL URBAN CONT PRE CONT POST EXPER PRE EXPER POST CONT PRE CONT POST EXPER PRE EXPER POST TOOL CENTERS 1 1 1 1 1 1 2 ECERS CHILDREN 10 7 9 9 10 10 22 CCF-H CHILD DEVEL. PARENTS 8 6 10 INTERVIEW TEACHERS 1 1 2 INTERVIEW COMMUNITY 4 1 4 FOCUS GROUP La Paz RURAL URBAN CONT PRE CONT POST EXPER PRE EXPER POST CONT PRE CONT POST EXPER PRE EXPER POST TOOL CENTERS 4 2 20 12 ECERS CHILDREN 20 15 102 71 CCF-H CHILD DEVEL. PARENTS 51 36 INTERVIE W TEACHERS 9 12 INTERVIE W COMMUNITY 4 8 FOCUS GROUP As can be seen in Table 1, Comayagua and Francisco Morazan had both an urban and a rural population. La Paz only had a rural population. Thus there were 5 distinct settings. July 7 2005 Education Development Center, Inc. Of the three communities Comayagua is the closest in proximity to the capital Tegucigalpa, followed by Francisco Morazan, and finally, La Paz which is the farthest and has no area that can be considered even somewhat urban. Instruments The instruments were adapted or created by EDC, who also provided training on the use of the instruments to the fifteen evaluators chosen by CCF-Honduras (CCF-H). The instrument selected and adapted to assess the learning environment, the ECERS (Early Childhood Environmental Rating Scale), rated centers on dimensions such as space and furnishings, the child’s personal care routine, the use of language and reasoning, the kinds of activities undertaken during the day, interactions (teacher-child and child–to-child) and the program structure. A set of interview instruments measured teacher, parent, and community attitudes towards education and their level of participation in the centers. Data were also collected on the family (the economic status and education level of adults in the home, for example) and the physical characteristics of the home. The evaluation also included a measure of each child’s development. The Child Development instrument created by Christian Children’s International (CCF) was chosen in anticipation that it was related to outcome measures associated with the basic requirements outlined in the Honduran National Curriculum: children graduating from the Community Centers for Preschool Education (Centro Comunitario de Educación Prebásica–CCEPREBS) will have acquired the skills to successfully complete the first grade. Table 2 outlines the tools used during the initial and comparative evaluation stages, what they measure, and how they were implemented. Table 2 The Instruments Used in the IRI Impact Study INSTRUMENT OUTCOME APPLICATION TIME STRATEGIES ECERS – Early Childhood Environmental Rating Scale Space & Furnishings Personal Care Routines Language – Reasoning Activities Interaction Program Structure Direct observation of environment by evaluator. Questions asked for clarification only Three (3) hours per center Outcomes and scoring system adapted to be relevant to communities and project Child Development Scale Large Motor Fine Motor Language/ Communication Activities performed with each individual child Two (2) hours per child July 8 2005 Education Development Center, Inc. Social/Emotional Cognitive INTERVIEWS PARENT TEACHER COMMUNITY Education level Previous education experience of child Family size Previous information about IRI Attitudes towards children’s learning One–on–one interviews with parent or guardian Focus group with community members One (1) hour each Interview questions were designed by EDC and FEREMA staff according to goals HOME OBSERVATION PARENT Economic level of family Direct observation by evaluator of physical home. This took place during interview 10 minutes The observation instrument was used to gain a general idea of the economic status of the family and community in general without having to ask the parent directly what their annual income may be. Each of the tools was used for the pre-test completed in April 2004 and for the post-test completed in November 2004. All attempts were made to evaluate the same groups and individuals during the initial and later data collection in both the experimental and control groups, in the rural and urban settings. However, for a variety of reasons, ranging from families moving to different communities, children not being of age when the evaluation was completed, parents not sending child to the preschool center or simply as a result of desertion, there was a significant reduction in the number of children evaluated and the number of parents interviewed during the post-test. (A copy of each evaluation instrument can be found in Annex I). RESULTS The results of the pre-tests provide the baseline information for the evaluation of the project and were used to compare with the results of the post-test to determine any impact that the interventions have had. This report includes data on all experimental and control groups on the ECERS and the Child Development Scale. The data resulting from the parent, teacher and community interviews are reported on for the urban and rural experimental groups only. The following is a summary of the data collected, with a presentation of comparative results by experimental and control groups in urban and rural settings. EARLY CHILDHOOD ENVIRONMENTAL RATING SCALE – ECERS Using the ECERS, each center was evaluated on six categories: Space/Furnishings, Personal Care Routines, Language/Reasoning, Activities, Interactions, and Program Structure. Within each category there were between one to seven indicators under which a center could receive a score between one and five, with one being “inadequate”, three being “minimal”, and five being “good”. Evaluators observed each center over a three hour period beginning with level one (1). If a center successfully completed all items July 9 2005 Education Development Center, Inc. under level one the evaluator would go on to level three (3). If the center successfully completed all items under level three, the evaluator would go on to level five. Only if the center successfully completed all the items described under levels one (1), three (3) and five (5) could they receive a score of five (5). If a center completed all the items under level one but only a few items under level three, then the center would receive a score of two (2). If the center completed all the items under level three and some of the items under level five, then the center would receive a score of four (4). The total number of centers included in the post-project study is slightly lower than the original number of centers in the sample either due to the fact that the center was temporarily closed or the evaluators were not able to get to the center due to lack of transportation and distance. (All data collected by rural and urban areas can be found in Annex II). What did the results show? On the pre-test the urban centers (control and experimental) scored higher than the rural centers. This was true across all the items being rated. On average during the pre-test the urban centers scored a three (3) indicating the environments had minimal equipment that was in good condition and child centered activities. Urban centers in the community of Francisco Morazan scored the highest. The rural community of La Paz, being the farthest from the capital Tegucigalpa, scored the lowest in regards to environment. In the rural area, the combined score for the experimental group was somewhat higher than the score for the control group, but in the urban area schools the pre-test scores were essentially the same for the control and experimental groups (See Table 3). Table 3 Scores on the ECERS Rural and Urban, Control and Experimental July 10 2005 Education Development Center, Inc. 1.73 2.59 2.04 3.07 3.28 3.58 3.25 2.51 0 0.5 1 1.5 2 2.5 3 3.5 4 Control Rural Pre Control Rural Post Experiem tnal Rural Pre Experiem tnal Rural Post Control Urban Pre Control Urban Post Experim ental Urban Pre Experim ental Urban Post The post test shows some surprising results. In the urban areas, the overall pre and post scores for the control schools are virtually the same, and this would be expected. However there was a large increase in scores for both experimental and control schools in the rural areas. While there is quite a significant decrease in the scores in the experimental urban schools from pre- to post-testing, this is likely the result of the post-test sample. For the pre-test, experimental urban scores were made up of schools from Comayagua and Francisco Morazan. For the post-test, however, only the schools in Comayagua were included in the sample, and they scored essentially the same on both the pre- and post-test. Since on the pre-test the Comayagua scores were considerably lower than the scores for Francisco Morazan, it is not surprising that the overall experimental urban school scores would be relatively low on the post-test. In essence, because Francisco Morazan schools were not in the post sample, the data do not allow us to say anything about the impact of the project on the ECERS score for the urban experimental schools. Table 4 Scores on the ECERS By Geographic Area ECERS Post-Test by Geographic area LaPaz Comayagua Francisco Morazan July 11 2005 Education Development Center, Inc. Rural - Control 2.29 2.75 2.88 Experimental 3.14 2.82 3.25 Urban - Control 2.83 4.33 Experimental No Sample 2.51 No DATA While it would be anticipated that the ECERS score would increase in the experimental schools across time, it was not expected that the control schools in the rural areas would show an increase over time. Discussions among the participants in the Juego y Aprendo project and the researchers did not yield any insights into why this might be true. One explanation, however, might be that there was informal contact between the teachers in the experimental and control groups, which resulted in the control school teachers picking up some of the ideas related to classroom environment from the experimental teachers. This would need to be explored. CHILD DEVELOPMENT TOOL The Child Development Scale designed and produced by Christian Children’s International Honduras (CCF-H) was used to measure the development of each child participating in both the control and experimental groups. The evaluators were quite familiar with the tool since it is an instrument that they use often with the children participating in the preschool centers managed by CCF-H. CCF-H staff was responsible for tabulating the data collected and providing summaries of the data related to child outcomes. The Child Development Scale is designed to assess children’s development in a number of areas (large motor skills, fine motor skills, language/communication, social/emotional and cognitive). According to children’s performance on a variety of tasks their development can be rated as Advanced, Developed (age appropriate), Needs Attention, or High Risk. What is interesting from the data gathered is that the pre- and post-test findings were consistent across all the groups (rural control and experimental and the urban control and experimental groups). In essence there was an overall improvement in terms of development. This is most notable in the upward shift from the ‘Needs Attention” category to ‘Developed” for all groups, regardless of location or whether or not they were a part of the project. (See Table 5). Table 5 Child Development Results for children in the experimental rural and urban settings involved in the Juego y Aprendo Project July 12 2005 Education Development Center, Inc. Large Motor Skills Fine Motor Skills Language/ Communications Social/ Emotional Cognitive 0 10 20 30 40 50 60 70 80 Child Development Tool Experimental Rural Pre-Test High Risk Needs Attention Developed Advanced Large Motor Skills Fine Motor Skills Language/ Communications Social/ Emotional Cognitive 0 10 20 30 40 50 60 70 80 90 100 Child Development Tool Experimental Rural Post-Test High Risk Needs Attention Developed Advanced July 13 2005 Education Development Center, Inc. Large Motor Skills Fine Motor Skills Language/ Communications Social/ Emotional Cognitive 0 10 20 30 40 50 60 70 80 Child Development Tool Experimental Urban Pre-Test High Risk Needs Attention Developed Advanced Large Motor Skills Fine Motor Skills Language/ Communications Social/ Emotional Cognitive 0 10 20 30 40 50 60 70 80 90 100 Child Development Tool Experimental Urban PostTest High Risk Needs Attention Developed Advanced When looking at the subgroups within the overall developmental scores, the greatest shift from “Needs Attention” to “Developed” is in the area of Large and Fine Motor skills with children making the least progress in Language/Communication skills. (See Annex III for the complete tables). July 14 2005 Education Development Center, Inc. What does this mean? It would suggest that at the end of one school year of participating in the project, more than half of the children were ‘on-track’ developmentally, with rural children making more progress than their urban peers, contrary to what might be predicted. The fact that scores for children in the experimental and control groups were not significantly different is interesting and raises the question of what it means to be ‘trained’ as a preschool teacher considering that the teachers/educators in the existing preschool programs have received more training and have more experience than the teachers/educators volunteering in the Juego y Aprendo program. Based on the amount of training that teachers in the control group had, it could be expected that the control group children would do better than the experimental group children. Table 6 Percentage of Respondents in Categories of Development, By Urban/Rural and Control/Experimental Groups High Risk Needs Attention Developed Advanced Pre Post Pre Post Pre Post Pre Post Control Urban 64 0 86 0 13 99 0 0 Experimental Urban 2.0 0 78.39 8.1 19 92 41 0 Control Rural 4.4 0 68 14 24 92 4 7.2 Experimental Rural 1.3 0 65 5.4 34 90 0 4.8 To determine if there are longer-term gains for children participating in preschool programs, FEREMA has begun the process of designing and implementing an evaluation that will track a sample of children who attended the pilot centers through the second grade. This may provide additional insights into the impact of the IRI program on children’s development and achievement. PARENT INTERVIEW – PRE AND POST The following information pertains to all points of the interview process for parents/families. The full questionnaire and tables of results for both the baseline and comparative data can be found in Annex IV. The parent questionnaire for the pre-test was implemented to get an idea of the educational experience of parents, the economic and family situation in the home, the attitudes of parents towards education and types of materials and activities available in the home to support the child’s education. The parent questionnaire for the post-test was slightly modified to be able to ascertain the experience of the child and parent throughout the seven months of the child’s participation in the Juego y Aprendo project, especially to measure the parent’s July 15 2005 Education Development Center, Inc. participation in the education of their child either through direct participation in the center or contact with the educator, to get a sense of the parent’s opinion of the Juego y Aprendo project, whether or not the parent felt the project had an impact on the child’s learning, and finally to gauge whether or not there was a change in the parent’s attitude toward education. The following is a summary of the data collected. A. Experimental Urban – Pre And Post During the pre-test a total of 23 parents/families were interviewed in the two communities of Comayagua and Francisco Morazan. During the post-test, twelve families were interviewed in Comayagua only so that comparative data reflects this particular community. Family characteristics In terms of basic demographics, all 23 families (100%) reported that Spanish is the primary language spoken at home. One family reported that they also spoke an indigenous as well as a second foreign language in the home (French). Thirteen percent said they had attended some high school. Forty-three percent of parents said that they had completed the sixth grade. Thirteen percent of parents indicated that they had completed the third grade. Thirty percent said they have not had any formal education. One of the things that is a good predictor of children’s success in school in the USA is the availability of reading materials at home. In order to know to what extent children had access to reading material, parents were asked whether or not there were things to read in the home. On the pre-test 57% said that they had some kind of reading material available in the home on a daily basis, while 43% said that they had no reading material available in the home. In comparison, on the post-test 42% of families said that they had some reading material in the home. (The drop in number of families with reading materials in the home may very well have been affected by the change in total number of families interviewed.) On the post test, of the 57% who responded that there were reading materials in the home, 59% said that they had books available, 30% said that they had newspapers available, and 11% indicated that they had other types of reading material available. Family composition Family composition was another area explored during data collection. Thirty percent of the families reported that two – four individuals live in the home. Forty-eight percent reported that five to eight individuals lived in the home, and 22% reported that between nine to twelve individuals live in the home. In terms of how many of those in the home are children, three parents (13%) reported having one child in the family. Thirteen families (57%) reported having between two and four children. Of the remaining seven parents/families, six reported having between five July 16 2005 Education Development Center, Inc. and seven children in the household (26%) and one parent reported having more than seven children (4% approximately). The maximum number of children reported was eight. Information on the children On the pre-test, the total number of children reported in all 23 households interviewed was 84. Of the children participating in the Interactive Radio Instruction program 25% were first born; 22% were second born; 26% were third born; and the remaining 26% were fourth born and younger. Parents were asked questions related to their child’s health status. At no time did a parent report that she/he would describe the child’s health as “poor”. In fact, on the pre-test, 17% described their children’s health as “excellent”. During both the pre (52%) and posts (58%) tests, the majority of parents described the child’s health as “good”. The comparative data revealed that no families described their child’s health as “excellent” (See Table 7). One hundred percent of families reported that their children were up to date on their immunizations. Table 7 Parent’s Report on Children’s Health Status Health of child (%) Pre-Test Post-Test a. poor 0% 0% b. average 30% 17% c. good 52% 58% d. excellent 17% 0% In order to get some understanding of the child’s nutritional status, parents were asked how many meals the child ate yesterday. During the pre-test one parent interviewed reported that their child had only had one meal the day before. One parent reported that their child had eaten two meals the day before. Twenty-one parents (92 %) reported that their children had eaten at least three meals the day before. At the end of the year, all 12 families in Comayagua reported that their child ate three meals the day before. Another indicator of health is a child’s activity level. Parents were asked to rate their child in terms of activity. At no time did families report their child as ‘inactive’. Baseline data revealed that two parents (8%) described their children’s activity level as somewhat active and fifteen parents (approximately 65 %) described their children as ‘active’. Only seven parents (30%) described their child as being very active. The comparative data showed that 50% of parents thought their child was “active” and 30% described their child’s activity level as “very active” (See Table 8). July 17 2005 Education Development Center, Inc. Table 8 Parent’s Rating of Children’s Activity Level Child’s Activity Level (%) Pre-Test Post-Test a. inactive 0% 0% b. somewhat active 5% 20% c. active 65% 50% d. very active 30% 30% Children and preschool In terms of children’s previous participation in a preschool program, approximately two thirds of the children in the experimental group had never participated in a preschool program, while about a third of the children had previously participated in a preschool program. Parents and the preschool In terms of the child’s current participation, parents were asked about their reasons for sending the child to preschool and what they expected the child to learn or gain from being in a preschool setting. Again these questions, as well as the next two, helped to gauge the attitude of parents towards education as well as their expectations for what children should be doing and learning from attending “formal schooling”. In terms of enrollment, there were a total of 28 responses to the question as some parents indicated several reasons for enrolling their child in the IRI project. All twenty-three parents said that they enrolled their child in the IRI project to prepare them to enter the first grade. In addition, five parents provided a second reason for enrolling their child in the project: one parent said it was because there was no other alternative; one parent said they enrolled their child in the IRI project to play; and three families enrolled their child in the IRI project for unspecified reasons. Parents were asked two related questions: What do you think your child will learn while enrolled in this preschool program? And what would you like your child to learn while enrolled in this preschool program? Not surprisingly the responses were quite similar for the two questions. For these two questions many parents had several things that they hoped and expected their child to learn. Parents were provided with five options, the first three being related to cognitive skills that are readily recognized as necessary for school readiness (write, count, read). The 4th option was “making friends” and the 5th was ‘other’ where parents could specify a reason. Almost all parents checked the 1st three. A number of theories July 18 2005 Education Development Center, Inc. can be generated for parent’s emphasis on the cognitive responses, such as parents being concerned that their child will have the skills to graduate from first grade and beyond, for which they would need skills such as learning to read, write and count. Thirty-five percent said that they thought or would like for their child to make friends. Parents feel that school is not necessarily a place to be social but it is a place to learn, and possibly because the communities are small and close-knit geographically the majority of children already know and play with each other on a regular basis. When asked what they thought their child would learn, some parents named more than one thing, resulting in a total of 55 responses. Forty percent said that they thought their child would learn to write (38 % would like their child to learn to write). Nine percent think that their child will learn to count (14% want their children to learn to count) and 29 percent said that their child will learn to read (34% would like for their child to learn to read). Fourteen percent also said that their child will make friends while participating in the program (10% percent want their child to make friends). Before the project began, the great majority of parents (91%) were aware that the children would be listening to audio programs. Only 9% responded that they did not know the children would be listening to audio programs. When asked what they thought the child would learn through listening to the radio, 24% indicated that they thought their child will learn to write (30% said they would like the children to learn to write). Eight percent of parents said that they thought their child will learn to count (15% said they would like this to happen) and 26% said that they thought their child will learn to read (25% said they would like children to learn to read). Thirty￾one percent indicated that their child will make friends (23%) and eleven percent (7%) thought that their child will learn other things. It is interesting that so many parents thought that the child will make friends as a result of listening to the radio program. The post-test questionnaires included the same latter questions on what parents thought their child had learned from attending the preschool center and listening to the audio programs, as well as what parents would have liked for their child learn. Like the baseline data, responses were similar (See Table 8). However, the majority of parents would have liked for their child to learn to read and a small percentage thought that their child learned to read or developed some skills associated to beginning reading. Table 8 What Urban Parents Think and Would Like Their Child to Learn POST-TEST DATA (%) Preschool Center Audio Programs Think Child Learned Liked for Child to Learn Think Child Learned Liked for Child to Learn a. write 32% 50% 26% 31% b. count 26% 4% 21% 15% c. read 5% 29% 6% 15% July 19 2005 Education Development Center, Inc. d. make friends 32% 13% 26% 27% e. other 5% 4% 21% 11% Of interest is the extent to which parents are involved in the preschool. There are several levels of participation that were explored. At a minimum level at the beginning of the year, only a quarter of the parents participated in the opening of the center. Furthermore, at that point, only 30% of the parents had received any written information about the preschool program. On the other hand 91% of parents/ families said that they had visited the preschool classroom before classes had begun. All the parents stated that they had met the community educator, and half the parents said that they communicate daily with the community educator, with about a quarter of the families stating that they communicate once a week with the community educator. At the end of the year, 100% of parents stated that they had been involved at the center in one way or another. All of the parents had attended regular meetings and several parents indicated that they had participated in additional ways, such as volunteering in the classroom (7%), donating resources or money (5%), building toys (4%) and in other unspecified ways (1%). In addition, all of the parents stated that the community educator had shared some information directly with them regarding their child, and half of the parents stated that they had received some information about the center in writing - specifically flyers announcing the opening of the center and other activities to take place throughout the year. Communication between the parents and the educator was maintained throughout the year with approximately half of the families continuing to have daily communication with the educator and the remaining families having weekly communication. B. Experimental Rural – Pre And Post During the baseline data collection a total of 69 parents/families were interviewed in all three communities of Comayagua, Francisco Morazan and La Paz. During the comparative data collection a total of 54 families were interviewed throughout the communities. The following is a summary of the data collected. Family characteristics All 98 families (100%) reported that Spanish is the primary language spoken at home. Only three (4%) parents had attended high school. Thirty-eight percent of parents said that they had completed the sixth grade. Forty-six percent of parents interviewed indicated that they had completed the third grade, and 12% of parents said they have not had any formal education As noted above, one predictor of children’s success in school is in-home access to reading materials. When asked if there were things to read in the home, 61% said there were, while 39% said there was nothing to read in the home. Among reading materials to be found in the home, 46% said that they had books available, 32% said that they had July 20 2005 Education Development Center, Inc. newspapers available, and 22% indicated that they had other types of reading material available. End of the year interviews revealed that families continued to have reading material in the home, particularly newspapers and books. Family Composition When asked about the composition of the household, 14% reported that two – four individuals live in the home. Seventy-two percent said that five to eight individuals lived in the home and 14% reported that between nine to twelve individuals live in the home. The total numbers of individuals living in homes of children participating in the IRI project are 430. From that total, 47% are adults, 31% are children between the ages of zero to six years old, and the remaining 22% are between the ages of seven to 16 years of age. Sixteen percent of the families reported having one child in the family. Sixty-five percent reported having between two and four children. Of the remaining, 19% of parents/ families, 92% reported having between five and seven children in the household, and one parent (8%) reported having more than seven children. The maximum number of children reported was eight. Of the children participating in the Interactive Radio Instruction program 41% were first born; 19% were second born; 17% are third born; and the remaining 23% are fourth born and younger. Information on the children An important contributor to a child’s being able to learn, is his/her health. Parents were asked how they would describe their child’s health. Only 13% reported that they would describe their children’s health as poor, while 17% described their children’s health as excellent. Twenty-nine percent described their children’s health as average and 41% described their children’s health as good. 100 percent of families reported that their children were up to date on their immunizations. At the end of the year, 32 percent of parents said that their child’s health was “good”, followed by 13 percent describing the child’s health as “excellent” and 9% saying it was “average”. While it is hard to assess the child’s nutritional intake, one way commonly used to judge this is to get an idea of how many meals a child eats during the day. If asked this directly, most parents will say ‘3’ – what they think the interviewer wants to hear. However, if parents are asked how many meals the child had yesterday they are more likely to respond in terms of what actually happens. No one reported that their child had only had one meal the day before, while five parents reported that their child had eaten two meals the day before. Ninety-three percent reported that their children had eaten at least three meals the day before. In comparison, by the end of the year only 48% of adults said that the child had eaten three meals the day before, while two parents each did say that the child had only eaten one and two meals the day before. Activity level is another indicator of health. Parents were asked to describe their child in terms of his/her level of activity. No families reported their child as being inactive. July 21 2005 Education Development Center, Inc. Twenty-nine percent described their children’s activity level as somewhat active and 49% described their children’s activity level as active. Only 22% described their child as being very active. During the post-test interviews 50 percent described their child’s activity level as “active” while 30 percent and 20 percent described the child’s health as “very active” and “somewhat active” respectively. Children and Pre-school Of interest in the study is whether or not children have had any pre-school experience prior to their involvement in the IRI project. Of the families interviewed, 62% reported that their child had not previously participated in a pre-school program, while 38% reported that their child had previously participated in a pre-school program. Parents and the Preschool Parents were asked why they enrolled their child in the program. There were a total of 80 responses to the question as some parents indicated several reasons for enrolling their child in the IRI project. Fifty-four percent said that they enrolled their child in the IRI project to prepare them to enter the first grade, while 6% said it was because there was no other alternative. Twelve percent said they enrolled their child in the IRI project so the child could play, and the remaining 28% enrolled their child in the IRI project for other (unspecified) reasons. Two related questions were asked: What do you think the child will learn and what would you like the child to learn in the pre-school? As noted, these are not unrelated questions. During the pre-test 29% said that they thought their child will learn to write (31% want their child to learn to write). Nine percent of parents thought that their child would learn to count (10% want their child to learn to count) and 22% said that their child will learn to read (23% would like for their child to learn to read). Twenty-one percent also said that their child will make friends while participating in the program (14% want their child to make friends), and 19% thought that their child will learn something else (22% want their child to learn other things). Of interest is how much the parents actually knew about the IRI project. A question that sought to get some basic information was, do you know that your child will be listening to audio programs? Eighty-eight percent of parents responded that, yes, they were aware that the children would be listening to audio programs. Only 12% responded that, no, they did not know the children would be listening to audio programs. To further assess the parents’ understanding of what it might mean to listen to a radio program, they were asked what they thought the children would learn from listening to the radio (and what they would like the children to learn). There were a total of 124 responses during the pre-test since several parents said that they think their child will learn several things from listening to the audio programs (148 responses to what parents would like the children to learn). Twenty-two percent indicated that they thought their child will learn to write (24% would like for their child to learn to write). Twelve percent said that they thought their child will learn to count (14% would like for their child to July 22 2005 Education Development Center, Inc. learn to count) and 14% said that they thought their child will learn to read (18% would like for their child to learn to read). Twenty-one percent of parents indicated that their child will make friends (20% for what parents would like their children to learn) and 35% thought that their child will learn other (unspecified) things (24% would like for their child to learn other things). It would be interesting to know what is included in the ‘other’ category, since it represents a relatively high percentage of the responses. Results achieved during the post-test are displayed in the box below. Again, parent responses were similar in regards to the two questions (See Table 9). Table 9 What Rural Parents Think and Would Like Their Child to Learn POST-TEST DATA (%) Pre-School Center Audio Programs Think Child Learned Liked for Child to Learn Think Child Learned Liked for Child to Learn a. write 22% 24% 22% 22% b. count 24% 23% 24% 21% c. read 11% 18% 20% 17% d. make friends 23% 21% 22% 21% e. other 20% 14% 20% 20% Of interest is the level of interaction between the parents and the preschool program. More than half of the parents (58%) participated in the opening of the center. Many attended the opening in spite of the fact that 83% of the parents/families said that they had not been provided with any written information about the preschool program. Eighty-five percent of parents/ families said that they had visited the preschool classroom before classes had begun. Throughout the year, 96% of parents participated in the center either through attending meetings (36%), donating resources or money (38%), building toys (19%) and/or volunteering and getting involved in other ways (17%). In terms of their relationship with the teacher, at the beginning of the project 58% of the parents indicated that they communicate with the teacher on a daily basis and 22% said that they have communication with the teacher at least once a week. One parent said that she did not know the teacher. The trend continued through the project, with 26% of parents indicating that they communicate with the teacher weekly, 24% had communication daily and only one parent stated that they had never met the teacher. TEACHER/EDUCATOR INTERVIEW A. Experimental Urban – Pre and Post In the urban areas of the communities of Comayagua and Francisco Morazan a total of five teachers were interviewed at the beginning of the year. At the end of the project year July 23 2005 Education Development Center, Inc. three community educators were interviewed in the community of Comayagua. None were interviewed in Francisco Morazan. (Complete results can be found in Annex V). Teacher Characteristics Two thirds of the teachers had only a 6th grade education, with 1/3rd having more than that. (The grade level completed by the latter was not specified. Nonetheless all the teachers claimed that they “read well”, with the majority claiming that they “read very well”. When asked about their ability to write, all the teachers said that they could write. However, some of them were not very comfortable with writing, as indicated by the fact that one of them stated they could “not write very well”. The majority, however, claimed they could “write quite well”. In terms of their own background, the majority of the group had children (80%). Those with children had fewer than 6 children, with half the group having 1-2 children and the other half having 3-5 children. In terms of the teacher’s previous experience working with young children, less than half of them had worked with children (40%). When asked why they decided to be part of the project, responses included some of the following: FRANCISCO MORAZAN “I want to help children get ready for school.” “So that the children will be able to learn lots of things.” “I was chosen by the parent committee.” COMAYAGUA “I like to work with the community.” “For more experience.” “There is a need in the community.” Since the experience of teaching young children was new for the majority of the teachers, questions were asked regarding what the teachers wanted to learn about and the kinds of skills they wanted to develop for their work at the beginning of the year and at the end of the year they were asked what they felt they had learned. They were provided the following options to choose from: child development; activities to do with children; professional development and other. All of the teachers gave the first three options high priority, as well as expressed an interest in learning about other items. Comparative data revealed that overall teachers felt they had been provided training, information and support on their topic priorities (See Table 10). Table 10 Skills/Knowledge Teachers wanted in order to Support July 24 2005 Education Development Center, Inc. Children’s Development and What They Gained Percentage Like to Learn (Pre-test) Have you Learned (Post-Test) a. child development 25% 33% b. activities 25% 17% c. professional development 25% 50% d. other 25 0 Teacher perception of children Also of interest was what the teachers thought that children would learn from their experience in the preschool. Teachers could choose from the following options: Write (21% chose this category), Count (21%), Read (16%), and Make Friends (21%), with an ‘Other’ (21%) category for individual responses. At the end of the year, teachers were asked the same question with the following responses: write (27%); count (27%); read (9.5%); make friends (27%) and other (9.5%). Teachers were asked the same question in relation to what children could learn from radio. They were given the same options as the previous question. In reviewing the responses, it is hard to tell what teachers thought children would learn from the radio, since about half of the teachers responded with ‘other’. Three percent thought children would learn to write, while 27% thought children would learn to count, 16% said they would learn to read and 27% say the project will help children make friends. The overall picture here would suggest that teachers did not have a clear understanding of what could be taught via the radio. At the end of the year the teachers were asked the same question in terms of the audio programs with the following results: Write (25%); count (25%); read (12.5%); make friends (25%) and other (12.5%). The results were very similar between the preschool and audio program experiences. Teachers were also asked a set of questions to assess their opinions and attitudes about what children should know. Out of the 20 topics related to what it is important for children to learn in preschool, at the beginning of the year, several educators felt that it was not “very important” for a child to know how to read or write. (Now that’s an interesting response.) The good news is that by the end of the project year 100% of teachers felt that it was “very important” for a child to know how to read and write. In fact, at the end of the year, most topics were rated by the teachers as “very important” for a child to know (See Table 11). Table 11 How important do you think it is for a child to know how to: Percentage Not important Important Very important July 25 2005 Education Development Center, Inc. Read: Pre-test 20 40 40 100 Post - Test 0 0 100 100 Write: Pre-test 0 20 80 100 Post - Test 0 0 100 100 Identify part of the body and their functions: Pre-test 0 100 0 100 Post - Test 0 0 100 100 Follow directions: Pre-test 0 100 0 100 Post - Test 0 0 100 100 Identify elements in nature: Pre-test 0 100 0 100 Post - Test 0 0 100 100 Identify and imitate musical sounds: Pre-test 0 40 60 100 Post - Test 0 33 67 100 Identify different tastes/flavors: Pre-test 0 40 60 100 Post - Test 0 33 67 100 Identify different smells/odors: Pre-test 0 100 0 100 Post - Test 0 0 100 100 Implement personal positive hygiene habits: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Implement positive hygiene habits regarding food and/or meals: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Learn and implement safety measures: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Express themselves verbally: July 26 2005 Education Development Center, Inc. Pre-test 0 40 60 100 Post - Test 0 33 67 100 Express their feelings: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Recognize growing stages: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Identify numbers: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Identify elements that pollute the environment: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Enjoy playing: Pre-test 0 40 60 100 Post - Test 33 67 100 Can relate to other children: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Knows their own name: Pre-test 0 40 60 100 Post - Test 0 0 100 100 Can use different art materials: Pre-test 0 20 80 100 Post - Test 0 0 100 100 Relationship with parents Parent participation was another area of interest. This was a focus within the research since much of the early childhood development literature indicates that the preschool experience is most efficacious when parents are involved in the process. However, parents are not going to get involved if teachers are not actively encouraging and supporting their involvement. A lot depends on the teacher’s attitude. Thus a series of questions were asked to ascertain whether or not teachers saw a role for parents and if so what they thought that role could/should be. July 27 2005 Education Development Center, Inc. The first question in relation to parent involvement had to do with whether or not teachers thought that there were ways that parents could participate in the preschool classroom. While 60% said “Yes”, 40% said “No”. This is not very encouraging. At the end of the year, 67% of teachers responded that there were opportunities for parents to participate in the center and all had weekly or more frequent communication with the parents, sharing information with them regarding what their child was doing during the day as well as providing information about children’s behavior and activities that parents can do at home to support the child’s education. Information was shared during regularly held parent meetings or in the community of Comayagua during home visits. A further set of questions had to do with relationships between the preschool and parents to date. For example, had parents been given any information about the preschool project? Teachers responded that parents had been given information (15 out of 16). And the majority of the teachers said that parents had been invited to the preschool center before classes began. When asked how they would describe their relationship with parents, all said that they had met the parents. A third stated they communicated daily with parents, but about half said it was only once a week. For 80% of the teachers the communication with parents includes information about the child. Operation of the classroom Another set of questions dealt with what kinds of materials were available to a teacher as she worked with young children, and how she was able to take care of the materials, as it is important that the teacher have a safe place to store the CD player and the CDs. More than half of the teachers responded that they did not have a place to store the CD player and other materials. When asked where they did store things, teachers provided multiple answers. The most common responses noted were: “I store the materials at home.” “Some things in the closet in the classroom and others at home.” “Some things in the classroom, others at home, some in the drawer of the desk, and in a cardboard box.” Training, Supervision and Support While all of the teachers responded that they had been trained, the interviews attempted to define more specifically what it was that the teachers learned during the training(s). For example, all teachers responded that they received some training regarding the operation of the preschool center. End of the year interviews revealed that teachers felt that they had received training in all areas of interest, from child development to using the CD player and the audio programs to managing the center environment. Responses revealed teachers had set up learning centers in the classroom and were using activities such as songs and games to continue to support children’s learning after the audio program had ended for the day or on alternate days. July 28 2005 Education Development Center, Inc. While the interviews were meant to establish a baseline and serve as a pre-test, some of the teachers had already been involved in the project when the interviews occurred .The teachers interviewed had been listening and using the audio programs since February 2004 when the centers were established. In fact all of the teachers responded that they had already used some of the activities that they heard or learned from the audio programs. One of the keys to quality teaching is that teachers feel they have appropriate support from their supervisor. So, teachers were asked if they ever had an opportunity to discuss any concerns or issues that they are having with Proyecto de Educación Prebásica Interactiva (PREPI) staff. All of them answered in the affirmative, which is very positive. On the other hand, there appears to be little attempt by PREPI to organize events that would bring the teachers together to share their experiences with one another. In response to the question, “has PREPI organized events in which you can participate with other staff and teachers?”, 81% said no. (Since the initial information was collected, PREPI has in fact sponsored a toy fair in Comayagua where parents and children built toys that were sold and judged for prizes.) One point to clarify is that teachers were provided with quarterly trainings from PREPI. The trainings focused on the use of the audio programs, managing the center, setting up the environment, activities to do with children and parent involvement. All but one of the teachers said that she received supervision and had time to plan her work with the children. Again, these are very positive aspects of the project. However, supervision is only one way of improving one’s work. Although teachers were visited by several individuals from many agencies and organizations and were provided the opportunity to ask questions and discuss concerns, there seems to be no system in place for follow-up so that whatever suggestions were provided to the teacher, she was left to her own devices to resolve it. Considering the overall situation of the teachers and the communities, resources are scare and opportunities few. The teachers were asked what they did if they wanted to improve their work. As this was an open-ended question, teachers provided multiple answers. The most common responses noted were: FRANCISCO MORAZAN “I have a meeting with the parents to solicit funds.” “I try again.” “I ask another person.” COMAYAGUA “I review the teacher manual.” “I ask for help.” In addition to supervision, teachers need materials and supplies to support their work. So teachers were asked: “Have you received any resources from the project that you can use to implement new ideas or activities”, the great majority of the teachers (88%) said that they did receive other materials to assist them in their work with children. The materials included basic supplies such as crayons, scissors and paper. July 29 2005 Education Development Center, Inc. At the end of the year, it was important to get a sense of what teachers felt their experience in the project had been and so a series of questions was designed in order to determine if teachers had used the program as it was intended and if provided the opportunity would they use the Juego y Aprendo program again. One hundred percent of teachers said that they would use the program again because it was easy to use and clearly the children enjoyed it while they listened and learned. The majority of educators described their experience as “well” and the children’s experience in the project as “excellent”. Responses reveal that the educators were on schedule as all indicated that to date they had listened to 80% of the audio lessons. Educators were able to provide the major themes of the audio programs, including the environment and family and community. The best program was said to be “A Happy Day” while the worst program was “The Day of the Child”. B. Experimental Rural– Pre And Post The educators are all volunteers who are at least 16 years old with a minimum of a sixth grade education; they can read and write very well. They are members of the community who have been approved by the Committee of Support to teach and facilitate the audio programs with the children enrolled in the project. During the baseline data collection, 13 community educators were interviewed in all three communities. At the end of the year 12 interviews were completed. Complete data results can be found in Annex V. Teacher characteristics The majority of the teachers (69%) have not had more than a 6th grade education; a few have had additional studies, but the highest level of education was not ascertained. Nonetheless, all of the teachers said that they could read, with 61% saying they could read very well; 39% said they could read well. The teachers also all claimed to be able to write, although 15% said they could not write very well, with 23% being comfortable with writing and the remaining 62% saying they could write very well. In terms of their personal lives, teachers were asked if they themselves had any children. Fifty-four percent responded in the affirmative. Thus quite a high percentage of the teachers do not have their own children. For those who had children 57% have 1-2 children, with 43% having 3-5 children. No one had more than 5 children. Thirty-eight percent of the teachers stated that they had worked with children before getting involved in this project. Thus a high percentage has had no experience working with children. When asked why they decided to work with this project, a variety of responses were given: LA PAZ: July 30 2005 Education Development Center, Inc. “To help out my parents financially.” “To try out the project.” FRANCISCO MORAZAN “I like to work with young children.” “I want to help children get ready for school.” “So that the children will be able to learn lots of things.” “I was chosen by the parent committee.” “I felt motivated to work with children.” COMAYAGUA “I like to work with the community.” “For more experience.” “There is a need in the community.” Teacher perceptions of children Since the majority of the teachers had no experience working with young children, one of the things that it is important to know is what they expect of young children and what do they think that the preschool experience will provide for children. The teachers were asked: What do you think the children will learn while participating in the pre-school center? The teachers chose between 4 responses (read, count, write, make friends). They could choose more than one and they could indicate other areas, but the precise responses included in the ‘Other’ category were not recorded. The total number of responses was 33 as some teachers marked more than one option. The responses were as follows: Write – 21%; Count – 9 %; Read – 12%; make friends – 36%; other – 21%. Of interest is whether or not teachers think that children can learn these same things through a radio program. In essence the pattern of responses is quite similar, although the sense is that it would be harder to learn to write as a result of the radio program (12%). It is hard to draw many conclusions from the responses to this question, since 36% of the responses fall in the ‘Other’ category – a quite sizeable number of responses where the specific topics are not known. At the end of the project, teachers were asked what they thought children learned from attending the preschool center and from listening to the audio programs. Teachers were able to choose from the same five options as stated above and again the responses were similar (See Table 12). Table 12 What Teachers Think Children Learned in a Preschool and through Audio Programs POST-TEST DATA (%) Pre-School Center Audio Programs Child Learned Child Learned a. write 30% 22% July 31 2005 Education Development Center, Inc. b. count 28% 19% c. read 15% 15% d. make friends 18% 25% e. other 9% 19% The rationale for asking these two similar questions was to determine if teachers could distinguish between the preschool center as a space where children and parents are active participants and the IRI program being one tool that the teachers use to help children develop skills and abilities. Although the IRI Juego y Aprendo project was indeed the reason for the existence of the preschool center and the main activity taking place in the classroom, it is hoped that throughout the year, with additional training and classroom management skills learned from listening to the audio programs, teachers would be able to provide additional activities that would support what the children are learning from the audio programs and have the center open, where the teachers were developing and implementing their own activities when the audio programs were not being used. As noted, the methodology was purposely designed so that teachers would only use the IRI program three days per week and as the year progressed and their skills increased teachers would be able to support children’s learning through teacher-planned activities. Some of this occurred through teachers establishing “learning centers” (58%), where activities were planned that included art, dramatic play and citizenship and using songs and other activities (92%) and games that they had learned from the audio programs. These were offered on alternate days when the audio programs were not being heard. Teachers were also asked the following questions to assess their opinions and attitudes about children’s learning. (In the rural communities only 12 out of the original 13 responded.) In general teachers’ attitudes about how important it is for children to have the skills and abilities noted in Table 13went from “important” to “very important”. An interesting change from pre-to post testing is that under “follow directions” teachers were split 50/50 as compared to the beginning of the year when the majority 75% believed that it was “very important” for children to follow directions. In regards to hygiene matters teachers’ beliefs remained constant considering hygiene habits in all ways to be “very important”. Table 13 How important do you think it is for a child to know how to: Percentage Not important Important Very important Read: Pre-test 8 8 84 100 Post - Test 0 8 92 100 Write: Pre-test 0 16 84 100 Post - Test 0 8 92 100 July 32 2005 Education Development Center, Inc. Identify part of the body and their functions: Pre-test 0 25 75 100 Post - Test 0 8 92 100 Follow directions: Pre-test 0 25 75 100 Post - Test 0 50 50 100 Identify elements in nature: Pre-test 0 33 66 100 Post - Test 0 8 92 100 Identify and imitate musical sounds: Pre-test 0 42 58 100 Post - Test 0 33 67 100 Identify different tastes/flavors Pre-test 0 16 84 100 Post - Test 17 83 100 Identify different smells/odors Pre-test 0 16 84 100 Post - Test 0 17 83 100 Implement personal positive hygiene habits: Pre-test 0 0 100 100 Post - Test 0 0 100 100 Implement positive hygiene habits regarding food and/or meals: Pre-test 0 0 100 100 Post - Test 0 0 100 100 Learn and implement safety measures: Pre-test 0 50 50 100 Post - Test 0 8 92 100 Express themselves verbally : Pre-test 0 8 92 100 Post - Test 0 0 100 100 Express their feelings: Pre-test 0 50 50 100 Post - Test 0 8 92 00 Recognize growing stages: July 33 2005 Education Development Center, Inc. Pre-test 0 33 66 100 Post - Test 0 25 75 100 Identify numbers: Pre-test 0 25 75 100 Post - Test 0 17 83 100 Identify elements that pollute the environment: Pre-test 0 8 92 100 Post - Test 0 8 92 100 Enjoy playing: Pre-test 0 25 75 100 Post - Test 0 42 58 100 Can relate to other children: Pre-test 0 8 92 100 Post - Test 0 17 83 100 Knows their own name: Pre-test 0 8 92 100 Post - Test 0 25 75 100 Can use different art materials: Pre-test 0 0 100 100 Post - Test 0 25 75 100 Relationship with parents In the same way that there was an attempt to get the parents’ understanding of the kind of relationship they should/could have with the project, it was thought important to get a sense of what teachers see as appropriate ways for parents and teachers to work together. Teachers were asked at the beginning of the year: Is there some way that parents can participate in the preschool classroom? There was an almost balanced response: 54% said yes while 46% said no. Teachers were under the impression that parents had been given information about the preschool project (92% saying that parents had received information vs. 83% of the parents/families said that they had not been provided with any written information about the preschool project). Seventy-seven percent said that parents had been invited to visit the preschool center before classes began. All teachers stated that they had met the parents, with 38% saying they communicate daily with parents; the same percentage said they communicated once a week with parents, with 85% of the teachers stating that they shared information about the child with the parents when they met with them. The same July 34 2005 Education Development Center, Inc. questions were asked at the end of the year to determine if teacher – parent relationships had changed. A large majority (58%) stated that they communicate with parents weekly while the remainder communicates with parents daily. The teachers stated that parents were provided information on the progress of their child, the community of La Paz going so far as preparing and giving to parents “report cards”. Other information shared with parents was in regards to other activities that the children were doing besides listening to the audio programs and some examples of activities that parents can do at home to support their child’s education. Only one teacher said that she had not shared any information with parents. Operation of classroom Since the project involves the use of technology, it is important to those who operate the project to know if the teachers have a safe place to store the materials. Teachers were asked directly, “Where are you are able to keep materials – personal and educational?”. Only 38% of the teachers said they had a safe place to store the CD players. As this was an open-ended question, teachers provided multiple answers. The most common responses noted were: “I store the materials at home.” “Some things in the closet in the classroom and others at home.” “Some things in the classroom, others at home, some in the drawer of the desk, and in a cardboard box.” Training, Supervision and Support At the beginning of the year, teachers were asked what skills they would like to learn while working with this project. Choices included: Child development (21%) Activities to do with children (37%) Professional development (37%) and other (5%). Thus there appears to be interest in receiving training in key areas related to working with young children. At the end of the year, teachers were asked what they had learned throughout the year. They were provided with the same options as stated above. Based on the responses it would seem that the teachers were provided with training and support in the areas of interest (See Table 14). All teachers responded that they had already had an opportunity to use the activities that they have heard or learned about on the audio programs. Table 14 What Teacher Would like to Learn and What they Learned Percentage Like to Learn (Pre-test) Have you Learned (Post-Test) a. child development 21% 33% b. activities 37% 33% c. professional development 37% 29% July 35 2005 Education Development Center, Inc. d. other 5% 5% Not only is there a need for training related to the context of a preschool program, but there is a need for training regarding the operation of the preschool center. All teachers responded that they had received appropriate training in terms of managing and organizing the center and in the use of the IRI Juego y Aprendo program. In terms of supervision, 85% responded that their work was supervised at the beginning of the project and 92 percent said that their work was supervised at the end of the year. During the comparative data interview, teachers were able to provide more information in regards to how they were supervised, by whom, how feedback and follow-up was provided and also what they themselves did with the suggestions they received. Teachers were visited throughout the year by PREPI staff whose role was to observe and provide support to the teachers. At the beginning of the year a cooperative agreement had been signed with each of the three school districts and teachers stated that they had been receiving regular visits from local school personal throughout the year. Some preschool centers were located in a classroom in the local school and a few teachers said that they were able to talk with the local school teacher to get some feedback and suggestions. Teachers also stated that during their conversations with each of the supervisors they talked about a variety of things, including classroom management and activities, concerns and strategies for working with children’s behavior and personal issues. Teachers also said that the project was structured in such a way that they had time to plan their activities with children. While the initial training is key to a quality program, in order for that quality to be maintained, it is critical to have an opportunity to discuss any concerns or issues that teachers have over time. When asked if they had an opportunity to discuss issues with PREPI staff, 85% of the teachers said Yes, although the majority (85%) did not feel that PREPI had organized events in which they could participate with other staff and teachers. At the end of year, in the community of Comayagua, a toy fair was planned and held. Besides this event no other events were held in other communities besides the quarterly training for the community educators. Teachers were asked what it was they do in order to improve their work. As this was an open-ended question, teachers provided multiple answers. The most common responses noted were: LA PAZ “I try to read and ask questions.” “I dedicate more time to it.” FRANCISCO MORAZAN “I have a meeting with the parents to solicit funds.” “I try again.” “I ask another person.” July 36 2005 Education Development Center, Inc. COMAYAGUA “I review the teacher manual.” “I ask for help.” By the end of the project year, teachers’ response included looking for local resources to support their work and making sure to ask the supervisor(s) when they arrived. At the end of the year, a set of questions was asked in order to determine the teachers experience in using Juego y Aprendo. Teachers were asked to rate their experience as well as the children’s experience in the project. Fifty percent said that their experience was “good” while 42% described their experience as “excellent” and one teacher said her experience went “well”. In regards to the children, the majority of teachers (67%) described the experience as “good” and the remaining 33 percent felt the experience was “excellent”. As noted, a calendar was created showing teachers on what date the programs were to be listened to, with Juego y Aprendo scheduled for three days a week. At the time of the final interviews, 80% of teachers had listened to 88 programs, indicating that they were on schedule. It was also clear that teachers were familiar with the programs because all were able to specifically state what the major themes of the Juego y Aprendo program were as well as indicate which the best and worst programs were. Juego y Aprendo included four categories which related to the national primary school curriculum and under which all the audio lessons were created. Several community educators mentioned that the audio programs in the Family and Community category were the best as well as the Environment. Many educators specifically stated “The Day in the Jungle” as the best audio program. Only one educator said that a program was bad and she specifically said the program about “The Garden” was the worst. The teachers’ description of their experience in the project supports the data stating that when asked, 100% of teachers said that they would use the program again because it is easy to use and the children clearly enjoyed it and made progress. One interesting point is that 42% of teachers said that they would not use the program exactly the same way if provided the opportunity to use again. Unfortunately no information as to the how they would change it and why was collected. COMMUNITY INTERVIEW – PRE-PROGRAM At the beginning of the evaluation process, in each community, the evaluators facilitated two focus groups: one in the rural area and one in the urban area. If possible a community interview was conducted where a center was located. For example, during the pre-test in Comayagua, two focus groups were held in the rural area. One at the Buenos Aires Taulabe Center and another at the Los Pozos Center. During each community interview, there was between 1 and 4 individuals who participated in the focus group. These individuals were residents of the particular community and included parents of children enrolled in the PREPI project, community leaders, and government representatives, specifically from the Ministry of Education. The meetings lasted July 37 2005 Education Development Center, Inc. approximately 30 minutes. The CCF-H evaluators served as note taker and facilitator, simply presenting the questions to the group and providing clarification if necessary. In the community of La Paz, which is described strictly as a rural area, the goal was to conduct one large focus group for the entire community, but because no transportation was available, parents and other participants would have had to walk one to two hours to reach the chosen center. So it was decided that one-on-one interviews would be completed instead. As a result, the evaluators conducted individual interviews with parents, community leaders, and school directors. The following is a summary of responses in relation to each of the questions asked. Overall, responses did not differ from community to community or from rural to urban areas. Participants had many of the same concerns, level of involvement, and comments. 1. Tell us how you became aware of the pre-school center: The majority of the participants attended community meetings where information was provided about the PREPI program, its objectives, curriculum (audio programs), and duration. Participants in the Focus Groups who were not parents of children enrolled in the project learned about the pre-school center from having direct contact with a representative from FEREMA. One participant did say that they were not aware that a project was going to exist until the teacher/educator began holding classes. 2. What do you think the purpose of the pre-school center is? All the participants stated that the purpose of the center was to prepare the children for first grade so that they will be successful. There were no other responses given. 3. Were you involved in the opening of the pre-school center? The responses here were generally split 50-50. Half of the participants said that yes they had in some way participated in the opening of the center. Depending on their role in the community, certain individuals attended the meetings held by the CCPREBS to inform the community about the PREPI program, while other participants such as community leaders or church clergy, assisted in negotiating the opening and timing of the center and gathering materials. Other individuals such as school directors met with FEREMA staff and made arrangements for payment of educators/teachers salaries. 4. Do you intend to volunteer in the pre-school classroom? If yes, what will you do? July 38 2005 Education Development Center, Inc. The majority of the participants reported that they plan to volunteer in the center by helping to collect materials, and parents specifically said that they would assist their children with any work and make sure the children arrived on time and ready. 5. Do you intend to participate on the parent committer or advisory committee? The majority of participants reported that they intended to participate either on the parent or advisory committee. 6. What type of impact do you think the pre-school center will have on the community? A: Educational B: Economic C: Social D: Family E: Health F: Other The responses to this question were interesting. Participants reported that they thought the center would either have an educational impact on the community or an educational, economic, social, family, and health impact on the community. It was either educational or all. No other options were reported on their own. 7. Do you know if there are other pre-school programs in the community? All communities reported that no other program for pre-school children was available at the moment. 8. Have there been other pre-school programs in the community that no longer exist today? Only one community, an urban area, reported that there had been a pre-school program available for children in the past. All other communities reported that no such programs were ever available. Other comments The majority of the comments made were that the center was important as it would support and advance the education of the children. Other comments made were that support was needed from the local municipality to build a school house and to provide basic services such as sanitation and water. COMMUNITY INTERVIEW – POST-PROGRAM July 39 2005 Education Development Center, Inc. The post-test interview was close-ended when compared with the pre-test interviews. Questions were direct and participants were asked to choose their responses from several options. The purpose of the interview was to get a sense of whether or not the community had been involved in the preschool center throughout the year as well as to determine how the community may be able to support the sustainability of the preschool center with the continued use of Juego y Aprendo. The post-test was only completed in the rural areas and so it will be difficult to compare data collected. In the community of Francisco Morazan an interview was conducted with one individual. In the communities of Comayagua and La Paz, between 3 and 8 individuals were present for the focus group. 1. What would you say is the purpose of the preschool center: All of the interviewees described the preschool center as a place where children can familiarize themselves with the routines of going to school so that they can learn and grow and be prepared to succeed in the first grade. 2. Were you involved in the preschool center?: A. yes B. No Ninety one percent said they were involved with the preschool center. If yes, how were you involved? Data are presented in Table 15. Table 15 Ways in Which Community Members were Involved in the Preschool A: meetings B: volunteering C: committee member D: other Total 8 6 3 2 19 Percentage 42% 32% 16% 10% 100% Several interviewees participated in the preschool centers in more than one way. The majority of the six respondents participated in the center by volunteering. 3. What type of impact do you think the center had on the community?: All of the interviewees believe that the center has had an educational impact on the community. Several also thought that the preschool center has had a social and familial impact on the community (See Table 16) Table 16 The Community’s Perception of the Impact of the Project A: B: C: D: E. F. July 40 2005 Education Development Center, Inc. educational economic social family health other Total 12 7 3 2 24 Percentage 50% 29% 12.5% 8.5% 100% 4. If the project is to continue, what type of resources or support do you think the community is able to provide to the center? Choices included: educator’s salary, toys or other educational materials, furniture for the classroom, and ‘other’, None of the interviewees felt that the community could provide the educator’s salary but noted that the educator had been well trained and now had the ability to work with children and teach them and therefore should be paid a higher salary. Several community members mentioned that the community is a poor area with little or no resources of its own. However, several participants felt that the community could provide some sort of educational materials, such as toys, and build furniture that could be used in the center. One community member mentioned that the community at large may be able to supplement a snack, so that all the children are receiving some food. Despite these issues, all the participants agreed that the center is a necessity for the children of the community and expressed hope that the project would continue to support the center with educational materials, trainings, and the educator’s salary. CONCLUSION The summative evaluation completed its objective of attempting to determine what kind of short-term impact (i.e. at the end of seven months of implementation) the Juego y Aprendo project had on the children, parents, community educators, classroom environment and the community at large. The results support the importance of the project, but do not provide definitive information on the long-term impact of the Juego y Aprendo approach to preschool education. Principal conclusions are: . • A change was exhibited in the classroom environment. Preschool centers were purposely established so that children and the community educator would have a space to meet. The ECERS revealed that changes in the environment took place throughout the year, with educators becoming more conscious of room arrangement, the use of local materials for activities and teacher–to-child interactions. The greatest change was seen for the rural groups who scored lowest at the beginning of the year; they evidenced a greater increase in their scores over time than the urban centers. • In all groups, interviews revealed that parents had many of the same expectations in terms of what they thought their child would learn from preschool (e.g. to read, write and count). All parents sent their child to the preschool so that they would be prepared for the first grade. Post-test interviews revealed that, with the exception of two to three parents in the entire sample, parents were involved in the preschool. Many attended regular parents meetings and a large majority volunteered or contributed to the center in one way or another. Data also showed July 41 2005 Education Development Center, Inc. July 42 2005 that parents were interested in how their child was progressing as indicated by the fact that communication between the teacher and parent occurred at least weekly, and for some there was daily communication with the community educator. • The Juego y Aprendo project had a direct impact on the community educators who were the group who received continuous training and support from the project. Many educators had attempted to establish “learning centers” within the classroom environment. The fact that teachers began to create their own activities was an indication of their ability to transfer the knowledge they had acquired into concrete actions. Through interviews and observations it was possible to see overall progress in the attitudes, skills and information of the educators. It was through them and their use of the audio programs and supporting materials that changes were made throughout the project. • Community interviews revealed that for the communities involved, the preschool center was important as it presented an opportunity for the young children that was not otherwise available. Community members obviously want the project to continue but they are not aware of how to go about allocating the resources to maintain it. So, without the financial and educational support of the project there is some question as to whether or not the preschool centers will be maintained. • In terms of outcomes on the Child Development Scale, the results are positive for the children participating in the Juego y Aprendo project as the data revealed that the program made a positive impact on the children in the experimental group across all developmental areas bringing the majority of the children from “Needs Attention” and “High Risk” to “Developed” and even a small group into the “Advanced” category. Finally, the local foundation, FEREMA is taking up the cause of continuing to follow a sample of children from the original 53 pilot centers. This follow up information will be useful in determining whether the program will have a lasting impact on the children. In January 2005, the Juego y Aprendo project was granted an extension for six months to perform the following activities: • Revise and produce 108 audio lessons for radio format as well as revise all educational materials so that they are radio ready, including the teachers guide. • Design an expansion and promotion plan for the CD and radio versions of Juego y Aprendo. • Create a specific parent and community involvement training for the community educators to accompany the Juego y Aprendo educational package. Since January FEREMA has been able to secure additional funding and throughout 2005 an additional 2000 preschool centers using Juego y Aprendo will open, helping the country of Honduras to provide quality preschool education to children not currently served by the existing system.