Interactive Teaching & Learning Program in Pakistan Cooperative Agreement No. 391-A-00-6-01075-00 A Report on Evaluation Findings 1st March 2006 – 28th Feb 2007 (Year One) May, 2007 Prepared by: Monitoring and Evaluation Section Children’s Resources International, Pakistan (Guarantee) Limited Table of Contents Executive summary Chapter 1: Introduction and objective of the program.........................................................1 Chapter 2: Attendance and promotion rate of the students..................................................4 2.1 Daily attendance in % age.........................................................................4 2.2 Retention rate ............................................................................................5 Chapter 3: Early Childhood Classroom Observation (ECCO) Results. ..............................7 Chapter 4: Technical Assistance Observation Results: .....................................................13 Chapter 5: Heads, Teachers and Parents surveys .............................................................16 5.1 Heads and teachers responses.................................................................16 5.1.1 Positive responses about the educational program..................17 5.1.2 Percentage of positive responses about teachers practices......21 5.1.3 Community Involvement.........................................................22 5.1.4 Important factors for the success of the program....................23 5.2 Parents responses....................................................................................25 Chapter 6: Family literacy-Pre& Post inventories results .................................................30 Chapter 7: Annual statistics (1st March 2006 – 28th February 2007).................................34 Chapter 8: Conclusion and recommendations...................................................................37 Annex: Questionnaires A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan i Executive summary CRI Pakistan initiated an Interactive Teaching and Learning Program in Pakistan with the funding support of USAID aim to improve the quality of education through teachers training, faculty development, family literacy, and parents’ involvement etc. This program commenced from 1st - Basic Education with strong ECE foundation March 2006 for the period of three years (2006- 2009). This program is being implemented in all the schools in Islamabad district, and selected schools in Rawalpindi and Karachi district. The Interactive Teaching and Learning Program in Pakistan was a direct result of another USAID funded project called Creating Democratic Schools (CDS) which was implemented by CRI successfully in Pakistan from February 2002 to Sep 2005. CRI’s Interactive Teaching and Learning Program has the following major components: - Family Literacy Program - Family Involvement Program - Higher Education Program The prime objective of the evaluation was to determine the efficiency, effectiveness and impact of the program. This evaluation report has been prepared to assess and analyze the project achievements and outputs by using different instruments i.e. classroom observation form; early childhood classroom observation (ECCO) form; parents, teachers and head teachers survey; and attendance & retention form. The report has been divided into seven chapters each chapter describes analysis and feedback of various program stakeholders about the program implementation. Chapter 1 describes the introduction of the program and its objectives. Chapter 2 of the report highlights the statistical analysis based on the attendance data collected by Master Teachers Trainers (MTTs), a three times in a year, at the beginning, middle and end of the school year. Schools were selected at random keeping in mind the geographical coverage and gender aspects. Analysis was done by phases (Phase-0 & Phase-1) and districts too. The information in this chapter shows the CRI methodology has a positive impact on attendance and grade retention rate as well. 96.71% of students enrolled in CRI old partner schools (Islamabad Capital Territory) were promoted to the next grade, compared to 83% of students in non-CRI classrooms. Chapter 3 of the report, based on early childhood classroom observation form, showing the areas of greatest change in CRI classrooms are also exemplars of the teaching/learning approach promoted by CRI training for the development of child centered classrooms and activity based methods. The number of items and their sizable differences in terms scores are strong indicators of the strides made by the CRI teachers to put their training into practice in their classrooms. Chapter 4 focuses on the observations made by MTTs during technical assistance (TA) visits. In new partner schools the average score of first TA was 7.49 and in the 3rd TA it was 10.92 against the maximum points of 23. This shows the change for ICT new schools is 3.43 points. This represents a good rate of progress within the five months after training of Grade 1 and Grade II teachers to implement quality of child A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan ii centered practices. But it needs a lot of effort to attain the maximum points 23. As for as old CRI partner schools are concerned the overall points was 14.62 during the first TA and found 16.82 in the second TA (January-February 2007). There was a change of 2.22 points. According to data Karachi schools are leading with the average score of 18.77, followed by Rawalpindi with an average score of 18.38 against the target of 23 points whereas Islamabad schools have an average score of 13.37. In chapter 5 of the report, overall 76.2% heads and 79.2% teachers (Phase I) said there is a positive change in the schools due to CRI interventions. As for as Old partner schools are concerned, almost the same results came out with a better percentage. 78.4% heads’ and 84.7% teachers’ responses indicated improvement in the schools. Overall 80.7% of parents saw positive changes in the outcomes associated with the CRI program, indicating how deeply the CRI methodology has been translated into action. The results of the heads, teacher and parents’ survey clearly indicate that the CRI methodology has been embraced by schools, teachers and parents in the target areas. In its first year of the expansion phase CRI has instigated great changes in the ways that young children learn. Chapter 6 shows Family Literacy Program’s impact on the learners on the basis of pre and post inventories. The average scored by the participants before attending the family literacy sessions was 24.15 and after attending the sessions it was 46.5 out of 50 which shows a significant change among the learners in respect of their Urdu literacy and basic math skills. Chapter 7 consisted of number of teachers trained and trainings conducted from March 2006 to February 2007. According to this information, CRI has trained 1454 teachers/head teachers of partner schools and 429 teachers/head teachers of other organizations during the year. Conclusion: Overall results encourage CRI to continue the methodology. They also provide guidelines for the planning and development of the next phase. Recommendations: In light of the experience from the program, following measures are recommended: o In order to improve the progress further, it is important to make sure the provision of technical assistance visits by MTTs on regular basis. o FDE should assign their own supervisors to monitor the work of its teachers and classrooms’ environment on regular intervals. o Teachers and heads should be advised on how to make a strategy to reach out more effectively to both families and communities. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 1 Chapter 1: Introduction and objective of the program CRI Pakistan initiated an Interactive Teaching and Learning Program in Pakistan with the funding support of USAID aim to improve the quality of education through teachers training, faculty development, family literacy, and parents’ involvement etc. This program commenced from 1st - Basic education with strong ECE foundation March 2006 for the period of three years (2006-2009). This program is being implemented in all the schools in Islamabad district, and selected schools in Rawalpindi and Karachi district. The Interactive Teaching and Learning Program in Pakistan was a direct result of another USAID funded project called Creating Democratic Schools (CDS) which was implemented by CRI successfully in Pakistan from February 2002 to Sep 2005. CRI’s Interactive Teaching and Learning Program has the following major components: - Family literacy - Family involvement - Higher education The education sector, in particular, is in a critical state of disarray. The sector's systemic failure stems from issues of both access and quality. While substantial investments have been made in building schools, access is not uniform and the overall quality of education remains very poor. Teachers are poorly qualified, often hired through political patronage, and receive little in-service training in different regions/districts. CRI program has been designed to improve teaching methodology and students’ performance through teacher training, faculty development and family literacy together with classroom material. An interactive teaching method is adopted with the objective of moving from a teacher centered to a student-centered classrooms environment. By the skills of teachers/heads though training and on going technical assistance, interactive teaching and learning program aids in meeting some of the major objectives of education reforms as well. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 2 By improving the quality of education through teacher education and training - a crucial element of the Government’s Education Sector Reforms-the program will support: 1. Increased use of child centered methods in the classrooms 2. Increased participation of families in the classrooms 3. Increased attendance and retention rates 4. Increased literacy among parents 5. Increased quality of preparation for teacher-in-training This report is based on the data collected by the CRI teams during March 2006-April 2007 for various exercises and this report presents the evaluation findings that reflect gains during the first year of the program (Phase-1). This report also presents the progress done in Old CRI partner schools (Phase-0) in Karachi, Rawalpindi and Islamabad. Breakdown of schools by regions are as follows: Table 1: # of partner schools Old partner schools (Phase 0) New partner schools (Phase I) Public Private Total Public Private Total Islamabad 35 4 39 140 -- 140 Rawalpindi 32 7 39 Karachi 29 11 40 Total 96 23 118 140 140 This report has been compiled by using the several key evaluation tools which are as follows: 1. ECCO has been adapted by CRI from a structured observation tool developed by National Association for the Education of Young Children (USA) to determine the quality of classroom practice. 2. CRI Classroom Observation is being used to monitor the physical environment, parents’ involvement and adaptability of child centered practices. 3. Parents survey is completed by parents, give information about changes in children and family participant in school life since introduction of CRI methodologies in the classroom. 4. Teachers’ survey is completed by teachers. The questionnaire seeks information about changes in classroom practice, student learning and behavior, and family participation since the introduction of CRI intervention. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 3 5. Principal and administrator survey, completed by principals/heads, is the questionnaire which seeks similar information as that gathered from teachers. 6. Students’ attendance form will be used to get attendance and retention rate of the children. 7. Registration form has been used to obtain the number of teachers/heads/principals etc trained. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 4 Chapter 2: Attendance and promotion rate of the students This section of the report highlights the statistical analysis based on the attendance data collected by MTTs, a three times in year, at the beginning, middle and end of the school year. Schools were selected at random keeping in mind the geographical coverage and gender aspects. Analysis was done by phases and districts too. The data shows the CRI methodology has a positive impact on attendance and grade retention rate as well. 2.1 Daily attendance in % age In old partner schools of Islamabad and Rawalpindi, results show significant change in attendance due to CRI methodology whereas Karachi schools are reflecting low attendance. Security, law & order situation and strikes in Karachi could be of those factors which might be affecting students’ attendance rate. Table 2: Students’ daily attendance in %age (Academic year 2006-07) Districts Phases Boys Girls Total ICT Phase I 81.91 82.70 82.30 ICT Phase O 89.02 91.23 90.12 Rawalpindi Phase O 88.68 90.93 89.80 Karachi Phase O 67.81 72.64 70.22 Table 3: Baseline attendance (2005-2006) of non CRI schools in ICT Districts Phases Boys Girls Total ICT Non CRI Schools 82.66 81.25 81.95 In ICT new partner schools (Phase I), there is improvement in daily attendance rate but not significantly, as teachers were trained in July and August 2006 and the material was delivered to all classrooms by the end of November/December 2006. In new partner schools (Phase I), 1.94% attendance increased among those students who were not attending the schools. The data also shows that girls’ attendance also increased around 4.3% whereas in boys attendance there is a slight decline (0.2%). The target is to improve the attendance up-to 15% among those who are not attending the schools, over the period of 3 years (2006-2009). It has also been observed that overall girls’ attendance was above than boys in “Phase O” and “Phase I” schools. In “Phase I” schools of ICT the daily attendance of the girls A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 5 was 82.70% and boys were on 81.91%. Same trend was observed in “Phase-0” schools of ICT, Rawalpindi and Karachi schools.. 2.2 Retention rate Retention or non-promotion is the practice of requiring a child to repeat a particular grade or requiring a child of appropriate chronological age to delay entry to the next grade (Setencich, 1994). Research shows a large correlation between dropouts and retention. Students who repeat two grades have a probability of dropping out of nearly 100 percent. So it is important to improve the retention rate to decrease the drop-out. FDE introduced students promotion policy to promote all students of KG and Grade I to the next grade since 2004. CRI methodology has visible effect on students’ promotion to the next grade which brings confidence among students as well. According to the baseline information students’ promotion rate to the next grade was 83% (Source: CRI attendance and promotion rate survey 2005-2006). The baseline retention data shows third and fourth graders has been retained more than children in other grades and retention rate declines in grade KG, I & V. Table 4: %age of students enrolled in CRI classrooms and promote to the next grade (2005-2006) Districts Phases %age of students promote to the next grade ICT Phase I -- ICT Phase O 96.71 Rawalpindi Phase O 85.88 Karachi Phase O 91.52 (Source: Attendance and retention survey Sep. 2006) A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 6 96.71% of students enrolled in CRI schools (ICT) were promoted to the next grade, compared to 83% of students in non-CRI classrooms. The students’ regularity and promotion rate shows parents, teachers and heads positive involvement in the child life. This data also shows relax classroom and school environment due to the child centered CRI methodology A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 7 Chapter 3: Early Childhood Classroom Observation (ECCO) Results. ECCO has been adapted by CRI from a structured observation tool developed by National Association for the Education of Young Children (USA) to determine the quality of classroom practices. This information has been gathered by the senior MTTs of CRI who had prior experience of conducting ECCO. In each team there were two MTTs who visited randomly selected schools for ECCO. The MTTs discussed their rating after completing each form. This exercise took 2 hrs times per classroom. This exercise was carried out in CRI and non-CRI classrooms to compare the effect of CRI methodology. For this purpose 10% of the CRI schools were selected at random keeping in the mind the rural and urban stratification where it was applicable. Non-CRI schools were selected by the concerned government departments. According to the aforementioned sample methodology CRI team has selected following number of classrooms for CRI (Phase O and Phase I) and non CRI schools. o 14 classroom of each CRI and non CRI partner schools – Phase I - ICT o 6 classrooms of old partner schools - Phase O - ICT o 6 classrooms of each CRI and non CRI partner schools - Phase O - Rawalpindi o 6 classrooms of each CRI and non CRI partner schools - Phase O - Karachi Due to the children exams in Feb 2007 and CRI trainings in the month of March 2007 team has carried out ECCO in April 2007 and completed it in 26 CRI partner classrooms and 23 Non CRI classrooms which remained slightly below the target. For each classroom a total of 38 items were rated in four categories i.e.: 1) staff interactions with children; 2) curriculum; 3) physical environment; and 4) family participation. The scoring of individual assessment items from 1 to 5 reflected the degree to which the rater observed the criteria in each classroom (1-rarely, almost never; 2-ocassionally; 3- half of the time; 4 most of the time; 5 – almost all of the time). According to the data, CRI partner classrooms (Phase I schools) reflected a measurable achievement in implementing child centered practices. The CRI classrooms were practicing 44.74% of these early education principles most of the time and 42.11% about half of the time. These statistics shows 86.85% of the ECCO criteria was met in A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 8 the CRI schools. Table 5 shows the frequency of early education practices found in CRI classrooms very significantly as compared to non CRI schools. In non CRI school there is only 13.16% of early education principles’ practicing in the classrooms. In fact 86.85 percent of the early childhood classroom practices were observed either “rarely/not at all” (42.11%) or “occasionally” (44.74%) in non CRI schools. Table 5: Frequency of Early Education Principles observed in new partner classrooms in ICT and non-CRI classrooms (in %) Not met Occasionally met Partially met Nearly met Fully met CRI partner classrooms (phase I) 2.63 10.53 42.11 44.74 -- Non CRI classrooms in ICT 42.11 44.74 13.16 -- -- Source: ECCO survey April 2007 Table 6 reflects that CRI partner classrooms (Phase O) are meeting the early education principles criteria up-to 94.7%. The CRI classrooms (Phase 0) were practicing 78.9% of early education principles most of the time and 13.2% almost half a day whereas 97.4% of early education principles were not met in Non CRI schools. Table 6: Frequency of Early Education Principles Observed in old partner classrooms in ICT, Rawalpinidi and Karachi and non-CRI classrooms (in %) Not met Occasionally met Partially met Nearly met Fully met CRI partner classrooms (phase O) -- 5.3 13.2 78.9 2.6 Non CRI 47.4 50.0 2.6 -- -- Source: ECCO survey April 2007 Most frequent Practices Observed in New Partner CRI Classrooms The following 17 classroom practices were observed with the greatest regularity in the CRI schools (ICT) i.e. observed most of the time which depict the child centered classrooms and these classrooms have acquired a broad base of early education principles. Table 7 list these most frequent practices in rank order: A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 9 Table 7: Most frequent practices observed in new partner CRI classrooms (Score above 3.50) Most frequent practices Score Staff provide children of both sexes with equal opportunities to take part in all activities 4.29 Space is arranged to accommodate children individually in small groups and in a large groups. 4.23 All age groups play outdoors daily, weather permitting. 4.08 Staff are available and responsive to children 4.00 Overall sound of group is pleasant most of the time. 4.00 Interact frequently with children showing affection, interest and respect 3.92 Staff speak with the children in a friendly, courteous manner 3.92 Variety of age-appropriate materials and equipment are available for children indoors and outdoors. 3.85 Staff encourage independence in the children as they are ready 3.85 Encourage language and literacy development. 3.85 Space is arranged to facilitate a variety of activities. 3.85 Developmentally appropriate materials and equipment are available. 3.77 Staff develops socials skills. 3.77 Staff conducts smooth and unregimented transitions between activities. 3.69 Staff talk with individual children and encourage children of all ages to use language 3.69 Staff treat children of all races, religions, family backgrounds and culture equally with respect and consideration 3.54 Encourage children to think, reason, question and experiment. 3.54 Progress made by CRI classrooms compared to Non CRI classrooms According to ECCO data CRI program brought significant changes in student- teacher interaction, social and academic environment in the classroom along with the family participation. Fig. 1: Average score by categories 3.8 3.2 3.4 2.8 4.0 3.7 3.4 3.6 2.0 1.6 1.2 1.6 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 New CRI schools Old CRI school Non CRI schools Interaction amon staff and children Curriculum Physical env Family participation A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 10 The figure-2 shows that New CRI partner classrooms gained 3.23 average score and Old partner classrooms gained 3.62 whereas the non CRI classroom scored 1.61. The difference attributable to CRI interventions in new partner schools is equal to 1.60 as represented by the diagram which is around 47% and the changes in old partner schools is around 2.01 score (59.3%) Fewer score depicts the prevailing situation of teacher directed classrooms instead of child centered in Non CRI schools. They treat children as a large group rather then taking them as individualized manner to promote children leanings. Figure 2: Difference between the mean score of CRI and Non CRI schools 3.23 3.62 1.61 0 1 2 3 4 5 New partncer classrooms Old partner classrooms Non CRI classrooms Average score Average score Score 1 show none of the aspects of a developmentally appropriate classroom valued by the CRI approach whereas score 5 shows schools where all criteria are observed almost all of the time. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 11 Table 8: Areas of change made by CRI methodology New Partner CRI Classrooms Old Partner CRI Classrooms Score Differential Score Differential Staff provide children of both sexes with equal opportunities to take part in all activities 2.87 1.21 Space is arranged to accommodate children individually in small groups and in a large groups. 2.66 2.12 Space is arranged to facilitate a variety of activities. 2.59 2.74 Developmentally appropriate materials and equipment are available for preschoolers. 2.47 2.39 Staff encourage independence in the children as they are ready 2.37 2.29 Develop socials skills. 2.04 2.35 Staff conducts smooth and unregimented transitions between activities. 2.04 2.35 Enhance physical development. 1.98 2.06 Overall sound of group is pleasant most of the time. 1.87 2.33 Foster positive self concept. 1.83 2.52 More than one option for group activity (individual, small group, or large group) is available most of the day. 1.81 2.73 Encourage children to think, reason, question and experiment. 1.8 2.11 A balance of large muscle / small muscle activities is provided in the daily schedule. 1.75 2.14 Staff does not use physical punishment or other negative discipline methods that hurt, frighten or humiliate children. 1.72 2.64 Staff are available and responsive to children 1.7 2.08 Staff speak with the children in a friendly, courteous manner 1.7 1.93 The schedule provides for alternative periods of quiet and active play. 1.7 2.55 Staff talk with individual children and encourage children of all ages to use language 1.69 2 Multiracial, nonsexist, non stereotyping pictures, dolls, books and materials are available. 1.68 2.29 Encourage language and literacy development. 1.68 1.83 Encourage creative expression and appreciation for the arts. 1.66 2.74 Staff provides materials and time for children to select their own activities during the day. 1.64 2.64 Private areas where children can play or work alone or with a friend are available indoors and outdoors. 1.55 1.78 Interact frequently with children showing affection, interest and respect 1.53 1.76 All age groups play outdoors daily, weather permitting. 1.53 1.53 A balance of child-initiated / staff initiated activity is provided while limiting the amount of time spent in large group, staff initiated activity. 1.51 2.05 Staff uses positive approaches to help children behave constructively. 1.45 1.84 Staff is flexible enough to change planned or routine activities. 1.37 1.99 Routine tasks such as toileting, eating, dressing and sleeping are handled in a relaxed and individualized manner. 1.32 2.01 The environment includes soft elements. 1.31 1.31 Staff treat children o all races, religions, family backgrounds and culture equally with respect and consideration 1.28 2.28 Program has procedures for orienting parents and children. 1.27 2.58 Opportunities are available for family members to become involved in the 1.2 2.05 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 12 New Partner CRI Classrooms Old Partner CRI Classrooms Score Differential Score Differential program. Staff uses a variety of informal communication strategies with parents. 1.1 1.64 Respect cultural diversity. 0.96 1.65 Staff uses a variety of formal communication strategies with parents. 0.95 1.64 Encourage and demonstrate sound health, safety and nutritional practices. -0.16 0.45 These classroom measures showing the areas of greatest change in CRI classrooms are also exemplars of the teaching/learning approach promoted by CRI training for the development of child centered classrooms and activity based methods. The number of items and their sizable differences in terms scores are strong indicators of the strides made by the CRI teachers to put their training into practice in their classrooms. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 13 Chapter 4: Technical Assistance Observation Results: This report is based on the classroom observation forms completed by Master Teachers Trainers (MTTs). The classroom observation form is used to monitor the physical environment, parents’ involvement, and adaptability of child centered practices. The forms are filled by MTTs during their technical assistance visits to CRI classrooms on a bimonthly basis. The data in this report was collected during the month of September and October 2006 and January-February 2007 from CRI Old partner as well as new partner schools. Table 9: Average classroom score by district: Islamabad New Partner Schools Islamabad Old Partner Schools Rawalpindi Old Partner Schools Karachi Old Partner Schools Total of Old partner schools September-October 2006 *1 7.49 12.66 15.12 16.08 14.62 January-February 2007 *2 10.92 13.37 18.38 18.77 16.84 Change ( points) 3.43 0.71 3.26 2.69 2.22 Source: *1. Classroom Observation data (First TA) *2 Classroom Observation data (3rd TA – Jan/Feb 2007) As reflected in table 9, in new partner schools the average score of first TA was 7.49 and in the 3rd TA it was 10.92 against the maximum points of 23. This shows the change for ICT new schools is 3.43 points. This represents a good rate of progress within the five months after training of Grade 1 and Grade II teachers to implement quality of child centered practices. But it needs a lot of effort to attain the maximum points 23. As far as Old CRI school are concerned the overall points was 14.62 during the first TA and found 16.82 the second TA (January-February 2007). There was a change of 2.22 points. According to the table 1, Karachi schools are leading with the average score of 18.77, followed by Rosalinda with an average score of 18.38 against the target of 23 points whereas Islamabad schools have an average score of 13.37. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 14 Table 10: Average classroom score and percentage by items and district during First TA: Islamabad - New Partner Schools Islamabad Old Partner Schools Rawalpindi Old Partner Schools Karachi Old Partner Schools Score gained %age gain Score gained %age gain Score gained %age gain Score gained %age gain Physical environment 2.16 24.00 6.28 69.83 6.39 71.00 6.71 74.60 Child centered Practices 5.02 41.83 6.15 51.27 7.78 64.84 8.40 69.98 Parent involvement 0.31 15.50 0.22 11.02 0.95 47.56 0.97 48.55 Total average score/%age gain 7.49 32.54 12.66 55.03 15.12 65.75 16.08 69.92 Source: Classroom Observation data (First TA) During the First TA, Table 10 shows new partner schools gained overall 32.54 percentage points in comparison to old schools in Islamabad which gained 55.03 percentage points. The Rawalpindi schools showed a gain of 65.75 percentage points where as Karachi schools were leading by a gain of 69.92 percentage points. Table 11: Average classroom score and percentage by items and district during 3rd TA: Islamabad - New Partner Schools (Phase I) Islamabad Old Partner Schools (Phase 0) Rawalpindi Old Partner Schools (Phase 0) Karachi Old Partner Schools (Phase 0) Score gained %age gain Score gained %age gain Score gained %age gain Score gained %age gain Physical environment 4.79 53.21 5.94 65.97 7.67 85.19 7.53 83.72 Child centered Practices 5.98 49.85 7.07 58.93 9.49 79.06 9.84 81.98 Parent involvement 0.14 7.24 0.36 17.86 1.23 61.54 1.40 69.77 Total average score/%age gain 10.92 47.46 13.37 58.11 18.38 79.93 18.77 81.60 Source: Classroom Observation data (3rd TA – Jan/Feb 2007) A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 15 Table 11 shows new partner schools gaining overall 47.46 percentage points in comparison to old schools in Islamabad which gained 58.11 percentage points. It requires a lot of improve through regular Technical Assistance. The Rawalpindi schools showed a gain of 79.93 percentage points where as Karachi schools were leading by a gain of 81.60 percentage points. The parents’ involvement score appears to be the lowest followed by child centered practices in ICT schools. Conclusion According to the study results (3rd o In order to improve the progress further, it is important to make sure the provision of technical assistance visits by MTTs on regular basis. TA), new partner schools scored overall 47.46 percentage points in comparison to old schools in Islamabad which scored gained 58.03 percentage points which reflects that old partner schools in ICT are not performing well. According to the data for which MTTs must concentrate on old schools along with the new partner school. FDE should also ensure the quality aspects by sending its own monitors to classrooms on regular basis. The Rawalpindi schools showed a score of 79.93 percentage points whereas Karachi schools were leading by the score of 81.60 percentage points. These statistics shows CRI partner classrooms need further attention during the TA visits to attain at least 80% target of applying child centered methods in the classrooms. Recommendations: On the basis of the TA results, following recommendations are made: o FDE should assign their own supervisors to monitor the work of its teachers and classrooms’ environment on regular intervals. o Head teachers should also be motivated to play their role in ensuring the adaptation of interactive teaching and learning methodologies by their teachers during the technical assistance. o Each MTT should provide tasks sheet to the teacher during their technical assistance visits and ensure its follow-up during next visits. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 16 Chapter 5: Heads, Teachers and Parents surveys Heads, teachers and parents self administered surveys were conducted in each region. 10% of CRI schools were selected randomly to carry out this exercise. CRI teams were provided a list of randomly selected schools. Teams visited each school and handed over questionnaires to the head/principal. They were requested to fill-in his/her forms and make sure that their CRI trained teachers filled the form and return to CRI office along with parents filled forms from their respective classes. In each new partner school at least two teachers’ forms were filled whereas in old partner schools at least 5 teachers of different grades filled the form. 10 parents of CRI classroom’s students filled the form in each school. Most responses were coded into one of five categories best describing the extent of changes the respondents had experienced or seen as a result of the CRI intervention: Much more; More; No change; Less; and More less. Much more and more categories reflect the positive change in areas associated with the work of CRI in developing quality early education in Pakistan. Table 12: Sample Distribution by Districts Districts Teacher (CRI trained teacher in each schools) Principal/Heads Parents (10 parents from each CRI partner schools) Islamabad (New partner) 28 14 14 schools x *10 parents= 140 Islamabad (Old parents) 25 5 5*10=50 Rawalpindi (Old parents) 25 5 5*10=50 Karachi (Old parents) 25 5 5*10=50 Total 103 29 290 In heads and teachers’ survey most of the items are similar and there is correlation among the responses of the heads and teachers so that the data is presented under the following sections: 5.1 Heads and teachers Reponses 5.2 Parents responses 5.1 Heads and teachers responses As mentioned in the above table 12, 14 heads and 28 teachers from the new partner schools gave feedback regarding the change in schools in lieu of CRI program. On the whole heads and teachers were positive about the level of improvement. Overall 76.2% A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 17 heads and 79.2% teachers said there is a positive change in the schools due to CRI interventions and only 2.4% heads and 1.7% teachers said there is no change. 21.4% heads and 19.1% teachers’ responses indicating almost negative view of CRI implementation by combining less or less categories. Table 13: Overall results of the heads/teachers (Phase-1 schools) Rankings Heads responses in %age Teachers responses in %age Much more 33.3 35.9 More 42.9 43.3 No change 2.4 1.7 Less 12.4 12.9 More Less 9 6.2 Total 100 100 As for as Old partner schools are concerned, almost the same results were come out with a better percentage. 78.4% heads’ and 84.7% teachers’ responses indicating improvement in the schools. CRI interventions were undertaken in these schools in 2002 whereas the in aforementioned schools CRI interventions introduced in 2006. Table 14: Overall results of the heads/teachers (Phase-0 schools) Heads responses in %age Teachers responses in %age Much more 35.3 43.7 More 43.1 41 No change 2.2 0.8 Less 14.4 11.1 More Less 5 3.3 Total 100 100 5.1.1 Positive responses about the educational program In New CRI Partner schools heads and teachers responses show positive change in nineteen items whereas in the following three items the score was generally low: o Parents visit and participate in the classroom. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 18 o There are enough adults in the class to work with individual or small groups of children o Learning activities occur outdoors The results in the table are in rank order according to the responses of the schools heads. ‘Much more’ and ‘more’ categories are clubbed into one to show the results meaningful. ‘Less’ and ‘more less’ categories were not reflected in the table which can be extracted by excluding much more, more and no change results from 100. Table 15: Positive responses about the educational program (Phase I) S.# Items Much more & more (response in %) No change (Response in %) Head Teacher Head Teacher 1. The mood in the classroom is relaxed 100.0 100.0 2. Children in the classroom are busy 100.0 100.0 3. Children like to come to school 100.0 100.0 4. Children feel good about what they can do 100.0 95.0 5 5. Classroom activities prepare children to read 100.0 95.0 5 6. Girls' school attendance is good 100.0 94.1 5.9 7. Children show respect for the adults in the classroom. 100.0 75.0 5 8. Boys' school attendance is good 100.0 72.2 9. Children in my classroom will return to school next year 85.7 95.0 5 10. Activity centers reflect themes that the children are interested in 85.7 80.0 14.3 11. There is a variety of learning materials in the classroom. 85.7 75.0 12. Children do things and figure out things for themselves 85.7 65.0 5 13. Children gain academic skills that they will need for their continued education 85.7 61.1 14. Children express their feelings 71.4 90.0 15. Children make choices about what they do in the classroom 71.4 68.4 16. The classroom contains activity centers 71.4 26.3 17. Children use critical thinking skills 71.4 73.7 5.3 18. Children treat one another with respect 71.4 45.0 10 19. Adults pay attention to the children's questions and comments 71.4 75.0 20. Learning activities occur outdoors 42.9 45.0 21. There are enough adults in the class to work with individual or small groups of children 28.6 30.0 5 22. Parents visit and participate in the classroom. 28.6 15.0 5 Mean 79.86364 71.62727 0.65 2.6 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 19 As depicted in table 15, average score of heads and teachers seem close which shows both have the same view point regarding the change in schools. In other words they have interpreted that much of the CRI methodology is into action. For further discussion these results may also be converted into 4 outcomes: I) Child outcomes; ii) Adult-child interaction and environment; iii) Educational outcomes; IV) Attendance outcomes. i) Child outcomes Overall average score of heads and teachers regarding the child outcomes seem close which shows both have the same view point regarding the change in schools, as mentioned in table 8. Most of the child outcomes are being met, according to heads and teachers responses. The following results show that child outcomes exhibit a high degree of positive change, in the perceptions of both the heads and teachers. This shows the physical and psychological environment of the CRI classes is welcoming. Only a slight difference on the extent of change occurs on three items. Heads expressed a higher degree of certainty in two items than did the teachers that the Children show respect for the adults in the classroom. (100% VS 75%) and Children treat one another with respect (71.4% VS 45%). 90% Teachers and 71.4% heads reported that children express their feelings. As teachers spent most of the time with the children so their assessment probably has more validity. Table 16: Child outcomes Items Head (Response in %) Teacher (Response in %) Children like to come to school 100.0 100.0 Children show respect for the adults in the classroom. 100.0 75.0 Children feel good about what they can do 100.0 95.0 Children express their feelings 71.4 90.0 Children treat one another with respect 71.4 45.0 Mean 88.6 81.0 ii) Adult-child interaction and environment; Again responses of heads/teachers are identical in 4 items. Among them three were showing higher degree of positive change as indicated in table 9. Both groups show the low prevalence of parents involvement (28.6% VS 15%); enough adults in the class to work with individual or small groups of children (28.6% VS 30%); and learning activities occurs outdoors (42.9% VS 45%). Discrepancies in the views of A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 20 heads and teachers appeared for two items i.e.: Parents visit and participate in the classroom; the classroom contains activity centers. Table 17: Adult-child interaction and environment Items Head (response in %) Teacher (Response in %) The mood in the classroom is relaxed 100.0 100.0 There is a variety of learning materials in the classroom. 85.7 75.0 Activity centers reflect themes that the children are interested in 85.7 80.0 Adults pay attention to the children's questions and comments 71.4 75.0 The classroom contains activity centers 71.4 26.3 Learning activities occur outdoors 42.9 45.0 There are enough adults in the class to work with individual or small groups of children 28.6 30.0 Parents visit and participate in the classroom. 28.6 15.0 Mean 64.3 55.8 iii) Educational outcomes According to heads and teachers’ data most educational outcomes are being met. Overall results are positive regarding educational outcomes. In three cases the results were identical and positive and in other three cases these are positive but not identical among the two groups (heads and teachers). As per data activities are designed to keep children busy, prepare them to read, help them make individual choices and develop critical thinking skills, and gain other academic skills. At 61.1%, it is curious that not more teachers report that the CRI program is providing children with academic skills needed for their future, as the heads seem quite assured that it is 85.7%. Perhaps future teacher training and technical assistance can re-emphasize how the child-centered classroom promotes children’s intellectual development. Table 18: Educational outcomes Items Head (Response in %) Teacher (Response in %) Classroom activities prepare children to read 100.0 95.0 Children in the classroom are busy 100.0 100.0 Children do things and figure out things for themselves 85.7 65.0 Children gain academic skills that they will need for their continued education 85.7 61.1 Children use critical thinking skills 71.4 73.7 Children make choices about what they do in the classroom 71.4 68.4 Mean 85.7 77.2 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 21 iv) Attendance outcomes Heads and teachers report high attendance among girls. As for as boys are concerned 100% heads and 72.2% teachers stated that boys attendance is good. Retention by boys and girls almost alike in CRI classrooms, as evidenced by the following responses: Items Head (response in %) Teacher (response in %) Boys' school attendance is good 100.0 72.2 Girls' school attendance is good 100.0 94.1 Children in my classroom will return to school next year 85.7 95.0 5.1.2 Percentage of positive responses about teachers practices Following statistics indicate that heads and teachers believe CRI program has made progress in improving teaching and learning in the classrooms. Team work is in compliance, with all of the heads and nearly all of the teachers reporting that teachers share their ideas; information and problems with other school staff. Majority of head teacher permits teachers to make decisions about classroom environment and activities. All the heads observed that teachers assess children’s individual skills and interest. Mostly teachers know how to set up classroom to keep children active and interested and to adapt activities to children’s skills and interests. Table 19: Percentage of positive responses about teachers practices Heads responses in % Teachers responses in % Team work Team work Teacher work and plan as a team 100% I share my ideas and information about my classroom 100% Teachers share information and solve problems with one another 100% I plan activities and share responsibilities with other adults in my classrooms 94.1% I discuss my ideas and problems with my head teacher/principal 95% Decision making Decision making Teachers are permitted to make decisions about classroom environments and activities 100% I enjoy trying out new ideas for classroom activities 100% I make good decisions in the classroom 100% A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 22 Heads responses in % Teachers responses in % Individualization Individualization Teachers assess children’s individual skills and interest 100% I know how to set up my classroom to keep children active and interested 95% I know how to adapt activities to children’s skills and interests 100% I notice differences in individual children’s skill and interests 95% I understand how children grow and develop 100% 5.1.3 Community Involvement Table 20 shows responses of heads/principal regarding community involvement in the school and classrooms. Which is not showing much progress even in the Phase I and Phase 0 schools. Unlike the responses reported in table 7, several of these items received significant amounts of negative response. Table 20: Community involvement (Phase I) Much More & more No Change Less More Less People in the community have donated goods and services to the Schools. 33.3 33.3 33.3 People in the community visit and participate in classroom activities. 28.6 57.1 14.3 The staff helps parents locate services in the community to address family needs. 66.7 16.7 16.7 The school building is used by the community 16.7 16.7 66.7 Table 21: Community involvement (Phase 0) Much More & more No Change Less More Less People in the community have donated goods and services to the Schools. 26.6 13.3 33.3 40.0 People in the community visit and participate in classroom activities. 29.4 17.6 41.2 11.8 The staff helps parents locate services in the community to address family needs. 76.4 17.6 5.9 The school building is used by the community, 13.4 20.0 26.7 40.0 These results indicate the need for CRI to focus on and continue to develop schools’ abilities to strengthen their bonds through the community. It is hoped that schools will reach out to the community. CRI seeks the inclusion of community because the whole A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 23 community must understand the CRI goals and support them if the changes in the early education of children are to be sustainable. 5.1.4 Important factors for the success of the program In the end of ‘heads and teachers questionnaires’ 6 items were asked to know the important factors which play a significant role to make the program successful. In the following figure, there was not a single answer reported as “not important” from the heads. In heads view point all the factors i.e. classroom material, ongoing support from CRI, classroom resource book, training, furniture, support form the ministry of education are important factors for the success of the program. However top three factors are considered utmost important for the successful implementation by the heads. These are ongoing support from CRI (100%) and “classroom material (100%)” followed by the support from the ministry of education (85.7%) etc. Fig. 3: Important factors for the success of the program (Phase I) (Heads responses) 57.1 42.9 0 100 0 0 28.6 71.4 0 100 0 0 42.9 57.1 0 85.7 14.3 0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 % Furniture Classroom materials Training Ongoing support from CRI Creating Child Centered Classroom resource book Support from the local Ministry of Education Critically Important Important Not important A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 24 Teachers were also having the same opinion as mentioned by the heads that classroom materials and ongoing support followed by support from the ministry etc were the critically important factors for the success of the program. Fig. 4: Important factors for the success of the program (Phase I) (Teachers responses) 60 40 0 90 10 0 65 30 5 90 10 0 75 25 0 80 20 0 0 10 20 30 40 50 60 70 80 90 % Furniture Classroom materials Training Ongoing support from CRI Creating Child Centered Classrom resource book Support from the local Ministry of Education Critically Important % Important % Not important % Conclusion: Overall heads and teachers were positive about the level of improvement. 76.2% heads and 79.2% teachers said there is a positive change in the schools due to CRI interventions in Phase I. In other words they have interpreted that much of the CRI methodology is into action. However groups (heads and teachers) show the low prevalence of parents involvement (28.6% VS 15%), enough adults in the class to work with individual or small groups of children (28.6% VS 30%); learning activities occurs outdoors (42.9% VS 45%); classrooms contains activity center (26.3%). A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 25 CRI team will try to improve the aforementioned characteristics by advising teachers and heads on how to make a strategy to reach out more effectively to both families and communities. CRI team will also advise to the teachers and head to improve the situation regarding learning activities occurs outdoors and classrooms contain activity centers. 5.2 Parents responses This study was undertaken only in CRI schools after the implementation of one year of the program for ‘phase I’ and 6 years for ‘phase 0’. The objective of this study was to get feedback about changes in children and family participation in child school life since introduction of CRI methodologies in the classroom. Parents questionnaires were delivered to the head teachers and made request to give them to the parents, around 10 questionnaires were filled in randomly selected schools. A total of 223 parents responded to 19 questions. Parents answered these questions using one of five response categories (Much more, More, No change, less, and much less). In new partner schools (Phase I), Table 14 shows that parents were very positive about the implementation of CRI methodology. By combining the score for “much more and more”, 80.7% of the parents reported positive changes as a result of CRI intervention. Report of no change was only .8 percent. Responses indicating a negative view of CRI implementation were reported by 18.5% Table 22: Summary results of the parents survey (Phase I) Much More More No Change Less More less All items 47.6 33.1 0.8 13.6 4.9 In old partner schools by combining ‘much more’ and ‘more’ categories 87% parents were of the positive view about the implementation of CRI methodology. Only .5% parent said that there was no change and only 12.5% parents reported negative view of CRI implementation. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 26 Table 23: Summary results of the parents survey (Phase 0) Much More More No Change Less More less All items 49.1 37.9 0.5 8.5 4.0 Table 24 indicates that parents reported high positive response in 15 items out of 19 items ranging from 72.2% to 100%. As table 14 much more and more responses have been combined, since together they signify the percentage of parents who attributed positive changes to the aims and outcomes of the CRI programs. The majority of the parents experienced significant positive changes in outcomes promoted by the CRI program. 100% parents stated that they felt welcome when they visited the classroom. Overall 80.7% of parents saw positive changes in the outcomes associated with the CRI program, indicating how deeply the CRI methodology has been translated into action. On average there is 0.8% parents said there is no change whereas negative responses are dealt with a separate table 24. Table 24: % of positive parents’ responses (Phase I) Much More + more No Change I feel welcome when I visit the Classroom. 100.0 I feel like my ideas are respected at the school. 98.6 I enjoy being with my child. 98.6 My child goes to school every day when he is well. 98.6 1.4 My child likes to go to School. 95.8 1.4 My child does things and figures out things for her/himself. 94.4 I feel like I make good decisions about my child. 94.3 I interact with and talk to my child. 93.1 2.8 My child receives a good education at School. 93.1 2.8 I know how my child grows and develops. 85.9 1.4 My Child expresses emotions. 83.3 1.4 I talk to the teacher(s) about my child. 73.2 I know about what my child does at School. 77.8 My child expresses his/her opinions. 77.5 1.4 My child makes good choices about what he wants. 72.2 I Visit and participate in my child's Classroom. 56.9 I receive support from other parents. 47.7 1.5 I participate in activities for families at School. 44.3 I share information about child rearing with other parents. 43.7 1.4 Mean 80.7 0.8 Table 25 indicates higher degree of positive response regarding the changes in outcomes promoted by the CRI program. Overall 87.5% of parents saw positive changes in the outcomes associated with the CRI program, indicating how deeply the A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 27 CRI methodology has been translated into action. On average there is 0.5% parents said there is no change. Table 25: % of positive parents’ responses (Phase 0) Much More + more No Change My child goes to school every day when he is well. 100.0 I feel like my ideas are respected at the school. 98.6 My child likes to go to School. 98.6 0.7 I feel welcome when I visit the Classroom. 98.0 My child receives a good education at School. 98.6 My Child expresses emotions. 95.2 I know how my child grows and develops. 92.5 I enjoy being with my child. 91.9 0.7 My child expresses his/her opinions. 91.8 I feel like I make good decisions about my child. 91.2 I interact with and talk to my child. 91.1 2.1 My child does things and figures out things for her/himself. 90.5 I talk to the teacher(s) about my child. 87.8 My child makes good choices about what he wants. 83.4 0.7 I know about what my child does at School. 82.1 I Visit and participate in my child's Classroom. 72.3 0.7 I share information about child rearing with other parents. 70.7 1.4 I participate in activities for families at School. 65.5 0.7 I receive support from other parents. 52.8 2.1 Mean 87.5 0.5 The aforementioned items of table 24 can be divided into three categories for further analysis. These relate to the benefits to children, to parents, and to the children’s education. Benefits to Parents In expressing their opinions about outcomes promoted by CRI which related to parents themselves from 85.9 to 100 %, which is a positive change in their own knowledge, behavior and feelings as shown in the following table. Items Much more+ more Parents responses in % I feel welcome when I visit the classroom. 100.0 I enjoy being with my child. 98.6 I feel like I make good decisions about my child. 94.3 I know how my child grows and develops. 85.9 Benefits to children 77.5% to 95.2 % parents attributed positive changes in this area, as described by the following table. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 28 Items Much more+ more Parents responses in % My Child expresses emotions. 95.2 My child does things and figures out things for her/himself. 94.4 I interact with and talk to my child. 93.1 My child makes good choices about what he wants. 83.4 My child expresses his/her opinions. 77.5 Benefits to children’s education Parents also attributed to the CRI program positive changes related to the educational program. From 73.2% to 98.6% responded positively, as shown in the following responses. Items Much more+ more Parents responses in % My child goes to school every day when he is well. 98.6 I feel like my ideas are respected at the school. 98.6 My child likes to go to School. 95.8 My child receives a good education at School. 93.1 I know about what my child does at School. 77.8 I talk to the teacher(s) about my child. 73.2 Negative parents’ responses Table 26 indicated 10 items from the survey attributed negative changes to the CRI program. These items are listed in rank orders. These items range across all of the outcomes, some relate to the children and others to the parents and the educational program/schools. Table 26: Phase I- Negative parent’s responses in %age Less More less Less + More less % % % I participate in activities for families at School 32.9 22.9 55.7 I share information about child rearing with other parents. 40.8 14.1 54.9 I receive support from other parents. 38.5 12.3 50.8 I Visit and participate in my child's Classroom. 26.4 16.7 43.1 My child makes good choices about what he wants 23.6 4.2 27.8 I talk to the teacher(s) about my child 22.5 4.2 26.8 I know about what my child does at School 18.1 4.2 22.2 My child expresses his/her opinions 18.3 2.8 21.1 My Child expresses emotions 11.1 4.2 15.3 I know how my child grows and develops 7.0 5.6 12.7 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 29 Keeping in mind the above table, it is important for CRI to think about the strategies designed to increase parents’ involvement. With time and some additional focused strategies, implementation of the CRI methodology should be able to affect more change with parents as it continuous work in schools. Conclusion: The results of the heads, teacher and parents’ survey clearly indicate that the CRI methodology has been embraced by schools, teachers and parents in the target areas. In its first year of the expansion phase CRI has instigated great changes in the ways that young children learn. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 30 Chapter 6: Family literacy-Pre& Post inventories results CRI designed and implemented a family literally program in CRI partner schools with non-literate and low-literate families of children in CRI’s program. The objective of the lessons is to provide them an opportunity to learn basic literacy and math skills. It is very important to make a relationship between their homes and their schools. With that in mind, the literacy lessons link the classroom learning to the home where families reinforce their own learning with that of their children. The Family Literacy Program offers 100 lessons in basic literacy and math skills developed by literacy experts. These lessons are divided over two sessions, each of six months duration. The first session is devoted to practices in lessons 1-50 and the second session to practice in 51-100 lessons. The lessons include reading, writing, speaking, listening and computation activities for parents. Two partner schools teachers’ form each school is trained in adult learning These are the same teachers who teach children in the morning. The family literacy inventory (1-50 lessons) has two parts. In part I there is 10 items related to Urdu literacy and basic math skills. The highest possible score is 50. In part II of the inventory there is only three open ended questions. The family literacy inventory (51-100 lessons) has the same aforementioned two sections in addition to ‘dictation’ and ‘reading’ of Urdu paras. Table 27 shows that before attending the family literacy classes only 19% respondents scored in between 41 to 50 whereas after taking the literacy sessions 83.6% scored in between 41-50 which shows a significant change among the non-literate and low￾literate families in CRI’s program in respect of Urdu literacy and basic Math skills. Table 27: Comparison of scores -Pre and Post Situation of Family Literacy Lessons score Pre Post Percent respondents Percent respondents 0-10 21.5 - 11-20 29.3 0.53 21-30 19.6 3.18 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 31 score Pre Post Percent respondents Percent respondents 31-40 10.6 12.7 41-50 19.0 83.6 100 100 The average scored by the participants before attending the family literacy sessions was 24.15 and after attending the sessions it was 46.5 which again shows a significant change among the learners in respect of their Urdu literacy and basic math skills. For further analysis data is analyzed by average score per test item. A comparison of table 28 with its maximum score represents the average score for each item. i.e. the average percentage of the sample that got the item correct. The score 75.1% correct in identifying the written names of the days of the week. Letter names and addition & subtraction were on the lowest side. Table 28: Average score by items (Baseline) Items Average Max. Average % of respondents with correct score Day 2.254 2 75.1 Counting with objects 2.203 3 73.4 What number is this 4.399 8 55.0 Name writing 2.939 6 49.0 Counting 2.119 4 53.0 Sentence reading 1.585 3 52.8 Hearing sounds 1.833 4 45.8 Rhyming words 1.772 4 44.3 Letter names 3.788 12 31.6 Addition and subtraction 1.257 4 31.4 Total 24.15 50 48.3 Table 29 reflects that almost all the items were above 89% with correct score. Counting with objects and to identify the names of the days of the week was on top i.e. 99% with correct answers. Remaining items were also on the higher side of the correct answers. Table 29: Average score by items (follow-up results) Items Average Max. Average % of respondents with correct score Counting with objects 3.06 3 99.0 Day 1.98 2 99.0 Counting 3.82 4 95.5 Sentence reading 2.86 3 95.2 Name writing 5.67 6 94.6 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 32 Items Average Max. Average % of respondents with correct score What number is this 7.438 8 93.0 Addition and subtraction 3.713 4 92.9 Rhyming words 3.658 4 91.4 Hearing sounds 3.565 4 89.1 Letter names 10.78 12 89.8 Total 46.55 50 92.8 As table 30 shows parents scores increased for all items and these increases ranged from 61.4 to 25.6 percentage points. For the most part, parent made their greatest gains on those literacy tasks where baseline scores had shown greatest weaknesses, most notably in addition & subtraction which is increased by 61.4% points, letter names (58.3%), rhyming words (47.2%) and so on. The extent of the gains per item is so impressive that the progress made by parents can also be presented as a factor of change over time. The lesser increases over time related to those tasks that respondents were more successful with prior to the CRI literacy lessons i.e. days, counting with objects. Table 30: Change over time due to CRI FLP Items %age change over time Addition and subtraction +61.4 Letter names +58.3 Rhyming words +47.2 Name writing +45.6 Hearing sounds +43.3 Counting +42.5 Sentence reading +42.4 What number is this +38.0 Day +23.9 Counting with objects +25.6 In six months the CRI family literacy curriculum increased parents knowledge in the full complement of literacy tasks assessed. If the parent population attending this CRI family literacy program is representative of parents elsewhere, and the instruction itself is as effective, there is very good reason to expect that parent literacy would successfully increase in the target area in a relatively short timeframe by means of replication of the CRI family literacy program. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 33 Overall results of the studies encourage CRI to continue the methodology. They also provide guidelines for the planning and development of the next phase. A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 34 Chapter 7: Annual statistics (1st March 2006 – 28th February 2007) According to M&E database, CRI has trained 1454 teachers/head teachers of partner schools and 429 teachers/head teachers of other organizations in this year. For these purpose 18 trainings has been conducted for partner schools’ teachers/head teachers and 16 trainings for other organizations’ teachers/heads. The target was to train 1126 teacher/heads teachers in first year of expansion phase which was over achieved. Children's Resources International, Pakistan Facts and Figures Sheet 1st March 2006-28th February 2007 Basic Education Districts Islamabad Rawalpindi Karachi Total Existing Partner Schools 140 140 Partner Classroom 471 471 Beneficiary Children 18840 18840 Type of Trainings Districts No. of Trainings Male Female Total Initial Training Islamabad 11 222 920 1142 Rawalpindi 41 77 118 Karachi 1 1 88 89 Sub Total 12 264 1085 1349 Fallow up Training Islamabad 1 0 Rawalpindi 0 Karachi 1 0 Sub Total 2 0 0 0 Grand Total 14 264 1085 1349 Orientation Training Islamabad 6 19 258 277 Rawalpindi 38 85 123 Karachi 0 Sub Total 6 57 343 400 Grand Total 20 321 1428 1749 No. of trainings Participants Facility training (HEC) Family Involvement Training Workshop Male Female Total 0 Bi-Monthly Meetings with Heads on Parental involvement Districts Islamabad Rawalpindi Karachi Total A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 35 No. of Meetings 1 1 1 3 Training for Trainers Workshop Male Female Total 0 Family Literacy Districts Islamabad Rawalpindi Karachi Total Existing Partner Schools 56 25 15 96 No. of learners 899 406 468 1773 Type of Trainings Districts No. of Trainings Male Female Total Training Islamabad (1-50 lessons) Rawalpindi 1 3 102 105 Karachi 0 Sub Total 1 3 102 105 Training Islamabad (51-100 lessons) Rawalpindi 0 Karachi 0 0 Sub Total 0 0 0 0 Orientation Training Islamabad Rawalpindi 2 20 70 90 Karachi 1 0 37 37 Sub Total 3 20 107 127 Total 4 23 209 232 Monthly Family Literacy Meetings with Trainers Districts Islamabad Rawalpindi Karachi Total No. of Meetings 4 4 4 12 Other Organizations (Non-CRI) Organizations Districts No. of Trainings Male Female Total Army Public School All 1 5 6 NISE (Special Education) Islamabad 2 20 24 44 Girls Guide Association Islamabad 2 81 81 PPAF-CUP Bannu 3 11 38 49 Fatima Jinnah University Rawalpindi 1 50 50 PPAF-SAFWCO Hyderabad 3 0 80 80 PPAF-NRSP Mianwali 1 23 23 PPAF-NRSP Khushab 1 20 20 PPAF-NRSP Bahawalpur 1 18 19 37 PPAF-Taraqee Foundation Sibbi 1 7 2 9 Creative Associates USA Balakot 1 18 12 30 Partners From Afghanistan / Bangaladesh 0 DEO/AEO/Educationist Islamabad 0 Parents, SMC/PTA members Islamabad 0 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 36 Donors, Inistitues & NGOs Islamabad 0 others Islamabad 0 0 0 0 0 Total 16 75 354 429 Summary of teachers trained and trainings imparted CRI Other org. Total No. Training Conducted 18 16 34 No. Teachers Trained 1454 429 1883 A Report on Evaluation Findings- March 2006 to February 2007 Interactive Teaching and Learning Program in Pakistan 37 Chapter 8: Conclusion and recommendations 8.1 Conclusion: Overall results encourage CRI to continue the methodology. They also provide guidelines for the planning and development of the next phase. 8.2 Recommendations: In light of the experience from the project, following measures are recommended: o In order to improve the progress further, it is important to make sure the provision of technical assistance visits by MTTs on regular basis. o FDE should assign their own supervisors to monitor the work of its teachers and classrooms’ environment on regular intervals. o Head teachers should also be motivated to play their role in ensuring the adaptation of interactive teaching and learning methodologies by their teachers during the technical assistance. o Learning aid material should be provided well in time to each classroom after the trainings. o Each MTT should provide tasks sheet to the teacher during their technical assistance visits and ensure its follow-up during next visits. o Teachers and heads should be advised on how to make a strategy to reach out more effectively to both families and communities. o Services of a professional development worker must be employed to ensure community participation and mobilization o Government must be involved to ensure sustainability and further progress