SUMMATIVE EVALUATION OF THE ELSA TEXT2TEACH PROJECT: FINAL REPORT Josefina N. Natividad, Sc.D. Demographic Research and Development Foundation, Inc. Palma Hall, UP Diliman Quezon City 2 Executive Summary: The ELSA text2teach Project is an intervention strategy for improving the teaching of English, math and Science in grades 5 and 6 in selected schools in the province of Maguindanao, Cotabato and South Cotabato. We undertook a summative evaluation of the Project two years into its implementation to assess learning gains among pupils exposed to this intervention relative to a comparative group of controls. We adopted a quasi-experimental design with a pre- and post test administered to selected study schools in the text2teach project area and a comparable set of control schools in the same provinces. We picked a 10 per cent sample of the ELSA text2teach schools (12 out of 122 schools) and matched each chosen sample with a control school. Matching criteria were accessibility for testing, relative proximity to the counterpart study school and enrolment size. The 12 schools were distributed across province proportional to number of text2teach schools by province. Thus we had 5 schools from Maguindanao, 4 from Cotabato and 3 from South Cotabato with matching controls for a total of 24 schools. All in all, there were 3297 pupils who participated in the evaluation by taking the pre- tests, 59 per cent from the study schools and 42 per cent from the control schools. Of these 1674 were grade 5 and 1623 were grade 6. At the end of the school year, 85 per cent of grade 5 and 89 per cent of grade 6 pupils took the post test. The pre-test was conducted July 31 to August 3,2006 while the post-test was done in the period March 11-16,2007 The test instruments used for the pre- and post test consisted of 50-item multiple choice type exams in English, Math and Science- all written in English. The items for these tests were based on the competency framework appropriate to the grade according to the Revised Basic Education curriculum. The test instruments were field tested in the Old Balara Elementary School, a public school in Quezon City. 3 Midway through the school year we also administered the same test instruments to a sample of teachers of text2teach from the various schools where the project was being implemented. The teacher participants did not necessarily come from the school in our study. Using the Department of Education yardstick of 75 per cent mastery as indicator for acceptable performance in the grade level, we set the cut off of 37.5 points as the “passing mark” for the teachers ( 75 per cent of the 50-item test). Using this rough criterion, the teachers posted mean scores below 75 per cent in English (36.1 points) and Science (33.1 points) and at the cut point of 75 per cent (37.5 points) for Math. Per province, Cotabato teachers posted passing scores in all subjects, Maguindanao teachers had the lowest average scores. South Cotabato teachers were in between. Regression analysis shows that the most important predictor of teacher score is province, being Maguindanao takes away points from predicted scores in English, Math and Science, being from Cotabato adds points to the predicted score. Being young (age 20-29) and having fewer years of experience in teaching also adds points to the predicted score in Math and Science at both grade levels. Evidently, experience as measured by age and teaching years does not lead automatically lead to higher scores in the diagnostic tests. To assess the effect of text2teach as an intervention to improve learning gains we used the paired t-test, comparing each child’s score at pre- and posttest. Our hypothesis was that children exposed to text2teach will show higher learning gains compared to controls. Learning gain is operationally defined as the difference between post and pre-test score. We also performed multiple regression with dummy variables to examine the effects of the intervention in combination with other factors such as the pupil’s basic characteristics (e.g. age, sex, past academic performance, family and parent characteristics). The major findings of the comparison of pre- and post test performance of pupils in the study, either as exposed to the text2teach intervention or as controls are the following: 4 • Exposure to text2teach as an intervention leads to significantly higher learning gains in English , Math and Science at both grade levels. The gains are very impressive for English and Science but less so in Math although still highly significant. • Although exposure to the intervention leads to learning gains in general, the magnitude of the gain is modified by the province where the school is situated. Being from Cotabato adds to predicted learning gains in all subjects at both grade levels, while being from Maguindanao has the negative effect of subtracting from predicted learning gains. • There are province-specific findings such as the following: 1. It is possible to overcome perceived learning handicaps attributed to ethnic affiliation as shown by the outstanding performance of predominantly Muslim and IP populated study schools in South Cotabato compared with the predominantly Christian control schools. 2. In an environment where pupils already manifest learning gains even without the text2teach intervention, the effect of text2teach is to further enhance the gains such as in the case of the Cotabato schools. 3. In an environment not optimally conducive to effective learning such as in Maguindanao, text2teach can spell the difference between some learning and no learning at all. Evaluation of effects of exposure to the intervention other than through learning gains as we measured it should be explored in order to document possible qualitative change that may have resulted from exposure to text2teach. 4. For Cotabato, the pattern in learning gains differs between grade 5 and grade 6. The most dramatic gains are posted at grade 5. This means that probably the initial exposure to text2teach has the greatest effect on learning but the effect of a second exposure is no longer as dramatic, although still positive. In other words, the effect of length of exposure is not linear but probably pleateaus somewhat with longer exposure. Considering that Cotabato schools whether study or controls are already the highest performers of the three provinces, this result can not be generalized to South Cotabato and Maguindanao- but most likely to other schools where learning gains with or without intervention is normal and expected. 5. For Maguindanao, the pattern of learning gains also varies between grade 5 and grade 6. At grade 5, the learning gains are minimal; they shoot up at grade 6. Unlike Cotabato, longer exposure to the intervention works better for the Maguindanao pupils as it takes two years of exposure to text2teach to observe its effect. Thus for Maguindanao pupils, optimal results may be obtained with longer exposure beginning at lower grade levels. 5 • The regression results produced to one unanticipated finding which is that being male has a negative effect on learning gains. It has been customary to focus on female children as the more disadvantaged, especially in the Muslim-dominated regions. Our finding in this study of a male disadvantage in learning outcomes highlights the need to use more empirical bases for deciding on actions to take for programs that target gender-based disparities instead of relying routinely on currently popular notions. The negative effect on learning of being male needs further study as it has many possible implications. 6 Summative evaluation of the ELSA text2teach project in Maguindanao, North Cotabato and South Cotabato: Report on the Pre-test Background: The overall objective of the summative evaluation of the ELSA text2teach Project in Maguindanao, Cotabato and North Cotabato is to estimate the impact of this intervention on learning in English, Mathematics and Science among grades 5 and 6 pupils in the schools where the text2teach project is being implemented. To achieve this objective, we adopted an evaluation design which is a quasi-experiment comparing a number of text2teach schools (the study schools) with a comparable set of non-text2teach schools (the control schools) in the same localities. We assess learning gains via a pre-test post-test design where pupils in grades 5 and 6 in both study and control schools take a diagnostic test in English, Science and Math appropriate to the grade level at the beginning of the school year (the pre-test) and the same test toward the end of the school year ( the post test). We also record background characteristics of the pupils for use as additional explanatory factors that may account for learning gains over and above the effect of the main independent variable which exposure to text2teach lessons in English, Science and Mathematics for the duration of school year. The test instruments: The test instruments in English, Science and Math were prepared by faculty members of the University of the Philippines Integrated School who themselves are currently teaching in these grade levels, namely: Melanie Donkor- English Rolando Villablanca – Science Aurora Zuniga – Mathematics Two sets of tests were prepared, one for grade 5 and one for grade 6- all in English. Altogether there were a total of six tests, one per subject area and grade level. Test items 7 were based on the competency framework for the respective grade level according to the Revised Basic Education Curriculum. Thus each test consisting of a 50-item multiple choice type of examination was designed to assess competencies that should have been acquired at the end of the grade level. The teachers who prepared the tests also viewed the actual video lessons used in the intervention to give them an idea how the lessons are presented in the text2teach classes. Pre-testing the test instruments: The test instruments were first field-tested at the Old Balara Elementary School, a public school located along Tandang Sora Avenue in Quezon City. Being a public school located in a lower class neighborhood with enrollment comng from lower to lower middle classes we believed the type of students enrolled in this school would be comparable to our sample schools. Two sections were chosen per grade level, the high section and a regular section. The tests were conducted on June 29 and 30 and July 3 and 4, 2006. The set of three subject area tests were administered in one sitting, in the following order: English, Science and Mathematics. In all testing time was approximately two and a half hours. Results are shown in table 1. Table 1. Mean scores in 50-item tests, by subject, grade level and type of section High section Regular section All Sections Subject Mean s.d. n Mean s.d. n Mean s.d. N Sig. English 5 23.3 5.48 41 17.7 4.01 52 20.1 5.46 93 .00 6 27.4 4.08 44 17.7 3.54 45 22.6 6.18 89 .00 Science 5 20.9 5.61 41 14.4 3.79 52 17.3 5.67 93 .00 6 25.8 4.19 44 15.9 3.29 45 20.8 6.25 89 .00 Math 5 22.6 7.76 41 18.2 4.74 52 20.1 6.60 93 .001 6 25.8 4.19 44 15.9 3.29 45 20.8 6.25 89 .00 Table 1 shows the results of the tests as mean scores in the 50-item examinations broken down by subject, grade level and type of section- whether high or regular. Differences in 8 mean scores between the high and regular sections were highly significant (p<.00) for each test at both grade levels. This means that the diagnostic instrument could successfully discriminate among pupils of different levels of ability (based on the a priori criterion of type of section). As expected, pupils from the high sections scored higher than those from the regular sections. Given that the tests contained items that were still to be learned during the school year it was not expected that the pupils will score very highly. Nevertheless, pupils from the high section had mean scores that ranged from 21 to 27 out of a possible score of 50 (42 to 54 per cent correct answers) , while the range of mean scores for regular section students was 14 to 18 (28 to 36 per cent correct answers.). We intend to administer the same exams at the end of the school year to assess learning gains in this set of pupils, even if they are not part of the actual study. This group can serve as another control group, one composed of pupils exposed to the public school system as are the actual test subjects on the study. Their main distinguishing characteristic is that they are from a highly urbanized area. Based on feedback from the Old Balara pupils, minor corrections were made on the test instruments consisting mainly of correcting typographical errors. Appendix A presents the item analysis results for each of the tests. Reliability as measured by Cronbach’s alpha ranged from .69 for English 5 to .79 for Math 6. Reliability for the other tests fell between these two values. All 50 items in the original test were retained for the field pre-test despite some items with low item-total correlation since these exams are considered mastery tests and each item is deemed as testing for a specific competency that is expected to have been acquired at the end of the grade level. 9 The conduct of the pre-test in the field sites: After the validation of the test instruments the next step was to conduct the actual pre-test in the provinces of Maguindanao, North Cotabato and South Cotabato. As proposed in the original protocol, 24 elementary schools were chosen to participate in the summative evaluation exercise, 12 text2teach and 12 non text2teach schools. The conduct of the tests in these schools was done in close partnership with a collaborating local institution, the Cotabato City Polytechnic State College (CCSPC), a state college offering only tertiary level courses and with main campus in Cotabato City. CCSPC was identified by SEAMEO INNOTECH as the partner for the field tests because the school and the personnel have no direct link to the elementary education system and thus have no stake in the outcome of the tests. Furthermore, there were no text2teach schools in Cotabato City and this further bolstered the impartiality of our partners. The choice of which schools to include in the summative evaluation was accomplished through the following procedure. First, the President of CCSPC , Dr. Dingan Ali and his deputy, Dr. Felipe Balucas Jr. chose candidate participant schools purposively from the 122 ELSA test2teach schools during the initial meeting between myself and the CCSPC officials at the SEAMEO Innotech head office in Quezon City. They based their preliminary selection of study schools on enrolment size and accessibility for testing through its proximity to the national highway. Further in identifying potential schools for inclusion they were mindful to keep the choice of schools widely dispersed around the Project sites, i.e. the provinces of Maguindanao, Cotabato and South Cotabato albeit with clustering around the national highway cutting through these provinces accessibility . This was to ensure representativeness of the entire geographic spread of each province while still fulfilling the requirement of accessibility . Dr. Ali who has worked in government prior to his stint at CCSPC was very familiar with the Maguindanao-Cotabato area and the location of the public elementary schools. For each study school that he and Dr. Balucas picked, he also suggested a control school that was as close as possible in size of enrollment and proximity to the national highway as the selected study school from among ELSA text2teach schools. 10 The tentative list of schools drawn at SEAMEO INNOTECH with Dr. Ali was sent to the district superintendents per province for their evaluation and approval. They in turn suggested control schools in those areas where the CCSPC team had not identified one, mainly in South Cotabato. Other minor changes were made on the CCSPC generated list in most cases due to concerns from the field about accessibility and security. Based on the foregoing procedure we arrived at the final listing of study and control schools shown in table 2. Table 2. Study and control schools for the ELSA text2teach summative evaluation STUDY SCHOOLS CONTROL SCHOOLS MUNICIPALITY/PROVINCE Tanuel Datu Pingiaman Datu Odin Sinsuat, Maguindanao Making Molina Parang, Maguindanao Datu Sangki Ampatuan Datu Dumpao Banog Ampatuan, Maguindanao Malangit, Buluan Datu Paglas Buluan & Paglas, Maguindanao Layug (Mamasabulod) Matalam Farm Pagalungan and Datu Montawal, Maguindanao Sinawingan Cocal Libungan, North Cotabato Katidtuan Central Osias Kabacan, North Cotabato Saguing Malasila Makilala, North Cotabato Dalipe Lika Mlang, North Cotabato Dumadalig San Felipe Tantangan, South Cotabato Ambalgan Teresita Sto Nino, South Cotabato Koronadal Proper Silway 8 Polomolok, South Cotabato For the fielding of the pre-test two teams were formed. Team A headed by Dr. Sema Dilna of CCSPC conducted the tests in Maguindanao while team B headed by Dr. Francisco Balucas Jr covered North and South Cotabato. Each team was accompanied by a field supervisor from the Population Institute/Demographic Research and Development Foundation Inc. A half day training session on the conduct of the pretest was held at CCSPC a day prior to the field work. The teams were also trained how to guide the pupils in filling up the self-filled questionnaire about the respondents’ background characteristics. Furthermore, the need to secure the test instruments was emphasized. The teams were made aware of the 11 contamination of research results that could possibly ensue should teachers in the study and control schools gain access to the test instruments and ask their pupils to practice for it in the post test. The teams went on field simultaneously from July 31 to August 4, 2006. Due to the schedule of classes in Maguindanao (Sunday to Friday) team A conducted the pretests from July 31 to August 3. In all 3297 pupils, 1674 grade 5 and 1623 grade 6 pupils, took the pre-test. Table 3 shows the distribution of test takers by school and province. As evident in table the ELSA test2teach schools tended to have bigger enrollment than the control schools, a bias that is built in into the process of selection by program administrators from SEAMEO INNOTECH in consultation with Department of Education officials as to which schools in the three provinces will participate in the text2teach program. One major consideration for inclusion of a school was a large enrollment in order to maximize the gains of the program. Consequently, control schools which we picked on the basis of their relative proximity to an ELSA text2teach study school will tend to be smaller in size. Altogether,of the 3297 pupils who took the test, 59 per cent were from the study schools, 41 per cent were controls (table 4). 12 Table 3. Distribution of test takers by grade level, province and school Province Type School Grade 5 N Grade 6 N Maguindanao study Tanuel ES 62 45 study Making ES 126 127 study Datu Sangki ES 108 79 study Malangit ES 41 30 study Layug ES 65 53 control Datu Pingiaman ES 20 20 control Molina ES 20 21 control Datu Dumpao ES 49 48 control Paglas ES 36 31 control Matalam Farm ES 66 72 Cotabato study Sinawingan ES 60 69 study Katidtuan ES 64 56 study Saguing ES 109 113 study Dalipe ES 64 65 control Cocal ES 28 23 control Osias ES 35 35 control Malasila ES 92 91 control Lika ES 99 98 South Cotabato study Dumadalig ES 59 57 study Ambalgan ES 85 74 study Koronadal Proper ES 146 175 control San Felipe ES 59 63 control Teresita ES 32 37 control Silway 8 ES 149 141 Total 1674 1623 Table 4. Distribution of test takers by type of school Grade 5 N Grade 6 n Total Study 59.1 989 58.1 943 58.6 1932 Control 40.9 685 41.9 680 41.4 1365 Total 100.0 1674 100.0 1623 100.0 3297 13 RESULTS: Prior to taking the test, pupils filled in a questionnaire on background information about themselves. Results from these questionnaires provide us a picture of some basic characteristics about grade 5 and 6 pupils in the three provinces. We present the descriptive results broken down by province and tabulated separately for grades 5 and 6. GRADE 5 A. Background characteristics: A total of 1674 grade 5 pupils took the test, 989 (59 per cent) from the study schools, 685 (41 per cent) from the control schools. However, while all of them filled up the questionnaire prior to the test, we noted during the data processing that there were some gaps in the information that they provided thus leading to some missing data. As the number of respondents with missing information did not add up to a significant proportion of total respondents we proceeded with the data analysis and dropped cases with missing information for the variable being described. Missing information is the reason why, despite a total of 1674 grade 5 respondents, the number of cases in tables are likely to be less than 1674. We also noted during the administration of the self-filled questionnaire that pupils in some of the schools needed a lot of coaching from their teacher on how to fill in the questionnaire. Furthermore, a number of Maguindanao students expressed difficulty in giving information on their birth dates claiming they did not know when they were born. This problem seemed confined to Maguindanao, none of this difficulty was observed in respondents from Cotabato and South Cotabato. Table 5 shows a description of the grade 5 pupils based on selected background characteristics: gender, age, number of siblings, mother and father’s education, mother and father’s occupation and past school performance ( ever repeated a grade and for grade repeaters, what grade/s was repeated). The rightmost value in each table indicates the level of significance of the chi square statistic that tests for the significance of differences in distribution across categories of a given background characteristic by province. Over all, the 14 table shows that there are significant differences in pupil characteristics across the three provinces. The only characteristic with no significant difference is gender distribution. In general there were more girls than boys in each of the three provinces. The ratio of boys to girls is roughly equivalent to the sex ratio in these age groups at the national level and indicates that, using this simple indicator there is no apparent gender bias in school attendance. In terms of age, we present percent distributions across given age because there were peculiarities in some of the data that may render a measure like the mean uninterpretable when used for all. For example, pupils from Cotabato and South Cotabato had an age distribution that clustered around age 11, the normative age for a grade 5 pupil. For pupils in these provinces, their given age is probably their true age. However, pupils from Maguindanao had a more dispersed age distribution with no clear clustering at age 11. More of the Maguindanao respondents also either did not fill in their birthdates or gave improbable years of birth. This finding was not unexpected given what we observed during the test administration when many pupils claimed they did not know their birth dates. Conscious of this situation, we plan to request for school records to verify the age of students in the Maguindanao schools so that we can use the age variable meaningfully in later analysis. In terms of grade repetition, Maguindanao students were more likely to have ever repeated a grade; in fact more than half (53.7 per cent) of the Maguindanao pupils reported they repeated a grade compared with 13.5 per cent and 24.6 per cent in Cotabato and South Cotabato respectively. In all three provinces the most commonly repeated grade was grade 1. In fact, for all areas taken together half of those who ever repeated a grad, repeated grade 1. This proportion was higher in Maguindanao with 6 out of every 10 repeaters repeating grade 1. Finally, about 1 in 10 repeaters in Cotabato repeated more than one grade. In terms of family-related characteristics, Maguindanao pupils came from significantly larger families reporting a mean of about 5 siblings (4.95), significantly higher than the average number of siblings in either Cotabato (4.34) or South Cotabato (4.36). Including the 15 respondent therefore, the mean number of children among our respondents in Maguindanao is about 6, and about 5 in Cotabato and South Cotabato. Compared with the national average of 3.5 children, the grade 5 respondents definitely come from families that are much larger than the national average. In terms of their mother’s education as reported by the respondents, the mothers in Maguindanao had the lowest level of educational attainment as a group compared with mothers from Cotabato and South Cotabato with about 13 per cent reported by their children as having no schooling compared with only 1 per cent in the two other provinces. Only Maguindanao mothers were reported as having attended Arabic schooling. About a fourth of Maguindanao respondents did not know their mother’s educational attainment, three times more the percentage in Cotabato and South Cotabato. The profile of father’s education is similar with mother’s education although the levels vary. As with mothers, Maguindanao fathers in general have a lower level of education compared with fathers in the two other provinces although the proportion with no schooling is lower than that of Maguindanao mothers. As with their mothers, a higher proportion of respondents in the Maguindanao schools did not know their father’s educational attainment. As to occupation, a higher proportion of Maguindanao mothers were reported by their children to have an occupation compared with Cotabato and South Cotabato. The highest proportion of mothers reported to have no occupation or are nonworking is in South Cotabato. More Maguindanao mothers are reported to be in farming/fishing and in business; there are more mothers in Cotabato and South Cotabato in services/sales. Maguindanao and South Cotabato had about the same proportion of mothers who are OFWs (about 5 per cent). As to father’s occupation, the highest proportion who are reported to have no occupation are in Maguindanao although the figures are not high (4 per cent for Maguindanao, 3 per cent for South Cotabato). The most commonly reported occupation of fathers in all three provinces is farming/fishing followed by trade/skilled work in Cotabato and South Cotabato, while in Maguindanao the second most common occupation is as police/barangay police/CVO. There 16 are also more fathers in Maguindanao in the Armed Forces of the Philippines. The proportion of fathers who are OFWs is lower at 1 per cent compared with mothers (4 per cent). B. Test results: Table 6a presents the mean scores of grade 5 pupils in the three subjects across the three provinces. All differences are highly significant (p < .000) and shows that Maguindanao schools scored lower than the two other provinces in all subjects. Cotabato schools had the highest scores of the three. Table 6b meanwhile shows the test results by province this time adding type of school as an additional layer for comparison. Taking all cases together (the first panel in table 6b), results show that overall the difference in mean scores between pupils from the study and the control schools are statistically significant for English and Math but not for Science. For English and Math the means of the control group are higher. Testing separately by province, table 6b shows that the significant differences in the overall scores are mostly attributable to South Cotabato where the mean scores of students in the control group was 2 points higher than the study group in English and Math and 1 point higher in Science. In contrast, within Maguindanao, there was no significant difference between study and controls while in Cotabato the only significant difference is in Science with the mean score of study schools recorded as less than 1 point higher than the controls. The test results by school are presented in table 6c. In Maguindanao, Paglas Elementary School had the highest score in English, Molina Elementary School in Math and Making Elementary School in Science. For Cotabato, the highest scorers were Sinawingan Elementary School (English and Math) and Saguing Elementary School (Science)- both text2teach schools while for South Cotabato these were Teresita Elementary School (English), Silway Elementary School (Math and Science)- both control schools. 17 GRADE 6 A. Background characteristics: A total of 1623 pupils took the tests, 943 (58 per cent) from the study schools and 680 (42 per cent) from the control schools. Compared with grade 5 pupils, there were fewer cases with missing information per characteristic. In general, the profile of grade 6 pupils does not radically differ from that of the grade 5 pupils. As with the grade 5 data, all differences per characteristic by province were significant except for gender. This similarity in findings is not unexpected as both groups come from the same geographic areas and are only differentiated by their grade level (table 7). On the whole, females outnumber males in each province. The age distribution clusters around age 12, the normative age at this grade level. As with the grade 5 responses, we need to validate birth dates from school records in Maguindanao to resolve unrealistic age reports resulting from the pupils’ not knowing their exact birthdates. The average number of siblings is also higher in Maguindanao while the combined average for the three provinces (5.27 children) is higher than the national average (3.5 children). The overall pattern in grade repetition is similar with that at grade 5. The proportion who ever repeated a grade is likewise highest in Maguindanao and lowest in Cotabato. The one notable difference is that unlike the grade 5 respondents , the most frequently repeated grades in Maguindanao among grade 6 respondents are 1 and 2 (at 41 per cent and 45 per cent of repeaters, respectively). In terms of parents’ education, Maguindanao mothers also have lower educational attainment with more of them reported as having no schooling; half had elementary schooling and below. Cotabato had the highest proportion of mothers who are high school graduates. Compared with the grade 5 pupils, very few in Maguindanao reported their mother as having gone thru Arabic school (less than 1 per cent vs. 5 per cent for grade 5). Finally fewer respondents from Maguindanao said they did not know their mother’s education. The 18 education profile of fathers closely resembles that of mothers with Maguindanao fathers having a lower level of educational attainment as a group. Parents’ occupation also shows a predominance of work in farming/fishing among fathers in the three provinces and among mothers in Maguindanao. More mothers than fathers are reported to be OFWs in all provinces with Maguindanao mothers having the highest proportion in this occupational category of the three. Maguindanao fathers have the highest proportion in such security-related work as barangay police/CVO. B. Test results: As shown in table 8a Cotabato schools again scored significantly higher of the three provinces in all subject areas. Differentiated into study and control schools table 8b shows that overall, the difference between study and control schools is significant for English and Science with control schools having a higher mean in these subjects. The difference for Science is not statistically significant. On a within province comparison (table 8b), the study schools in Maguindanao have a higher mean score in English and the difference is significant but not so for Math and Science although the pattern is for the mean of the study schools to be slightly higher than the controls. For Cotabato, the difference between study and controls is significant for Math but not for Science or English. As with Maguindanao, the study schools have slightly higher mean scores than control schools in all subjects. For South Cotabato, the contrast between study and controls is more evident and confirmed further by highly significant p- levels ( p≤.001). In this province, the means for the controls in English and Math are almost three points higher than for the study respondents and 1 point higher in Science. Broken down further into specific schools, table shows that some schools that scored high at grade 5 also did at grade 6. In Maguindanao, Paglas Elementary School recorded the highest mean in grade 6 English as it did with the grade 5 test. Making Elementary School had the highest mean score in Math while Layug and Making scored highest in Science. However, 19 compared with Cotabato and South Cotabato, these scores are generally lower. For Cotabato, Sinawingan Elementary School had the highest mean for English and Science while Saguing Elementary School is the topnotcher for Math. Both are study schools. In South Cotabato, San Felipe Elementary School scored the highest in English; Silway Elementary School topped Math and Dumadalig Elementary School was the highest for Science. San Felipe and Silway are controls. Other observations: While conducting the exam, the field team members noted that in Maguindanao the testing time took much shorter than the two hours we allotted. In some the pupils finished in under an hour. We observed that the pupils did not really read the questions and were obviously guessing. This observation was further bolstered in the data encoding when many answer sheets showed signs of guessing (crisscrossing, parallel answers, double answers for an item). 20 Table 5. Descriptives for Grade 5 respondents Sex Maguindanao Cotabato South Cotabato Total N Sig. Female 57.12 52.73 53.58 54.50 883 0.300 Male 42.88 47.27 46.42 45.50 739 Total 100.00 100.00 100.00 100.00 100.00 N 555 550 530 1635 Age no info 7.54 3.12 0.95 3.93 64 0.000 < 10 3.41 1.47 0.38 1.78 29 > 16 0.00 0.18 0.19 0.12 2 10 13.29 24.59 21.14 19.61 319 11 28.37 45.69 45.90 39.83 648 12 21.18 14.86 15.43 17.21 280 13 14.18 6.42 9.14 9.96 162 14 7.90 2.75 4.19 4.98 81 15 2.69 0.92 2.10 1.91 31 16 1.44 0.00 0.57 0.68 11 Total 100.00 100.00 100.00 100.00 N 557 545 525 1627 Number of siblings Mean 4.95 4.34 4.36 4.56 0.000 s.d. 2.616 2.761 2.588 2.670 N 557 543 521 1621 Minimum 0 0 0 0 Maximum 13 16 12 16 Ever repeated a grade Yes 52.38 18.90 26.06 25.56 412 0.000 No 47.62 81.10 73.94 74.44 1200 Total 100.00 100.00 100.00 100.00 N 525 545 545 1615 Repeated grade grade 1 40.50 35.51 52.45 42.72 226 0.000 grade 2 44.80 11.21 6.29 27.60 146 grade 3 1.79 3.74 4.90 3.02 16 grade 4 3.23 4.67 4.20 3.78 20 grade 5 2.15 10.28 4.20 4.35 23 grade 6 0.72 7.48 2.10 2.46 13 >1 grade 0.00 3.74 2.80 1.51 8 No information 6.81 23.36 23.08 14.56 Total 100.00 100.00 100.00 100.00 N 279 107 143 529 21 Table 7. Descriptives for Grade 6 respondents Sex Maguindanao Cotabato South Cotabato Total N sig. Female 55.7 52.4 55.3 54.4 883 0.482 Male 44.3 47.6 44.7 45.6 739 Total 100.0 100.0 100.0 100.0 100.00 N 526 550 546 1622 Age no info 0.4 0.7 0.6 0.6 9 0.000 < 10 6.1 0.4 0.6 2.3 37 > 16 0.2 2.7 0.6 1.2 19 10 1.3 1.6 0.2 1.0 17 11 12.4 25.1 18.2 18.6 302 12 29.1 47.3 46.7 41.2 667 13 27.6 13.5 16.2 19.0 307 14 15.0 6.2 9.6 10.2 165 15 6.3 1.1 5.7 4.3 70 16 1.7 1.5 1.8 1.7 27 Total 100.0 100.0 100.0 100.0 N 526 550 544 1620 Number of siblings Mean 4.65 3.92 4.25 4.27 0.000 2.486 2.405 2.506 2.482 N 517 546 545 1608 Minimum 0 0 0 0 Maximum 14 11 12 14 Ever repeated a grade Yes 53.4 19.6 26.2 32.8 529 0.000 No 46.6 80.4 73.8 67.2 1083 Total 100.0 100.0 100.0 100.0 N 522 545 545 1612 Repeated grade grade 1 40.50 35.51 52.45 42.72 226 0.000 grade 2 44.80 11.21 6.29 27.60 146 grade 3 1.79 3.74 4.90 3.02 16 grade 4 3.23 4.67 4.20 3.78 20 grade 5 2.15 10.28 4.20 4.35 23 grade 6 0.72 7.48 2.10 2.46 13 >1 grade 0.00 3.74 2.80 1.51 8 No information 6.81 23.36 23.08 14.56 Total 100.00 100.00 100.00 100.00 N 279 107 143 529 22 Table 6a. Mean grade 5 pre-test scores by province and subject Maguindanao Cotabato South Cotabato Total Subject* Mean s.d. N Mean s.d N Mean s.d. N Mean s.d. N English 13.27 3.612 593 15.49 4.024 551 14.99 3.786 530 14.55 3.924 1674 Math 12.69 3.229 593 14.07 4.321 551 13.52 4.306 530 13.41 4.002 1674 Science 12.32 3.398 593 13.99 3.799 551 13.92 3.666 530 13.37 3.701 1674 *All differences are significant; p < .000 Table.6b. Mean grade 5 pretest scores by province,subject and type of school Type Mean s.d. Sig. Minimum Maximum N English control 14.90 3.99 0.0022 1 27 685 All study 14.30 3.86 1 30 989 Math control 13.89 4.21 0.0000 0 37 685 study 13.07 3.82 0 37 989 Science control 13.52 3.78 n.s. 2 27 685 study 13.27 3.65 1 30 989 English control 13.19 3.68 n.s. 1 22 191 study 13.31 3.58 1 24 402 Maguindanao Math control 12.60 3.21 n.s. 3 21 191 study 12.73 3.24 1 23 402 Science control 12.10 3.52 n.s 2 21 191 study 12.42 3.34 1 21 402 English control 15.35 3.96 n.s 6 26 254 Cotabato study 15.61 4.09 5 30 297 Math control 14.37 4.22 n.s 6 29 254 study 13.82 4.39 0 31 297 Science control 13.59 3.55 0.0213 5 26 254 study 14.33 3.97 6 30 297 English control 15.78 3.87 0.0000 5 27 240 South study 14.33 3.60 6 30 290 Cotabato Math control 14.40 4.68 0.0000 0 37 240 study 12.78 3.83 3 37 290 Science control 14.58 3.85 0.0001 4 27 240 study 13.37 3.42 6 24 290 n.s. – not significant 23 Table6c. Mean grade 5 pretest scores by province,subject and school Province School English Math Science Tanuel ES Mean 12.71 12.48 12.35 s.d. 3.06 3.57 3.41 N 62 62 62 Making ES Mean 14.00 12.88 12.91 Maguindanao s.d. 3.60 3.05 3.31 N 126 126 126 Datu Sangki ES Mean 13.13 12.90 12.27 s.d. 3.51 3.23 2.82 N 108 108 108 Malangit ES Mean 13.00 12.41 11.51 s.d. 3.99 3.07 3.61 N 41 41 41 Layug ES Mean 13.06 12.60 12.32 (Mamasabulod) s.d. 3.74 3.44 3.85 N 65 65 65 Datu Pingiaman ES Mean 12.95 13.4 12.15 s.d. 3.90 3.38 3.39 N 20 20 20 Molina ES Mean 13.7 13.95 12.8 s.d. 3.20 3.73 3.83 N 20 20 20 Datu Dumpao ES Mean 12.29 12.12 11.67 s.d. 3.46 3.05 3.63 N 49 49 49 Paglas ES Mean 14.75 12.22 12.39 s.d. 3.42 3.42 3.56 N 36 36 36 Matalam Farm ES Mean 12.94 12.50 12.05 s.d. 3.87 2.94 3.43 N 66 66 66 24 Table 6c (Cont.). Mean grade 5 pretest scores by province,subject and school Province School English Math Science Sinawingan ES Mean 16.40 15.18 15.08 s.d. 4.09 4.42 3.35 N 60 60 60 Katidtuan ES Mean 16.25 14.41 13.67 Cotabato s.d. 3.67 3.73 3.34 N 64 64 64 Saguing ES Mean 15.60 13.79 15.84 s.d. 4.42 4.59 4.08 N 109 109 109 Dalipe ES Mean 14.23 12.00 11.72 s.d. 3.60 4.10 3.45 N 64 64 64 Cocal ES Mean 15.5 13.32 13.11 s.d. 3.11 3.14 3.42 N 28 28 28 Osias ES Mean 13.91 13.80 13.29 s.d. 3.18 2.82 2.79 N 35 35 35 Malasila ES Mean 15.85 14.99 13.51 s.d. 4.45 4.60 3.48 N 92 92 92 Lika ES Mean 15.34 14.28 13.90 s.d. 3.85 4.49 3.90 N 99 99 99 Dumadalig ES Mean 14.63 13.32 13.49 s.d. 3.83 3.62 3.48 N 59 59 59 South Ambalgan ES Mean 14.42 12.32 13.65 Cotabato s.d. 2.86 3.27 3.22 N 85 85 85 Koronadal Proper ES Mean 14.16 12.83 13.16 s.d. 3.89 4.18 3.51 N 146 146 146 San Felipe ES Mean 14.88 12.92 14.20 s.d. 3.54 3.91 3.14 N 59 59 59 Teresita ES Mean 16.78 14.41 15.47 s.d. 4.06 4.57 3.80 N 32 32 32 Silway 8 ES Mean 15.92 14.99 14.54 s.d. 3.90 4.87 4.10 N 149 149 149 All Mean 14.55 13.41 13.37 s.d. 3.92 4.00 3.70 N 1674 1674 1674 25 Table 8a. Mean grade 6 pre-test scores by province and subject Subject* Maguindanao Cotabato South Cotabato Total Mean s.d N Mean s.d N Mean s.d N Mean s.d N English 14.86 4.505 526 19.13 5.37 550 18.23 4.935 547 17.44 5.280 1623 Math 12.65 3.290 526 16.04 4.98 550 15.44 4.504 547 14.74 4.571 1623 Science 12.72 3.518 526 15.96 4.55 550 14.73 4.382 547 14.50 4.386 1623 *All differences are significant; p < .000 Table 8b. Mean grade 6 pretest scores by province, subject and type of school Type Mean s.d. Sig. Minimum Maximum N English control 17.79 5.49 0.0250 1 33 680 All study 17.19 5.11 0 34 943 Math control 15.14 4.69 0.0026 3 36 680 study 14.45 4.46 1 39 943 Science control 14.65 4.43 n.s. 3 32 680 study 14.39 4.35 4 31 943 English control 14.27 4.27 0.0227 1 26 192 Maguindanao study 15.19 4.61 0 33 334 Math control 12.43 3.48 n.s. 3 21 192 study 12.77 3.17 1 20 334 Science control 12.53 3.49 n.s. 3 21 192 study 12.84 3.54 4 24 334 English control 18.70 5.35 n.s 7 32 247 Cotabato study 19.50 5.36 7 34 303 Math control 15.43 4.13 0.0089 6 29 247 study 16.54 5.54 7 39 303 Science control 15.55 4.38 n.s. 5 29 247 study 16.30 4.66 7 31 303 English control 19.69 5.19 0.0000 6 33 241 South study 17.09 4.41 8 29 306 Cotabato Math control 17.00 5.07 0.0000 8 36 241 study 14.21 3.56 6 25 306 Science control 15.43 4.60 0.0009 5 32 241 study 14.18 4.13 5 28 306 26 Table 8c. Mean grade 6 pretest scores by province, subject and school Province School English Math Science Maguindanao Tanuel ES Mean 13.49 11.98 11.80 s.d. 4.47 3.85 3.06 N 45 45 45 Making ES Mean 17.06 13.34 13.63 s.d. 4.59 3.21 3.41 N 127 127 127 Datu Sangki ES Mean 13.38 12.68 11.94 s.d. 4.52 3.12 3.69 N 79 79 79 Malangit ES Mean 14.70 12.13 11.97 s.d. 3.58 2.39 2.57 N 30 30 30 Layug ES Mean 15.15 12.58 13.64 (Mamasabulod) s.d. 3.74 2.73 3.89 N 53 53 53 Datu Pingiaman ES Mean 13.20 13.20 12.40 s.d. 2.91 3.91 3.30 N 20 20 20 Molina ES Mean 15.24 13.10 12.62 s.d. 2.47 3.14 3.57 N 21 21 21 Datu Dumpao ES Mean 13.19 12.38 12.69 s.d. 2.73 3.29 3.70 N 48 48 48 Paglas ES Mean 19.45 12.58 13.35 s.d. 3.66 3.55 3.41 N 31 31 31 Matalam Farm ES Mean 12.76 12.00 12.08 s.d. 4.37 3.57 3.43 N 72 72 72 27 APPENDIX A Item analysis Item Statistics English 5 Item Number Proportion with correct answer Std. Deviation N 1 .23 .420 93 2 .11 .311 93 3 .28 .451 93 4 .33 .474 93 5 .30 .461 93 6 .38 .487 93 7 .18 .389 93 8 .39 .490 93 9 .29 .456 93 10 .41 .494 93 11 .42 .496 93 12 .15 .360 93 13 .56 .499 93 14 .39 .490 93 15 .49 .503 93 16 .46 .501 93 17 .42 .496 93 18 .20 .405 93 19 .66 .478 93 20 .91 .282 93 21 .46 .501 93 22 .73 .446 93 23 .29 .456 93 24 .31 .466 93 25 .15 .360 93 26 .75 .434 93 27 .56 .499 93 28 .82 .389 93 29 .13 .337 93 30 .38 .487 93 31 .43 .498 93 32 .18 .389 93 33 .41 .494 93 34 .30 .461 93 35 .19 .397 93 36 .33 .474 93 37 .42 .496 93 38 .15 .360 93 39 .27 .446 93 40 .26 .440 93 41 .84 .370 93 42 .24 .427 93 43 .74 .440 93 44 .16 .370 93 45 .61 .490 93 46 .69 .466 93 47 .65 .481 93 48 .68 .470 93 49 .16 .370 93 50 .29 .456 93 Scale Statistics Mean Variance Std. Deviation N of Items 20.14 29.774 5.457 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .680 .661 50 29 Item Statistics Science 5 Item Proportion with correct answer Std. Deviation N 1 .62 .487 93 2 .34 .478 93 3 .27 .446 93 4 .25 .434 93 5 .33 .474 93 6 .51 .503 93 7 .45 .500 93 8 .78 .413 93 9 .52 .502 93 10 .37 .484 93 11 .23 .420 93 12 .26 .440 93 13 .41 .494 93 14 .40 .492 93 15 .31 .466 93 16 .19 .397 93 17 .37 .484 93 18 .39 .490 93 19 .59 .494 93 20 .10 .297 93 21 .40 .492 93 22 .52 .502 93 23 .35 .481 93 24 .17 .379 93 25 .06 .247 93 26 .43 .498 93 27 .22 .413 93 28 .20 .405 93 29 .39 .490 93 39 .35 .481 93 31 .34 .478 93 32 .61 .490 93 33 .28 .451 93 34 .31 .466 93 35 .35 .481 93 36 .31 .466 93 37 .16 .370 93 38 .12 .325 93 39 .39 .490 93 40 .41 .494 93 41 .37 .484 93 42 .35 .481 93 43 .41 .494 93 44 .16 .370 93 45 .54 .501 93 46 .29 .456 93 47 .20 .405 93 48 .29 .456 93 49 .25 .434 93 50 .37 .484 93 Scale Statistics Mean Variance Std. Deviation N of Items 17.29 32.187 5.673 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .690 .680 50 30 Item Statistics Mathematics 5 Item Proportion with correct answer Std. Deviation N 1 .28 .451 93 2 .10 .297 93 3 .18 .389 93 4 .77 .420 93 5 .43 .498 93 6 .38 .487 93 7 .37 .484 93 8 .31 .466 93 9 .73 .446 93 10 .34 .478 93 11 .68 .470 93 12 .59 .494 93 13 .48 .502 93 14 .52 .502 93 15 .32 .470 93 16 .43 .498 93 17 .52 .502 93 18 .33 .474 93 19 .82 .389 93 20 .68 .470 93 21 .38 .487 93 22 .41 .494 93 23 .58 .496 93 24 .65 .481 93 25 .35 .481 93 26 .24 .427 93 27 .25 .434 93 28 .16 .370 93 29 .49 .503 93 30 .33 .474 93 31 .31 .466 93 32 .25 .434 93 33 .37 .484 93 34 .31 .466 93 35 .25 .434 93 36 .39 .490 93 37 .25 .434 93 38 .29 .456 93 39 .47 .502 93 40 .40 .492 93 41 .19 .397 93 42 .19 .397 93 43 .10 .297 93 44 .39 .490 93 45 .29 .456 93 46 .25 .434 93 47 .55 .500 93 48 .81 .397 93 49 .81 .397 93 50 .18 .389 93 Scale Statistics Mean Variance Std. Deviation N of Items 20.13 43.548 6.599 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .777 .768 50 31 Item Statistics English 6 Item Proportion with correct answer Std. Deviation N 1 .58 .496 89 2 .78 .420 89 3 .58 .496 89 4 .45 .500 89 5 .72 .452 89 6 .33 .471 89 7 .44 .499 89 8 .64 .483 89 9 .25 .434 89 10 .73 .446 89 11 .72 .452 89 12 .45 .500 89 13 .65 .479 89 14 .54 .501 89 15 .28 .452 89 16 .54 .501 89 17 .51 .503 89 18 .11 .318 89 19 .26 .440 89 20 .18 .386 89 21 .28 .452 89 22 .54 .501 89 23 .55 .500 89 24 .52 .503 89 25 .47 .502 89 26 .46 .501 89 27 .40 .494 89 28 .34 .475 89 29 .69 .467 89 30 .25 .434 89 31 .93 .252 89 32 .74 .440 89 33 .58 .496 89 34 .52 .503 89 35 .64 .483 89 36 .38 .489 89 37 .44 .499 89 38 .21 .412 89 39 .18 .386 89 40 .54 .501 89 41 .25 .434 89 42 .36 .483 89 43 .36 .483 89 44 .54 .501 89 45 .17 .376 89 46 .21 .412 89 47 .29 .457 89 48 .40 .494 89 49 .47 .502 89 50 .10 .303 89 Scale Statistics Mean Variance Std. Deviation N of Items 22.55 38.182 6.179 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .734 .717 50 32 Item Statistics Science 6 Item Proportion with correct answer Std. Deviation N 1 .84 .366 89 2 .17 .376 89 3 .35 .479 89 4 .87 .343 89 5 .35 .479 89 6 .35 .479 89 7 .42 .496 89 8 .45 .500 89 9 .19 .395 89 10 .71 .457 89 11 .62 .489 89 12 .53 .502 89 13 .46 .501 89 14 .40 .494 89 15 .70 .462 89 16 .16 .366 89 17 .71 .457 89 18 .54 .501 89 19 .26 .440 89 20 .38 .489 89 21 .48 .503 89 22 .49 .503 89 23 .44 .499 89 24 .18 .386 89 25 .39 .491 89 26 .21 .412 89 27 .64 .483 89 28 .29 .457 89 29 .38 .489 89 30 .39 .491 89 31 .20 .404 89 32 .33 .471 89 33 .15 .355 89 34 .45 .500 89 35 .04 .208 89 36 .35 .479 89 37 .44 .499 89 38 .27 .446 89 39 .51 .503 89 40 .29 .457 89 41 .28 .452 89 42 .26 .440 89 43 .42 .496 89 44 .37 .486 89 45 .45 .500 89 46 .47 .502 89 47 .58 .496 89 48 .48 .503 89 49 .61 .491 89 50 .49 .503 89 Scale Statistics Mean Variance Std. Deviation N of Items 20.79 39.033 6.248 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .740 .719 50 33 Item Statistics Mathematics 6 Item Mean Std. Deviation N 1 .87 .343 89 2 .22 .420 89 3 .43 .497 89 4 .46 .501 89 5 .55 .500 89 6 .53 .502 89 7 .70 .462 89 8 .17 .376 89 9 .53 .502 89 10 .70 .462 89 11 .39 .491 89 12 .30 .462 89 13 .30 .462 89 14 .26 .440 89 15 .33 .471 89 16 .58 .496 89 17 .18 .386 89 18 .44 .499 89 19 .49 .503 89 20 .35 .479 89 21 .34 .475 89 22 .24 .427 89 23 .44 .499 89 24 .55 .500 89 25 .49 .503 89 26 .19 .395 89 27 .46 .501 89 28 .42 .496 89 29 .28 .452 89 30 .34 .475 89 31 .30 .462 89 32 .79 .412 89 33 .24 .427 89 34 .42 .496 89 35 .34 .475 89 36 .33 .471 89 37 .53 .502 89 38 .28 .452 89 39 .47 .502 89 40 .61 .491 89 41 .51 .503 89 42 .21 .412 89 43 .35 .479 89 44 .25 .434 89 45 .47 .502 89 46 .22 .420 89 47 .48 .503 89 48 .52 .503 89 49 .49 .503 89 50 .42 .496 89 Scale Statistics Mean Variance Std. Deviation N of Items 20.73 49.836 7.059 50 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .794 .786 50 A Report on the Diagnostic Test for Teachers in the ELSA Text2Teach Schools in Maguindanao, Cotabato and South Cotabato 34 Background: As part of the summative evaluation of the ELSA Text2teach Project in Mindanao the same set of tests in grades 5 and 6-level English, Math and Science administered to pupils in the study and control schools were given to teachers in the ELSA Text2teach schools in Maguindanao, Cotabato and South Cotabato. The objective of the testing was to gauge teacher competency in the skills and content areas of the subjects that they are expected to handle. Results: A total of 98 teachers took the grade 5-level tests while 95 teachers took the grade 6-level tests. Tables 1 and 2 present the description of the two samples of teachers in terms of basic background characteristics. Because both samples were overwhelmingly female, we do not disaggregate by sex. Table 1 shows that the teachers who took the grade 5 level tests were mostly BSE/BSEEd graduates; half did not have a major or specialization; most obtained their degrees from a private university/college. Their mean age is 43 years and the average years of teaching experience is 15.8 years. A high 75 per cent have either obtained Masters’ degrees or earned Masters’ units; about a quarter had a rank of Master Teacher. The majority handle a combination of subjects, a comparatively smaller proportion taught only one subject￾either English, Math or Science. The highest proportion of teacher test takers came from Cotabato (39.8 per cent), followed by Maguindanao (34.7 per cent) and South Cotabato (25.5 per cent). 35 Table 1. Descriptives for Grade 5 teachers N Percent Province Maguindanao 34 34.7 Cotabato 39 39.8 South Cotabato 25 25.5 Graduated from State College/Univ 31 31.6 Private College/Univ 64 65.3 Both 3 3.1 Basic degree BSE/BSEed/BEED 82 83.7 BSE + others 5 5.1 Other non BSE 11 11.2 Major/specialization None 49 50.0 Math 8 8.2 Science 7 7.1 English 3 3.1 Others 31 31.6 Advanced degree None 24 24.5 With Masters units 47 48.0 With Masters degree 27 27.6 Age 25-29 6 6.1 30-39 34 34.7 40-49 31 31.6 50+ 27 27.6 Mean age 43 years s.d. 8.74 Range 25 to 61 Years of 0 to 5 12 12.2 teaching 6 to 10 20 20.4 11 to 15 18 18.4 16 to 20 20 20.4 21 to 39 28 28.6 Mean years of teaching 15.83 s.d. 9.27 Rank Master Teacher 24 24.5 Teacher 74 75.5 Grade level taught Grade 5 31 31.6 Grade 6 45 45.9 Multi grade 22 22.4 Subject taught Mathematics 17 17.3 English 14 14.3 Science 12 12.2 Combination 55 56.1 Total 98 100.0 36 Table 2 shows the descriptives for teachers who took the grade 6-level diagnostic tests. The characteristics of this sample does not differ much from the grade 5 test takers except for being slightly younger (mean age is 41 years), with slightly lower mean years of teaching experience (15 years). The proportion who taught a combination of subjects was lower while the proportion who taught multiple grades was slightly higher. There were more teachers in this sample who had earned either Master’s units or Masters’ degrees with two respondents having earned doctoral units. Table 2. Descriptives for grade 6 teachers Characteristic N Percent Province Maguindanao 20 21.1 Cotabato 44 46.3 South Cotabato 31 32.6 Graduated from State College/Univ 36 37.9 Private College/Univ 57 60.0 Both 2 2.1 Basic degree BSE/BSEed/BEED 57 60.0 BSE + others 19 20.0 Other non BSE 19 20.0 Major/specialization None 31 32.6 Math 10 10.5 Science 4 4.2 English 14 14.7 Others 36 37.9 Advanced degree None 14 14.7 With Masters' units 46 48.4 With Masters' degree 33 34.7 With doctoral units 2 2.1 Age 20-29 10 10.5 30-39 36 37.9 40-49 30 31.6 50+ 19 20.0 Mean age 41.3 s.d. 8.59 Range 24 to 59 Years of teaching 1 to 5 14 14.7 6 to 10 16 16.8 10 to 15 26 27.4 16 to 20 14 14.7 21 to 36 25 26.3 Mean years of teaching 15.0 s.d. 8.68 Rank Master Teacher 18 18.9 Teacher 77 81.1 Grade level taught Grade 5 31 32.6 Grade 6 36 37.9 Multi grade 28 29.5 Subject taught Mathematics 20 21.1 English 20 21.1 Science 10 10.5 Others/combination 45 47.4 Total 95 100.0 38 Tables 3 and 4 show the performance of the teachers in the diagnostic tests broken down by subject and selected teacher characteristics. Each table shows the mean score in the 50-item test and the specific p-value for the differences that reached at least the .05 level of significance. Over all teachers who took the grade 5 level test obtained a mean score of 36.1 points ( 72.2 per cent correct answers) in English, 37.5 points (75 per cent correct answers) in Math and 33.1 points (66.2 per cent correct answers) in Science while for teachers who took the grade 6-level test, the mean scores were 36.2 (72.4 per cent correct answers), 37.6 (75.2 per cent correct answers) and 32.6 points (65.2 per cent correct answers), respectively. At both grade levels, the teachers registered the highest mean scores in Math and the lowest in Science. Looking at differences in mean scores across teacher- related characteristics, these differences were most striking across province. Teachers from Cotabato scored the highest in all three subjects while Maguindanao teachers scored the lowest- both at grade 5 and 6 levels. All differences are highly significant (see tables 3 and 4). As to the other teacher characteristics, the differences in scores across type of college attended, basic degree major/specialization, age, years of teaching, rank grade level and subject taught were not significant in the grade 5-levels tests. However, having advanced training significantly increased mean scores for Math and Science but not for English. The pattern is somewhat different in the grade 6-level test with more teacher characteristics registering significant effects on mean scores. Aside from province, there was significant difference in mean scores in English, Math and Science across years of teaching. Those with the longest experience in teaching (21+ years) consistently scored the lowest. For Math, the highest scorers were those with fewer years of teaching experience (up to 10 years experience). Also, for Math, the youngest teachers registered the highest mean scores. Test participants who were handling grade 5 but took the grade 6 level test scored significantly lower than those who were teaching grade 6 or multigrade. 39 Table 3. Mean scores in grade 5 English, Math and Science by teacher characteristics Characteristic English Mathematics Science Province Maguindanao 31.3 32.5 27.5 Cotabato 39.5 41.4 38.5 South Cotabato 37.3 38.3 32.3 p-value 0.000 0.000 0.000 Graduated from State College/Univ 37.0 39.1 32.7 Private College/Univ 35.4 36.5 32.8 Both 40.0 43.7 42.3 p-value n.s. n.s. n.s. Basic degree BSE/BSEed/BEED 35.6 37.2 32.4 BSE + others 37.6 39.8 38.4 Other non BSE 39.2 38.7 35.9 p-value n.s. n.s. n.s. Major None 35.3 37.7 32.6 Math 38.0 39.5 36.3 Science 36.0 37.3 35.4 English 41.7 42.0 34.3 Others 36.2 36.3 32.4 p-value n.s. n.s. n.s. Advanced degree None 35.5 35.5 30.0 With master's units 35.9 37.0 32.5 With master's degree 36.8 40.2 36.9 p-value n.s. 0.029 0.003 Age 20-29 37.3 39.0 38.0 30-39 36.2 37.9 33.4 40-49 35.5 35.3 31.7 50+ 36.3 39.3 33.2 p-value n.s. n.s. n.s. Years of teaching 0 to 5 36.4 38.4 35.4 6 to 10 35.9 35.4 32.1 11 to 15 37.8 39.7 33.5 16 to 20 35.5 35.3 32.8 21 to 39 35.4 38.9 32.7 p-value n.s. n.s. n.s. Rank Master Teacher 37.8 40.6 36.2 Teacher 35.5 36.5 32.1 p-value n.s. 0.007 0.019 Grade level taught Grade 5 37.0 37.9 34.9 Grade 6 36.2 38.2 32.2 Multi grade 34.6 35.7 32.2 p-value n.s. n.s. n.s. Subject taught Mathematics 36.7 40.6 33.8 English 38.0 38.4 34.4 Science 37.3 39.6 33.8 Combination 35.1 35.9 32.4 p-value n.s. n.s. n.s. TOTAL 36.1 37.5 33.1 N 98 98 98 Table 4. Mean scores in grade 6 English, Mathematics and Science by teacher characteristics Characteristic English Mathematics Science Province Maguindanao 29.7 29.3 24.4 Cotabato 38.2 40.8 36.5 South Cotabato 37.7 38.5 32.3 p-value 0.000 0.000 0.000 Graduated from State College/Univ 36.3 38.3 32.9 Private College/Univ 36.1 37.0 32.2 Both 40.0 41.5 39.5 p-value n.s. n.s. n.s. Basic degree BSE/BSEed/BEED 36.1 38.2 33.5 BSE + others 36.6 34.3 29.5 Other non BSE 36.2 37.4 31.1 p-value n.s. n.s. n.s. Major none 36.2 37.6 33.9 math 35.3 37.5 33.7 science 35.5 41.8 37.8 english 39.3 38.4 32.6 others 35.4 36.8 30.5 p-value n.s. n.s. n.s. Age 20-29 35.6 41.4 33.6 30-39 37.9 39.9 34.5 40-49 35.0 34.9 31.3 50+ 35.3 35.5 30.4 p-value n.s. 0.0232 n.s. Years of teaching 1 to 5 36.3 41.1 34.7 6 to 10 38.7 42.4 35.4 10 to 15 36.8 36.4 32.2 16 to 20 38.4 38.4 34.5 21 to 36 32.8 33.4 29.0 p-value 0.0134 0.0030 0.0370 Advanced degree None 36.8 35.5 33.2 with master's units 35.7 36.3 31.0 with master's degree 36.6 40.1 34.3 with doctoral units 38.5 40.5 36.0 p-value n.s. n.s. n.s. Rank Master Teacher 37.0 39.9 33.6 Teacher 36.0 37.1 32.4 p-value n.s. n.s. n.s. Grade level taught Grade 5 33.0 33.9 29.8 Grade 6 37.6 39.4 34.9 Multi grade 38.0 39.3 32.8 p-value 0.0013 0.0087 0.0213 Subject taught Mathematics 36.2 40.1 31.3 English 37.8 35.3 32.1 41 Science 37.0 40.3 36.6 Combination 35.4 37.0 32.5 p-value n.s. n.s. n.s. Total 36.2 37.6 32.6 N 95 95 95 After looking at the bivariate relationship between each teacher characteristic and performance in the subsets of the diagnostic test, the next step in the analysis of the teacher performance was to perform multiple linear regression where we model the simultaneous effects of teacher characteristics on scores in the English, Math and Science tests. Prior to performing the regression analysis, we first recoded all the teacher characteristics into dummy variables, except for advanced degree which we treat as an interval level variable (1=no advanced degree, 2=with Masters’ units, 3=with Masters’ degree, 4=with doctoral units). We recoded age and years of teaching into dummy variables in order to capture which specific age group or category of years of teaching exerted an effect on scores. The reference category for each dummy variable are as follows: Dummy Variable Reference category Cotabato Maguindanao South Cotabato Graduated from state university Graduated from private university Both Basic degree Non-BSE Major Others Age 20-29 Age 30-39 Age 40-49 50+ Years of teaching 0-5 6-10 11-15 16-20 21+ Master Teacher Teacher Teaching Grade 5 Teaching Grade 6 Teaching multigrade Teaching English only Teaching Math only Teaching Science only Combination 42 We did a stepwise procedure and report in tables 5 and 6 only the model with the significant predictors of scores in each of the subjects and grade levels. We also report R square for the specific model to indicate the proportion of variance accounted for by the given model. Table 5a. Regression model predicting grade 5 English scores of teachers B Std. Error p-value (Constant) 38.6094 0.6361 0.0000 Maguindanao -7.3153 1.0800 0.0000 R square 0.3234 Table 5b. Regression model predicting grade 5 Math scores of teachers B Std. Error p-value (Constant) 39.0328 1.1032 0.0000 Maguindanao -5.8741 1.3307 0.0000 Cotabato 2.5439 1.3072 0.0547 Age40 -3.5761 1.0992 0.0016 Master teacher 2.6731 1.2169 0.0305 R square 0.4374 Table 5c. Regression model predicting grade 5 Science scores of teachers B Std. Error p-value (Constant) 29.6700 1.3227 0.0000 Cotabato 5.1153 1.4427 0.0006 Maguindanao -6.1031 1.4566 0.0001 Age20 5.9655 2.7269 0.0313 0 to 5 4.0759 2.0126 0.0458 Graduate studies 1.7483 0.7864 0.0287 Teaches grade 5 2.6149 1.2107 0.0334 R square 0.5341 43 Results for the grade 5 test (tables 5a to 5c) shows that, taking all characteristics together, province was the only significant predictor of English scores. Specifically, being from Mindanao subtracted 7.3 points from one’s predicted score in English, compared with the reference category (South Cotabato). There was no significant effect of being from Cotabato. For grade 5 Math scores, being from Maguindanao likewise shaved off points from one’s predicted Math score (almost 6 points) while being from Cotabato added 2.5 points (significant at .055). Being a Master Teacher added about 2.7 points but being age 40-49 subtracted 3.6 points compared with the reference category of age 50+. There are more significant predictors of Science grades compared with English and Math. As can be seen in table 5c, being from Maguindanao again resulted in shaving off points from one’s predicted score in Science but being from Cotabato, being in the age group 20-29, having 1-5 years of teaching experience and graduate studies all have a positive effect on predicted Science scores. For the grade 6 level test (tables 6a to 6c) there are two significant predictors of English score, being from Maguindanao and teaching grade 5. Both factors have a negative effect on predicted English score. For Math, being from Maguindanao takes away 9 points from one’s predicted score (compared with the reference category South Cotabato). Being age 20-29 and 30-39 adds to predicted Math score (compared with being age 50+); being in the younger age group (20-29) in fact adds more points (7.9 vs. 3.2). Having 6-10 years teaching experience also adds to predicted math score as does teaching Math. Each incremental increase in having an advanced degree adds 2 points to one’s predicted Math score. Teaching grade 5 on the other hand has a negative effect on predicted Math score. Again, being from Maguindanao has a negative effect on predicted Science score while being from Cotabato, having 1-5 years of teaching experience and teaching Science all have a significant positive effect on predicted Science score. 44 Table 6a. Regression model predicting grade 6 English scores of teachers B Std. Error p-value (Constant) 39.1077 0.6442 0.0000 Maguindanao -7.7175 1.2360 0.0000 Teaches grade 5 -3.8671 1.0747 0.0005 R square 0.3781 Table 6b. Regression model predicting grade 6 Math scores of teachers B Std. Error p-value (Constant) 34.5972 1.6807 0.0000 Maguindanao -9.0877 1.6236 0.0000 Age20 7.9111 2.1936 0.0005 Age30 3.1698 1.4175 0.0279 6 to 10 3.5121 1.7720 0.0506 Teaches grade 5 -3.5253 1.3784 0.0123 Teaches Math 3.4656 1.5913 0.0321 Graduate studies 2.1828 0.8894 0.0161 R square 0.4554 Table 6c. Regression model predicting grade 5 Science scores of teachers B Std. Error p-value (Constant) 30.4918 1.1541 0.0000 Maguindanao -7.2427 1.6770 0.0000 Cotabato 5.2146 1.3874 0.0003 0 to 5 4.0570 1.6749 0.0174 Teaches Science 5.7896 2.0052 0.0049 R square 0.4303 45 Discussion: The diagnostic tests that we administered to two sample of teachers in Cotabato, Maguindanao and South Cotabato were the same diagnostic instruments that we used in the summative evaluation of pupil performance in grades 5 and 6 using a pre-test,post￾test design. Test items were based on the competency framework for the respective grade level according to the Revised Basic Education Curriculum. Thus each test consisting of a 50-item multiple choice type of examination was designed to assess competencies that should have been acquired at the end of grade level 5 and 6. Using the DepEd yardstick of 75 per cent mastery of subject content as a measure of acceptable performance for elementary pupils, the results of this diagnostic test shows that overall the scores of the teachers who took the tests fall short of this standard in English and Science at both grade levels (below 75 per cent mean correct answers) and just at the cut point for Math (75 per cent mean correct answers). The scores in Science tend to be the lowest The low mean performance of the aggregate sample is attributable mostly to the low scores of test participants from Maguindanao. Of the three provinces in the study, Cotabato teachers scored the highest and their mean scores in the three subjects are all above the 75 per cent passing mark (although they also scored lowest in Science). South Cotabato teachers are in between. The contrast in performance between Cotabato and Maguindanao is quite striking. For example, in grade 6 Math, the mean correct answers for Cotabato is 81.6 per cent; the comparable score for Maguindanao is 58.6 per cent while South Cotabato’s mean correct answer is 77 per cent. The regression analysis produced other notable findings such as the manifest negative effect of being older and having longer years of teaching experience on performance in Math and Science- at both grade levels. This finding implies that training older teachers to improve subject competency may be less effective than training the younger teachers and those with fewer years of experience. For grade 6 Math and Science, teaching the appropriate subject (Math or Science) compared with reaching a combination of subjects has a positive effect on scores. 46 There is a need to further pursue the ramifications of these findings especially their implications on future intervention strategies to improve teacher competency in handling their classes. Clearly when teachers themselves are unable to reach the acceptable standard of performance for the grade level the repercussions on teaching and learning in the classroom are immense. The fact that the teachers who participated in the diagnostic exams are teaching in the text2teach classes imply that further improvement of knowledge and skills, especially among Maguindanao teachers is called for. Perhaps improving competence in the English language is a good entry point. When teachers are unable to master the material they will teach because of a weakness in comprehending the language in which the material is presented, then some potential for effective teaching is already lost. For example, we repeatedly heard comments from the text2teach teachers during the pre- and the post test about their difficulty in comprehending the lessons in Science narrated in American-accented English. One suggestion we heard was to dub the Science lessons in Filipino-accented English, just like the Math and English lessons. The relatively poor performance of Maguindanao text2teach teachers compared with the Cotabato and South Cotabato counterparts merit special attention especially in light of the equally poor performance of Maguindanao pupils at the pre-and post tests relative to pupils from the other provinces. What may work for Cotabato and South Cotabato to improve teacher competency may not be the best strategy for Maguindanao.