r f r_ f ~ I . [ [ I L I L l Final Report of the Comun1dades Mayas Alfabetizadas (COMAL) Project Guatemala 1999 - 2001 Monitoring and Evaluation Final Report prepared by DataPro, S.A. for Save the Children and USAID/G-CAP Cooperative Agreement 520-A-00-99-00004-0 May 2002 TABLE OF CONTENTS Introduction ................................................................................... 1 Overview ....................................................................................... 4 Monitoring and Evaluation Methodology .......................................... 5 Participant's Motivations and Expectations before Beginning Literacy Training ........................................................................................ 6 Initial Enrollment .................................................. · ......................... 8 Initial Enrollment: 1999 ............................................................................... 8 Initial Enrollment: 2000 ............................................................................... 9 Initial Enrollment: 2001 ............................................................................. 10 Initial Enrollment by Department ................................................................ 12 Initial Enrollment by Gender and Age .......................................................... 14 Literacy Strategies ....................................................................... 16 Strategies .................................................................................................. 16 Methods and Techniques ............................................................................ 16 Literacy Materials and Supplies ................................................................... 20 Community Materials .............................................................................. 21 Materials provided by the COMAL Project and the NG Os ........................... 21 The Integrated Community Literacy (!CL) Model ............................ 22 Community Development and the COMAL Model ......................................... 23 Community Projects ................................................................................... 23 Group Characteristics ................................................................... 24 Group Size ................................................................................................. 25 Duration of Literacy Sessions ...................................................................... 27 Number of Sessions per Week .................................................................... 27 Number of Hours of Sessions per Week ...................................................... 29 Results by Language Used in Group ............................................................ 31 Facilitator Characteristics .............................................................. 34 Facilitator Gender ...................................................................................... 35 Facilitator Age ........................................................................................... 35 Facilitator Educational Level ....................................................................... 35 Leadership ................................................................................................ 37 Training Received by Facilitators ................................................................. 37 Impact in the Life of the Facilitators ............................................................ 39 Project Results ............................................................................. 41 Results by NGO .......................................................................................... 41 The Institutional Support of the NGOs of the COMAL Project ....................... .42 Materials and Supplies ............................................................................ 42 Selection and Training of Literacy Facilitators ........................................... 43 Literacy Technicians and Supervision of Facilitators and Their Groups ........ 43 Results by Year ......................................................................................... 44 Results by Department ............................................................................... 47 Results by Area ......................................................................................... 49 Gender Interaction of Facilitators and Participants ....................................... SO Age Effects of Facilitators and Participants .................................................. 54 Limitations of the Adult Literacy Process ........................................ 57 Impact at the End of the Project ................................................... 59 Drop-out .................................................................................................... 61 Those Who Kept Going .......................................... , ................................... 62 Achievements and Knowledge Retention ..................................................... 63 The Impact of the Literacy Process on Self, Family and Community 64 The Impact in the Lives of the Participants .................................................. 64 Participant Self-esteem .............................................. , ............................... 64 Education of their Children ......................................................................... 64 Economy, Work and Commerce .................................................................. 65 The Utility of Numbers ............................................................................... 65 Travel Outside of the Community ............................................................... 65 Legal Formalities ....................................................................................... 65 Dealing with Health Problems ..................................................................... 66 Social Participation in the Community ......................................................... 66 Religious Practice ........................................................................................ 66 Interethnic Communication ........................................................................ 66 Interest Satisfaction ..................................................................... 67 Facilitators ............................................................................................. 67 Participants ............................................................................................ 67 Introduction The purpose of this report is to document the achievements of the COMAL Project during the three years of implementation of literacy groups in rural Guatemala. As we will see, Project implementation was systematically modified over the three years as a result of a tightly integrated monitoring and evaluation system that identified implementation weaknesses as well as successful strategies for carrying out literacy work. This report, therefore, describes how implementation was carried out each year, what results were achieved, and with which factors success was associated. To give the reader a sense of the dramatic evolution of the Project from 1999 to 2001, this section presents a series of figures that highlight the major changes. Subsequent sections present detailed tables that help understand the intricacies of Project implementation and the impressive results achieved in the final year. Figure 1 shows how much the Project expanded between 1999 and 2001. The number of literacy groups formed each year quadrupled from 313 in 1999 to 1,233 in 2001. Figure 1 3,000 Literacy Groups Formed each Year 2,500 The number of groups formed in a 2,000 '"o~r f'!!liHinJ.p/orl bF!tweon 1000 anrl 2001: half of the groups were formed in 2001 1,500 12,549 'i 1,000 500 ~ @ I IJ 313 u 1999 2000 2001 Total Figure 2 shows the same comparison for the number of participants enrolled each year. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 1of68 Figure 2 Participants Enrolled each Year bl Men II Women 50,000 -,------------============='.___----------, Total 1999-2001: 45,000 +----------------------'--'-"-'--'- 62"--'--'-- 34~-'-'--------1 Enrollment per year 40•000 -+------in re se fiiv fold ________ _______ _ 35,000 +--------i!Je{i'N*- -'t-9-997~-; 2""""":-------------- 20,000 +-------------= 1 _ ____ _ 15,000 +------------ 1,754 4,338 1999 2000 2001 Total Figure 3 shows the eight- to ten-fold increase in the number of literates produced between 1999 and 2001. Figure 3 Literates Produced each Year L_g !_~ ____ mi ome~ 30,000 ----------- ---------·- ···-----··--·---- ----------~ Total J999..200l: 37,784 25,000 +-- ]J.e. J.!lfilb.!U.J:>..LJiJN_;JJru;_pLO.IiJJ.C.eJ:i i.,...__ __________ _ a year increased 8- to 10-fo/d between 1999 and 2001 1999 2000 2001 Total As Figure 4 shows, the efficiency of the literacy process was greatly improved for both men and women. Although the Project was more effective with women than with men in 1999, by 2001 men and women both had efficiency rates above 65%. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 2of68 Figure 4 Efficiency each Year OMen • Women 70"!. -.-------- 60% +-- 'ffki.ency improv..e.dJJ¥------------~ 52% to 76% between 50% +----'"1 =..a.ui"--'J,(l.U...------1 40o/o +------ 30%+----t 1-----1 I 6s.4cr. I 20% +----I· In .23 r 10% +----I· Oo/o _.____........_......__ 1999 2000 2001 But perhaps the greatest success of the Project was to reduce the principal source of failure of participants to become literate: drop-out. As Figure 5 shows, drop-out among men was cut from 59.1 % in 1999 to 28.4% in 2001, and among women from 49.5% in 1999 to 26.0°/o in 2001. Consistent with the principles of the integrated community literacy model, more relevant content and use of active learning techniques resulted in higher retention of participants. 60% 50% 40% 30% 20% 10% 0% Figure 5 Drop-out each Year .-- -;:;:==::;;;-- ~~ ~~ w~~~ :J ------- ise.1cr. I ----I E4±!J 1999 2000 Drop-out was cut in h.a./f_ between 1999 and 2001 2001 COMAL Project Monitoring and Evaluation Report 1999-2001 Page 3of68 Improvement of access to educational services, especially to bilingual education services in rural areas from Guatemala, was established in the peace process agreements on Identity and Indigenous Rights, Socioeconomic and Agrarian issues, and Human Rights. These agreements called for a concerted effort to achieve a 70% adult literacy, including bilingual literacy in those Mayan languages where technically feasible. The COMAL Project contributed to meeting this goal through its focus on implementing in Guatemala's Zonapaz a literacy model known as Integrated Community Literacy (ICL). "There was a project for war-disabled persons, some of them had visited me at my home and told me about the literacy classes, and I agreed to participate. We came here for v some small projects, such as credits for stores and other small business ventures." (Focus group with participants. Man from Ixil area) Overview Between 1999 and 2001, the COMAL Project was developed as a model for adult literacy education in communities that previously had been socially marginalized and/or suffered from Guatemala's many years of internal armed conflict. While contributing to human resource reconstruction and development in those communities, the COMAL Project achieved extraordinary results in terms of enrollment, retention and production of literate adults. For example, in 1999 there were only 6,092 participants enrolled. This was increased in 2000 to 23,601 participants and in 2001 to 33,141 participants for a total life-of-project enrollment of 62,834 adults provided access to an alternative, integral and community-focused educational system that, in the words of participants, allows them to "open the mind." It became evident that when participants sense that their learning expectatives are being met, they tend to continue with the literacy process, sharply reducing drop-out: the major cause of failure of most adult literacy projects worldwide. Where links exist with short-term community-based development activities, this tendancy is reinforced, and participants become aware of changes in their personal, family and community lives. The COMAL Project's significant accomplishments over the 1999-2001 period became a source of great pride for all the organizations and personnel involved in the Project. Simultaneously expanding coverage (literacy goups four-fold, participants five-fold) while improving efficiency (the number of literates produced increased by eight- to ten-fold) attests to the effectiveness of the COMAL Project Monitoring and Evaluation Report 1999-2001 Page 4of68 internal review and redesign mechanisms. Key elements of the successful strategy implemented include: 1. Linking literacy training with the practical needs of adults 2. Improved access to literacy centers located close to participants 3. The literacy methodology itself as applied by facilitators and participants 4. Project implementation of financing, training, support and supervision 5. Extensive methodological and operational monitoring and evaluation 6. Extensive use of local institutions, their experience in community work and access to community leadership The COMAL Project results indicate that in order to be successful, literacy projects should satisfy the interests of young adults integrating, at a minimum elements of gender equity, community development, human rights, personal and intercultural strengthening. Monitoring and Evaluation Methodology Monitoring and evaluation of the COMAL Project involved both quantitative and qualitative methods. Quantitative results were derived from a custom-designed database system that tracked individual participants from all literacy groups formed, and generated CONALFA-required reporting forms. In addition, facilitator profiles were kept and used in data analysis. For the first two years of the Project, training plans and participant logs were used to identify specific training elements received by each facilitator. The qualitative monitoring, formative evaluation and research activities were conducted in the native language of the community by trained bilingual personnel. Between participants and facilitators, a total of 623 persons from nearly all of the municipalities where the COMAL Project was implemented. Thirteen complete studies were produced from the qualitative studies. v The techniques used by the qualitative monitoring and evaluation activities included: focus groups with participants and facilitators, observations of literacy groups and training events, in-depth interviews and followup interviews with successful women participants, in-depth interviews with drop-outs, participatory action research, rapid interviews, support to training workshops and feedback to NGOs on issues resulted from research. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 5of68 Literacy Group Formation The majority of facilitators used home visits to announce, promote, convince, and organize participants into literacy groups. Because settlement in most communities is widely scattered, this phase of literacy group organization was often one of the most difficult of the entire process. The best results came from the enrollment of people known by the facilitators: families, friends and neighbors. Facilitators also enrolled persons from organizations they belonged to, such as women's or religious groups. Another option employed was to request assistance from community leaders and authorities. Generally the facilitators followed some strategies to organize their participants groups: 1. Make an inventory of the illiterate population in the community to develop an invitation plan 2. Meet with known people or organized groups 3. Obtain consent and support from community leaders 4. Make household visits 5. Give an extensive explanation about the literacy project 6. Promote a dialog with potential participants about their needs 7. Motivate the participants to enroll and to participate in the sessions 8. Make a list of participants enrolled in the group 9. Hold an organizational meeting with participants The facilitators generally felt that it was important to know about the needs of the participants, and what they want to know and learn, in order to improve the learning process. Facilitators underscored the importance of encouraging participants through formal recognition of their achievements, such as through the use of participant's certificates. Participant's Motivations and Expectations before Beginning Literacy Training The participants' interests relate to the practical application of literacy to their daily activities in different areas: economy, communication, health, religion and community participation. Those participants already involved in a development or group activity generally have more clearly expressed ideas as to their goals for learning and what they want to learn, such as: COMAL Project Monitoring and Evaluation Report 1999-2001 Page 6of68 1. Carry out purchase and sale transactions in Spanish 2. Learn to read the bible and assume active roles in church activities 3. Participate in community projects 4. Listen to political speeches In all the groups, participants said that their highest priority was to learn to rea , write and speak in Spanish, to have "no fear" when communicating with others and to achieve economic benefits. They state that they want to learn Spanish because they feel that the use of the local language limits them in terms of their ability to communicate with other regions. Participants were also interested in obtaining their own identity card or birth 1< certificate, in order to stop dependency on other persons: "stop paying a day to someone else in order to deal with an emergency." This is very important, taking into consideration the lack of identity cards in the ZONAPAZ, where many people, especially women and children, aren't counted in the population of Guatemala. Another point of interest is to obtain the "study acknowledgement" through the literacy certificate. The participants in the COMAL Project often link their motivations to achieve literacy with life issues: 1. Self-improvement 2. Community development projects 3. Responding to encouragement from the facilitator 4. Access to socio-economic opportunities 5. Access to schools 6. Access to and participation in community organizations 7. Reduce the generation gap in terms of access to education 8. Overcome the effects of internal armed conflict In community projects, there are interest in integral literacy that involves the persons and their community relationships. In the COMAL Project, one of the ways employed to link literacy and individual practical needs was through the addition of parallel projects. Some of the NGOs participating in the COMAL Project seamlessly integrated such parallel projects with community literacy in their coverage areas, while in other cases it was the facilitator who developed and promoted community projects as a way of enhancing participant enrollment, retention and performance. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 7of68 One of the most popular focuses of community organization is a schou · 1 committee. Community members are generally interested in their children's education, and often see a potential for improving their own education as adults. An important benefit of the literacy process is an increased appreciation of the value of education generally. This represents a ma·or attitudinal change that will benefit the next generation of children, eciall ir s. Adult literacy training thus has the potential for breaking the eye nd under-education in the future. The integrated community literacy model offers adults an opportunity to increase their participation, improve self-esteem and overcome timidity. Being organized and participating in group activities empowers the participant to exercise their rights to free expression and decision-making. Speaking out ("the use of the word") also helps to mend the profound damages individuals a ~ommunities suffered through restrictions imposed during the prolonge vii conflict Initial Enrollment The history of the COMAL Project reflects the goal to expand access to literacy services in the Zonapaz. Actual literacy group implementation was carried out through direct agreements with a variety of non-governmental organizations (NGOs) and through sub-agreements managed through certain NGOs. NGOs exhibited a wide range of capability, community presence and involvement in other development activities, and as we shall see, these factors were not unrelated to the success they achieved with the literacy groups they managed. Initial Enrollment: 1999 Table 1 shows the modest beginnings in 1999, where four primary organizations (ADEJUC operated as Save the Children's primary partner, and entered into sub￾agreements with other NGOs in the department of El Quiche) organized 313 literacy groups with 6,092 participants enrolled. In 1999, ADEJUC and its partners accounted for 78% of all groups formed and 71 % of participants enrolled. Although the number of groups formed was 96% of the target for the year, small groups sizes (averaging less than 20) caused the Project to enroll only 80% of the projected participants. CCAM, FUNDADESE, ALMG and COINDI were significant under-performers in 1999, while DICOR outperformed relative to its sub-agreement. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 8 of68 ADEJUC DICOR FUNDADESE IXMUKANE COINDI FUNDAZUCAR FUND ES Total 11 NGOs NGO Initial Enrollment: 2000 325 7,597 Actual 502 47 43 1,18 827 20 68 19 69 6,092 The 2000 literacy cycle saw a very significant expansion of the COMAL Project. The number of implementing NGOs nearly doubled to 21, the number of groups formed more than tripled to 1,003 (beating the target by 12%) and 23,601 participants were enrolled (9% above expectations). Average group size increased to 23.5. In 2000, ADEJUC and its partners contributed 45% of the groups and 46% of participants enrolled. ADEJUC CRS CCAM COPROJUC FUNDADESE IXMUKANE HAD IT A CORO COINDI CODIN NGO DES MA ECODESARROLLO FE ALEGRIA FUNDAP FUNDAZUCAR COMAL Project Monitoring and Evaluation Report 1999-2001 Page 9of68 ------1 Table 2: 2000 Literacy Cycle Initial Enrollment Primary I Grou:>s Participants Organization Implementing NGO Programmed Actual Programmed Actual FUN DES 6C 8€ 1,50C 2,074 FUN RURAL 31 37 684 896 PRODESCA 47 4€ 1,026 960 SIDERGUA 13 13 378 378 Total (21 NGOs) 896 1,003 21,705 23,601 CCAM, ASOREMA and FUNDADESE fell far short of enrollment expectations, but others, notably IXMUKANE, ADISA, FUNDES and FUNRURAL, enrolled participants well in excess of those programmed. IXMUKANE both increased the number of groups formed three-fold and averaged 26.4 participants per group. ADISA added 60% more groups and 81°/o more participants. FUNDES upped its group count by 42% and its participant count by 38% with an average of 24.1 per group. FUNRURAL increased the number of groups by 19°/o, but achieved a 31 % increase in participants relative to their agreement through an average group size of 24.2. Initial Enrollment: 2001 In the final year of the Project, an NGO selection process was implemented that included careful examination of past performance and unit cost analysis and comparison based upon more realistic NGO- and department-specific data. The objective was to maximize the results obtained with limited funding while maintaining the Project's geographical, linguistic and gender focus. Furthermore, all agreements were entered into directly by Save the Children, eliminating intermediary agreements such as had existed with ADEJUC. Table 3 shows that through the 17 NGOs, 1,233 groups were formed (5% above target) and 33,141 participants were enrolled {also exceeding the target by more than one thousand). Average group size increased to 26.9. Virtually all of the NGOs met or exceeded the number of groups formed and participants enrolled established in their agreements with Save the Children. Four NGOs (IXMUKANE, ADEJUC, FUNDAZUCAR and FUNRURAL) accounted for 40% of groups formed and 43% of participants enrolled in 2001, while FUNDES, SIDERGUA and CDRO contributed another 21 % of participants between them. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 10of68 Implementing NGO DEJUC DERSO DES MA DISA SORE MA ORO OINDI onse·o de Pueblos Ma as de Guatemala FEY ALEGRIA Foro de la Mu· er FUNDAZUCAR FUND ES FUN RURAL IXMUKANE RODESCA IDERGUA otal 17 NGOs 1,175 Table 4 consolidates the three literacy cycles into a single, NGO-specific table. Twenty-seven NGOs carried out implementation, forming 2,549 literacy groups (6% above target) with 62,834 participants (2% above target). The 2001 literacy cycle alone contributed 48% of groups and 53°/o of participants. Six NGOs (IXMUKANE, ADEJUC, FUNDES, CDRO, FUNDAZUCAR and FUNRURAL) together enrolled 32,587 participants (52% of the total) over the life of the Project. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 11of68 Table 4: 1999-2001 Literacy Cycles Initial Enrollment Literacy Groups Participants Cycles Implementing NGO Programmed Actual Programmed Actual 00 CODIN 19 18 432 310 99,00,01 ~DEJUC 209 224 5,740 5,661 01 ~DERSO 41 41 1,230 1,242 00,01 ~DES MA 93 100 2,325 2,162 00 HADITA 18 18 540 429 99,00,01 ~DISA 70 86 1,703 2,207 99 ~LMG 30 22 750 436 00 ~SOD EM I 90 103 2,520 2,781 00, 01 ~SO REMA 59 56 1,750 1,658 01 Belejeb Batz 30 39 900 897 99,00 ~CAM 193 117 3,758 1,939 00, 01 ~ORO 203 208 5,075 4,801 99,00, 01 ~CIND I 102 115 3,040 2,888 01 onseio de Pueblos Mayas de Guatemala 30 35 750 982 00 ~OPROJUC 38 38 950 916 99 DICOR 36 61 579 827 00 ECODESARROLLO 25 24 375 356 00, 01 FEY ALEGRIA 140 125 2,800 2,484 01 Faro de la Muier 15 15 450 419 99,00 FUNDADESE 54 32 1,165 600 00 FUNDAP 45 51 1,250 1,454 99,00,01 FUNDAZUCAR 156 156 3,737 3,857 99,00,01 FUND ES 161 204 4,530 5,547 00,01 FUN RURAL 134 138 3,259 3,583 99,00,01 IXMUKANE 212 311 6,722 9,138 00, 01 PRODESCA 111 121 2,626 2,655 00,01 SIDERGUA 82 91 2,448 2,605 Total (27 NGOs) 2,396 2,549 61,404 62.834 Initial Enrollment by Department The COMAL Project was designed to address literacy needs in the war-ravaged departments of the Zonapaz with a focus on providing literacy services to women and youth. Table 5 shows the distribution of the 1999 initial enrollment by department and participant gender, also indicating the number of municipalities served. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 12of68 The department of El Quiche accounted for 73°/o of groups formed in 1999, and 65% of the participants enrolled. Women made up 71 % of participants from all departments, indicating that the Project was meeting its gender target correctly. A factor of some interest to observe is the wide variance in average group size between the departments during the first year of the Project. Table 5: 1999 Literacy Cycle Initial Enrollment bv Department Department Number of Number of Participants Avg. Group Municipalities Groups Male Female Total Size El Quiche 16 227 1,268 2,714 3,982 17.5 Quetzaltenango 4 7 - 149 149 21 .3 Solola 6 58 375 1,119 1,494 25.8 Suchiteoeouez 5 10 69 124 193 19.3 Totonicaoan 2 11 42 232 274 24.9 Total 33 313 1,754 4,338 6,092 19.5 29% 71% Table 6 shows that major expansion of literacy services in 2000 occurred in the departments of El Quiche, Totonicapan and Quetzaltenango, while activities were initiated in Alta Verapaz and San Marcos. El Quiche had 46°/o of participants, followed by Totonicapan with 23%. The number of municipalities served increased to 52. Women made up 70% of participants overall, ranging from 61°/o in Suchitepequez to 83% in Quetzaltenango. Average group size differences between departments were reduced in the 2000 literacy cycle. Table 6: 2000 Literacv Cvcle Initial Enrollment bv Department Department Number of Number of Participants Avg. Group Municipalities Groups Male Female Total Size Alta Verapaz 3 16 3 358 361 22.6 El Quiche 15 451 3,433 7,463 10,896 24.2 Quetzaltenango 16 121 54€ 2,621 3,167 26.2 $an Marcos 5 55 437 888 1,325 24.1 $olola 3 74 653 1,146 1,799 24.3 Suchitepequez 6 25 233 367 600 24.0 ITotonicapan 7 261 1,755 3,698 5,453 20.9 !Total 52 1,003 7,060 16,541 23,601 23.5 30% 70% The strategy employed during the 2001 NGO selection process sought greater concentration and focus of the Project, especially given the goal of providing high-quality literacy mat~rials in the local languages. This consideration led to dropping the departments of Alta Verapaz (K'ekchi') and San Marcos (M'am) for the 2001 literacy cycle. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 13of68 Table 7 shows that literacy services were provided in 56 municipalities in the five departments with an average of 22 groups and 592 participants per municipality. The participation of women averaged 69%, ranging from 64°/o in Solola to 84°/o in Quetzaltenango. All departments averaged group sizes of over 24, and two (El Quiche and Quetzaltenango) averaged 28. Table 7: 2001 Literacy Cycle Initial Enrollment by Department Department Number of Number of Participants Avg. Group Municipalities Groups Male Female Total Size El Quiche 19 516 5,146 9,700 14,846 28.8 QuetzaltenanQo 15 140 633 3,282 3,915 28.0 Solola 8 211 1,914 3,336 5,250 24.9 Suchitepequez 7 120 862 2,183 3,045 25.4 Totonicapan 7 246 1,622 4,463 6,085 24.7 Total 56 1,233 10, 177 22,964 33,141 26.9 31% 69% Table 8 presents the combined totals of the three literacy cycles in the same format. A total of 74 municipalities in seven departments were served with an average of 34 groups and 849 participants each. El Quiche remained a key Project focus (47% of participants), followed by Totonicapan (19°/o), Solola (14°/o), Quetzaltenango (12%) and Suchitepequez (6°/o). A total of 43,843 women were enrolled, making up 70% of all participants. Table 8: 1999-2001 Literacy c, cles Initial Enrollment by Department Department Number of Number of Participants Avg. Group Municipalities Groups Male Female Total Size Alta Verapaz 3 16 3 358 361 22.6 El Quiche 21 1,194 9,847 19,877 29,724 24.9 QuetzaltenanQo 20 268 1,179 6,052 7,231 27.0 San Marcos 5 55 437 888 1,325 24.1 Solola 10 343 2,942 5,601 8,543 24.9 Suchltepequez 11 155 1,164 2,674 3,838 24.8 Totonicapan 7 518 3,41S 8,393 11,812 22.8 Total 74 2,549 18,991 43,843 62,834 24.7 30% 70% Initial Enrollment by Gender and Age Figure 6 shows how well the COMAL Project targeted more youthful participants over the three years of implementation. Over 50% of participants were between 15 and 24 years of age. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 14of68 Figure 6 Age Distribution of Participants - 1999-2001 403 35% 30% 25% 203 135.93 [ 15% - ,_..-- ,__ !lA.83 J 12.13 ~. . J - J1143 r?. ,____ ,____ - 19.13 ~ n[§] r .__ 0% ~ < 15 15 • l9 20. 2 4 25. 29 30 . 34 35. 39 40 . 44 45 + Age R~nge Table 9 shows the average age of male and female participants by literacy cycle. Men were slightly younger than women, and the average age for each dropped somewhat over the life of the Project. This resulted primarily from a relative reduction in participants 40 years and older for both men and women and an increase in those under 30 years of age. Table 9: Age of Participants by Literac / Cycle and Gender Avera~e Age % Less than 30 % 30-39 % 40 and Older Literacy Cycle Male Female Male Female Male Female Male Female 1999 27.5 28.7 62% 58% 18% 20% 20% 21% 2000 26.7 27.4 66% 63% 17% 19% 17% 17% 2001 26.8 27.3 66% 63% 17% 19% 17% 17% Total 26.8 27.5 66% 63% 17% 19% 17% 17% Participants frequently cited the age of other participants (too young, too old) as one of the characteristics that they did not like about their literacy group: participants generally preferred a group relatively homogenous in age. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 15 of 68 Literacy Strategies Strategies Facilitators were very aware that earning the trust of participants in their groups was very important for project success. They employed a number of techniques and strategies to reinforce participant's self-esteem and encourage them to continue even when they were tired or frustrated. Facilitators employed dynamics and felt that they made the classes more interesting. Nevertheless, most participants were opposed to the use of dynamics because they felt that their purpose in attending literacy sessions was "for learning, not play." Adult participants have rather formal views of what an .educational process should be, and are especially sensitive to doing "child-like" things as adults. The rather negative reaction of participants to the use of dynamics could also be related to the quality of their application. Facilitators were not always clear about the differences between the purpose and content of dynamics or techniques. Another strategy used by some facilitators consisted in dividing the group by apparent learning ability or level to allow the facilitator to focus attention on participants having more trouble. While this strategy can create a better relationship between facilitators and participants, it generally also leads to a situation in which those who finish their work faster receive less attention from the facilitator. The level of learning achieved depends largely on the participant's interest in reading, writing and numerical ability. Facilitators expressed the view that it is easier for participants to count and recognize numbers, but they have great difficulty writing them. In order to evaluate participants' learning, facilitators most frequently called them one-by-one to the blackboard. The reading and writing of K 'iche ' was considered difficult by both facilitators and participants. Methods and Techniques Receiving training in a topic by itself does not ensure that it will be effectively used--the duration and quality of the training, post-training follow-up and supervision, etc. are also important considerations. Nonetheless, knowing what training topics and content were provided to how many facilitators and the results they achieved helped the Project focus on more effective training elements. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 16of68 Table 10 looks at those training topics or contents that were received by at least 10°/o of the facilitators in 1999 and that produced above average pass rates among facilitators who had been exposed to them. Table 10: 1999 Uteracv Cycle Final Results by Trainina Topic Received by Facilitators Results shown only for the 35 topics with above average results and received by at least 10% of facilitators. 89 different topics were given in 1999, 62 of which were received by at least 10% of facilitators. The average pass rate for 1999 was 41 .6%. Pass Rate if Percentage of Received Facilitators Trained in Trainina Content Tvpe Trainina Topic/Content this Topic/Content Project Information lnformaci6n sobre evaluaci6n final 49.0% 16.7% Dynamic Pasando el marcador 48.5% 10.8% Dynamic La tela arana 48.5% 10.8% Mathematics Bingo {loteria) de resta 48.2% 14.6% Mathematics Bingo {loteria) de suma 48.2% 14.6% Project Information Recomendaciones 47.9% 18.1% General Knowledge Educaci6n formal - no formal 47.6% 30.5% Dynamic Rompecabezas 47.6% 14.5% Reinforcement Exercises Carta del alcalde 47.4% 16.6% Techniques Nuestras fiestas 47.4% 16.6% Techniques Bingo {loteria) de letras y palabras 47.3% 13.6% Techniques Diario personal y diario de suenos 47.3% 13.6% Dynamic El f6sforo 47.3% 13.6% Techniques Mapa del cuerpo humano 46.9% 14.9% Icebreakers ldentificaci6n con letras 46.9% 14.9% Techniques T em as de interes 46.8% 13.3% Dynamic La papa caliente 46.6% 15.7% General Knowledge Escritura y actualizaci6n de K'iche' 46.2% 42.9% Educaci6n bancaria y educaci6n General Knowledge participativa 46.0% 31.6% Reinforcement Exercises lntercambio de frases 45.9% 35.8% Project Information Test de formadores 45.8% 16.1% Reinforcement Exercises Leer en tiras de papel 45.1% 35.8% Techniques Historias colectivas 44.1% 64.0% Mathematics Secuencia de numeros 43.9% 67.9% Techniques Dinamica de analisis 43.9% 64.9% General Knowledge Logros, problemas y alternativas 43.4% 64.9% Mathematics El reloj 43.4% 44.1% Techniques Secuencia de oraciones 43.2% 48.5% Techniques Secuencia de palabras 43.2% 48.5% Techniques Ri achlal [la familia] 43.1% 47.1% Kab' awil winaq [personalidades del Techniques ser humano] 43.1% 47.4% Mathematics El calendario 43.0% 45.9% Techniques Curaci6n de enfermedades 42.4% 55.5% Techniques Ri utzilal qab' aquil [la salud] 42.4% 48.1% Techniques Ri wuqub' q'ii rel dia domingo] 42.4% 48.1% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 17 of 68 Of the 89 topics or techniques given in 1999 training events, 62 were received by at least 10°/o of the facilitators. Of these 62, 35 were associated with results above average, although only the first 21 contributed a 10% or greater relative improvement in pass rates (equivalent to a 4.2% absolute increase). Unfortunately, these topics and techniques were not received by a very high percentage of facilitators in their 1999 training-frequently fewer than one in six. Table 11 examines, using the same criteria as above, the training topics and content associated with above-average pass rates in the 2000 literacy cycle. There were 97 different topics or contents given in 2000, 80 of them received by at least 10% of facilitators. Of these, 68 were associated with above-average pass rates among facilitators who had received them. Only the first 12, however, are associated with a relative improvement of 10% or more (equivalent to a 5.7°/o absolute increase above the average for the year). Generally, about one in four facilitators received training in these topics. Table 11: 2000 Literacy Cycle Final Results by Training Topic Received by Facilitators Results shown only for the 68 topics with above average results and received by at least 10% of facilitators. 97 different topics were given in 2000, 80 of which were received by at least 10% of facilitators. The averaae oass rate for 2000 was 56.9%. Pass Rate Percentage of Facilitators if Received Trained in this Training Content Type Training Topic/Content Topic/Content Project Information Asistencia tecnica 70.0% 14.6% Project Information Perfil de entrada de los tecnicos 67.5% 11.1% Project Information Perfil de salida de los tecnicos 67.5% 11 .1% Hoja de instructive de General Knowledge evaluaci6n 65.9% 27.5% Reinforcement Exercises Evaluaci6n de escritura 65.7% 27.4% Reinforcement Exercises Evaluaci6n de lectura 65.7% 27.4% Project Information Aspectos administrativos 65.6% 27.8% Manual ACI: tecnicas y juegos Techniques para fluidez/asentar destrezas 65.5% 20.5% Evaluaci6n del calculo Mathematics matematico 65.4% 27.6% Perfll de entrada de los Project Information facllitadores 65.0% 24.3% Perfll de salida de los Project Information facilitadores 65.0% 24.3% Capacidades del buen Project Information facilitador(a) 62.8% 21 .9% General Knowledge Logros, problemas y alternativas 61 .6% 63.9% lnformaci6n sobre evaluaci6n Project Information final 60.9% 56.5% Techniques Ri kinsancho [mis gastos] 60.7% 32.2% Ri upatanil ri q'ayes [la utilidad Techniques de las hierbas) 60.5% 31 .6% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 18of68 Table 11: 2000 Literacy Cycle Final Results by Training Topic Received by Facilitators Results shown only for the 68 topics with above average results and received by at least 10% of facilitators. 97 different topics were given in 2000, 80 of which were received by at least 10% of facilitators. The averaae pass rate for 2000 was 56.9%. Pass Rate Percentage of Facilitators if Received Trained in this Trainina Content Tvpe Trainina Topic/Content Topic/Content Techniques l Que podemos hacer? 60.5% 28.2% Techniques El mercado [Ri k'ayib'al] 60.5% 31 .6% Detectemos nuestros problemas Techniques comunales 60.5% 27.8% Icebreakers Presentaci6n por parejas 60.3% 28.7% Sugerencias para utilizar Techniques graficos 60.3% 31 .2% Ub' anik ri qab'aqil [el cuerpo Techniques humano) 60.3% 48.2% Diario personal y diario de Techniques suefios 60.2% 28.9% Dynamic El f6sforo 60.1% 30.5% Presentaci6n del Proyecto Project Information COMAL 60.0% 22.1% Techniques Ri utzilal qab' aquil [la salud] 59.8% 34.8% Mathematics Secuencia de numeros 59.6% 72.8% Utikik taq ri qatiko'n [la siembra Techniques de nuestras cosechas) 59.5% 32.5% Dynamic Juego de tarjetas de numeros 59.5% 35.7% Techniques RI wuqub' q'ij [el dia domingo] 59.4% 33.9% Techniques La lluvia de ideas 59.3% 32.2% Jawi Ek'o ri qatiko'n [en d6nde Techniques estan nuestras siembras] 59.2% 32.6% Ri jupq ri ek6 pa ri komon [grupo Techniques que esta en mi comunidad] 59.1% 32.5% Dynamic Rompecabezas 59.1% 29.9% Kab' awil winaq [personalidades Techniques del ser humano] 59.0% 30.5% Xpe wi ri Qak'aslemal [el origen Techniques de nuestra vida] 59.0% 30.5% Icebreakers ldentificaci6n con letras 59.0% 46.5% Dynamic Juego de objetos 59.0% 31.9% Ri qachak ronojel q'ij [nuestros Techniques trabajos diarios) 58.8% 33.7% Project Information Test de formadores 58.7% 45.3% Kapewi uchuq; qb'il qak'aslemal Techniques [de d6nde nuestras fuerzas) 58.3% 32.5% Techniques Dinamica de analisis 58.3% 45.6% Techniques Los trabajos de la casa 58.3% 34.3% Mathematics Aplicaci6n de las matematicas 58.3% 62.2% Lo que hacen las nifias y los Techniques ninos 58.2% 34.0% Educaci6n bancaria y educaci6n General Knowledge participativa 58.1 % 25.6% Techniques Nuestras fiestas 58.1% 81 .9% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 19of68 Table 11: 2000 Literacy Cycle Final Results by TraininQ Topic Received by Facilitators Results shown only for the 68 topics with above average results and received by at least 10% of facilitators. 97 different topics were given in 2000, 80 of which were received by at least 10% of facilitators. The average pass rate for 2000 was 56.9%. Pass Rate Percentage of Facilitators if Received Trained in this Training Content Type TraininQ Topic/Content Topic/Content Project Information Elementos de ACI 58.1% 85.7% Reinforcement Exercises Carta del alcalde 58.1% 81.3% Techniques Curaci6n de enfermedades 58.0% 34.3% Techniques El mapa de mi familia 58.0% 33.7% Escritura y actualizaci6n de General Knowledge K'iche' 58.0% 51.4% Dynamic La papa caliente 58.0% 44.8% Techniques C6mo me llama 57.9% 38.1% Project Information Recomendaciones 57.9% 23.2% Dynamic Hilo conductor 57.7% 94.3% Techniques Historias colectivas 57.7% 75.9% Techniques Temas de interes 57.7% 22.3% Expectativas y compromisos del General Knowledge taller 57.6% 93.0% General Knowledge La educaci6n de adultos 57.5% 62.6% General Knowledge Educaci6n formal - no formal 57.5% 37.8% Reinforcement Exercises Lectura individual y grupal 57.5% 94.9% Reinforcement Exercises Trazo de letras 57.4% 93.4% General Knowledge C6mo aprenden los adultos 57.3% 60.8% Reinforcement Exercises lntercambio de frases 57.2% 72.0% Mapa de la comunidad [Le Techniques tin am it] 57.1% 63.2% Mathematics Bingo (loterfa) de resta 57.0% 45.0% Mathematics Bingo (loterfa) de suma 57.0% 45.0% Literacy Materials and Supplies Material provision and use can enhance or impede the literacy process. The materials used in the COMAL literacy Project may be divided into three categories: 1. Material that can be obtained in the community. 2. Material provided by COMAL Project 3. Material provided by the NGO COMAL Project Monitoring and Evaluation Report 1999-2001 Page20of68 Informal education uses and encourages the use of appropriate teaching materials. Depending on the environment of the learning process, common objects known to the participants are employed. For example, beans or corn seeds can be effectively used as a substitute for the abacus used in a formal education setting. Even animals known to the participants can be used for learning. Community Materials Among these are those that the facilitators found and took advantage of primarily for teaching mathematics or developing participant knowledge of their environment. These materials were seen as an important resource for teaching, but were viewed as an alternative resource that could not substitute for the lack of formal supplies such as notebooks, pens, worksheets, writing boards, etc. Community materials identified as useful for teaching mathematics included beans and corn grains, small rocks and coins. Facilitators also use fruits, vegetables, and ornamental and medicinal plants for: 1. Learning how to read and write and read the names of fruits and plants 2. Motivate participants' interest in practical issues of daily use or application 3. The use of medicinal plants for healing The locale and furniture where literacy sessions were carried out is also included in this category because inadequate conditions in some cases made both facilitators and participants uncomfortable. Materials provided by the COMAL Project and the NGOs The teaching/learning process carried out in the literacy sessions was felt by facilitators to require a minimum set of materials: a blackboard, pencils, erasers, notebooks, sharpeners, etc. The Project, for methodological reasons, downplayed the importance of blackboard use. This is because the adults need to memorize and to set up in their mind what is they are to write or that they had learned before. Because a blackboard is subject to erasure at any moment, erasing as well the written conceptual product. Paper-based writing survives indefinitely, allowing for review and improvement with time. Currently, I am using a blackboard, because the house where we have our sessions has plastering you can't write on paper over. What I did to solve the problem was to buy a blackboard, even though someone told us that it is not necessary, but I am using the COMAL Project Monitoring and Evaluation Report 1999-2001 Page 21of68 blackboard to put the papers on so participants can write on the paper. (Focus Group with Facilitators. Woman Totonicapan, 2001) The lack of materials or their running out caused difficulties in carrying out the literacy sessions. In some cases, this situation accelerated participant drop-out or group disintegration. To enhance the probability of success in literacy efforts, it is important to pay proper attention to the quality, adaptability, quantity and delivery time of materials. When faced with the lack of materials the facilitators looked for alternatives to provide them to the participants, such as obtaining materials from another institution, borrowing materials from the local school, or purchasing them themselves. The delivery of literacy materials should not be limited to a single delivery at the beginning of the literacy cycle, but must also foresee materials use over the entire cycle. Facilitators also requested improvement in the delivery of the literacy certificates. Specific requests from facilitators for materials included blackboards, some way to keep participant papers and notebooks, wood crayons for each participant, more pencils, erasers and sharpeners. Participants were highly critical of beginning literacy classes without materials yet being available, again emphasizing the need for literacy projects to efficiently handle these logistical aspects. The Integrated Community Literacy (ICL} Model The ICL model implemented by the COMAL Project did not limit itself to the literacy process alone, but it promoted development of individual and community values, as evidenced through the inclusion of subject material covering human rights, culture, sexuality and others pertaining to health. Learning to read and write is certainly a central focus of a literacy project, but the why of becoming literate is equally important. Learning to read and write does not necessary imply the satisfaction of the immediate and strategic needs of adult participants. ICL represents an alternative form of literacy training, and the methodology combines increase in knowledge with the acquiring of literacy skills. The ICL model appears to have had good results, particularly in terms of participant attendance, retention and promotion. Inclusion of themes relevant to participants, such as embroidering by women or agricultural activities, received a high degree of acceptance by participants. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 22of68 Because of this interest, interinstitutional coordination becomes very important, especially in places in which government investment is scarce and the NGOs are the often the only force supporting community organization and development. Facilitators often take the initiative to establish interinstitutional contact in order to obtain supplies, initiate projects, and obtain economic and technical support. Community Development and the COMAL Model One of the basic principles of the COMAL Project and the ICL methodology consisted in the integration of the literacy process with individual life projects and community development. This integration wasn't always achieved and sometimes unfulfilled NGO and facilitator promises created false expectations among facilitators and the participants in their groups. After an participatory needs assessment, a group might decide to combine literacy training with development activities such as farming, poultry, hog breeding, health, community organization, food aid projects or community development project proposals such as electric energy, potable water and roads. Because of this, facilitators suggested their need for additional training in such topics as simple nutritious cooking recipes, or how to form committees. Some techniques promoted by the Project, such as "the basket" and "health" have had considerable success. Facilitators also worked with generative themes based on problems found in the community and how they could be solved. Facilitators generally felt that it was important for the literacy process be accompanied from the beginning to the end with visible, practical projects for the participants. This ideal situation, however, was not often achieved. Community Projects Some projects carried out by women in conjunction with their ICL literacy classes include: handicrafts, needlework, darning, machine embroidering, crochet, belts with embroidered beads, small carpets, tablecloths, purses, cooking, meals and pastry, the calendar and the days of the month, use of medicinal plants, talks about diarrhea, homemade whey, family planning, nutrition and hygiene. The projects carried out by men included: contacts with community improvement committees, contacts with auxiliary mayors, pastors and church elders, contacts with other institutions, greenhouses, vegetables gardens, organic and chemical fertilizer production and use, and vaccination of pigs and chickens. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 23of68 Group Characteristics Absenteeism, drop-out and group disintegration have been shown to be the largest threats to the success of literacy groups. Most facilitators agreed that combining practical activities with the literacy activity was the best way to avoid drop-out and group disintegration. Activities such as handicrafts or embroidery functioned well with women groups. Others had used techniques of natural medicine with plants and farmer techniques, but this practice had not been well received by men, basically because it requires physical effort like their daily labors in the field. To reduce the absenteeism, facilitators use several strategies: 1. Home visits 2. Support those who are frequently absent 3. Individually, motivate to learn 4. Use motivational techniques in teaching 5. Teaching through drawing Facilitators indicated that during the home visits they also had to inquire about any problems or disagreements the participants might have, and try to make adjustments such as changing the session schedule. Facilitators also indicated a number of attitudinal aspects important for enhancing attendance, retention and promotion: 1. Deliver what you promise 2. Arrive at the session first 3. Never miss a session 4. Encourage the participants 5. Be patient with the participants 6. Understand how adult learning works 7. Collaborate with the participants 8. Hold the participants' hand to teach them how to use the pencil 9. Write jointly with the participants 10. Don't be angry 11. Don't humiliate participants The monitoring and evaluation system tracked many characteristics of the literacy groups formed and the facilitators who led the groups. Subsequent sections of this report will show how these characteristics relate to the success of participants, and these analyses were continually used to improve Project performance. The following tables frequently show major changes in group and facilitator characteristics that resulted from implementing the results of these analyses. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 24of68 Group Size The relatively poor results obtained in 1999-far fewer participants than expected-focused early attention on the size of groups formed. Table 12 shows that in 1999 24% of all groups had fewer than 15 participants and only 8% had more than 30. The results section will show the serious consequences of small initial group size in terms of drop-out and disintegration of groups. In the 2000 literacy cycle, very small groups were reduced to 5°/o of the total, and by 2001 to 3%. At the same time, groups with more than 30 participants comprised 11°/o in 2000 and fully 23% in 2001. An important part of the efficiency gains of the Project derived from this shift in the size of the groups. Table 12: Patterns of Group Size by Year Group Size Year 1999 2000 2001 Total Less than 15 24% 5% 3% 7% 15-30 68% 84% 74% 77% More than 30 8% 11 % 23% 16% Figure 7 shows this pattern graphically. Small groups went from almost a quarter of all groups in 1999 to 3% in 2001, while large groups increased from 8% of the 1999 groups to 23% in 2001. Figure 7 Patterns In Group Size C o 1999 000 02001 90% --- 80% 70% t 60% ~ 50% f ::: 20% 10% 0% Less than 15 15. 30 More th an 30 COMAL Project Monitoring and Evaluation Report 1999-2001 Page 25 of68 Table 13 shows that in 1999, small group sizes had poor results. High drop-out (46%) and group disintegration (16%) rates caused small group pass rates to be only 34%, compared to the 1999 average of 42%. Drop-out and disintegration were lowest in groups of 30 or more participants, although, in accordance with expectations, the failure rate rose slightly with group size. This was interpreted as a "critical mass" phenomenon rather than one of educational quality, and reinforced the need to initiate groups with sufficient participants to avoid disintegration due to dwindling numbers of participants. Table 13: 1999 Literacy Cycle Final Results by Size of Group Literates Result Size of Group Produced Pass Fail Drop-out Disintegrated Less than 20 814 34% 5% 46% 16% 20 -29 1,117 46% 6% 43% 5% 30 or more 605 48% 8% 41% 3% Total 1999 2,536 42% 6% 44% 9% AveraQe Size 1999: 19 In 2000, better group management brought an overall reduction in drop-out and group disintegration. Table 14 shows that above-average pass rates and lower drop-out rates were now occurring in smaller groups. The increase in drop-out rates with increasing group size is quite marked (although group disintegration falls at the same time). The 2000 results suggest that overall efficiency is best achieved with group sizes of between 20 and 29. Table 14: 2000 Literacy Cycle Final Results by Size of Group Literates Result Size of Group Produced Pass Fail Drop-out Disintegrated Less than 20 2,429 63% 5% 24% 7% 20 - 29 7,965 57% 5% 31% 6% 30 or more 3,042 52% 6% 41% 2% Total 2000 13,436 57% 5% 33% 5% Average Size 2000: 24 This interpretation appears to be validated by Table 15, which presents the same analysis for the 2001 literacy cycle. Pass rates are virtually identical for small￾and medium-sized groups, but groups with more than 30 participants have markedly higher literacy examination failure rates, leading to a reduced efficiency. Small groups continue to be at a higher risk of suffering group disintegration. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 26 of68 Table 15: 2001 Literacy Cycle Final Results by Size of Group Literates Result Size of Group Produced Pass Fail Drop-out DisinteQrated less than 20 957 67% 3% 27% 3% rlO- 29 10,480 68% 5% 27% 1% ~O or more 10,375 64% 9% 25% 1% Total 2001 21 ,812 66% 7% 26% 1% AveraQe Size 2001: 27 Duration of Literacy Sessions Although participants expressed satisfaction with the number of hours and days per week of their literacy sessions, they were also very aware that they needed more time to learn. While uncertain how much more time they needed, many felt that years could be necessary (one to three years was often expressed). Some participants explained that this was the third literacy course they had participated in, and still did not feel that they had learned all they needed to. Because of work commitments, most participants preferred session to be held in the evening hours or on weekends. In my case, we organized our group and then established the schedule, because the participants could not attend during day hours, all have responsibilities. But then they set the time, because if they said in the night hours then in the night, most requested that we have our sessions on Saturday and Sunday, from three p.m., but during weekdays only from four p.m. (Focus Group with Facilitators. Woman, Quiche, 2001) While evaluating the time that they spent in the literacy sessions, participants reflected on what they had learned as key to whether the time was well spent, or was too little or too much. Those participants whose experiences in the group were less than satisfactory often explained their absenteeism or drop-out as due to a "lack of time." The lack of time for literacy activities was clearly associated with the individual's priorities. Number of Sessions per Week Table 16 looks at the relationship between the number of sessions per week and the outcome of the groups in 1999. Most groups held sessions 2 or 3 times a week. Lower failure rates generally were associated with more sessions per week. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 27of68 Table 16: 1999 Literacy Cycle Final Results by Number of Sessions per Week Literates Result Sessions per Week Produced Pass Fail Drop-out Disintegrated 1 66 49% 14% 37% 0% 2 1,059 39% 7% 45% 10% 3 932 43% 6% 43% 10% 4 423 44% 3% 48% 5% 5 27 54% 10% 36% 0% 6 0 0% 0% 0% 100% 7 6 33% 0% 67% 0% Total 1999 2,513 42% 6% 44% 9% Table 17 repeats the analysis for the 2000 cycle. There were more groups held three or more times per week, and these usually had higher pass rates due to lower failure rates rather than through lower drop-out or group disintegration. Table 17: 2000 Literacy Cycle Final Results by Number of Sessions per Week Literates Result Sessions per Week Produced Pass Fail Drop-out Disintegrated 1 339 41% 10% 41 % 7% 2 3,696 55% 6% 33% 6% 3 7,553 58% 5% 32% 4% 4 813 57% 3% 38% 2% 5 918 65% 2% 28% 5% 6 44 48% 0% 0% 52% Total 2000 13,363 57% 5% 33% 5% The 2001 literacy cycle, shown in Table 18, does not produce a very clear picture. Groups that met two, three or four times per week all had identical pass rates. Table 18: 2001 Literacy Cvcle Final Results by Number of Sessions per Week Literates Result Sessions per Week Produced Pass Fail Drop-out Disintegrated 1 198 70% 6% 24% 0% 2 4,313 66% 7% 26% 2% 3 12,086 66% 7% 26% 1% 4 3,067 66% 6% 28% 0% 5 1,782 64% 7% 29% 1% 6 124 62% 3% 20% 15% Total 2001 21,570 66% 7% 26% 1% Table 19, which shows the life of Project results, suggests that groups that meet three to five times a week enjoy higher pass rates. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 28of68 Table 19: 1999-2001 Literacy Cycles Final Results by Number of Sessions per Week Literates Result Sessions per Week Produced Pass Fail Drop-out Disintegrated 1 603 49% 10% 37% 5% 2 9,068 57% 7% 32% 5% 3 20,571 62% 6% 29% 3% 4 4,303 61% 5% 33% 1% 5 2,727 64% 5% 29% 2% 6 168 55% 2% 13% 30% 7 6 33% 0% 67% 0% Total 1999-2001 37,446 60% 6% 30% 3% Number of Hours of Sessions per Week It was noted that from 1999 to 2001 the average numbers of session hours increased from 6.3 to 9.1. Table 20 examines the relationship in 1999 between session hours per week and outcomes. Most groups had session hours between five and seven per week in 1999, achieving a 39% pass rate. Groups holding sessions between eight and ten hours per week had an average pass rate of 43°/o as a result of lower group disintegration rates. Table 20: 1999 Literacy Cycle Final Results by Number of Hours of Sessions per Week Literates Result Hours per Week Produced Pass Fail Drop-out Disintegrated 4 or less 490 51 % 12% 28% 9% 5-7 1,230 39% 5% 45% 12% 8 - 10 685 43% 5% 50% 3% 11 or more 98 39% 3% 52% 6% Total 1999 2,503 41 .7% 6.0% 43.7% 8.7% Average Hours 1999: 6.3 The 2000 cycle, shown in Table 21, shows the effect of increased session hours. Groups with more session hours generally had higher pass rates, and lower failure, drop-out and group disintegration rates. Contributing to the improved results in 2000 was a shift in the number of groups with more session hours per week. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 29of68 Table 21: 2000 Literacy Cycle Final Results by Number of Hours of Sessions per Week Literates Result Hours per Week Produced Pass Fail Drop-out Disintegrated 4 or less 1,177 48% 9% 31% 11% 5-7 4,283 56% 6% 32% 6% 8 - 10 5,870 57% 5% 36% 3% 11 or more 2,011 67% 4% 25% 4% Total 2000 13,341 57% 5% 33% 5% Average Hours 2000: 7.9 With an average of 9.1 session hours per week in 2001 (Table 22), Project performance continued to increase. Groups with eight or more session hours per week had the best results, due to lower drop-out rates rather than any other factor. The combination of the number of sessions per week and total session hours offered per week would appear to accommodate participant's needs best. Table 22: 2001 Literacy Cycle Final Results by Number of Hours of Sessions per Week Literates Result Hours per Week Produced Pass Fail Drop-out Disintegrated 4 or less 305 64% 5% 32% 0% 5-7 4,015 63% 7% 28% 1% 8 - 10 11,346 66% 7% 26% 1% 11 or more 5,687 67% 7% 24% 1% Total 2001 21,353 66% 7% 26% 1% Average Hours 2001 : 9.1 Table 23 confirms this view for the combined three-year set of data. Pass rates rise with more session hours per week, but due primarily to reduced drop-out and group disintegration rather than lower failure rates. If participants feel that insufficient learning is being accomplished over a period of time, they will experience an increased tendency to drop out rather than tolerate the slow pace of learning. Table 23: 1999-2001 Literacy Cycles Final Results by Number of Hours of Sessions per Week Literates Result Hours per Week Produced Pass Fail Drop-out Disintearated 4 or less 1,972 51% 9% 30% 9% 5-7 9,528 56% 6% 33% 5% 8-10 17,901 61% 6% 31% 2% 11 or more 7,796 67% 6% 25% 2% Total 1999-2001 37,197 60% 6% 30% 3% Averaae Hours 1999-2001: 8.3 COMAL Project Monitoring and Evaluation Report 1999-2001 Page 30of68 Another problem detected with the 1999 groups was the tendency to offer too few session hours per week. In 1999, 22% of all groups were scheduled to operate with fewer than six hours per week and only 12°/o of the groups were offered nine or more hours per week. This sort of scheduling did not adapt itself well to the needs of the participants, and thus had serious impacts on attendance and learning. Table 24 shows the significant changes in subsequent years-by 2001 the average number of hours per week had increased 44% to 9.1, groups held fewer than six hours per week comprised only 3% of the total while those operating nine or more hours per week made up fully 61 % of all groups. Table 24: Session Hours per Week by Year Session Hours per Year Week 1999 2000 2001 Total Less than 6 22% 13% 3% 9% ~-8 66% 38% 37% 41% Sor more 12% 49% 61% 50% !Average 6.3 7.9 9.1 8.3 Figure 8 shows clearly how groups that held sessions nine hours or more per week increased from 3% in 1999 to 61 % of all groups in 2001. Figure 8 Session Hours per Week ~~~ _ _rti2o~? D2"001J 70% ----- ------- -·--·- - -------- - 60%+---- 503+---- t ~40%i---- !30%-t---- • u l 20% l0% 0% __ _.___ Leu lhan 6 : . l .. ~ .' : . -;: i .• j ' 1 Efii I ._, ., . • : :;·, i ~~" I I ' 6·8 Results by Language Used in Group I ! . i I :g i I .m.LJ 9 or rTDre The general philosophy of the COMAL Project was to allow the group to decide in what language the group would become literate. Fieldwork indicated that most COMAL Project Monitoring and Evaluation Report 1999-2001 Page 31of68 participants favored acquiring literacy skills in Spanish, usually in combination with the local language. Table 25 looks at the results from 1999 by language. Table 25: 1999 Literacy Cycle Final Results by Languaae Used in Group Literates Result Group Language Produced Pass Fail Drop-out Disintegrated Spanish 342 49% 14% 30% 7% K'iche' 743 44% 6% 48% 2% K'iche'-Spanish 1,370 39% 4% 44% 13% lxil 20 22% 24% 53% 0% Other 61 50% 5% 45% 0% Total 1999 2,563 42% 6% 44% 9% In 1999, high drop-out and group disintegration rates among groups held in either the local language or in combination with Spanish caused Spanish-only groups to achieve a significantly higher pass rate ( 49%) than the others. Much in terms of materials and techniques were improved in the 2000 literacy cycle, especially in terms of language support in K'iche'. Table 26 shows the results for 2000, where Spanish-only groups did not fare as well and local language or bilingual groups did. Table 26: 2000 Literacy Cycle Final Results by Language Used in Group Literates Result Group Language Produced Pass Fail Droo-out Disintearated Spanish 2,384 51% 6% 35% 8% K'iche' 1,036 56% 3% 31% 10% K'iche'-Soanish 6,591 63% 5% 28% 4% lxil 625 66% 1% 24% 9% lxil-Soanish 2,465 48% 7% 42% 4% Other 335 65% 3% 32% 0% Total 2000 13,436 57% 5% 33% 5% In 2001, the difference in outcome rates between languages becomes smaller, as shown in Table 27. Spanish-only groups had higher drop-out rates (30%), leading to a lower pass rate in 2001. K'iche-Spanish groups achieved the highest pass rate, due to a low drop-out rate of 19%. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 32of68 Table 27: 2001 Literacy Cycle Final Results by Lanr:iuar:ie Used in Group Literates Result Group Language Produced Pass Fail Droo-out Disintegrated Spanish 7,368 62% 7% 30% 1% K'iche' 2,009 67% 7% 26% 0% K'iche'-Soanish 9,913 69% 7% 23% 1% lxil 329 67% 14% 19% 0% !xii-Spanish 1,889 67% 10% 23% 0% Other 304 53% 1% 46% 0% Total 2001 21,812 66% 7% 26% 1% Table 28, combining all years of the Project, generally confirms this pattern. The highest pass rate (61 %) is observed among bilingual (Mayan-Spanish) groups, followed by Spanish-only and Mayan only groups, tied at 59% pass rates. The advantage enjoyed by the bilingual groups derives from the lower drop-out rate (all language groupings had similar failure and group disintegration rates). Reflecting both the Project's geographical focus and participant preferences, 47% of participants made literate by the Project did so in K'iche'-Spanish groups, 27°/o in Spanish-only groups, 12% in !xii-Spanish groups, 10°/o in K'iche'-only groups and 3°/o in Ixil-only groups. Table 28: 1999-2001 Literacy Cycles Final Results by Language Used in Group Literates Result Group Language Produced Pass Fail Drop-out Disinter:irated Spanish 10,094 58% 7% 31% 4% K'iche' 3,788 58% 5% 33% 3% K'iche'-Soanish 17,874 63% 6% 27% 4% I xii 974 64% 7% 24% 6% I xii-Spanish 4,354 55% 8% 35% 3% Other 700 58% 2% 40% 0% Total 1999-2001 37,784 60% 6% 30% 3% Among the difficulties expressed by participants in learning to read and write in their native language, the most often mentioned were: 1. They feel that the Spanish language is easier 2. That the writing of K'iche' gets complicated by the "little dots", (diacritical marks for the long vowels), the apostrophes and the pronunciation 3. That the K'iche' language has more letters than does Spanish Participants say they wish to learn Spanish because, since K'iche' is their native language, they say they know it "from birth." The true reason, however, is that COMAL Project Monitoring and Evaluation Reporl 1999-2001 Page 33of68 participants have a series of social and economic motivations to learn Spanish. Participants also express a belief that they will still be illiterates because of what they identify as dialectic differences of the K'iche' language between regions and villages. It appears, nonetheless, that there is a tendency among facilitators to favor bilingual teaching. In this approach, for example, they will first write a word in Spanish, and then write it and explain it in K'iche'. Although facilitators recognized that bilingual teaching slows down the class, it was also seen as usual for those participants less adept with verbal Spanish, especially women. Facilitator Characteristics It was often difficult for the NGOs to recruit facilitators from the community who had a high education profile. In general, however, they sought out facilitators with at least a third grade formal education, had exercised a certain level of community leadership, and were felt would fulfill requirements on time and at a performance level that each NGO established. Thus, the majority of the facilitators had prior experience as community leaders, health promoters or religious facilitators, and even included former literacy teachers from CONALFA. The uncertainties and insecurities mentioned by first-time literacy facilitators were primarily based on mistrust of NGO promises for accompanying projects and becoming held accountable in their community for failing to deliver on those promises. In addition, many facilitators were concerned about the secuelae of the psychosocial armed violence that still created resistance among community members to being placed on a list or to provide personal information on information about community leaders. This section looks at pass rates by a number of facilitator characteristics by year where available. Facilitator gender and age issues are examined separately in a later section. In Table 29, the reported prior experience of facilitators in conducting literacy work was examined in relation to pass rates by year. In all years, prior literacy experience contributed to better results. This was particularly so in the 2000 literacy cycle, where experienced literacy workers achieved an absolute increase in the pass rate of 10%. Table 29: Pass_R.ates by Facilitator Prior Literacy Experience by Year Prior Literacy Experience Year 1999 2000 2001 Total Yes 44% 62% 67% 63% No 40% 52% 65% 58% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 34of68 Facilitator Gender Although the Project has always targeted women and youth appropriately, there was an early tendency to favor male facilitators rather than women. Early analyses indicated that female facilitators tended to be more successful, especially among the target population. Table 30 shows how, between 1999 and 2001, the relative participation of male and females facilitators was exactly reversed. In 2001, women made up 54°/o of all facilitators. Table 30: Facilitator Gender by Year Facilitator Gender Year 1999 2000 2001 Total Male 54% 49% 46% 48% Female 46% 51% 54% 52% Facilitator Age Fieldwork raised some concerns regarding the age of the facilitator and the response of the group. The general impression was that older facilitators were less effective with a younger target population, while older participants felt uncomfortable when led by very young facilitators. Table 31 shows some fairly small changes in the composition of the facilitators: due mostly to an increase in younger facilitators, the average age was reduced from 28 in 1999 to 26 in 2001. Table 31 : Facilitator AQe bv Year Facilitator Age Year 1999 2000 2001 Total Less than 18 9% 19% 14% 16% 18-45 88% 78% 82% 81 % Older than 45 3% 3% 4% 3% Average 28 25 26 26 Facilitator Educational Level Facilitators experienced doubts regarding their capabilities, academic preparation, and the support the NGOs would give them. Both the economic benefits the NGOs offered them and a certain degree of social conscience to their support for the community eventually convinced them to become facilitators. An early concern identified in the COMAL Project was the high percentage of facilitators with limited formal education. Analyses of results identified facilitator education level to be a significant factor to group success, even when significant COMAL Project Monitoring and Evaluation Report 1999-2001 Page 35 of68 training had been provided. Table 32 shows that in 1999, fully 42% of all facilitators had less than a sixth grade education. This proportion was sharply reduced to 16% in the 2000 literacy cycle, and cut yet more, to 6%, in 2001. Table 32: Facilitator Educational Levels bv Year Education Level Year 1999 2000 2001 irotal Less than sixth orade 42% 16% 6% 14% Sixth grade or higher 58% 84% 94% 86% The dramatic change in education level of facilitators is shown in Figure 9. Figure 9: Education Leve Is of the Facilitators 100% ..---- ---~~ ~r. f-- ------------------1 80% +---- - --- ----------- i 103+------------------- .i ~ r.t-- -------- ---;;=:::::::==: .. l 503+--------- - - --- --1 : ~ r.i-.--r- ~1-------------'----~ i 0. 30%----t 20,. l•2r. I J0%i-----I 6% +--~....._ __ Less than slllh gr1de Sixth gr•de or hllh•r Table 33 looks at pass rates by the educational level of the facilitator. In the 1999 cycle, facilitators with sixth grade education or more achieved a pass rate 12% higher than their less educated counterparts. In both the 2000 and 2001 cycles, however, less educated facilitators had greater success with their groups. The general experience was that primary school teachers did not work well with adult literacy groups, mostly due to higher drop-out rates among the participants. Primary school teachers were often observed to use the same teaching style with adults as they use in the primary school classroom with children. As will be seen in the next section, however, additional training improved performance among all facilitators. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 36of68 Table 33: Pass Rates bv Facilitator Educational Levels bv Year Education Level Year 1999 2000 2001 Total Less than sixth grade 31% 64% 70% 57% $ixth grade or higher 43% 57% 66% 61% Leadership The COMAL Project was envisioned as having wide geographic coverage and was targeted to small, rural communities where illiteracy was most endemic. By design, then, the Project faced the challenge of finding suitable candidates for literacy facilitators. "What I see is that my group, even though I have only studied until sixth grade, has a lot of confidence in me and say that they do not want another person coming from far to teach, because they have taken into account what the school teachers do, that sometimes they do not show up for class in a week because of the distance. That is why they want somebody from town, so they can talk with confidence with him ... " (Focus Group with Facilitators. Agreement 2001) Ideally, facilitators should be from the same community as the participants. Because of the lack of educated persons in some communities, however, NGO's at times had to recruit facilitators from other communities in the area. The lack of reliable transportation often made this a less-than-ideal solution and resulted in facilitator absences at group sessions. In the facilitators' opinion, it is not age, gender or academic level that matters for a successful group. What they see as most important is to have good communication with the participants and take an interest in the problems of their communities. The "good attitude" is also key to reduce facilitator desertion. "Of the people in here [facilitator training course], the majority have only reached sixth grade, but I think that it depends on our experiences with the groups, because even if we had a great title in our hands, if the people don't have confidence in us, I cannot unite them." (Focus Group with Facilitators No. 14 2000) Training Received by Facilitators Facilitator interest in expanding their own knowledge supported their role as motivators of the educational process for their participants. Some facilitators expressed a desire to further their own educational preparation by finishing primary school or initiating their high school studies. The facilitators recognized that the training provided by the COMAL Project helped them to motivate and keep the literacy groups intact. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 37of68 Table 34 looks at the combined effects of facilitator education and levels of training received on the outcome variables in 1999. The lowest pass rate (19%) was observed among facilitators with little formal education and little literacy training, who often had groups that disintegrated. Better-educated and better￾trained facilitators achieved more respectable results (43%). Comparing the pass rates of less-educated facilitators with more training (34%) with better￾educated, less-trained facilitators (35%) suggested that adequate training could make up for the weakness observed in less-educated facilitators. But the superior results of better-educated and better-trained facilitators nonetheless emphasized the need for careful facilitator selection and training. Table 34: 1999 Literacy Cycle Final Results by Facilitator Traininr:i and Education Literates Result Education Days Training Produced Pass Fail Drop-out Disintegrated Less than six 53 19% 1% 35% 46% Less than sixth grade Six or more 404 34% 5% 60% 1% Total 457 31% 5% 55% 10% Less than six 277 35% 4% 52% 9% Sixth grade or more Six or more 573 43% 5% 44% 9% Total 850 40% 4% 47% 9% Less than six 169 33% 5% 46% 16% No data Six or more 537 56% 4% 40% 0% Total 706 48% 4% 42% 6% No data 523 51 % 15% 23% 12% Total 1999 2,563 42% 6% 44% 9% Table 35 looks at the same issues for the 2000 literacy cycle, the last year training of facilitators was managed directly by Save the Children. As mentioned earlier, the formal education advantage ceased to exist, but the effect of training-particularly among less educated facilitators-becomes clearer. Having received six or more days of literacy training consistently improved performance of the facilitator. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 38of68 Table 35: 2000 Literacy Cycle Final Results by Facilitator Trainina and Education Literates Result Education Days Training Produced Pass Fail Droo-out Disintearated Less than six 256 47% 5% 25% 22% Less than sixth grade Six or more 1,444 67% 4% 27% 2% Total 1,700 63% 4% 27% 6% Less than six 2,011 54% 6% 31 % 9% Sixth grade or more Six or more 7,471 58% 5% 34% 3% Total 9,482 57% 5% 34% 4% Less than six 737 48% 6% 32% 13% No data Six or more 951 63% 6% 31% 0% Total 1,678 55% 6% 32% 7% No data 576 46% 4% 32% 17% Total 2000 13,436 57% 5% 33% 5% This focus on training was enhanced by an examination, in 1999 and again in 2000, of which specific training topic or content could be associated with positive results. Monitoring and evaluation obtained the detailed daily schedule of each training event, and also obtained the daily attendance logs for all literacy workers trained. This allowed us to determine whether a given facilitator had been trained in a given topic or content. Tables presented earlier in this report examine the relative impact of having received training in particular topics. Facilitators generally felt that, to be effective at promoting literacy in the Mayan language, they required more training in the writing of K'iche'. Most facilitators had their first experience at writing their native language with the COMAL Project. As with participants, facilitators had difficulty and expressed resistance to the learning written K'iche', and argued that the dialect differences in each sociolinguistic region would simply increase illiteracy in the Mayan communities. Facilitators showed interest in receiving training in various teaching techniques and content areas, some of which correspond to participant interests, such as community organization, and making minutes and arrangements. Facilitators also expressed interest in receiving courses on how to reduce desertion in literacy groups. The lack of a socialization strategy for facilitators did not help reinforce the effectiveness or knowledge base among the literacy promoters of the NGO's. Impact in the Life of the Facilitators Facilitators experience a wide variety of satisfaction, strengths and motivations, frustrations, weaknesses and dislikes. These feelings cause an impact in individual and group outcomes in the community. In general, facilitators express COMAL Project Monitoring and Evaluation Report 1999-2001 Page 39of68 positive feelings about "being fortunate to have been taken into account," a sense of personal worth and pride for the contribution they are making to their community. "You used to hear that we were not worth anything because we are indigenous, and that our language has no value, but now that it is being taken into account because it is being written, they are teaching letters. It is hard but we have to make the effort." (Focus Group with Facilitators No. 1 2000) The facilitators expressed frustration when: • there is little progress by the participants • absenteeism or drop-out is high • they feel pressured by the NGO to keep the group intact • inadequate or insufficient supplies are provided • they do not receive the institutional support that they ask for • training do not cover their perceived needs • they see the final results of the literacy process • they cannot respond to participants' request to provide follow-up literacy training One of the most discouraging aspects for facilitators was participant absenteeism and drop-out, because they felt it reflected on their own capabilities. Some facilitators also associated poor results with low expectations and motivation of their participants. The lack of supplies and equipment created pressure and tension between participants and facilitators, and between facilitators and the NGO. There was also institutional pressure applied by the NGO on facilitators to keep participants initially enrolled in the group and to ensure that they all passed at the end of the process. This pressure made facilitators conscious of these issues, but also created a situation in which the group's progress and results reported to the NGO might not coincide with reality. Some NGOs also used a discounting mechanism, where the facilitator pay was tied to the number of participants reported to still be in the group. Frustration was also expressed at differences of opinion between facilitators, NGO literacy supervisors and CONALFA supervisors because the latter are critical about participant learning levels, causing discouragement inside the group and criticism of the facilitator. The national political and economic situation also negatively impacted the COMAL Project, disrupting arrangements facilitators negotiated with other institutions for community projects to accompany literacy training. Unfulfilled commitments by FIS and FONAPAZ, institutional bureaucracy and skeptical community participation affected literacy group cohesion and interest levels. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 40of68 Confusion and delay also characterized negotiations with governmental and other authorities, including CONALFA, where drastic budgetary cuts and personnel changes occurred. The experience of teaching to read and write causes significant impacts on the facilitator in terms of personal economics, work expectations, social status, personal and community relationships and individual capabilities, and these impacts cause facilitators to develop alternative plans, goals and motivations. Facilitators perceive that the training they have received has increased their level of knowledge. This is felt to be true even though the qualitative change in their life is not always immediately apparent. Underlying this attitude is a belief in the economic potential derived from education. While facilitator experiences vary greatly and range from frustration to immense satisfaction, there is no question that the lives of facilitators are significantly changed as a result of the literacy process. Project Results Results by NGO The COMAL Project was implemented through agreements entered into with a large number, and variety, of NGOs operating in the Zonapaz. Table 36 shows the final results, by NGO, for the three-year period of the Project. A total of 37,784 persons were made literate by the 27 NGOs. The largest numbers of new literates came from IXMUKANE (5,546), ADEJUC (3,918), FUNDES (3,357), CORO (3,305), FUNRURAL (2,804) and FUNDAZUCAR (2,214). Table 36: 1999-2001 Literacy Cycles Final Results by Organization Literates Result Implementing NGO Produced Pass Fail Drop-out Disintegrated ACODIN 174 56% 15% 25% 4% ADEJUC 3,918 69% 5% 25% 1% ADERSO 894 72% 2% 26% 0% AD ESMA 1,563 72% 7% 18% 3% HAD IT A 276 64% 4% 32% 0% ADISA 1,256 57% 6% 30% 8% ALMG 127 29% 6% 52% 13% - - ASODEMI 953 34% 6% 52% 8% ASOREMA 1,1 83 71 % 7% 20% 2% Beleleb Batz 439 49% 6% 44% 1% CCAM 527 27% 4% 46% 23% CORO 3,305 69% 3% 28% 1% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 41 of68 Table 36: 1999-2001 Literacv Cycles Final Results bv Oraanization Literates Result Implementing NGO Produced Pass Fail Droo-out Disintegrated COINDI 1,631 57% 4% 38% 2% Conseio Pueblos Mavas Guatemala 651 66% 8% 26% 0% COPROJUC 542 59% 14% 27% 0% DICOR 357 43% 5% 49% 3% ECODESARROLLO 263 74% 7% 19% 0% FEY ALEGRIA 1,300 52% 6% 40% 2% Foro de la Mujer 131 31% 5% 52% 12% FUN DAD ESE 406 68% 5% 27% 1% FUNDAP 279 19% 5% 62% 14% FUNDAZUCAR 2,214 57% 4% 38% 0% FUND ES 3,357 61% 9% 25% 6% FUNRURAL 2,804 78% 7% 13% 1% IXMUKANE 5,546 61 % 10% 27% 3% PRODESCA 1,817 68% 2% 27% 3% SIDERGUA 1,871 72% 9% 19% 0% Total (27 NG Os) 37,784 60% 6% 30% 3% The Institutional Support of the NGOs of the COMAL Project Particularly during the early stages of the COMAL Project, there were gaps in terms of the institutional support provided by NGOs to the facilitators. As a result, facilitators would seek support from other institutions, groups or individual people in the community. Facilitators expressed that in some cases they sought out schoolteachers or priests for support, supplies, and matters such as "the use of the dieresis in K'iche'." Materials and Supplies The beginning of the Project was characterized by a paucity of materials and a general sense of informality about the ICL methodology itself. The lack of materials (particularly in K'iche') led to innumerable problems, including the use of material incompatible with the ICL model. Facilitator morale was negatively affected by participant perception of the "threadbare" quality of the Project, and facilitators were frustrated at the ineffectiveness of their own attempts to purchase or acquire materials that turned out to be inappropriate for adult education. The final year of the Project created a wealth of materials in Spanish and Mayan languages with content tuned to adult interests and motivations. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 42 of68 Selection and Training of Literacy Facilitators NGOs steadily increased the proportion of facilitators with at least a sixth grade primary education level, while maintaining requirements regarding leadership skills, community credibility and commitment. Training provided by the COMAL Project tied to provide facilitators with a broad understanding of the ICL process, while emphasizing an active learning methodology that provoked questioning and participation. During the first two years of the Project, however, there was a predominance of memorization of techniques and dynamics, often with little understanding by the facilitators of why or when these should be used. Furthermore, most of these techniques and dynamics were untested and as a result were ineffective or even rejected by participants. Early training was not well adapted to the educational and knowledge level of the facilitators, causing problems in replication at the literacy group level. Nevertheless, facilitators always expressed that the training they received stimulated their creativity and expanded their overall knowledge base. Working with Mayan communities implies the need for competent technical personnel with linguistics experience. Almost all literacy technician and facilitator training was conducted in Spanish. Grammatical issues regarding the writing of K'ekchi' were also not resolved until the final year of the Project. The training to the facilitators about the ACI, made an emphasis in the search of satisfactions of the expectations of an adults, trying that the subjects could be practiced in the participants daily life. The Project tried that the literacy process would impact the life of the participants and facilitators in the economic, social, thoughtfulness and other projects' life. The ACI methodology demanded the facilitators to achieve empathy and stimulation with their groups. Have an emphasis in certain progresses: measuring and visualizing of tangible products. But it also has relative flexibility during the sessions on: attendance, subjects, and supplies. Literacy Technicians and Supervision of Facilitators and Their Groups In the latter part of the Project, more intensive use of NGO and Project literacy technicians provided better support for the facilitators. This support consisted of carrying out home visits with the facilitators, undertaking motivation visits to the groups, clearing up of facilitators' doubts and reinforcing facilitators' training. Follow-up, rather than "supervision," characterized this phase of the Project, and COMAL Project Monitoring and Evaluation Report 1999-2001 Page 43of68 the relationship between NGO literacy technicians and facilitators was enhanced in 2001 when the former became directly responsible for the training of the latter. During the first two years of the Project, Save the Children personnel carried out the training of facilitators, and insufficient attention was placed on the role of the NGO literacy technicians. The potential for adequate supervision has always been limited by the distance and inaccessibility of the literacy centers, combined in most cases with a lack of transportation facilities to the communities. During the last year of the Project, great emphasis was placed on improving the effectiveness of supervision through early detection of facilitator weaknesses in knowledge and group management, and appropriate support and follow-up. During COMAL's last year, NGOs greatly strengthened Project implementation based on feedback and insights provided by the Monitoring and Evaluation team and improved NGO support activities carried out by Save the Children personnel. This resulted in positive action by NGOs based on lessons learned about the technical, methodological and administrative requirements and limitations of the ICL model, improvements in materials and training techniques, and a more effective support/supervision model. Results by Year As was presented earlier, Save the Children implemented the COMAL Project through a total of 27 NGOs over a three-year period in seven departments, forming 2,549 literacy groups with 62,834 participants. The diversity of implementing mechanisms, locales, literacy workers and participants provided the Project with ample opportunity to employ the monitoring and evaluation system to identify successful strategies as well as weaknesses, and through frequent presentation of quantitative and qualitative results and analyses to the NGO network, the Project underwent continuous improvement. The previous sections identified some of the changes that were implemented in response to these analyses. This section examines the results of the Project by year, frequently in conjunction with the many factors found to relate to success. It was a goal of the Project to develop a methodology that not only increased success, but did so in as many settings and with as wide a range of conditions and participants as possible. The results presented in this section highlight how the Project methodology came to fulfill that goal. All groups were evaluated in terms of outcomes and conditions. In order to understand how different factors relate to the success of a group, we used the monitoring and evaluation system to report the key outcome variable for each individual in the group based upon the final evaluation performed at the end of COMAL Project Monitoring and Evaluation Report 1999-2001 Page 44of68 the literacy cycle. The outcome for an individual, therefore, can be characterized as either: • Passing the literacy evaluation examination • Failing the literacy evaluation examination ("In process," in CONALFA terminology) • Dropping out from the group before the literacy evaluation examination is given • Belonging to a group that disintegrates While it may seem that attention should be placed upon only the measure of success-passing the literacy evaluation exam-it is actually more important to determine why a participant might fail to do so. As we will see, the success of the COMAL Project had more to do with increasing retention-avoiding drop-out and group disintegration-than any other aspect. While many factors contributed to this success, this section of the report seeks to identify which factors affected which aspect of success, and with whom. Table 37 presents the results for 1999 by organization. As mentioned earlier, 1999 was something of a disappointment for the Project; both because of shortfalls in the initial enrollment and a pass rate lower than expected. Only 2,536 ( 42°/o) of participants passed the literacy test. Particularly of concern was the high drop-out rate (44%) and group disintegration rate (the groups of 9% of participants fell apart). The fact that 6% of the participants failed the literacy test pales by comparison with these figures, focusing our attention on how to improve retention. If the Project managed to increase retention, one could expect around 88% of those who took the literacy test to pass it. Table 37: 1999 Literacv Cvcle Final Results bv Organization Primary Literates Result Organization Implementing NGO Produced Pass Fail Drop-out Disintegrated ADEJUC 209 42% 5% 53% 0% ADISA 318 67% 6% 27% 0% ALMG 127 29% 6% 52% 13% ADEJUC CCAM 232 20% 3% 50% 28% DICOR 357 43% 5% 49% 3% FUNDADESE 85 42% 4% 54% 0% IXMUKANE 299 44% 7% 50% 0% COINDI COINDI 384 44% 4% 47% 5% FUNDAZUCAR FUNDAZUCAR 106 55% 3% 43% 0% FUND ES FUND ES 419 60% 17% 12% 12% Total (11 NGOs) 2,536 42% 6% 44% 9% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 45of68 The wide range of pass rates the different NGOs obtained in 1999 was both a cause for concem-CCAM had a miserable pass rate of 20%--and a source of hope-if ADISA could achieve 67%, then why couldn't all of the NGOs? Very clearly, the key was to reduce drop-out by keeping the interest of participants throughout the literacy cycle and holding groups together. NGOs had a quite wide range of organizational capacity, especially with regard to internal training experience and skills, and this clearly affected their ability to implement the Project. The 2000 literacy cycle reflects the Project's response to the 1999 results. Table 38 shows that, overall, a pass rate of 57% was achieved, mostly through reducing drop-out (from 44% in 1999 to 33% in 2000) and avoiding group disintegration (down to 5% in 2000 from 9% the prior year). Table 38: 2000 Literacy Cycle Final Results by Organization Primary Literates Result Organization Implementing NGO Produced Pass Fail Oroo-out Oisinteorated AOEJUC 1,051 67% 5% 24% 4% ADISA 466 65% 0% 25% 11% ASOOEMI 953 34% 6% 52% 8% ADEJUC ASOREMA 163 80% 2% 18% 0% CCAM 295 39% 5% 41% 15% COPROJUC 542 59% 14% 27% 0% FUN DAO ESE 321 81% 5% 13% 1% IXMUKANE 2,334 66% 2% 28% 5% ADIAT AOIAT 276 64% 4% 32% 0% CORO CORO 1,832 67% 4% 28% 1% COINOI COINDI 496 59% 4% 37% 0% ACOOIN 174 56% 15% 25% 4% CRS ADES MA 732 71% 9% 14% 6% ECOOESARROLLO 263 74% 7% 19% 0% FEvALEGRiA FEvALEGRIA 293 48% 7% 45% 0% FUN OAP FUN OAP 279 19% 5% 62% 14% FUNOAZUCAR FUNOAZUCAR 351 57% 3% 40% 0% FU NOES FU NOES 1, 131 55% 10% 26% 10% FUNRURAL FUN RURAL 664 74% 3% 19% 5% PROOESCA PROOESCA 580 60% 5% 31% 4% SIOERGUA SIOERGUA 240 64% 11% 25% 0% Total (21 NGOs) 13,436 57% 5% 33% 5% What is perhaps the most impressive in 2000, however, is the appearance of highly successful NGOs. FUNDADESE achieved a pass rate of 81°/o, ASOREMA 80%, ECODESARROLLO and FUNRURAL 74%. In each of these cases, drop-out was cut to under 20% and failure to 2-5%. Very disappointing results were seen, nonetheless, for NGOs such as FUNDAP (19% pass rate), ASODEMI (34%) COMAL Project Monitoring and Evaluation Report 1999-2001 Page 46of68 and CCAM (39°/o). But the fact that 12 of the 21 NGOs achieved pass rates of 60% or better once again provided the Project with a new level of expectation. Many changes took place in the 2001 literacy cycle, including a more critical selection process for NGOs (although more groups and participants were serviced, this was done with fewer NGOs either with proven track records or selected much more carefully than in the past), improved materials and supervision, and wider dissemination of the strategies found to be successful in prior years. Table 39 shows that the Project achieved a pass rate of 66°/o and the drop-out rate was further cut to 26%. In 1999, drop-out and group disintegration amounted to 53%. In the 2001 literacy cycle, together they accounted for but 27% of participants. Table 39: 2001 Literacy Cycle Final Results by Organization Literates Result Implementing NGO Produced Pass Fail Drop-out Disinteorated ADEJUC 2,658 74% 5% 22% 0% ADERSO 894 72% 2% 26% 0% ADES MA 831 74% 5% 22% 0% ADISA 472 47% 10% 35% 9% Beleieb Batz 439 49% 6% 44% 1% ASOREMA 1,020 70% 8% 20% 2% CORO 1,473 72% 1% 27% 0% COINDI 751 64% 3% 31% 2% Conseio Pueblos Mavas Guatemala 651 66% 8% 26% 0% FEY ALEGRIA 1,007 54% 6% 38% 2% Foro de la Muier 131 31% 5% 52% 12% FUNDAZUCAR 1,757 58% 5% 38% 0% FUNDES 1,807 65% 6% 28% 1% FUN RURAL 2,140 80% 9% 11% 0% IXMUKANE 2,913 60% 16% 24% 1% PRODESCA 1,237 73% 9% 18% 0% SIDERGUA 1,631 73% 9% 18% 0% Total (17 NGOs) 21,812 66% 7% 26% 1% Of the 17 NGOs, only five of them failed to achieve a 60% pass rate and eight of them had pass rates of 70% or better. As a result, the 2001 literacy cycle produced 21,812 new literates, 58% of the total over the life of the Project. Results by Department As was seen earlier, most of the groups in 1999 were concentrated in the department of El Quiche. Table 40 shows that most (61 %) of the literates COMAL Project Monitoring and Evaluation Report 1999-2001 Page 47of68 r I produced in 1999 were from the same department. The pass rates varied widely between the departments, being highest (73%) in Totonicapan and lowest (27%) in Quetzaltenango, although the small numbers make interpretation difficult. Table 40: 1999 Literacy Cycle Final Results by Department Literates Result Department Produced Pass Fail Drop-out Disintearated El Quiche 1,550 39% 5% 48% 9% Quetzaltenango 40 27% 40% 16% 17% Solola 639 43% 7% 41% 9% Suchitepequez 106 55% 3% 43% 0% Totonicapan 201 73% 5% 16% 6% Total 1999 2,536 42% 6% 44% 9% Table 41 shows the same information for the seven departments corresponding to the 2000 literacy cycle. With the exception of Quetzaltenango, a tighter range of pass rates (56-84%) is observed. El Quiche accounts for 46% of all new literates produced by the Project in 2000. Table 41 : 2000 Literacy Cycle Final Results by Department Literates Result Department Produced Pass Fail Drop-out Disintegrated Alta Verapaz 303 84% 8% 8% 0% El Quiche 6,125 56% 4% 34% 6% Quetzaltenanao 1,107 35% 8% 44% 13% San Marcos 940 71% 3% 23% 3% Solola 1,076 60% 3% 40% 0% Suchitepequez 341 57% 3% 40% 0% Totonicapan 3,544 65% 7% 26% 2% Total 2000 13,436 57% 5% 33% 5% Table 42 presents the results of the 2001 literacy cycle by department. Pass rates range from 59°/o (Quetzaltenango) to 73% (Solola). El Quiche accounted for 44°/o of the new literates, followed by Totonicapan (20°/o) and Solola (18%). Table 42: 2001 Literacy Cycle Final Results by Department Literates Result Department Produced Pass Fail Drop-out Disintearated El Quiche 9,554 64% 10% 24% 2% Quetzaltenanco 2,310 59% 6% 33% 2% Solola 3,826 73% 3% 22% 2% Suchitepequez 1,757 58% 5% 38% 0% Totonicapan 4,365 72% 5% 23% 0% Total 2001 21,812 66% 7% 26% 1% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 48of68 Table 43 shows the life of Project production of literates by department. Of the 37,784 total, 17,229 (46%) correspond to El Quiche, 8,110 (21°/o) to Totonicapan, 5,541 (15°/o) to Solola, 3,457 (9%) to Quetzaltenango and 2,204 (6°/o) to Suchitepequez. Table43: 1999-2001 Literacv c, cles Final Results by Department Literates Result Department Produced Pass Fail Drop-out Disintegrated Alta Verapaz 303 84% 8% 8% 0% El Quiche 17,229 58% 7% 31% 4% Quetzaltenango 3.457 48% 8% 38% 7% $an Marcos 940 71% 3% 23% 3% Solo la 5,541 65% 4% 28% 3% Suchitepeauez 2,204 57% 4% I 38% 0% rrotonicapan 8,110 69% 6% 25% 1% Total 1999-2001 37,784 60% 6% 30% 3% Results by Area Although the Project focused on rural areas, some groups were formed in areas classified as urban (usually the municipal capital). Table 44 shows that 97% of literate produced in 1999 were members of groups in rural areas. Table 44: 1999 Literacy Cycle Final Results by Area Literates Result Area Produced Pass Fail Drop-out Disinteqrated Rural 2.462 42% 6% 44% 9% Urban 74 31% 20% 33% 16% Total 1999 2,536 42% 6% 44% 9% The expansion of Project coverage in the 2000 literacy cycle also increased the proportion of new literates from urban areas to 11 % (Table 45). Although failure rates were higher among rural participants (6% versus 2°/o), the higher drop-out rate among urban participants gave a slight advantage to the rural participants in terms of the pass rate. Table 45: 2000 Literacy Cycle Final Results by Area Literates Result Area Produced Pass Fail Drop-out Disinteqrated Rural 11.993 57% 6% 32% 5% Urban 1.433 56% 2% 37% 5% Peri-urban 10 59% 12% 29% 0% Total 2000 13.436 57% 5% 33% 5% COMAL Project Monitoring and Evaluation Report 1999-2001 Page 49of68 In the 2001 literacy cycle, 10% of participants were from areas classified as urban. Table 46 shows that pass rates continued to favor rural participants by a slight margin. Table 46: 2001 Literacy Cycle Final Results by Area Literates Result Area Produced Pass Fail Drop-out Disintearated Rural 19,689 66% 7% 26% 1% Urban 1,822 64% 8% 26% 1% Peri-urban 301 60% 5% 35% 0% Total 2001 21,812 66% 7% 26% 1% Table 47 shows the life of Project results by area. Ninety percent of all literates were from groups in rural areas, and there are no significant differences in any of the rates between rural and urban participants. Table 47: 1999-2001 Literacy Cycles Final Results by Area Literates Result Area Produced Pass Fail Drop-out Disintearated Rural 34,144 60% 6% 30% 3% Urban 3,329 59% 6% 31 % 4% Peri-urban 311 60% 5% 35% 0% Total 1999-2001 37,784 60% 6% 30% 3% Gender Interaction of Facilitators and Participants This section examines the effect of gender and age of both facilitators and participants. Table 48 shows that, in 1999, female facilitators achieved pass rates significantly higher (46%) than did their male counterparts (38%). Interestingly, by the last year of the Project male and female facilitators did equally well. Table 48: Pass Rates by Gender of Facilitator by Year Facilitator Gender Year 1999 2000 2001 Total Male 38% 56% 66% 59% Female 46% 58% 66% 61 % Similarly, in 1999 pass rates for female participants were higher than for men (see Table 49), due mostly to the higher drop-out rates for the latter. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 50of68 Table 49: 1999 Literacy Cycle Final Results by Participant Gender Literates Result Participant Gender Produced Pass Fail Drop-out Disintegrated Male 652 37% 4% 51% 8% Female 1,884 43% 7% 41% 9% Total 1999 2,536 42% 6% 44% 9% As shown in Table 50, in 2000 male participants still had higher drop-out rates than did women, but even with the higher literacy examination failure rates of women, female participants out performed males. Table 50: 2000 Literacy Cycle Final Results by Participant Gender Literates Result Participant Gender Produced Pass Fail Drop-out Disintearated Male 3,921 56% 4% 35% 5% Female 9,515 58% 6% 31% 5% Total 2000 13,436 57% 5% 33% 5% By 2001, participant gender equality was nearly evident, as shown in Table 51. Table 51: 2001 Literacy Cycle Final Results bv Participant Gender Literates Result Participant Gender Produced Pass Fail Drop-out Disintegrated Male 6,657 65% 6% 27% 1% Female 15,155 66% 7% 26% 1% Total 2001 21,812 66% 7% 26% 1% Table 52 shows the three-year combined results by participant gender. Female participants comprised 70% of both initial enrollment and literates produced, and overall performed somewhat better than men. Table 52: 1999-2001 Literacy Cycles Final Results by Participant Gender Literates Result Parti~ipant Gender Produced Pass Fail Drop-out Disintearated Male 11,230 59% 5% 32% 4% Female 26,554 61% 7% 29% 3% Total 1999-2001 37,784 60% 6% 30% 3% The 1999 gender results for both facilitators and participants brought to question the potential interaction of facilitator and participant gender. Table 53 looks at the pass rates, by year, based on the four combinations of facilitator and participant gender. Not surprisingly, in 1999 the best results were obtained by female facilitators with female participants. Male participants did better with COMAL Project Monitoring and Evaluation Report 1999-2001 Page 51of68 female rather than male facilitators. Gender sensitivity training carried out during subsequent years has mostly eliminated the effects of gender interactions. Table 53: Pass Rates by Facilitator and Particioant Gender by Year Gender of Year Facilitator/Participant 1999 2000 2001 Total Male I Male 36% 56% 66% 60% Male I Female 40% 55% 66% 59% Female I Male 40% 55% 64% 58% Female I Female 47% 59% 66% 62% We were also able to examine the gender composition of the group and gender￾specific outcomes. Table 54, for 1999, provided a startling insight into gender dynamics in literacy groups. If there were as few as one male participant in a group, the drop-out rate of women was 70% higher (46%) than in women-only groups (27%). This striking difference is shown graphically in Figure 10. Table 54: 1999 Literacy Cycle Final Results for Women by Grouo Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disintegrated Men Present 1,214 39% 5% 46% 9% Only Women 670 54% 12% 27% 8% Total 1999 1,884 43% I 7% 41% 9% Figure 10 Presence of Men and Results for Women -1999 1 -D-Men Present l!!!I Only Women IO% ..------------ --- - - -----------~ 50% 40% 30% 20% ,--------- ---t !§ 10% o~ Pass Fail Drop-out Disintegrated COMAL Project Monitoring and Evaluation Report 1999-2001 Page 52of68 Table 55 looks at the same thing for men in 1999. There were only a few men￾only groups in 1999, but it also appears that men-only groups performed better, again due to lower drop-out rates, than mixed-gender groups. Table 55: 1999 Literacy Cycle Final Results for Men by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disinteorated Women Present 625 37% 4% 51% 9% Only Men 27 49% 9% 42% 0% Total 1999 652 37% 4% 51 % 8% Table 56 repeats the analysis for women in the 2000 literacy cycle. Although the difference is not as dramatic as in 1999, higher drop-out and lower pass rates for women continue to be associated with the presence of men in the groups. Table 56: 2000 Literacy Cycle Final Results for Women by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disintearated Men Present 6,946 57% 6% 32% 5% Only Women 2,569 59% 7% 29% 6% Total 2000 9,515 58% 6% 31% 5% Table 57 does the same analysis for men in the 2000 cycle. Men do better in men-only groups due to a combination of lower drop-out, group disintegration and literacy examination failure rates. Table 57: 2000 Literacy Cycle Final Results for Men by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disinteorated Women Present 3.484 55% 4% 36% 6% Only Men 437 65% 2% 30% 3% Total 2000 3,921 56% 4% 35% 5% By the 2001 literacy cycle, the results for women with and without men in the groups were virtually identical, as shown in Table 58. Table 58: 2001 Literacy Cycle Final Results for Women by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disintearated Men Present 11,219 66% 7% 26% 1% Only Women 3,936 67% 8% 25% 0% Total 2001 15, 155 66% 7% 26% 1% Table 59 shows that, for men in 2001, there no longer were any differences in results between groups that were men-only or with women present. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 53of68 Table 59: 2001 Literacy Cycle Final Results for Men by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disinteorated Women Present 5,879 65% 6% 27% 1% Only Men 778 65% 6% 27% 2% Total 2001 6,657 65% 6% 27% 1% Table 60 summarizes the three years of experience for women. Overall, 27% of all new female literates came from women-only groups, where performance was 3% better due to lower drop-out rates. Table 60: 1999-2001 Literacy Cycles Final Results for Women by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disintegrated Men Present 19,379 60% 7% 30% 4% K:>nlv Women 7,175 63% 8% 27% 3% Total 1999-2001 26,554 61% 7% 29% 3% Table 61 shows that men perform better in men-only groups by a significant margin, although only about 11 % of new male literates came from such groups. Men-only groups had lower drop-out rates and lower group disintegration rates. Table 61 : 1999-2001 Literacy Cycles Final Results for Men by Group Gender Mix Literates Result Group Gender Mix Produced Pass Fail Drop-out Disintegrated Women Present 9,988 59% 5% 33% 4% Only Men 1,242 65% 5% 28% 2% Total 1999-2001 11,230 59% 5% 32% 4% Age Effects of Facilitators and Participants Pass rates by facilitator age are shown for all years of the Project in Table 62. In 1999, lower pass rates were observed for younger ( 41°/o) and particularly older (28%) facilitators. The highest pass rates were found in facilitators 21-30 years old. In 2000, the highest pass rates (61 % and 59%) were found in facilitators from both the 31-40 and 21·30 age groups, with younger (20 or less) and older (41 or more) facilitators having somewhat lower pass rates (57% and 56°/o, respectively). By 2001 only younger facilitators had pass rates slightly lower (64%) than any of the other age groups, who all performed equally (67%). COMAL Project Monitoring and Evaluation Report 1999-2001 Page 54of68 J Table 62: Pass Rates by Facilitator Aoe bv Year Facilitator Age Year 1999 2000 2001 Total 20 or less 41 % 57% 64% 59% 21 - 30 44% 59% 67% 62% 31 -40 36% 61 % 67% 61% 41 + 28% 56% 67% 59% Apart from the goal of targeting the Project towards youth, the 1999 literacy cycle raised concerns about efficiency problems with older participants. Table 63 shows that pass rates dropped off beginning about age 30, and fell to 27% among participants 45 or more years old. This drop off was due to a combination of literacy examination failure, drop-out and group disintegration. Table 63: 1999 Literacy Cycle Final Results by Age of Participant Literates Result Age of Participant Produced Pass Fail Drop-out Disinteorated No age specified 10 15% 9% 14% 62% 10-14 138 45% 7% 43% 6% 15-19 842 47% 4% 42% 7% ~0-24 332 44% 5% 42% 9% ~5-29 329 44% 6% 44% 6% ~0-34 243 42% 6% 44% 9% 35-39 263 43% 6% 42% 9% 140-44 150 38% 8% 45% 10% 145 + 229 27% 11% 50% 12% Total 1999 2,536 42% 6% 44% 9% In 2000, although younger participants continued to enjoy the highest pass rates, the drop off in pass rates among older participants is not so marked, as shown in Table 64. Drop-out continues to increase with age, but literacy examination failure and group disintegration are less affected by age than in 1999. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 55 of68 Table 64: 2000 Literacy Cycle Final Results by Age of Participant Literates Result Age of Participant Produced Pass Fail Drop-out Disintegrated No age specified 9 14% 3% 9% 73% 10-14 93 69% 5% 24% 2% 15-19 5,519 63% 5% 29% 4% 20-24 2,147 59% 4% 31 % 6% ~5-29 1,585 54% 6% 34% 6% ~0-34 1,334 55% 6% 34% 5% ~5-39 1, 113 52% 5% 38% 6% 140-44 699 50% 6% 38% 6% 145 + 1,343 48% 7% 39% 6% Total 2000 13,436 57% 5% 33% 5% This same pattern continues in the 2001 literacy cycle, as shown in Table 65. The new literacy examination applied in 2001, which were generally harder for all participants than the CONALFA-based tests used in 1999 and 2000, appear to affect participants 40 years or older somewhat more. Table 65: 2001 Literacy Cycle Final Results by Age of Participant Literates Result Age of Participant Produced Pass Fail Droo-out Disintegrated No age specified 8 67% 8% 25% 0% 10-14 101 68% 5% 22% 5% 15-19 8,367 70% 7% 22% 1% 120-24 3,392 69% 7% 23% 1% 125-29 2,696 68% 6% 25% 1% 30-34 2,024 65% 7% 28% 1% 35-39 1,884 64% 6% 29% 1% 40-44 1,207 60% 8% 31% 1% 145 + 2,002 56% 10% 34% 1% Total 2001 21,812 66% 7% 26% 1% Table 66 summarizes the three years of Project implementation. The pattern of increased drop-out with age is quite clear, rising systematically from 26% in the 15-19 age group to 38% in the 45 and older group. Older learners face significant challenges to acquiring literacy: opportunity costs associated with attending literacy sessions, poorer eyesight, poorly developed psychomotor skills (such as for holding a pencil), family pressures, social stigmas associated with "going back to school," etc. Increased literacy examination failure rates are also characteristic of the 40 and older age groups who do remain in literacy groups. Importantly, the table shows that 67% of all new literates produced by the Project were between 15 and 29 years of age. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 56of68 Table 66: 1999-2001 Literacy Cycles Final Results bv Aoe of Participant Literates Result Age of Participant Produced Pass Fail Drop-out Disintegrated No age specified 27 19% 6% 13% 62% 10-14 332 56% 6% 33% 5% 15-19 14,728 65% 6% 26% 2% 0-24 5,871 63% 6% 28% 4% 25-29 4,610 60% 6% 30% 3% 0-34 3,601 59% 6% 32% 3% 35-39 3,260 57% 6% 34% 4% 140-44 2,056 54% 7% 35% 4% 145 + 3,574 50% 9% 38% 4% Total 1999-2001 37,784 60% 6% 30% 3% Age and gender combined with marital status are important variables affecting the time dedicated to literacy sessions. Men often are instrumental in the decision-making process regarding the participation of all women in their family group. Even though participants express a belief that education can help them improve their economic, personal or community life, most participants, and especially those who are older, openly identified a fear of "forgetting what they have learned" due to the lack of follow-up literacy classes. A reality of life among the rural Mayan population is that the bonds formed in the ..; interethnic relationships between generations, male-dominated gender communication and sociolinguistic comprehension create a "fear" in the individual to express themselves in public. The ICL literacy sessions create a space in which expression and participation is encouraged. For participants, therefore, the "shame" and "fear" of speaking in public have diminished. They now participate in community meetings and are no longer reluctant to express their opinions. Limitations of the Adult Literacy Process Successful participants encountered widely varying degrees of difficulty in becoming literate. Prior experience and language ability were important factors in determining how difficult an individual might find the process. For example, participants felt that those who had the opportunity of attending at least a year of formal schooling found it easier to hold a pencil or draw. On the other hand, participants frequently compared the way formal primary school is taught with the way the ICL works with adults, and sometimes doubt COMAL Project Monitoring and Evaluation Report 1999-2001 Page 57of68 the process because it does not meet their expectations as to what education is about. This expectation bias, combined with self-doubt about their own ..; capabilities is more frequent in adults than in youth, and in women more than in men. Other perceived limitations of the adult learning process include: 1. A weak attitude towards the habit of studying, or "opening one's mind to knowledge" 2. The direction, interests and beliefs of participant's lives 3. Prioritization and distribution of participant's time 4. The manual ability to take a pencil and draw or form letters with it, or, as in the case of one women's group, letting a male facilitator guide their hand Participants suggested that in order to overcome these limitations, homework be assigned for practice in private. This suggestion, however, has two practical implications: 1. Participants must be willing and able to dedicate the necessary time at home to practice in spite of domestic commitments 2. They must have the supplies necessary to practice effectively Participants Identified their progress by saying that they had learned "to read some letters" such as vowels, but found it difficult to combine letters to form words. Most recognized that they could copy words. Others said that they had memorized the writing of some words, such as their name, the name of their town, municipality or department, or the names of their literacy classmates. The participants insisted that every letter of the alphabet should be taught, so they can read fluidly. They also question the ICL approach to teaching complete derivative words based on generative words. In terms of native language literacy training, participants universally compare its usefulness to Spanish. Participants indicate the utility of Spanish for communicating outside their own community, even if the other community is also Mayan speaking. Participants thus prefer learning to read, write and speak in Spanish, generally adding the argument that they want to learn Spanish ../ because they can already speak their own language. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 58of68 Participants sometimes criticized the knowledge or teaching skill of the facilitator, but were more often critical of the lack of appropriate materials to learn with. The 2001 literacy cycle started after a considerable delay. This moved the end￾of-cycle evaluation period as well, causing some conflict with seasonal migration movements to the coast. Impact at the End of the Project The shift in overall Project results may be observed in Figures 10-12 for 1999, 2000 and 2001, respectively. Clearly, Project success in 2001 was achieved by cutting drop-out in half and virtually eliminating group disintegration. Better NGO selection, improved facilitator and participant materials, effective dissemination of successful practices, improved supervision and more effective technical support and assistance from Save the Children all contributed significantly to these results. Figure 11 1999 Results Disintegrated 9% Drop-out 43% Fail 6% COMAL Project Monitoring and Evaluation Report 1999-2001 Pass 42% Page 59of68 Figure 12 2000 Results Disintegrated 5% Drop-out 33% Drop-out 26% 5% Figure 13 2001 Results Disintegrated 1% Pass 57% Although the Project clearly achieved an impressive success rate, still one out of three participants failed to pass. Drop-out affected more than one in four participants and is caused by a combination of national socioeconomic problems v common to all, personal problems that "are so private you are the only one who knows them" and some technical weaknesses of the COMAL Project. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 60 of68 Drop-out For some participants, when the reasons for which a person has an interest in enrolling in a literacy group lose importance or fail to materialize, their likelihood of staying in the literacy group also diminishes. Failure to attend was almost always stated to be due to "not enough time," but this reflects a number of underlying priorities and perceived progress and benefits of the literacy process. There are a large number of reasons why participants drop-out, but the four fundamental reasons revealed through interviews with drop-outs are related to: 1. Changes in the aspirations, individual limitations and perceptions of the q world can motivate a person into suddenly or gradually stopping their · participation in the literacy process. 2. Pressures associated with family, community culture, gender, marital status, community organizations and/or social participation accumulate to cause a person to stop their learning process. 3. Community poverty exerts economic and social pressure, such as the need for one or more family members to migrate. This type of pressure is beyond the control of the individual and the sphere of influence of the methodology or the implementing NGO. 4. Failure of the ICL methodology or the NGO to satisfy the needs of individuals and groups. In the case of women, the most powerful reasons for dropping out are related to family commitments, especially the care of children, an aspect especially influential when the children are young or of preschool age. Other frequently J ./ mentioned barriers include household work, the permission of the husband, getting married, caring for other people, becoming widowed or changes in household composition due to migration. Childcare is a problem not only generally, but can be a specific concern during "f the literacy sessions themselves when other arrangements must be made for their care. The approval of a husband is crucial for women to carry out any activity, and can provoke many problems, not only internally for the family, but also for the "v facilitator. A woman might decide to participate in the literacy sessions, but if the husband at any time says otherwise, the woman will drop out from the sessions. Similarly, if a woman participant becomes married or enters into union while she is in literacy training, she may be forced to drop out by her mate. Maternity can also interrupt the literacy process as can, to a lesser degree, provision of care for sick or elderly family members. Women who become COMAL Project Monitoring and Evaluation Report 1999-2001 Page 61of68 widowed or who must assume additional responsibilities when family members migrate often find it impossible to continue with sessions. Economic pressures cause individuals to seek work opportunities outside of their communities, effectively ending their participation in the literacy group. As with health problems, this is an area in which neither the facilitator nor the Project can have much influence. A literacy group heterogeneous in terms of age can also increase drop-out or even the disintegration of the group when: • Older adults feel pressured by a majority of younger participants • Participants include children not accepted by or reported by the facilitator • Children participate alternately between literacy and formal primary school • Participants believe that because of their age they can no longer learn • Neighbors ridicule adult participation in the literacy process Some participants that dropped out said they did so because they were not interested in learning in K'iche'. Facilitators also mentioned this factor in explaining drop-outs. Some participants said to be willing to learn and read, but did not perceive that their knowledge retention increased. After seeing no improvement over a period of time, they reached the conclusion that continuing to attend was a waste of time. Community participation projects require an additional investment of time, sometimes competing with the literacy session schedule itself. Finally, the rainy season adversely affects the attendance of the participants, and can lead to drop-out when conditions make it difficult to arrive at sessions with small children, for example. Those Who Kept Going Interviews with participants who did not drop out brought up a constant theme as to their reasons for continuing their literacy training: "because they want to # improve themselves." Continuing participants also mentioned the degree of empathy that the facilitator has achieved with the participants in the group, to the point that the participants do not wish to " mistreat" the facilitator or appear ungrateful. Another theme is the feeling that the group has found support for their problems. An example of this is the testimony given by a participant who explained that even though she cannot see very well in the afternoons, she keeps attending the sessions to help her to forget the disappearance of her ..f. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 62of68 children during the internal armed conflict. She feels that has not learned as much as she should because her sadness does not allow her to pay enough attention. Continuing participants said that they feel "happy and grateful to the facilitator" and because of this they were learning. Even participants who think that they haven't learned anything express this feeling, underscoring the empathy achieved by the facilitator. Facilitator empathy was most often mentioned when a group had a unifying purpose, such as religion. Participants who say they want to keep learning envision using their newly acquired skills to read and write and use numbers. Skills acquired in the Spanish language are often mentioned as useful to them. It is especially important for participants to be able to sense their progress, especially during the early stages of literacy training. Many expressed that drop￾out is due to participants not being able to clearly see enough progress during the difficult beginnings of literacy training. This suggests that facilitators must provide constant feedback and encouragement to reduce early drop-out. Achievements and Knowledge Retention Most successful participants can identify some form of positive change in their lives due to their literacy training. Women participants often identified feeling "more sure" or trusting of themselves and their abilities, suggesting that the ICL literacy training can help women develop the independence and self-reliance necessary to take a more active role in their lives. In terms of tangible achievements, participants mentioned results related to reading, writing and mathematics, such as: 1. Having learned "some [but not all] letters" of the alphabet 2. They can write and sign their name, no longer having to "sign" with their fingerprint 3. Writing the names of others 4. Writing the names of their community, municipality and department 5. Writing the names of some animals 6. Some believed they understand more words in Spanish 7. They learned the Spanish names for the parts of the human body, something seen as important to help them to ask for medicine and explain an illness 8. Writing numbers from 1 to 10 9. Perform addition, and with some difficulty, subtraction COMAL Project Monitoring and Evaluation Reporl 1999-2001 Page 63 of68 v Participants often expressed the feeling that they had learned so little of all there was to learn, suggesting both that they have become aware of a wider world but also that the literacy training is viewed as falling short of their (perhaps new) expectations. Interviews with successful participants about how they felt at the end of the literacy cycle generally yielded expressions of satisfaction combined with relief: "it was worth it learning read and write," happiness because soon they will finish the process, they will have more time to take care of work or do things at home, they won't have to worry about attending, that by finishing the commitment they will feel free and not under pressure to attend. There were a few participants who expressed indifference at the end of the literacy cycle, saying that it was not worth the time invested because they did not learn much. These participants said that, because of other responsibilities, they did not expect to follow up on the lessons learned once the process finished. For these participants, literacy does not appear to be a priority in their future plans and life projects. The Impact of the Literacy Process on Self, Family and Community The Impact in the Lives of the Participants the ICL literacy process the opportunity for change in their lives: improving their ../ In general, persons made literate through the COMAL Project identified through !I economic situation by means of management of new knowledge, greater self￾worth and self-esteem, feeling less "sad," "ashamed" or insecure, and identification of education as a means or model for progress. Participant Self-esteem Participants identified changes in their personal lives related to self-confidence and their relationships with other people in their family and community, where v they felt that their worth had increased. Most felt that they would also gain short term economic benefits. In all groups, participants identify as a great achievement the ability to write their names with accuracy. Many believe that being literate will help them defend themselves against ridicule and mistreatment by others within and especially outside of their community. Education of their Children - COMAL Project Monitoring and Evaluation Report 1999-2001 Page 64 of68 I I I I r One of the most outstanding changes in participants was an increased ;¥- appreciation of education, both for themselves and for their own children. Participants expressed interest in helping their children with homework although they felt that, because of their own limited knowledge, they were still not capable of orienting or guiding the child. Nonetheless, they clearly recognized that before becoming literate they could not even understand what their children did. Economy, Work and Commerce Participants believe that potentially the most important change in their lives resulting from the literacy process will occur at the economic level, especially in 1·1 v / commercial activities. Specifically, participants see the ability to work with numbers for the purpose of buying and selling as a major achievement. The Utility of Numbers With regard to the mathematical operations, participants recognized that in knowing the numbers they can count change. Participants expressed the absolute need to learn "the numbers in Spanish" for commercial purposes. When comparing the degree of difficulty of learning "the letters" to learning "the numbers," participants said that whereas they didn't believe that they would recognize all of the letters in street signs, they would be able to identify the numbers. On the other hand, they said that it was difficult to add and even more complicated to perform subtraction. The ability to tell time was also / mentioned as an accomplishment gained through literacy training. Travel Outside of the Community As a result of the literacy process, some participants said that they would now feel more confident traveling to town or the capital for market business or for work. Legal Formalities Learning Spanish gives participants a greater sense of security when dealing with the legal world outside of their community: participants feel that they are better able to deal with the need to travel, communicate and sign their names in order to conduct such formalities. Before becoming literate, participants felt that they could easily be "taken advantage of" by unscrupulous persons. Women participants mentioned going to the municipality to obtain their identity v/ card and being able to sign their names without having to use their fingerprints, COMAL Project Monitoring and Evaluation Report 1999-2001 Page 65of68 register a birth or request identity cards for their children, and handling inheritance matters. Dealing with Health Problems Participants said that becoming literate would help them in seeking out medical care because they could now identify parts of the body in Spanish and explain v V' symptoms better. This was seen as allowing them to more effectively using the services of the health post, consulting a doctor or requesting medication. Social Participation in the Community Some participants expressed interest in becoming part of a community council or committee, even mentioning expectations or desires to become presidents, v v secretaries, treasurers and representatives, or to assume other positions in the community, or to get a position in an office of the government. Participants also mentioned community work and seeking help for the community from the municipality or other organizations. Nonetheless, social and community participation with a representation position does not seem to be an important motivation in the lives of the participants, because in the majority of cases they see such positions as something that can take many years to achieve. Even when the participants don't expect to hold a formal position within the community, however, they express satisfaction at It being able to attend community meetings, understand what is going on and R express their opinion about the subject. Some participants said that, when they know more things, they want to help their neighbors by forming a literacy group to teach them to read and write. Religious Practice Inside the social organization of the community, religion, and especially evangelical religion, plays an important part of social life. Because literature is such an important part of all religions, many participants identified this as a strong motivation to learning to read. Successful participants mentioned that they now that they can read a little of the bible and hymns, they can more fully participate in church, including reading passages, singing in church and going to the altar. Interethnic Communication COMAL Project Monitoring and Evaluation Report 1999-2001 Page 66 of68 I l l l l l L The participants fell that the ability to speaking some Spanish is an important achievement: "being able to speak with a ladino, saying good morning, good afternoon, and good evening, being able to ask a ladino for coffee if you are thirsty, and if a bad word is said to you, being able to defend yourself." Interest Satisfaction Facilitators Key among aspects that characterize the positive experiences of the facilitators are: 1. 2. 3. 4. 5. 6. A greater appreciation for the value of education A positive sense of their contribution to a national development process Becoming aware of the needs and limitations of their communities Satisfaction at seeing the results of their efforts in the progress of their participants A sense of belonging to a group with common interests Receiving institutional support through training and supplies Facilitators express their satisfaction and joy when they can clearly see the progress achieved by the participants during the literacy sessions, as well as in the impact that can be observed on the life projects of the participants. Facilitators also obtain great satisfaction through the recognition by participants of their efforts on their behalf. Participants In the majority of the cases, a primary interest of participants was "learning to write and sign their name." This interest was nearly always satisfied as a result of the literacy process. A second interest focused on "learning to speak Spanish." This interest was quite strong but generally was not fully satisfied by the literacy process, creating dissatisfaction among some of the participants and in a very few cases led to late drop-out. In terms of participant priorities, "learning the numbers" and basic arithmetical operations ranks third. While knowledge gained through the literacy process in this area was deemed important, and participants clearly identified its practical application, most participants felt that they needed more practice to become proficient. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 67of68 Although most participants perceived that their lives had greater "value" or "worth" than before beginning the literacy classes, most felt that the lack of follow-up literacy training and/or practical application of their new knowledge v posed some risk to sustained benefits. Participants sensed some real or potential impact on their economic situation v through improved management of new knowledge. In some cases, the participants felt an immediate change in their lives. The major impact corresponded attitudinal changes related to the expansion of their knowledge frontiers. In many cases, participant interest in "knowing their surroundings" exceeded what the literacy sessions were able to offer. The majority of participants do value their literacy experience, even though they felt that they have learned "too little." This means that their expectations possibly were not satisfied, or that their expectations have increased as a result of what they have learned and reflected on during their literacy training. Most believe, nonetheless, that the little they have learned has already been of use to them. They recognize that in the past they could not read nor write anything, and now they can write words and even simple sentences. They now find significance in things that before had none. Especially for women participants, self-esteem and self-worth have increased. A sense of independence-especially from others whom they fear might mislead or take advantage of them￾characterizes the newly literate participants. COMAL Project Monitoring and Evaluation Report 1999-2001 Page 68of68