Governments USAID FROM T HE AMERICAN PEOPLE 0 Peace Corps Process Evaluation of Peace Corps/Guatemala Essential Nutrition Actions and Designing for Behavior Change Training of Trainers Program MARCH 2018 PEACE CORPS PUBLICATION NO. M0130 Peace Corps mmini;; Progra Overseas_ Support and Training Acknowledgments We would like to express our appreciation to the leadership and programming and training staff of Peace Corps/Guatemala for their support throughout this process evaluation. In addition, we thank the staff of Peace Corps/Washington, USAID, and the Government of Guatemala for their participation and assistance. We also want to thank Jorge Barrios, Sindi Escobar López, Mario Marroquín, María Eugenia Mendizabal, Yolanda Najera, Eduardo Santis, Rafael Sitan, Felipe Velásquez, Melisa Vásquez, and Angela Xiloj for serving as enumerators for all interviews with health center directors and staff and community members. Finally, we want to thank all the health center staff, Peace Corps/Guatemala Volunteers, and community members for taking time to answer our questions and provide us with valuable insight into the ENA&DBC TOT Program. We hope the time they devoted to this evaluation will result in an even more robust program and healthier children, women, and families in Guatemala. Danielle Niedermaier Feed the Future Evaluation Specialist Danny Murphy Program Assistant Joanie Cohen-Mitchell Chief of Programming and Evaluation All 10 enumerators and two of the three members of the evaluation team. Photo credit: Patty Rossell. Cover image provided by Nicole Mortenson (RPCV/Guatemala, 2014-2016) This evaluation was prepared by the Peace Corps with the support of the United States Agency for International Development (USAID) to further the Feed the Future initiative. Page ii Table of Contents Acknowledgments ......................................................................................................................................... ii Acronyms and Abbreviations ....................................................................................................................... vi Executive Summary .....................................................................................................................................vii Introduction.................................................................................................................................................... 1 1. Program Implementation........................................................................................................................... 2 Key Findings: Greatest Strengths and Weaknesses of the Program ....................................................... 3 Key Findings: ENA and DBC TOTs.......................................................................................................... 6 Recommendations for TOTs...................................................................................................................10 Key Findings: Support Group Meetings..................................................................................................19 Recommendations for Support Group Meetings ....................................................................................30 2. Program Complementarity With Other Programs ...................................................................................33 Key Findings ...........................................................................................................................................33 Recommendations..................................................................................................................................36 3. What the Peace Corps Can Learn From This Evaluation.......................................................................38 How to Make the Most of a TOT-Style Program.....................................................................................38 How the Peace Corps Contributes to Development............................................................................... 39 Annex 1. Overview of Program ...................................................................................................................40 Annex 2. Evaluation Questions ...................................................................................................................43 Annex 3. Methods ....................................................................................................................................... 44 Annex 4. How Directors and Staff Understand the Program ......................................................................47 Figures Figure A. Timeline of ENA&DBC TOTs evaluated ........................................................................................ 2 Figure E. Number of other staff trained at health center on ENA by staff who participated in an ENA TOT8 Figure K. Health center staff positions that would benefit the most from attending the ENA and DBC TOTs Figure B. The program is focused in these five departments in the Western Highlands of Guatemala .......3 Figure C. Greatest strengths of the program ................................................................................................ 4 Figure D. Greatest weaknesses of the program ........................................................................................... 5 Figure F. Number of staff trained at health center on DBC by staff who participated in a DBC TOT ...........9 Figure G. Changes to improve program .....................................................................................................10 Figure H. Health center staff: How to make TOTs more useful.................................................................. 11 Figure I. Directors, staff, and PCVs reporting health center staff trained at the TOTs who no longer work for the health center .................................................................................................................................... 12 Figure J. Word cloud of health center staff members' biggest dreams in life .............................................12 .................................................................................................................................................................... 14 Figure L. Why selected staff positions would benefit most from attending ENA TOT ................................15 Figure M. Why selected staff positions would benefit most from attending DBC TOT ...............................15 Page iii Figure N. Are the staff who are likely to benefit from the TOTs the staff who actually attend the TOTs?..16 Figure O. Breakdown of staff positions who are likely to benefit from the ENA TOTs and who actually attend the ENA TOTs..................................................................................................................................16 Figure P. Breakdown of staff positions who are likely to benefit from the DBC TOTs and who actually attend the DBC TOTs..................................................................................................................................17 Figure Q. How staff are conducting support group meetings .....................................................................19 Figure R. Health center staff who reported they could use best practices during support group meetings after the ENA TOT ......................................................................................................................................20 .................................................................................................................................................................... 21 Figure S. How often staff reported using best practices during support group meetings after the ENA TOT Figure T. People who support or do not support health center staff in using best practices......................21 Figure U. Advantages of using best practices cited by staff.......................................................................22 Figure V. Program target and actual beneficiaries .....................................................................................23 Figure W. Percentage of community members who reported difficulty in attending support group meetings .................................................................................................................................................................... 24 Figure X. Percentage of community members who reported difficulty in remembering to attend support group meetings ...........................................................................................................................................24 Figure Y. Norms that support community members in attending support group meetings and norms that hinder attendance .......................................................................................................................................25 Figure Z. Perceptions of God's will that people get sick, according to staff and community members......25 Figure AA. Likelihood that the support group meetings help community member or her family ................27 Figure BB. How well support group meetings are received in the community ...........................................27 Figure CC. Community members have support from family/friends to attend meetings ............................28 Figure DD. Who supports community members.........................................................................................29 Figure EE. How the ENA&DBC TOT Program is similar to other MSPAS programs.................................34 Figure FF. Posters promoting various healthy behaviors on the wall at a health center............................34 Figure GG. The 19 key behaviors comprising the Wheel of Practices for Better Living.............................35 Figure HH. A health center director holding up a poster of the Wheel of Practices for Better Living.........36 Figure II. Logic Model of ENA&DBC TOT Program as interpreted for this process evaluation..................42 Figure JJ. Number of directors, staff and community members interviewed in each department and community...................................................................................................................................................44 Figure KK. Health center directors and staff: years of work experience in Guatemala and nutrition .........45 Figure LL. Health center director and staff familiarity with program, by position........................................48 Figure MM. Familiarity with program by job title .........................................................................................48 Figure NN. Word cloud of health center directors and staff describing the program, including the purpose of the program.............................................................................................................................................48 Figure OO. Description of the population or people involved in the program.............................................49 Figure PP. Description of program benefit or outcome...............................................................................49 Figure QQ. Description of the purpose of the program (population or people) ..........................................50 Figure RR. Description of the purpose of the program (benefit or outcome)..............................................51 Figure SS. Cost and number of participants at the ENA and DBC workshops hosted by Peace Corps between January 2012 and July 2015 ........................................................................................................52 Page iv 1 2 3 4 5 6 7 Boxes Box : Motivating Health Center Staff........................................................................................................12 Box : Guatemala Culture and Laws Affect Staff Members' Use of Best Practices ...................................17 Box : Increase Director/All Staff Buy-In.....................................................................................................18 Box : Best Practices for Support Group Meetings ....................................................................................19 Box : How to Increase the Use of Best Practices During Support Group Meetings .................................22 Box : Guatemalan Law, Culture, and Religion..........................................................................................24 Box : Community Members' Biggest Dream in Life ..................................................................................32 Page v Acronyms and Abbreviations DBC Designing for Behavior Change ENA Essential Nutrition Actions MSPAS Ministerio de Salud Pública y Asistencia Social, Ministry of Public Health and Social Assistance (Guatemala) PC Peace Corps PCV Peace Corps Volunteer RPCV Returned Peace Corps Volunteer TOT training of trainers USAID United States Agency for International Development An enumerator typing an interview response into the DataWinners app on the Google Nexus 7 tablet. Photo credit: Danielle Niedermaier. Page vi An enumerator interviewing a community member in their home. Photo credit: Danny Murphy. Executive Summary Feed the Future is the United States government’s global hunger and food security initiative, leveraging the strengths and resources of 11 federal departments and agencies to transform lives toward a world where people no longer face the agony and injustice of extreme poverty, undernutrition, and hunger. The Peace Corps is an interagency partner in the whole of government initiative, working to address food security in a majority of countries in which Peace Corps Volunteers (PCVs) serve. From late 2015 to early 2016, Peace Corps/Washington and Peace Corps/Guatemala collaborated on a process evaluation of the Essential Nutrition Actions and Designing for Behavior Change Training of Trainers (ENA&DBC TOT) Program, which is supported through a partnership with the United States Agency for International Development (USAID) to support Feed the Future in Guatemala. The goal of Peace Corps/Guatemala’s ENA&DBC TOT Program is to improve undernutrition-related care at targeted health centers in 30 municipalities in the Western Highlands of Guatemala by increasing the knowledge and skills of health center staff, especially related to breastfeeding, complementary feeding, support groups, counseling, behavior change, and adult learning principles. The objective of the program is to improve the capacity of targeted health centers in Guatemala to carry out activities that improve child and maternal nutrition in the local communities.1 This program trains staff at the Ministry of Public Health and Social Assistance (Ministerio de Salud Pública y Asistencia Social, MSPAS) health centers on Essential Nutrition Actions (ENA) and Designing for Behavior Change (DBC), who in turn train their health center colleagues on these topics. Once trained, the appropriate health center staff implement support groups targeting two of the seven Essential Nutrition Actions:2 exclusive breastfeeding and improved complementary feeding practices. The purpose of this evaluation is to: 1. identify potential improvements in programmatic components of the delivery model, to assist the program to achieve its planned objectives and expected results; 2. identify best practices and recommendations that will support replicability; and 3. assess the applicability of the program model for other Peace Corps posts implementing food security and/or nutrition programming, as well as any type of training of trainers (TOT)-style program. 1 The complete logic model for the program can be found in Annex 1. 2 The program focuses on these two ENA because the rates of exclusive breastfeeding and improved complementary feeding practices are low in the Western Highlands of Guatemala. Peace Corps Volunteers have the ability to make a difference in these areas (as compared to other areas, such as vitamin supplementation and treatment of children with acute malnutrition). Page vii Methods The evaluation used a mixed-method approach for data collection, including quantitative and qualitative data from interviews, an online survey, and review of secondary information about the program. The 13- person evaluation team conducted structured interviews in January 2016 at 15 health centers and within the community with three participatory groups: • 11 health center directors • 35 health center staff • 85 community members (66 references from health center staff; 15 selected from support group meeting attendance lists; 4 unknown) The evaluation also included an online survey for PCVs. Six PCVs who attended at least one TOT completed the survey. Results and Recommendations The results of the evaluation reveal that the program is being implemented as it was designed. The health center staff reported that they are gaining the intended knowledge and skills during the TOTs. Community members reported that they enjoy the support group meetings and are also generally gaining the intended knowledge and skills. However, there are specific ways in which the program can be adjusted in order to be more effective. Some key recommendations include: • Train more health center staff • Provide more trainings and follow-up support • Adjust training logistics • Provide more materials and tools to staff and community members • Adjust support group meeting logistics • Broaden topics discussed • Increase community engagement in the program The evaluation also shows that the program was designed in such a way that it fits in with and complements other programs and activities around maternal and child health and nutrition. It appears that the change from using ENA to using the “Wheel of Practices for Better Living” approach3 that is in process will benefit the program. Some key related recommendations include: • Discuss opportunities for complementarity with similar organizations. • Ensure staff who have already been trained receive guidance on the new “Wheel” approach. Other Peace Corps posts who have a similar TOT-style approach for a program, or who are interested in adopting such an approach, would benefit from considering several factors, including: • Get buy-in from all levels of partner organizations, when applicable. • Ensure PCVs provide regular and consistent follow-up to TOT participants, and assist them in implementing the training with their peers upon their return to their work site. • Take into consideration local culture, norms, and laws during program planning and implementation; formative research to understand barriers and motivators is very useful. The Peace Corps works at the grassroots level, with PCVs focused on building capacity. However, this does not mean that Peace Corps’ contribution to development is small. Rather, when implementing evidence-based activities and positioned within a larger initiative or framework, like Feed the Future, PCVs can have a large, sustainable impact. 3 The “Wheel of Practices for Better Living” is a program to promote healthy practices among mothers, children, and homes promoted by the Guatemalan Ministry of Public Health and Social Assistance, the USAID Nutri-Health Project, and the Alliance for Nutrition. It focuses on improving breastfeeding practices, feeding and care, hygiene, and use of health services during the “first 1,000 days” window of opportunity, from pregnancy through a child’s second birthday. Page viii Introduction The process evaluation of Peace Corps/Guatemala’s Essential Nutrition Actions and Designing for Behavior Change Training of Trainers (ENA&DBC TOT) Program was carried out in January 2016 by OPATS, with funding from USAID through Feed the Future. The audience for this evaluation report is the Peace Corps/Guatemala staff implementing the ENA&DBC TOT Program as well as Peace Corps staff around the world implementing (or planning to implement) similar programs. The wider audience includes USAID staff and other government agencies and development organizations in an effort to share best practices and recommendations to improve program design and delivery. This report is organized into three sections that align with the main areas of study and themes of the evaluation: 1. Program implementation 2. Program complementarity with other programs 3. What the Peace Corps can learn from this evaluation Each section contains key findings of the evaluation and recommendations based on those findings. Two enumerators chatting with a community member and her son in their home garden. Photo credit: Sindi Escobar & Eduardo Santis. More details and information can be found in annexes. Annex 1 describes the ENA&DBC TOT program, including the logic model of the program as interpreted for this evaluation. Annex 2 includes all questions that guided the evaluation. Annex 3 outlines the methods used, and Annex 4 contains information about how directors and staff understand the program. Page 1 An enumerator helping a community member mark her thumbprint on a consent form, surrounded by several children and another enumerator. Photo credit: Sindi Escobar & Eduardo Santis. 1. Program Implementation Peace Corps/Guatemala’s ENA&DBC TOT Program began in 2012, and there have been several trainings held every year since then (see Figure A). Peace Corps Volunteers and their work partners from a variety of locations participated in the five-day TOT workshops. Figure A. Timeline of ENA&DBC TOTs evaluated NI\ I 0 1 ,lanu;i.rf ?01? FNA rn­ h'\IV;I 21) 12· FNATO￾H•,••111bs1 2013 ooc -or ~ i.:'l.llU~:r'j 2Ui 4 FNATO￾Jun•2014 Fr-IA- OT ,Jt1nv::11 y 2015 DDC TO￾M;.:rL112t> lt FHATDT r~Li,utu;, 2015 DOC TO￾Juti 2!i'o ENA TOTs focused on knowledge of breastfeeding and complementary feeding, and the techniques to impart this knowledge to coworkers via trainings and to community members via support group meetings. DBC TOTs focused on conducting community assessment to identify barriers to behavior change and designing community programs and interventions that address the barriers specific to the community. Participants are expected to train their coworkers at their health center shortly after returning to their community. The program is focused in five departments in the Western Highlands of Guatemala (see Figure B). Page 2 Figure B. The program is focused in these five departments in the Western Highlands of Guatemala Totonicapán Quezaltenango Sololá Chimaltenango Guatemala City Quiché Key Findings: Greatest Strengths and Weaknesses of the Program Greatest Strengths of the Program According to health center directors and staff, the most commonly mentioned greatest strengths of the ENA&DBC TOT Program are that it results in trained health professionals (3 directors, 19 staff) and benefits for communities (4 directors, 13 staff) and the population in general (7 staff) (see Figure C). Specific community-level benefits mentioned include education of community members via clubs and home visits (6 staff), training on specific health-related topics and having inputs such as Vita Cereal (2 directors, 4 staff), responsiveness of the program to community needs (1 director, 1 staff), and community acceptance of the program and need to improve maternal and child nutrition (2 staff). Over one-quarter (27%) of health center directors did not know what the greatest strength of the program was. This correlates well with seven of the directors being only somewhat familiar with the program and one director being not at all familiar with the program. Health center directors said the following about the program’s greatest strength: • “The Peace Corps Volunteers live in the community and that gives us greater opportunity to reach people.” • “Empowerment of the personnel who work in the field.” • “It responds to the real needs of the community through participatory activities.” A staff noted: • “PCVs, given that they bring strategies on how to train on nutrition topics.” Peace Corps Volunteers said: • “It breaks down behavior change in a very comprehensive way and focuses on the cycle of malnutrition in Guatemala and where health workers can step in to try and break this chain.” Page 3 I I -I -   • “One greatest strength is that it includes a wide range of topics, increases the capacity of health workers, and provides them with the basic resources to implement the strategies learned during the training sessions.” • “Fun, interactive trainings!” Figure C. Greatest strengths of the program Greatest Weaknesses of the Program 0% 10% 20% 30% 40% 50% 60% Don't know or No response Involves everyone Key authorities involved Program's methodology Have a Peace Corps Volunteer Population benefits Community benefits Trained health professionals Greatest Strengths of the Program Health Center Director Health Center Staff According to health center directors and staff, the greatest weaknesses of the program include limited funding for the program (2 directors, 16 staff), lack of robust community participation (12 staff), and inadequate human resources and institutional support (2 directors, 12 staff) (see Figure D). Four health center directors and one staff did not provide a response or said they did not know what the greatest weakness of the program is. A health center director said: • “There was not any continuation with the Peace Corps Volunteer and this considerably affected the program.” Staff noted: • “The health promoters don’t participate and few staff members are taken into account.” • “Male participation.” • “The lack of empowerment amongst the population.” • “A lot of people resist change because of their beliefs.” • “Dedication and parent collaboration.” • “That the basic resources were not available to hold the workshops or meetings.” • “There are no supplies, at times there are no macro-vitamins, the workforce instability, and lack of materials such as the weight and height scales make field work difficult.” • “The change in personnel, understaffing and command of Mayan language.” • “That there is no follow-up given.” Page 4 l - •   • “We workers do it with education, but we do not have materials for practice and food demonstration.” Peace Corps Volunteers said: • “The greatest weakness is that the program lacks some monitoring and evaluation aspects.” • “There isn’t much reinforcement after the one ENA workshop.” • “Lack of resources and difficulty of implementation on the local level, due to inconsistent support by counterparts.” • “We cannot be sure that health workers replicate the information with their constituents.” • “Program is very technical with lots of details that sometimes go over the heads of the health workers.” Figure D. Greatest weaknesses of the program 0% 10% 20% 30% 40% 50% 60% Don't know or No response None Coordination with other institutions Distance between communities Language barriers Human resources and institutional support Community participation Funding Greatest Weaknesses of the Program Health Center Director Health Center Staff Page 5 Key Findings: ENA and DBC TOTs The evaluation revealed several strengths of the ENA&DBC TOTs. In keeping with the design of the program, the TOTs were found to be training health center staff, and these staff were training their colleagues at their respective health centers. The TOTs also occurred at the appropriate time in Peace Corps Volunteers’ service. “Right now, it is a planned and structured program. In the community, the feedback is very positive. There is not a statistic [available] but there has been change since people are allowing themselves to be educated and sensitized.” – Health Center Director 1. Health center staff are being trained. From when the ENA&DBC TOT Program started in 2012 to the time of this evaluation (January 2016), there had been six ENA workshops with 153 unique participants (102 health center staff, 51 Peace Corps Volunteers) and three DBC workshops with 116 unique participants (76 staff, 40 PCVs). Health center staff reported that they enjoyed and found value in attending the ENA and DBC trainings. Even more importantly, they reported that they gained the knowledge and skills that the trainings aim to impart. They also reported going back to their health centers and training their peers on what they learned. Staff said they wished they could have more training and that more staff could be trained. Of the 35 health center staff interviewed, 33 participated in at least one ENA TOT prior to this evaluation. Several staff reported that they attended more than one ENA TOT. The average number of months between when they participated in their first ENA TOT and when the interviews for this evaluation were conducted was 14 months (range: 1-36 months). This suggests that there had been adequate time between the ENA TOTs and when this evaluation was conducted in order for most staff to train their colleagues back at the health center and to begin implementing what they had learned by using best practices. Similarly, of the 35 staff interviewed, 31 participated in at least one DBC TOT prior to this evaluation. Several reported that they also attended more than one DBC TOT. The average number of months between when they participated in their first DBC TOT and when the interviews for this evaluation were conducted was 12 months (range: 1-24 months). While a slightly shorter timeframe than the ENA TOTs, 12 months is still adequate time in which to train their colleagues back at the health center and to begin implementing what was learned at the DBC TOTs by using best practices. 2. Health center staff and Peace Corps Volunteers report that the training is useful and effective. ENA TOT: Staff Perceptions Of the 33 health center staff who attended at least one ENA TOT, 30 reported that the training was very useful and 2 reported that it was somewhat useful. Thus, 97 percent reported that the ENA TOT was very or somewhat useful. Staff reported that the ENA TOT was useful because of what was learned: facilitation/planning techniques or methodologies (11 staff) and technical knowledge (4). Specific technical topics that staff reported as learning included: complementary feeding, and food production and preparation (9); the adult learning principle ERCA (Experience, Reflection, Conceptualization, Application) (3); health and nutrition (3); exclusive breastfeeding (2); and behavior change (2). Page 6 With respect to facilitation/planning techniques or methodologies, one staff said 97% of the health center staff reported that the ENA TOT was very useful that the ENA TOT was very or “because knowledge and techniques were acquired to more adequately deliver somewhat useful; 100% of staff reported information to the population, in an that the DBC TOT was very or interactive and dynamic manner.” Another somewhat useful. staff said, “the strategy that Peace Corps uses is a lot easier. Their techniques allow for people to better retain the information so that they can apply it.” A third staff said, “I learned a lot about how to approach and speak with families in the community using a simple and adequate vocabulary.” Finally, another staff reported learning how to “adequately plan [and] formulate objectives, and different forms of teaching.” With respect to technical knowledge, two staff reported learning more about local food: One said, “local products or food should be taken advantage of for consumption and for planting, as well as the combination of foods.” And another reported learning “about the vitamins that greens and local vegetables contain. People can consume these without spending too much money.” DBC TOT: Staff Perceptions Of the 31 staff who attended at least one DBC TOT, 29 reported that the DBC TOT was very useful and 2 reported that it was somewhat useful. Thus, 100% reported that the DBC TOT was very or somewhat useful. Health center staff reported that the DBC TOT was useful because of what was learned: facilitation￾planning techniques or methodologies (9), technical knowledge (3), and behavior change (3). Specific technical topics that staff reported as learning included: behavior change (5), how to improve child growth and development (3), and the adult learning principle ERCA (Experience, Reflection, Conceptualization, Application) (2). One staff said that the DBC TOT was very useful “because they taught us how to facilitate the topics to different audiences.” Another said, “it helped me understand people on a community level in order to provide quality service and be empathic with patients.” Other staff said the DBC TOT was very useful “because we learned strategies to take the message to the people about how to improve health” and “you can put into practice the acquired knowledge.” With respect to technical topics that were learned, one staff reported learning “that different educational strategies exist without being so tedious, and are suited to the type of participants.” Others learned “an easier model on how to give an effective talk and efficient training” and “related aspects or tools of how to achieve behavior change in health and nutrition content.” Overall Staff Perceptions Of the 35 health center staff who attended an ENA and/or a DBC TOT, 34 staff (97%) reported that they learned at least one new skill from either of the trainings. The main things they reported that they learned were related to improved skills and methodologies around facilitation, interpersonal interactions, and session planning (28 staff). Other skills that staff reported they learned included: active listening and empathy (3), more dedication and teamwork (3), identifying strengths, weaknesses, and opportunities (2), improved nutrition practices (2), and how to motivate people (1). One staff reported gaining “the ability to train through interactive forms.” Another said, “I learned that the talks must include motivation and visual materials” and how “to give more effective training, through an organized plan.” Another reported learning about “ice breakers, concrete objectives, and teaching strategies.” Finally, another staff learned how to better trust, respect, and listen to others. Peace Corps Volunteer Perceptions Peace Corps Volunteers who participated in the TOTs reported that the trainings were effective and they learned something new. All reporting PCVs (5) said that they learned a new skill from attending a TOT. Of those five PCVs, one thought the training was very effective at teaching technical ENA knowledge and Page 7 four thought the training was somewhat effective. Four of the PCVs said they gained skills related to facilitation techniques. One PCV said, “working better with counterparts was a skill gained because there was so much one-on-one time with the counterparts. The actual information shared had already been thoroughly taught during pre-service training.” In addition, most PCVs reported thinking that, after the TOT, most health center staff could successfully run or manage support group meetings. One PCV said this is because “I think they come in with a good baseline for counseling and the trainings help them practice and improve.” However, one PCV said that only some staff could successfully run or manage support group meetings after the TOT because “they didn’t really understand the point. They had a hard time correcting [cultural] misconceptions in a proactive and positive way.” Most PCVs also reported thinking that most staff could successfully negotiate a nutrition behavior with participants. One PCV said, “they are equipped with the skills and knowledge to do so [after attending the TOT].” Another noted, “I feel that capacity was definitely improved, but there would still need to be a lot of support from the PCV before the attendee could do it on their own.” Overall, PCVs reported that they think the TOTs were very effective (40%) or somewhat effective (60%). One PCV said that trainings were very effective because “the trainings included a wide range of topics and educational approaches.” Another PCV said, “counterparts really enjoy these trainings because they are interactive and have a lot of good information for participants and opportunities for them to practice what they have learned.” However, one PCV said the trainings were only somewhat effective because there is “too much time lecturing and not enough practical experience.” 3. TOT participants are training other health center staff. ENA TOT The health center staff and PCVs who attended the TOTs were strongly encouraged to train their 79% and 77% of the health center colleagues at their respective health centers when they returned from the training. Of the 33 staff interviewed who had attended at least one ENA staff reported that they trained other staff at their health center on TOT, 26 of them (79%) reported that they trained other staff at their health center on ENA when they ENA and DBC, respectively. returned from the training. On average they trained 20 colleagues (range: 2–38) (see Table E). Figure E. Number of other staff trained at health center on ENA by staff who participated in an ENA TOT Number of other staff trained Number of ENA TOT participants who trained other health center staff after returning from the ENA TOT None 7 1-5 4 6-10 4 11-15 4 16-20 3 21-25 5 26-30 4 31+ 3 Four out of six PCVs reported that they trained other staff at their health center on ENA when they returned. An average of 13 staff were trained (range: 8–16). The PCVs reported that these ENA trainings differed slightly from the ENA TOTs in that they were always shorter than actual TOTs. Two PCVs reported that they did not do a training at their health center because staff did not have enough time to attend a training. Page 8 DBC TOT Similarly, of the 31 staff interviewed who had attended at least one DBC TOT, 24 of them (77%) reported that they trained other staff at their health center on DBC when they returned from the training. On average they trained 17 staff members (range: 2–38) (see Table F). Figure F. Number of staff trained at health center on DBC by staff who participated in a DBC TOT Number of other staff trained Number of DBC TOT participants who trained other health center staff after returning from the DBC TOT None 7 1-5 3 6-10 4 11-15 7 16-20 4 21-25 4 26-30 1 31+ 2 Three out of four PCVs reported training other staff at their health center on DBC when they returned. An average of 22 staff were trained (range: 8–50). One PCV reported that the training he/she did was shorter than the DBC TOT. Another PCV reported that the training was done at the departmental level. The one PCV who reported not doing a training on DBC for staff at his/her health center said he/she did not do the training because “the doctor thought that only the two participants who participated could benefit, and felt that every other staff member already had information on the topic.” 4. TOTs occur at an appropriate time during Peace Corps Volunteers’ service. ENA TOT Of the six PCVs who said they attended an ENA TOT, all of them said two staff from their health center attended with them, which is in line with the program design. The PCVs said they attended the ENA TOT an average of 4.8 months after being sworn in (range: 2–8 months). This is very close to when PCVs said they would ideally attend an ENA TOT: on average, PCVs said it would be best for Volunteers to attend the ENA TOT at 4.5 months after swearing in (range: 0–6 months). They gave two reasons for choosing this timing. First, it was good timing for the health centers: “Early February or in the fourth month [is best because] staff is normally chosen by January and no yearly plans are made until February, meaning this would be a great way to start off the new year.” Second, it was good timing for Volunteers; they have a good understanding of the health center and also have plenty of time to implement activities: • “It is at the beginning, so the Volunteer still has time to implement trainings.” • “They have reached a solid level of confidence/trust with their coworkers at that point and would have completed their health district diagnosis to understand the basic issues and health statistics of their community.” • “So there is some time in site to be familiar with the community and to get an idea of what counterparts would be best to bring.” One PCV said that it was best to be trained on the ENA TOT material during pre-service training, because “health center staff expect Volunteers to have this knowledge on arrival. If not, it’s hard for the staff to see the Volunteers’ expertise in [the] future.” DBC TOT Similarly, of the four PCVs who said they attended a DBC TOT, all of them said two staff from their health center attended with them. The PCVs said they attended the DBC TOT an average of 7.3 months after Page 9   being sworn in (range: 6–9 months). This is close to when PCVs said they would ideally attend an DBC TOT: On average, PCVs said it would be best for Volunteers to attend the DBC TOT at 5.5 months after swearing in (range: 5–6 months). The reasoning they gave was that it gives Volunteers time to get to know their health centers and also enough time to implement what is learned: • “It gives enough time to get to know coworkers, but also it is not a good idea to have the training at the end of the fiscal year because staff changes normally happen in December and January.” • “Because [ENA] and DBC are a huge part of the MCH [maternal and child health] project, it needs to be introduced early in service.” • “So you know who to bring with you that is the most active in site and would want to replicate this training with their fellow health center staff.” • “It’s more complicated and requires more confianza [trust]. At the same time, [behavior change] is a long process, so giving Volunteers time to promote these changes is also essential.” Recommendations for TOTs Based on recommendations from interviewees (see Figures G and H) and the findings of the evaluation, the following changes could be made to improve the TOTs: • Train more health center staff • Provide more trainings and follow-up support • Adjust the training logistics and implementation (make the trainings longer, closer to participants’ health centers, and more practical; include more topics) • Provide more materials and tools Figure G. Changes to improve program 0% 10% 20% 30% 40% 50% Don't know or No response None Use local language Improve coordination Improve meeting logistics Increase community participation Strengthen specific program activities Provide more resources and materials Provide more trainings and follow-up support Changes to Improve Program Health Center Director Health Center Staff Page 10 - I - I   Figure H. Health center staff: How to make TOTs more useful 0 1 2 3 4 5 6 7 8 9 10 No changes needed Better physical space for the training Reinforce teamwork & empathy Hold trainings in closer locations Make training longer Include more, related topics Learn & receive more materials & tools Train more staff Make training more dynamic & practical Receive more trainings & follow-up support DBC TOTs ENA TOTs 1. Train more health center staff. Health center directors, staff, and PCVs all agreed that more staff should be trained. This is because To make the TOTs more useful, there is high staff turnover and because those who one health center staff member are trained are not always the most appropriate staff to be trained. recommended that the program “take into account all the staff High Staff Turnover next time.” Of the 35 staff interviewed, 19 said that there were staff who had been trained at the TOTs but who no longer work at the health center. These 19 staff estimated that, on average, they knew of three staff who had been trained but who no longer work for the health center (median: 2; range: 1–15). Similarly, three of five PCVs said that there were health center staff who had been trained at either or both TOTs but who no longer worked at the health center (1 PCV said no, another PCV said he/she didn’t know). These three PCVs estimated that, on average, they knew of four staff who had been trained but who no longer worked for the health center (median: 3; range: 3–5). Because of this high rate of staff turnover (see Figure I), regular trainings would be useful to ensure that the right health center staff are trained and leading the meetings. Page 11 T maln-u riliOD￾Figure I. Directors, staff, and PCVs reporting health center staff trained at the TOTs who no longer work for the health center 0% 10% 20% 30% 40% 50% 60% 70% Health Center Director Health Center Staff PCVs Over 50 percent of health center staff and PCVs reported that there were staff who had attended at least one ENA/DBC training who no longer work for the health center. Health center directors were less likely than health center staff and PCVs to report this. Box 1: Motivating Health Center Staff Staff members’ biggest dream in life is to do their job well and to help others (see Figure J). It would be beneficial if the Peace Corps/Guatemala program team used this information to encourage and motivate staff to use best practices. For example, the program team could note staff members’ biggest dreams in life and refer back to them as motivators throughout the TOT and after when they’re implementing support group meetings. Figure J. Word cloud of health center staff members' biggest dreams in life Page 12 _.. SUSia tiealthi= we.. fi  ._,.e . deve op,. ... -m~e •=u-;gode::U SUDPOrl 1r women=s e e- • tore::.-- ; ontam0 - g O O -·" I ""ltlltbetter- IUdv; - ~ =ca mpact ufia,ua1e p ,.,helll!HI llllc.1111ec,~ =- ·-= = =-=Ji ...... - .= lam·1 •=1 r 1• .newi _ I J.!:--:r.:w~ !\mr=Ch 11 d re1e~H -=dcommun11v...:f POPUiation Training the Right Staff The evaluation found that sometimes the staff who were selected to attend the trainings were not actually the most appropriate staff to be trained. Directors and staff agreed that auxiliary nurses, health educators, and health promoters would benefit from attending either/both TOTs: at least 45 percent of directors and staff named those three positions (see starred columns in Figure K). These positions were selected because they have a close connection and contact with community members and they provide direct services to the communities (see Figures L and M). One staff member suggested improving the program by making “the trainings be continuous due to the constant change in personnel.” However, there was some disagreement between health center directors, staff, and PCVs around which staff positions would benefit most from attending the TOTs. Directors, who are often medical doctors, thought that medical doctors would benefit from attending these trainings at a much higher rate than staff, who are not medical doctors. Other positions mentioned who would benefit from the TOTs included community facilitator and technical health staff. Three directors and five staff said everyone (all health center staff) would benefit from being trained: • “Everyone should be involved so that the program can be taken advantage of more.” (director) • “Because the more trained people are, the better the service.” (director) • “Share [the trainings] with all of the staff, in the Mayan language.” (director) • In order “to develop good teamwork to offer good service to patients.” (staff) • “Because if there is not good health care delivery, every health worker should get involved in order to achieve change.” (staff) One director said that auxiliary nurses, health educators, and health promoters should be trained because “they are out on the ‘battlefield,’ in the communities, and in direct contact with the patients for more time than the rest of the staff. But for addressing inaccurate community beliefs, it would be the doctor or professional nurse.” One staff recommended that those same positions be trained for similar reasons: because “the auxiliary nurses give talks at the health center and the educators work in the community. The health promoters also work in the community with the people.” All PCVs thought professional nurses would benefit from attending both ENA and DBC TOTs. At least two PCVs thought that all of the other positions except medical doctors would benefit from attending the trainings; only one PCV said medical doctors would benefit. One PCV said that rural health technicians, a position that was not included in the question options, would also benefit. The only PCV who selected medical doctor said that doctors (and professional nurses) “would benefit most because they are usually in a position of power, therefore if they see an importance during the training, it is more likely that it can be replicated in site. If it is a person who does not have a lot of power in the health center, they are likely to utilize what they learned but it is unlikely that the trainings can be replicated with more staff.” PCVs said they selected the non-MD positions because, as one PCV put it, those positions “work directly with community members and [provide] education.” Page 13 * ** * * *    * Figure K. Health center staff positions that would benefit the most from attending the ENA and DBC TOTs 0% 20% 40% 60% 80% 100% Health Center Directors Health Center Staff PCVs Positions that would benefit from attending the ENA TOT Positions that would benefit from attending the DBC TOT Agreement among Directors, staff, and PCVs that these positions would benefit from attending the TOTs Directors, staff, and PCVs agreed that health educators, health promoters, and auxiliary nurses would benefit from attending the TOTs (see starred columns). However, directors, who are often medical doctors, were more likely than staff members to indicate that medical doctors would benefit from the training. Page 14   1 j - • j • I I l - I  1 - I - - , __  Figure L. Why selected staff positions would benefit most from attending ENA TOT 0% 10% 20% 30% 40% 50% 60% 70% Don't know or No response All decisions should be consensual Because they're involved with health and nutrition… These could support when needed These have higher education levels To develop teamwork & coordination &… Better trained staff leads to better service delivery… Because they provide services in the communities Close connection and contact with community… Staff Positions that would Benefit Most from ENA Training - Why Health Center Director Health Center Staff Figure M. Why selected staff positions would benefit most from attending DBC TOT 0% 10% 20% 30% 40% 50% 60% Don't know or No response To develop teamwork & coordination &… Because they're involved in health and nutrition… They speak the local language They're familiar with the problems the population… Because they provide services in the communities Better trained staff leads to better service delivery… Close connection and contact with community… Staff Positions that would Benefit Most from DBC Training - Why Health Center Director Health Center Staff Page 15 ...    Both health center directors and staff said just over 50 percent of the time that the staff positions which they had named as being the ones that would benefit the most from attending the TOTs were the staff positions who were actually attending the TOTs (see Figures N, O, and P). PCVs responded less favorably; they said 10 percent of the time that those staff positions who would benefit are the ones who were attending. Directors and staff said 40 percent of the time that these staff positions are sometimes the ones attending; PCVs said this 68 percent of the time. Finally, directors and staff said 7 percent of the time that these staff positions are never the ones attending; PCVs said this 23 percent of the time. Figure N. Are the staff who are likely to benefit from the TOTs the staff who actually attend the TOTs?    Directors - ENA TOTs Directors - DBC TOTs Yes, always Staff - ENA TOTs Yes, sometimes Staff - DBC TOTs No, never PCVs - ENA TOTs PCVs - DBC TOTs 0% 20% 40% 60% 80% 100% Figure O. Breakdown of staff positions who are likely to benefit from the ENA TOTs and who actually attend the ENA TOTs Technical Health staff Medical Doctor Auxilliary Nurse Health Educator Yes, always Health Promoter Yes, sometimes Professional Nurse No, never Other Community Facilitator 0% 20% 40% 60% 80% 100% Page 16 ...    One way to make the TOTs more useful, according to one staff member, is to ensure that “the Figure P. Breakdown of staff positions who are likely to benefit from the DBC TOTs and who actually attend the DBC TOTs 0% 20% 40% 60% 80% 100% Community Facilitator Other Medical Doctor Professional Nurse Auxiliary Nurse Health Promoter Health Educator Yes, always Yes, sometimes No, never For the few staff who said that the staff positions who would benefit are not the positions who are attending, they gave a few reasons: for example, because the Peace Corps did not consider them, they were not invited or aware of the training, or their workload and duties made it difficult to attend such a training. One staff said the positions she/he recommended were not the ones attending “because [the Peace Corps has] invited personnel who are distinct from those who are qualified.” PCVs had supporting responses for why the staff positions who would benefit are the positions attending: they said the doctor thought those staff underneath him would benefit from the training (rather than the doctor himself) and that those staff were the ones who were available. 2. Provide more trainings and follow-up support. Over 40 percent of health center directors and staff, as well as several PCVs, agreed that staff could benefit from more trainings and more follow-up support after the trainings. One Volunteer said, “There needs to be reinforcement of the topics.” Another Volunteer said that there should be a “follow-up workshop that participants can come to if they implement the nutrition workshop with the rest of their coworkers in their [health center].” A third Volunteer said it is important to “focus on next steps; since health workers already have the knowledge base, it would be good to focus more on community program implementation, in order to put into practice new skills and trainings be reinforced and occur monitor/evaluate program outcomes and more frequently.” impact.” Box 2: Guatemala Culture and Laws Affect Staff Members' Use of Best Practices It appears that Guatemalan culture and laws both help and hinder staff from using best practices. It would be beneficial for the Peace Corps/Guatemala program team to examine which aspects of the culture and law are barriers to and motivators for staff to use best practices in order to address the barriers and capitalize on the motivators in the future. 3. Adjust the training logistics and implementation. While several staff said that no changes were needed to make the TOTs more useful, many had recommendations (see Figure G above). Nine staff said that the trainings, especially the ENA TOTs, should be more dynamic and practical. One way to make the training more practical is by using simpler Page 17 language: one staff recommended “using a type of vocabulary that is more practical to remember.” A PCV also said that one way to improve the program is to limit the list of terms that are used, such as those related to adult learning principles. Some staff said that the trainings should be longer (5 staff), include more topics (5), and be closer to the health center at which they work (3). One staff recommended that there be a “schedule and day to hold the sessions in a neutral or closer community.” Another staff said that one way to improve the TOTs is to ensure that “we receive more trainings on different topics to teach the community.” Box 3: Increase Director/All Staff Buy-In If health center staff at all levels, and especially directors, understand the purpose of this program better, they will likely support the program more. More buy-in from higher levels at the health centers could result in more resources being allocated to the program. Additionally, more buy-in from the peers of the staff who implement the meetings could mean that the staff get more support from their peers, which will help them to continue to implement best practices and could mean best practices are shared more with their peers. In order to get more buy-in from directors, the program could consistently invite directors to the TOTs, or perhaps to a portion of the TOTs, such as the last day or so, when staff who attended can demonstrate the knowledge, skills, and attitudes they gained during the TOTs. 4. Provide more materials and tools. A commonly cited greatest weakness of the program by health center directors and staff was funding. One Volunteer cited limited resources as a key challenge in implementing the program. While it is not entirely clear what directors, staff, and PCVs think could be accomplished with more funding for this program, they provided some specific recommendation related to funding. Several directors (27%) and staff (34%) said that one way to improve the program is to provide more resources and materials. Eight staff recommended that they learn and receive more materials and tools. One staff noted wanting “to have sufficient materials that are specific to the program.” Other staff provided recommendations to “receive more tools that can be used with the people” and “give us visual materials.” In addition to these recommendations, additional funding could be used to provide incentives for meeting participants (such as snacks or meals) or to cover transportation costs for staff to meet in locations closer to community members’ homes. Since it is not entirely clear what would be implemented with additional funding, we recommend further investigation by asking health center staff and Peace Corps Volunteers how more funding would be implemented and what it would provide, before providing the additional funding. During these conversations regarding funding, best practices for maximizing resources may be uncovered; if that is the case, it would be expeditious to share and promote those best practices to make best use of the program’s limited funding. Page 18 Key Findings: Support Group Meetings Health center staff are conducting support groups meetings. Community members report that they find value in and gain important, intended knowledge and skills by attending the support group meetings. They generally have support from their families to attend, but could gain more support from the wider community. Guatemalan laws and culture both positively and negatively affect community members’ willingness and ability to attend the meetings. 1. Health center staff are conducting support group meetings. Thirty-two (32) out of 35 staff reported conducting support group meetings after attending the TOTs. Figure Q provides more detail about the number and nature of meetings held. Figure Q. How staff are conducting support group meetings # Staff Who Reported (out of 35) Question Mean Median Range 32 After the trainings, how many support group meetings have taken place? 24 12 0–200 32 How many community members have participated in support groups? 249 50 2–3,000 30 How many health center staff are leading those support groups? 11 7 2–64 Of the four PCVs who answered questions about the implementation of support group meetings, only one PCV said that support group meetings have taken place (24 meetings). That PCV reported that 75 community members have participated in support groups and that three staff are leading those support groups. Nearly all of the staff said that they think they could use best practices in their support group meetings, given their present knowledge, resources, and skills: 27 staff (77%) said they think they could use best practices and 5 staff (14%) said they possibly could use best practices (see Figure R). Box 4: Best Practices for Support Group Meetings Staff who attend the ENA TOTs learn the following best practices, to be utilized when they are leading support group meetings: • The facilitator arranges the group in a circle. • The facilitator presents himself/herself to the group. • The facilitator lists the potential members and characteristics of the group, as well as the role of the facilitator, before engaging with community members. • The facilitator reviews the objectives of each session before and at the conclusion of each meeting. • The facilitator writes the participants’ set of norms that will guide the group and the meetings, following them consistently and reviewing them regularly. • The facilitator uses adult learning techniques and interactive education methodology to explain the topic of the day clearly, promote learning among participants, reinforce important topics, and evaluate participants’ learning. • The facilitator solicits participants’ existing knowledge and encourages continuation of positive practices. • The facilitator promotes learning through a variety of teaching techniques, determined beforehand to be effective and culturally appropriate. • The facilitator uses effective listening techniques and asks open-ended questions. • The facilitator promotes a safe space of respect and confidentiality. Page 19 1-- J • The facilitator encourages all participants to share personal experiences and to support each other mutually, and strengthens healthy attitudes and practices while modifying unhealthy attitudes and practices. • The facilitator learns from the experiences of participants, and reflects and learns from his/her own experiences. • The facilitator thanks participants for their attendance and participation and encourages participants to share their experiences with others. • The facilitator documents group activities, members of the group and attendance, and invites and reminds participants of meetings. Staff who attend the ENA TOTs learn the following seven Essential Nutrition Actions, which are best practices promoted during support group meetings: • Promotion of optimal nutrition for women • Promotion of adequate intake of iron and folic acid and prevention and control of anemia for women and children • Promotion of adequate intake of iodine by all members of the household • Promotion of optimal breastfeeding during the first 6 months • Promotion of optimal complementary feeding starting at 6 months with continued breastfeeding to 2 years of age and beyond • Promotion of optimal nutritional care of sick and severely malnourished children • Prevention of vitamin A deficiency in women and children During these support group meetings, many staff are using the best practices that they learned during the 86% of health center staff ENA TOT (see Figure S). Fifteen (15) staff reported that they always use best practices during support group meetings, and 13 staff reported that they use best practices most of the time. Two staff reported that they use best practices just some of the time. In total, 86 percent of staff reported that they use best practices all or most of the time during their support group meetings. reported that they use best practices all or most of the time during their support group meetings. Figure R. Health center staff who reported they could use best practices during support group meetings after the ENA TOT 0 5 10 15 20 25 30 Don't know / Won't say Possibly Yes Page 20   Figure S. How often staff reported using best practices during support group meetings after the ENA TOT 0 2 4 6 8 10 12 14 16 Some of the time Most of the time Always Nearly all staff (88%) said that their family and friends know they are using best practices in their support group meetings, and of those staff, all said their family and friends definitely (75%) or sort of (25%) support them in using best practices. Similarly, staff get support to use best practices from many colleagues and other people, though some colleagues do not support them. Staff reported that their peer colleagues support them the most (see Figure T). Staff most often reported that they do not feel unsupported by anybody, though some staff said their peers, others, and leadership do not support them. Figure T. People who support or do not support health center staff in using best practices 0 5 10 15 20 25 Nobody Colleagues: Other Everybody Colleagues: Leadership Others Colleagues: Peers Supports Does Not Support Staff reported several advantages to using best practices (see Figure U). By using best practices, staff said they give more participatory, interactive talks and sessions and plan better for those sessions. Staff reported that this led to the community members being more engaged and active participants in the support group meetings—which then led to greater learning and understanding on the part of the participants, and resulting in greater outreach of the messages and achievement of community benefits. In addition, staff said that an advantage of using best practices was that it is easier than not using best practices. Finally, staff said that using best practices led to greater trust among participants and created a setting such that the support group meetings were not just for educational purposes but also were truly supportive situations for participants. Page 21 7 Figure U. Advantages of using best practices cited by staff 0 2 4 6 8 10 12 Greater outreach of the message Groups are not just educational but also act as… More trust It's easy Mothers are more engaged, active, participate… Achieve community benefits Staff give more participatory, interactive talks,… Greater learning and understanding by… PCVs similarly reported that they have seen changes in the way the health center is implementing its nutrition programming after the ENA TOTs. One PCV reported observing more cooking demonstrations and two new nutrition clubs. Another PCV reported that “[staff] use more interactive methods to engage women in the support groups.” Box 5: How to Increase the Use of Best Practices During Support Group Meetings There are a variety of methods that could be employed to help health center staff use best practices more regularly during their work. Volunteers recommended providing more training, follow-up support, and materials for staff, and to ensure support for using best practices from the director. One Volunteer also suggested that the program “use leaders in the health centers who have influence and those who are able to encourage others to implement and utilize the new methods.” Another method for increasing the use of best practices is to focus on laws and cultural aspects that affect the use of best practices. Three staff said that there are laws that make it more likely they will use best practices, such as the decentralization law, COMUSA; the law of water; the Political Constitution of the Republic of Guatemala; the Health Code about the right to health and well-being of the population; and the Integral Prevention for Children and Adolescents. Another three staff did not name specific laws but said that the right to health, food security, a sustainable life, and education, as well as women’s rights and children’s rights, make it more like they will use best practices. Just over half of staff said that there are cultural rules or taboos against using best practices during support group meetings. Five staff mentioned customs or beliefs related to feeding that prevent them from using best practices or inhibit their effectiveness if staff do use best practices—specifically, machismo, religion, the influence of family members, and beliefs around family planning (3 staff). Ten staff mentioned customs or beliefs related to non-food topics, and five staff mentioned general customs or beliefs. For example, two staff said that people believe that postpartum mothers should not drink cold water or eat green vegetables, because doing so will cause disease in the child. Program staff should conduct more research to see how laws and culture affect the use of best practices and how they can be utilized or diminished in order to increase the use of best practices. Page 22  ~  ~ ;;;;;;k flllll;;A I J)####/41 wu74'4'4'4'4'4'/41 2. The program is reaching the intended beneficiaries. Health center directors and staff think the demographics of the target population and the actual beneficiaries are similar (see Figure V). Both directors and staff think children and women/mothers are the primary target populations as well as the primary beneficiaries of the program. Directors and staff often specifically noted children beneficiaries as being children under 5 years of age, children under 2, or children in the “first 1,000 days” (from the start of pregnancy to a child’s second birthday), as well as According to one health center director, the target population is “mothers of families, but it would be good to have a workshop for fathers because they would strengthen the content as they are the provider and head of household.” Figure V. Program target and actual beneficiaries malnourished children. For women, they often specified women of reproductive age, pregnant women, mothers, or nursing women. While there was little difference in the general demographics of the target population and actual program beneficiaries, there were fewer directors and staff who thought the program was reaching the population to the extent that they thought they should be. In addition, 45 percent of directors did not know the demographics of the actual beneficiaries. 0% 10% 20% 30% 40% 50% 60% 70% 80% Don't know/No response Adolescents Family Population/People Professionals Children Women/Mothers Health Center Director Demographics of Actual Beneficiaries Health Center Director Demographics of Target Population Health Center Staff Demographics of Actual Beneficiaries Health Center Staff Demographics of Target Population Page 23       Most community members do not find it difficult to attend the support group meetings (see Figure W). Community members also do not find it too difficult to remember to attend the support group meetings every time (see Figure X). Figure W. Percentage of community members who reported difficulty in attending support group meetings 71% 25% 5% Not difficult at all Somewhat difficult Very difficult Figure X. Percentage of community members who reported difficulty in remembering to attend support group meetings 60% 35% 5% Not difficult at all Somewhat difficult Very difficult Box 6: Guatemalan Law, Culture, and Religion It appears that Guatemalan law, culture, and religion both help and hinder community members from attending meetings (see Figure Y). Eighteen community members said there are norms that make it easier to attend support group meetings; 17 said there are norms against attending the meetings. Community members named laws and individuals within the community, such as neighbors, facilitators of support group meetings, and local authorities, which provide support in some way to community members so it is easier for them to attend support group meetings. Five community members said that “women’s rights” makes it easier to attend. For example, one noted: “When we go to the meetings, the law says that women have a right for health.” Other noted community support includes motivation from local authorities, a warm welcome from the support group facilitators, and an invitation from neighbors to attend. Machismo appears to play a role in preventing community members from attending the support group meetings. One community member said, “Machismo and dependence on men [prevent women from attending]. They do not accept the new norms to better living and healthy lifestyles.” Religion also plays a role. For example, one community member said, “They told us that we must have our children according to God's will. The health center tells us there is nothing we can do. In terms of participating in society and talks, they do not give women the opportunity.” It is unclear who the community member is referring to specifically, but she raises important issues that need to be addressed in order for the program to be Page 24     successful. In addition, gossip in the community and unfriendly fellow participants in the meetings also make it more difficult for community members to attend meetings. Perception around God’s will regarding getting sick can also impact the success of the program. Nearly 30% of community members and a quarter of staff said that is it sometimes God’s will that people get sick (see Figure Z). Religion is a large aspect of Guatemalan culture, so the program team could identify a way in which these beliefs could be addressed in order to improve the program. Overall, it would be beneficial for the program team to examine in more detail which aspects of the culture and religion are barriers to and motivators for community members to attend meetings in order to address the barriers and capitalize on the motivators in the future. Figure Y. Norms that support community members in attending support group meetings and norms that hinder attendance4 0 2 4 6 8 10 12 14 Cultural/Religious Beliefs Machismo Community Support/Stigma Laws Norms against attending Norms supporting attending Figure Z. Perceptions of God's will that people get sick, according to staff and community members. 0% 10% 20% 30% 40% 50% 60% 70% 80% Definitely God's will Sometimes God's will Never God's will Staff Community Members 3. Community members find value in attending the support group meetings. Of the 85 community members interviewed, 83 of them said they attended a support group meeting in 2015. These 83 community members attended a total of 603 support group meetings— an average of seven meetings attended per community member (range: 1–20; mode: 12). The main aspect of the meetings that community members like the most is what they learn (59 community members). Other aspects of the meetings that they like include: the structure of the meetings (10), the meetings are motivating and encouraging (4), and the topics covered during the meetings (4). Fourteen 4 Machismo is a cultural/religious belief but it was not included under cultural/religious beliefs because of the frequency with which it was mentioned compared to other cultural/religious beliefs. Page 25 community members reported liking everything about the meetings. Specific topics they liked learning about include: • Food (18) o Food for children (8) o Food for pregnant women (1) o Local food (1) • Health (14) • Hygiene (13) • Caring for children (11) • Nutrition or malnutrition (8) • Pregnancy (5) Community members think the support group meetings are useful: 94 percent reported they are very useful, and 99 percent reported they are very useful or somewhat useful. Community members gave a variety of answers for why the meetings are useful, but the most common response was the topics that were taught and the knowledge they gained from attending the meetings (61). The specific topics taught and knowledge gained varied across community members, though the most common responses included food (especially for children) (19), health (16), nutrition or malnutrition (10), and caring for children (8). Similarly, the most common advantages to attending the support group meetings cited by community members was acquiring new knowledge (58). A wide variety of subjects/topics learned were mentioned, including food and food preparation (12), children’s health (9), health (9), and nutrition (7). The majority of community members (65%) said that there is nothing that they disliked about the support group meetings. In addition to gaining the knowledge and skills that the meetings aim to impart, community members reported that they gained unintended knowledge, skills, and attitudes by attending the support group meetings, such as feeling encouraged and motivated, overcoming shyness and gaining self-confidence, making new friends, and improving family dynamics: • “I like everything. I learned a lot. I feel motivated.” • “They taught us how to value ourselves. We have rights.” • “I like them because they have helped me be more participative and independent, and to learn more than just household chores.” • The meetings “help with self-esteem,” help “get rid of shyness,” and help with “no longer being afraid to speak with others.” • “We get to know more friends. We gain new experiences or share our experience of what we learn during the talks.” • The meetings “stabilize family relations” and “help us to listen and improve family life.” Nearly 90 percent of community members said that it is very likely that the support group meetings will help her or her family (see Figure AA). This is important considering 50 percent of community members said that it is very likely or somewhat likely that their child will get sick in the next six months. Page 26        Figure AA. Likelihood that the support group meetings help community member or her family 6% 4% 1% Very likely Somewhat likely Not likely No response 4. Community members generally have support from family and the community to attend support group meetings. Health center staff think that support group meetings are generally well received in the community (see Figure BB). Figure BB. How well support group meetings are received in the community 89% Yes, definitely Yes, sort of Don't know / Won't say Staff gave several reasons for why the support group meetings were well received: participants recognized the need and were motivated to learn and achieve health benefits (14), the meetings functioned well and involved punctuality, respect, and friendliness (4), and community authorities and/or leaders were involved (2). 66% 19% 16% Page 27 • • • Staff gave a few reasons for why the support group meetings were not as well received as they could be: people did not support or participate (4), people wanted an incentive (but the program doesn’t provide incentives) (2), and because of beliefs and cultural customs (1). The vast majority of community members said that their family and friends know that they attend the support group meetings (95%). The vast majority also said that they generally have support from their family and the community to attend support group meetings (see Figure CC). Family members, especially husbands, are the main people who support community members in attending the meetings (see Figure DD). One community member said, “My parents [support me] because they saw the difficulties I have with malnutrition.” According to one health center staff, meetings were well-received in the community “because it’s a practical way to teach and people don't feel the tension of a training. Instead, they view it as a way to share experiences.” Figure CC. Community members have support from family/friends to attend meetings 3% 1% 96% Yes No Doesn't know/ No response According to one community member, “There is a lot of envy on behalf of the community members, but for me it is not an obstacle that hinders me, but did say someone does not support them, instead keeps me going forward.” the most common responses were other Neighbors and in-laws are the main groups of people who do not support community members in attending support group meetings. Just over half of the community members (43) said that there was nobody who did not support them. Of those who community members (20), particularly neighbors (14), and family members (13), particularly in-laws (8). Community members often said their family members did not know the benefit of atttending the support group meetings, and that is why they do not support them. One community member said, “My in-laws and my husband do not support me because they say it does not benefit me at all.” Envy was also often cited as a reason certain people, especially neighbors and other community members, were not supportive of the community members attending the meetings. One community member said, “Our neighbors are envious and they are the ones who do not want to participate.” Another said, “The neighbors don’t like that I learn a lot of new things and sometimes they don’t speak to me.” Page 28 Figure DD. Who supports community members Those who do not support com munity members in attending support group meet ings 20 13 - 2 I 3  8- 1 I Source: (Community Member Survey) Relationship to community members Those who do support community members in attend ing support group meetings Community Members - s Neighbors Friends Family Members Most or All Family Members Husband Parents In-Laws Siblings Chi ldren Everybody Nobody - 14 20 - 13 - 11 - 16 I 2 I 2 50 43 78 When community members were asked if there are any community laws or rules in place that they know of that make it easier for them to attend support group meetings, the majority (72%) said no. However, 18 community members (21%) did say yes, naming a variety of different norms, with the most common focusing on women’s rights (5). When community members were asked if there are any cultural rules or taboos against attending the support group meetings, the majority (78%) said no. However, 17 community members (20%) did say yes, with the most common norms reported centered around cultural or religious aspects, such as machismo, specific limitations (often from husbands or grandparents) on women participating in activities, and religious or community beliefs such as that sexuality is a taboo topic (13). Page 29 Recommendations for Support Group Meetings While nearly 25 percent of community members said there are no changes needed to the support group meetings, some community members, health center directors, and staff provided recommendations for how to improve the meetings. Based on those recommendations from interviewees and the findings of the evaluation, the changes that could be made to improve the support group meetings are as follows: • Adjust meeting logistics. • Provide more materials. • Broaden topics discussed. • Increase community engagement. 1. Adjust meeting logistics. Recommendations related to meeting logistics are as follows: • Reduce distance required to travel to meetings (and/or provide travel stipend). • Have a consistent day of week and start and end time for meetings. • Adjust length of meeting based on the wishes of the participants in that specific group. • Ensure all leaders and participants of meetings are punctual. • Improve physical meeting space of meetings. • Use local language and less technical language during meetings. The majority of community members (65%) said that there is nothing that they dislike about the support group meetings. However, many of the things they did not like about the meetings revolve around the attendance and logistics of the meetings: poor punctuality (8), poor participation and motivation (4), meeting time and duration (3), inadequate meeting space (3), meeting starts late (2), they are always sitting down (1), insufficient material (1), and poor attendance (1). In addition, a few community members expressed dislike for various aspects of how the meetings were run: topics covered (2), language too sophisticated (1), and trainers unprepared (1). Health center directors and staff agree that language is sometimes an issue during the meetings, and they typically said that meetings should be conducted in the local language rather than Spanish. One director said that “it would be better to make materials in the communities’ own language.” The main recommended change to improve the program stated by community members was to improve meeting logistics (29%). Similarly, nearly half of the community members (47%) said a change to meeting logistics would make it easier for them to attend, namely changing the time at which the meetings are held, the duration of the meetings, the location of the meetings (i.e., the distance traveled and the mode of transportation), the physical space in which the meetings are held, and how punctually the meetings start and end. However, the specifics around each of these factors varied; for example, some people wanted shorter meetings while others wanted longer meetings, and some people wanted them in the afternoon and some wanted them earlier in the day. Therefore, some adjustments to meeting logistics could be made in order create a better experience for community members. For example, meetings could be held in locations closer to the homes of members attending; this may mean that more groups will need to be created so they can be more disperse and closer to the participants. Having the meetings closer to their homes would mean that participants would spend less time away from home, which would likely be seen as a bonus for family members. Fourteen percent (14%) of community members said being less busy and having more time would make it easier for them to attend the meetings, and 5 percent said having childcare and/or someone to help with responsibilities at home would make it easier. Another option would be to provide transportation funds to each participant. Similarly, each group should regularly assess the day and time when they meet to see how they can be adjusted so it is easier for community members to attend. Another way to improve the meeting logistics is related to language. Community members said language is sometimes a barrier to understanding everything during the meetings; therefore, staff leading the Page 30 According to one health center staff, in order to be more effective, meetings need to have local language skills. Health center staff said that the staff positions who work closest and most directly with community members should be the ones who attend the TOTs; these staff sometimes, but not always, have the necessary local language skills. Given the importance of being able to communicate during these meetings, language skills should be emphasized more in the selection process of who attends the TOTs and, therefore, who leads the support group meetings. 2. Provide more materials. Health center staff said that they would appreciate having more visual aids for the meetings, and community members said they would appreciate the use of more visual aids during the meetings. PCVs also agreed that both staff and community members would benefit if more materials were available for the meetings. Four staff said that it is difficult to use best practices during the support group meetings without proper materials. Given the language concerns that community members expressed, it would be important to make sure any visual aids or other materials are in local languages in addition to Spanish. 3. Broaden topics discussed. Community members said they were very interested in learning more and branching out into even more health-related topics during these meetings. Staff said they were also interested in learning more topics to discuss with community members during meetings. Staff could be encouraged to incorporate other relevant health topics during the meetings, as appropriate. Or, once support groups are well established and staff have covered the necessary nutrition information, they could continue to meet and discuss other health topics that participants are interested in. 4. Increase community engagement. Based on the evaluation results, the program should increase community engagement by: • Helping community members to better understand the program. • Securing more support from family, neighbors, and the wider community (for example, by enlisting their help with covering responsibilities at home so women can attend the meetings). Community members said they generally are supported by their family to attend the meetings, but often not from their in-laws nor from neighbors. If other community members understood the program better, they would likely support the women in attending the support group meetings and, if appropriate, attend themselves. Increased program knowledge may also lead to communities providing practical support to the women who attend the meetings. For example, several community members said that it is difficult for them to attend because of household responsibilities, but if families and neighbors understood the program better, they may be more willing to watch the women’s children or assist with her other household responsibilities so she would be able to attend the meetings. Several staff said that the program would also be more effective if family members were more involved in the program. According to one health the program should “involve all of center staff, the program should “involve all of the the family members surrounding family members surrounding the pregnant mother.” the pregnant mother.” Most community members said that their biggest dream in life is to support their families so they have healthy and happy families (see Box 7). Since this was so common among community members, it is likely that this is a common dream among the broader communities as well. This could be used as a motivator for community engagement in the program. Page 31 Box 7: Community Members' Biggest Dream in Life Community members reported that their biggest dream in life is to support their families so they have healthy and happy families. One community member said, “My biggest wish is that my children grow up sound and healthy.” Another said her biggest dream in life is to “be a good mother and wife, [and] be the best support for my children and see them graduate.” Program staff could use this to motivate community members so they attend meetings and implement what they learn at the meetings. One community member specifically mentioned being able to teach her children what she has learned at the support group meetings: “[My biggest dream is to] instruct my children based on what I have learned, tell them about the changes in women so that they don't experience what happened to me with my mother, who never spoke to me about menstruation or sex.” Page 32 Maternal and Child Health PCV showing off the garden she helped start at the health center Photo credit: Danielle Niedermaier. 2. Program Complementarity With Other Programs The ENA&DBC TOT Program fits in well with and complements other trainings that health center staff have attended and other programs in the areas in which the program is implemented. The new “wheel” approach will likely enhance the program further. Key Findings 1. The program fits in with and complements other services around maternal and child health. Health center directors and staff reported that they attended a variety of other similar trainings in 2015. These trainings were facilitated by a wide variety of organizations, both government and nongovernment, such as MAGA, UNICEF, and SESAN. The most common organization mentioned by directors and staff was MSPAS (Ministerio de Salud Pública y Asistencia Social, Ministry of Public Health and Social Welfare). Many different nutrition-related topics were covered during these trainings, as well as non￾nutrition topics, such as other health topics, agriculture, and communication. Health center directors and staff said that MSPAS provides other services for mothers and children around nutrition, such as growth monitoring and providing supplements. They noted that the ENA&DBC TOT Program fills a gap in the MSPAS programming, by assisting pregnant and lactating women to have good nutrition for themselves and their children so they can achieve regular growth, typically by using only local foods. The vast majority of staff and directors (90%) said that the ENA&DBC TOT Program fits in well or very well with other programs. They reported that the ENA&DBC TOT Program and other programs have the same or similar guidelines, methodologies, goals, and objectives (9% directors, 31% staff) and focus on at every level.” improving general health (9% directors, 20% staff), in addition to less reported similarities (see Figure EE). According to one director, “Everything is connected and goes to benefit the population Page 33   Figure EE. How the ENA&DBC TOT Program is similar to other MSPAS programs 0% 10% 20% 30% 40% Target population Focus on training or education Focus on improving diet & nutrition Focus on improving general health Guidelines, methodology, goals, objectives How the Program is Similar to Other MSPAS Programs Health Center Director Health Center Staff While some PCVs said they do not know as much about the other services that MSPAS provides that are similar or relevant, three PCVs did say that they think the EN &DBC TOT Program fits in well with and complements the other services provided by MSPAS: • “It complements the already established programs.” • “It is a good supplementary training to what MSPAS messages are.” Figure FF. Posters promoting various healthy behaviors on the wall at a health center Photo credit: Danny Murphy. 2. The “Wheel of Practices for Better Living” approach will enhance the program further. The Wheel of Practices for Better Living Program was formally launched by MSPAS in April 2014 and implementation began in August 2015, led by University Research Co., LLC (USAID/Nutri-Salud). This Page 34 multipronged approach is a social and behavior change communication strategy to address chronic malnutrition through 19 key behaviors for improving maternal and child nutrition and health at the household level (see Figures GG and HH). This new approach is distinct from the Essential Nutrition Actions (ENA) approach previously used by the ENA&DBC TOT Program by expanding its scope to include home visiting and counseling, health provider management and health facility activities, and improving the human resource capacity to provide quality health services at the community level.5,6 Figure GG. The 19 key behaviors comprising the Wheel of Practices for Better Living 5 University Research Co., LLC (USAID, Nutri-Salud) Annual Report, October 1, 2014–September 30, 2015. http://pdf.usaid.gov/pdf_docs/PA00MR4F.pdf 6 Pereira, M. et al. 2017. Addressing the Health of Guatemala’s Most Vulnerable: The Wheel of Practices for a Better Life. http://www.pmaconference.mahidol.ac.th/dmdocuments/PMAC2017_CONFERENCE_POSTER%203.pdf Page 35 Source: Pereira, M. et al. 2017. Addressing the Health of Guatemala’s Most Vulnerable: The Wheel of Practices for a Better Life. Figure HH. A health center director holding up a poster of the Wheel of Practices for Better Living Photo credit: Danielle Niedermaier. Health center directors and staff don’t think the Wheel approach differs significantly from the ENA approach. Where it does differ, they see it as an improvement over ENA, specifically because the Wheel groups everything together better and is more practical, more visual, and easier to monitor (1 director, 8 staff). Most directors and staff did not know how using the Wheel approach will affect the program. The few who did report on how the Wheel approach would affect the program most often said it will generally strengthen the program (1 director, 8 staff): “[The Wheel] will definitely benefit [the program] since it is a visual support and the person checks if he/she is complying and is able to see commitment in the wheel.” Most PCVs also said they think the Wheel is better than ENA and that the change to using the Wheel will improve the program. One PCV said, “The [Wheel] is much easier to follow, has a lot of training materials and is a great design.” Another PCV said, “I think [the Wheel] can be a great complement [to the program], in that it can assist in applying theories and methods learned during trainings on the community level.” Recommendations 1. Discuss opportunities for complementarity with similar organizations. It would be useful for the Peace Corps/Guatemala program team to continue to meet with program managers of similar programs in order to make sure activities align and complement each other as best as possible. This is particularly important as the Government of Guatemala, USAID/Guatemala, Peace Corps/Guatemala, and other organizations continue implementing food security programs under Feed the Future. Such conversations may open doors for future collaboration and strengthen programs for all organizations involved. 2. Ensure staff already trained receive guidance on the new “Wheel” approach. While some of the 178 health center staff who attended ENA or DBC TOTs no longer work for MSPAS, it is important that those staff who still work at the health centers receive some form of training on the new “Wheel of Practices for Better Living” approach. This training could take a variety of forms, including ensuring that the PCVs who work with these staff take time to update them on the Wheel. Given the Page 36 feedback from directors and staff, ensuring that staff who have already been trained understand the Wheel approach will likely improve the support group meetings. It will also ensure that the staff already trained and any future staff who are trained at these health centers are implementing the same best practices and are able to support each other in doing so. Page 37 Two enumerators with two health center staff and the PCV who works with them. Photo credit: Danny Murphy. 3. What the Peace Corps Can Learn From This Evaluation The results of this evaluation reveal several insights into how other Peace Corps programs can make the most of a TOT-style program and how the Peace Corps contributes to development, particularly with respect to larger initiatives or frameworks. How to Make the Most of a TOT-Style Program There are several factors that should be considered when designing and implementing a training of trainers-style program to make it more effective: • If the program involves coordination with another organization in any capacity, the program team should get buy-in from all levels of that organization in order to be able to effectively implement the program at the grassroots level. • PCVs need to provide regular and consistent follow-up in order to ensure topics that are learned during a TOT are being implemented. This follow-up needs to be planned for, as part of the design of the TOT, with monitoring tools to assist them in their follow-up. Page 38 • PCVs’ leadership, impetus, and encouragement to health center staff to conduct the training back at site with peers/colleagues is key. If Volunteers do not take steps to make sure the training happens, things will typically carry on in “business as usual” style like they were before the TOT. • Local culture, norms, and laws can have significant effects on programs, so PCVs need to do research to understand barriers and motivators in all situations. The Peace Corps’ PACA approach (Participatory Analysis for Community Action) contains many useful tools for Volunteers to use during their research. How the Peace Corps Contributes to Development The ENA&DBC TOT Program is one piece of the larger food security initiative that the U.S. Government, Government of Guatemala, and other actors have been implementing since the Feed the Future initiative began in 2009. The integrated approach of these various organizations and programs ensures sustainability, fosters synergies, and helps them achieve shared goals for rural development. Feed the Future’s goals are beginning to be realized, in part because of Peace Corps/Guatemala’s ENA&DBC TOT Program. The Peace Corps works at the grassroots level, with PCVs focused on building capacity. Therefore, PCVs are perfectly positioned to implement evidence-based activities within a larger initiative or framework. Page 39 Annex 1. Overview of Program Guatemala has the largest economy in Central America. However, with a Gini index of 53.7,7 it is also one of the most unequal countries in the world. The United Nations Development Programme (UNDP) ranked Guatemala 133 among 187 countries in the Human Development Index (HDI), placing it last among Central American countries. After dropping from 56 percent in 2000 to 51 percent in 2006, poverty rates8 rose again to 53.7 percent in 2011.9 Poverty is more extreme in rural areas (71% poor; 24% extremely poor), as income is below average. Food security is a serious concern in Guatemala, especially when it comes to children. Half of all children under 5 in Guatemala are chronically malnourished—the worst level of malnutrition in the Western Hemisphere and one of the highest in the world. The statistics are even more distressing in the Western Highlands of Guatemala where the majority of the population is indigenous. Feed the Future and Peace Corps/Guatemala focus their efforts in these target regions where chronic malnutrition among young children is over 67 percent.10 One of the main reasons Guatemala faces such high levels of chronic malnutrition is that families lack resources to produce or buy foods that meet their nutritional needs. In addition, families lack information on healthy feeding practices for infants and children.11 Peace Corps/Guatemala began implementing the Essential Nutrition Actions and Designing for Behavior Change (ENA&DBC) Program in an effort to improve these major food security concerns for families in the Western Highlands of Guatemala. The ENA&DBC TOT Program has no formal program documentation, as its design and implementation evolved organically from existing funding opportunities. A logic model has been created as interpreted for the ENA&DBC TOT Program, demonstrating the inputs, activities, outputs, outcomes, and impact of the program (see Figure II). The logic model is presented in two levels, to display the training of trainers (TOT) level, as well as the trainings at health centers level. It also displays program assumptions (beliefs the evaluation team has about the program, those involved, the context and the way in which the program is envisioned to work), as well as external factors that interact with and influence the program. The ENA&DBC TOT Program has the following theory of change: Problem statement: Child malnutrition is widespread in Guatemala. Goal: Improve stunting-related care at targeted health centers in 30 municipalities throughout Guatemala by increasing the knowledge and skills of health center staff, especially related to practices of breastfeeding, complementary feeding, use of support groups, counseling, and behavior change and adult learning principles. Objective: To improve the capacity of targeted health centers in Guatemala to carry out activities that improve maternal and child nutrition in their communities. 7 In economics, the Gini coefficient, which is sometimes expressed as a Gini ratio or a normalized Gini index, is a measure of statistical dispersion intended to represent the income or wealth distribution of a nation's residents. It is the most commonly used measure of inequality. 8 The international poverty line has a value of US$1.90 PPP. 9 World Bank Guatemala Country Overview (from World Bank Poverty Assessment): http://www.worldbank.org/en/country/guatemala/overview 10 Western Highlands Integrated Program Baseline Survey, 2013. https://www.measureevaluation.org/resources/publications/tr-14-100 11 www.feedthefuture.gov Page 40 Peace Corps/Guatemala employs Volunteers in three sectors: health, agriculture, and youth development. At the TOT level, the ENA&DBC TOT Program trains PCVs in the Health and Agriculture sectors as well as the Volunteers’ community counterparts, MSPAS health center staff. Of the three sector projects implemented by Peace Corps/Guatemala, the ENA&DBC TOT Program contributes most significantly to the Maternal and Child Health project. As part of Peace Corps/Guatemala’s food security efforts, the ENA&DBC TOT Program was implemented to reduce child undernutrition through the capacity building of health center staff who provide local nutrition education through support groups. Page 41 ENA & DBC TOT LOGIC MODEL FY 2013-FY 2016 Problem: Child malnutrition is w ide spread througho ut Guatemala. Goal: Improve stunting-related' care at targeted Health Centers in 30 municipalities throughout Guatemala by increasing the knowledge and skills of Health Center staff, especially related to practices of breastfeeding, complementary feed ing, use of support groups, counseling, and behavior change and ad ult learning principles. Objective: To improve the capacity of targeted Health Centers in Guatemala to carry out activities that improve maternal and child nutrition in their communities. 1 "stunting-related" because the ENA&DBC TOT Program focuses on reducing the prevalence of this result of chronic undernutrition, the most prevalent malnutrition problem in Guatemala. I Lev~ s IC::] ::=I ==Act=lvltl=es ====:I ;:::: I ==O====: ulpuls I ~ PC/ PC/Guatemala I!! Guatemala 41 Slaff C: ·; ~ PCVs 0 Health C) Cenler staff C: ·c Training lim 'E I- Funding for TOTS ..: ~ .. Center "E Staff .. 0 .&: Non-TOT .. Health .. 41 Center :i:: Staff ... .. C) Training C: time ·c ~ Funding for Trainings at <'i Health Centers Develop ENA TOT materials Develop DBC TOT materials Train PCVs Facilitate TOTS Revise/\Jpdate TOTs, as needed Provide ongoing support to PCVs (and Health Cef'lter Staff, as needed) CV & Select (TOT) Health Center Staff Select which Health Center Staff will attend TOT Attend ENA and DBC TOTs Provide ongoing support to PCVs and Health Center Staff, as needed PCV & Select (TOT) Health Center Staff Implement ENA training back at Health Center Implement DBC training back at Health Center PCV & All/Relevant Health Center Staff Carry out ENA action plan Carry out behavior change action plan Implement support groups for targeted oomen Implement ccunseling sessions for targeted oomen s spent on TOTS # of PCVs trained al TOTs oo ENA # of Health Center staff trained at TOTs on ENA # ENA action plans developed at TOT # of PCVs trained al TOTs oo behavior change and arull learning principles # of Health Genier staff trained at TO Ts on behavior change and adult teaming principles # behavior change aclion plans developed at TOT OulpUls $ spent on health center trainings # ENA trainings led by TOT participants at Health Centers # of Health Center staff trained at Health Centers on ENA # ENA action plans developed at training # DBC trainings led by TOT participants at Health Centers # of Health Center staff trained at Health Centers on behavior change and adult learning principles # behavior change action plans developed at training Assumptions: 1. Support groups and counseling sessions are successfully facilitated to lead to behavior change. # of Health Centers implemerting (?) their ENA ilt-tion plan # ot Heetth Centers implemerting (?) their behevior change .iction plen # support s,oups taking ~ace at lhe He.ilth Ce~ ers # Health Center ua!f leadi'lg a support g,oup # Beneficiaries pa.rtic~ting in a support group • c.ounseling sessions ta.king place at the H ea• h Centers • Health Center Stlllff leading eounuling sessions 1'- Benefic;iaries participating in counuling ses!liom • Beneficiaries lrained in exclusive bfeastfeeding praetiees # Beneficiori,n lroine d in complementary feeding pradices # Beneficiaries demonstraling knONledge of improved breastfeeding practices # Beneficiaries demonstrating knowledge of improved complementary feeding praclices # Beneficiaries demonstrating skills in providing impraved canplemenla.y foods ,_. ru1II # Beneficiaries implementing improved breastfeeding practices # Beneficiaries implementing improved ccmplementary feedin practices Public Benefit External Factors: Staff turnover at Health Centers, Health Center Staff attrition, lack of standardization at Health Centers, current Ministry of Health priorities, Volunteer emergencies, Volunteer attrition, Volunteer language level. Figure II. Logic Model of ENA&DBC TOT Program as interpreted for this process evaluation Page 42 Annex 2. Evaluation Questions Three key questions guided the evaluation: 1. How well-designed is the program? 2. How well-implemented is the program? 3. What is the overall value of the program? More specific evaluation questions incorporated into the evaluation design included the following: 1. How well-designed is the program? ENA&DBC TOTs • What other services does MSPAS provide at health centers regarding nutrition counseling, groups of community members, exclusive breastfeeding, and complementary feeding practices? • How well does the Peace Corps/Guatemala ENA&DBC TOT Program fit with and complement other services provided by the MSPAS health centers? What other trainings have TOT participants attended previously? • What other similar programs are being implemented in the same geographic areas as the ENA TOT Program? • How well is the ENA&DBC TOT Program reaching the appropriate target population of health center staff? • How many ENA&DBC TOTs have occurred? How many health center staff and PCVs were trained during these TOTs? • How can the TOTs be improved? 2. How well-implemented is the program? Capacity Building/Skills Transfer • Is there an effective transfer of knowledge, skills, and attitudes from these workshops? • Is the timing right for this workshop within the grander continuum of PCV training? • How many resulting ENA trainings and DBC trainings at health centers have occurred? How many health center staff were trained during these trainings? Are these trainings modified at all from how the information is taught during the TOTs? • How can the support group meetings be improved? 3. What is the overall value of the program? Content and Implementation • How many support group meetings have occurred? How many staff are leading those meetings? How many people have attended those meetings? What are the demographics of the people who have participated in those meetings? • How does the new content focus (La Rueda de Prácticas para Vivir Mejor, “The Wheel of Practices for Better Living”) differ from the ENA content focus? How will that affect the program moving forward? • How much did the TOTs cost? Page 43 I I I I Annex 3. Methods Evaluation staff from Peace Corps/Washington worked closely with Peace Corps/Guatemala staff to design the program evaluation. In January 2016, the data collection was conducted. Ten enumerators with the necessary local language skills were trained for one week on evaluation methods, interviewing skills, and mobile data collection using Google Nexus 7 tablets. The following week, the enumerators traveled to all five departments where PCVs were living and working and where staff had attended at least one workshop at the time of the evaluation. They interviewed 11 health center directors, 35 health center staff, and 85 community members at 15 health centers (out of 30 health centers involved in the program) (see Figure JJ). Figure JJ. Number of directors, staff and community members interviewed in each department and community. Department Community # Directors Interviewed # Staff Interviewed # Community Members Interviewed Quetzaltenango Cajolá 1 4 3 Quetzaltenango Concepción Chiquirichapa 1 0 0 El Quiché Joyabaj 2 5 6 Sololá Nahualá 1 1 5 Chimaltenango Patzicia 1 2 6 Chimaltenango Patzún 0 2 8 Totonicapan San Andrés Xecul 0 4 7 El Quiché San Antonio Ilotenango 1 2 5 El Quiché San Bartolomé 0 2 3 Totonicapan San Cristóbal 1 2 1 Chimaltenango San José Poaquil 1 3 5 Quetzaltenango San Miguel Sigüilá 1 2 15 Sololá Santa Clara La Laguna 1 2 8 Sololá Santa Lucía Utatlán 0 2 9 Totonicapan Totonicapan 0 2 4 Demographics of Health Center Directors Interviewed Health center directors spent an average of 17 years working in Guatemala, 10 of those in nutrition (see Figure KK). In comparison, health center staff spent an average of 7 years working in the country, 5 in the field of nutrition. Demographics of Health Center Staff Interviewed Health center staff interviewed consisted of 30 females and five males. Interviewed were conducted with ten health educators, seven professional nurses, three auxiliary nurses, two social workers, one health promoter, one person in charge of nutrition and food security, and 11 others. Demographics of Community Members Interviewed All community members interviewed were female. Sixty-six community members (78%) were selected by referral, 15 (18%) were selected randomly by support group meeting attendance lists, and four (4%) were missing information on how they were selected. Community members interviewed had lived in their community for an average of 21 (range 1-79; median 19) years. Community members had an average of 6 (range 2-13; median 6) people living in their household. Page 44 • • • The goal of the enumerators was to interview, per health center, two community members who were referrals from health center staff and three community members who were randomly selected from the support group meeting attendance lists. Figure KK. Health center directors and staff: years of work experience in Guatemala and nutrition 0 5 10 15 20 25 30 35 40 Years of work experience In Nutrition In Guatemala Maximum Average Minimum Directors Staff Directors Staff Key: Interviews with directors and staff were conducted in Spanish. Interviews with community members were typically conducted in a local language, with the responses recorded in Spanish. Enumerators generally interviewed directors and staff using tablets, but more often interviewed community members using pen and paper for cultural reasons and later transferred responses to the tablet. The surveys on the tablets were uploaded nightly when the enumerators had access to Wi-Fi. Enumerators received written consent from all interviewees to conduct all interviews. Two translators were hired to translate the Spanish interview responses into English. Their translations were verified by evaluation team members fluent in Spanish. Peace Corps/Guatemala Volunteers who had attended at least one ENA or DBC TOT were invited to complete an online survey through Survey Monkey. The survey was open from January 18 through April 18, 2016. Quantitative data analysis, including the production of summary statistics, was conducted in Microsoft Excel. Iterative thematic analysis of qualitative data was conducted in NVivo, a product of QSR International. Page 45 Left: A community member signing a consent form held by an enumerator. Photo credit: Sindi Escobar & Eduardo Santis. Right: Two enumerators practicing conducting an interview while other enumerators observe. Photo credit: Danny Murphy. Page 46 Annex 4. How Directors and Staff Understand the Program Health center staff are more familiar with the program than directors (see Figures LL and MM). Health center staff have a very good understanding of what the ENA&DBC TOT Program is about and the purpose of the program (see Figure NN). Health center directors appear to have a weaker understanding of the program than implementing staff. Many staff and directors said that the purpose of the program is to improve health and nutrition through food and/or to avoid/eliminate health problems, and that those who benefit are children, women/mothers, and the general population (see Figures OO, PP, QQ, and RR). The understanding that directors and staff have of the program generally aligns with the design of the program. Health center directors and staff typically specified children under 5 years of age, and of these respondents, some specifically mentioned children under 2 years or in the “first 1,000 days” (from pregnancy through a child’s second birthday). Similarly, for women, they typically specified pregnant women and/or mothers. Three health center directors referred to the people involved in the program as “people in need” or “patients,” which health center staff never did. This may reflect a difference in understanding of the program: staff see the program as targeting all mothers and children, whereas directors see it as targeting those mothers and children who are sick or have specific needs that others do not. When health center staff described the program, they mentioned women/mothers and children nearly equally, but when they explained the purpose of the program, they mentioned children substantially more frequently than women/mothers. This is perhaps because, while the program targets women/mothers through their support group meetings, the goal of this work is to improve the nutrition and health of children. Health center staff and directors most often said the program involves improving health and nutrition and improving education, and the purpose of the program is to improve health and nutrition and avoid/eliminate health problems, particularly malnutrition. The key ways in which the program improves health and nutrition, according to staff and directors, is through making better food choices, proper complementary feeding, and optimal breastfeeding practices. Health center directors, staff, and PCVs said the following about the program’s purpose: • “To improve children’s health and give guidelines so that households have tools to live better.” (director) • “Eradicate malnutrition and have a healthy community. Raise awareness of the actions they create and motivate them to take advantage of their own resources within reach.” (staff) • “The purpose of the ENA&DBC TOT Program is [to] increase the knowledge, skills, and abilities of health workers in behavior change techniques, nutrition education strategies, and adult learning methodologies to improve implementation of community programs and achieve sustainable behavior change.” (PCV) • “The purpose of these two programs is to teach a little bit of the theory of behavior change so health workers can better understand how to move their communities along the continuum. The ENA is a stepping stone to behavior change that uses nutrition as concrete examples of behavior change.” (PCV) Page 47 . promotefnHlJlemenl leedtn,8ctlvitles lb monllorssesslon know1eduelllO ersieachl~g ~ tnuoluedhehaviorll reg n a ntm, d1cali_ "' supportP. P 8ftpauentsa a k I I Ucalors iloctor 8~ contact now ~! eflectlvegood w rks :g !I,! understanJIOIWood8Uslnec• u rs a dflcei - groups pre'!!!.nl 'I I Uii en - - = personal = E women=;•- make §.a 1ackii: &·= = •!:tal!Jl~    .; =.; ~~=-: =.. - =~~ .. _ ... ,.. .,- =••- I a ._ m•- ao .. Ji-•:: .... C,S Hot nanei; needs I I~•= .. m= sii=== ==~;J mtaw.Ji's~ hiiii =.. c h ·11 d r I n c&worf:hops hvutenehelp O E de~lopment - - .----a11rac11ce tra1n1nnsg 01,·11·1 I gm -,ImJiememed ,•ervnneasu!Ks .. : .11:siirrces j8UXIIH~rvpeople ~ .. =.,,p,eYsonnel e mater1als ! = a:osi l!! COFPSfamilvchangels E i ig!~! i locus _ • •-=. populations Figure LL. Health center director and staff familiarity with program, by position 0% 20% 40% 60% 80% 100% Health Center Staff Health Center Director Not at all familiar Somewhat familiar Very familiar Figure MM. Familiarity with program by job title Not at all Familiar Somewhat familiar Very familiar Other 0 7 7 Health Educator 0 6 4 Professional Nurse 0 3 4 Director 1 7 3 Auxiliary Nurse 0 2 1 Health Promoter 0 0 1 Figure NN. Word cloud of health center directors and staff describing the program, including the purpose of the program. Page 48 .. - ..   I I - I - I - I -I - I -   Figure OO. Description of the population or people involved in the program 0% 10% 20% 30% 40% 50% 60% Patients People in need Adolescents Family Community Population/People Children Women/Mothers Describe Program: Population or People Involved Health Center Director Health Center Staff Figure PP. Description of program benefit or outcome 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Improve Housing Issues Improve Food Security Improve Child Development Train Health Professionals Promote Behavior Change Promote Healthy Lifestyle Improve Health Service Delivery Avoid/Eliminate Health Problems Improve Hygiene Improve Education Improve Health & Nutrition Describe Program: Program Benefit, Good Result, or Outcome Health Center Director Health Center Staff Page 49   Figure QQ. Description of the purpose of the program (population or people) 0% 10% 20% 30% 40% 50% 60% Men Community Patients People in need Family Population/People Women/Mothers Children Purpose of Program: Population or People Health Center Director Health Center Staff Page 50 "1 --- 1 ' -   Figure RR. Description of the purpose of the program (benefit or outcome) 0% 10% 20% 30% 40% 50% 60% 70% Behavior Change Better Quality of Life Train Health Professionals Improve Child Development Improve Health Service Delivery Improve Hygiene Change Attitudes Improve Education Avoid/Eliminate Health Problems Improve Health & Nutrition Purpose of Program: Program Benefit, Good Result, or Outcome Health Center Director Health Center Staff Page 51 Office of Overseas Programming and Training Support The Peace Corps Office of Overseas Programming and Training Support (OPATS) develops technical resources to benefit programming, training, and evaluation staff and the larger development community. This publication was produced by the Peace Corps with funding from the U.S. Agency for International Development (USAID) Bureau for Food Security. The Peace Corps encourages the public to freely use this publication and other technical and training materials produced by the agency. Peace Corps Office of Overseas Programming and Training Support 1111 20th Street, NW, Sixth Floor Washington, DC 20526 feedthefuture@peacecorps.gov Abridged Dewey Decimal Classification (DDC) Number: 363.8