EVALUATION AND RESEARCH DESIGN REPORT USAID/MALAWI APATSENI MWAYI ATSIKANA APHUNZIRE AMAA EVALUATION AND RESEARCH SCHOOL CONSTRUCTION ACTIVITY IN MACHINGA AND BALAKA DISTRICTS – PART 1 July 20, 2018 – Final version September 25, 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by NORC at the University of Chicago. AMAA EVALUATION AND RESEARCH USAID/MALAWI APATSENI MWAYI ATSIKANA APHUNZIRE AMAA EVALUATION AND RESEARCH SCHOOL CONSTRUCTION ACTIVITY IN MACHINGA AND BALAKA DISTRICTS – PART 1 EVALUATION AND RESEACH DESIGN REPORT July 20, 2018 – Final version September 25, 2018 Prepared under Contract No.: ID-OAA-M-13-00010 Submitted to: Ben Sylla, USAID E3/ED Niloufer De Silva, USAID, GenDev Submitted by: NORC at the University of Chicago DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Evaluation and Reseach Design Report | I AMAA EVALUATION AND RESEARCH CONTENTS EVALUATION AND RESEARCH PURPOSE, AUDIENCE, AND INTENDED USE ........ 1 Introduction.................................................................................................................. 1 Project Background ..................................................................................................... 1 AUDIENCES ................................................................................................................... 5 EVALUATION AND RESEARCH APPROACH AND DATA ............................................ 5 Evaluation and Research Questions and Indicators .................................................... 5 METHODOLOGY AND ANALYSIS PLAN....................................................................... 7 Selection of Treatment and Comparison Communities ............................................... 7 A. Machinga District................................................................................................. 8 B. Balaka District ..................................................................................................... 9 Methodology Used to Address Evaluation and Research Questions......................... 10 Question 1 ............................................................................................................. 10 Question 2 ............................................................................................................. 13 Question 3 ............................................................................................................. 16 Question 4 ............................................................................................................. 17 Question 5 ............................................................................................................. 17 Question 6 ............................................................................................................. 18 Question 7 ............................................................................................................. 19 DATA SOURCES AND COLLECTION METHODS....................................................... 19 Question 1 ............................................................................................................. 19 Question 2 ............................................................................................................. 20 Question 3 ............................................................................................................. 21 Question 4 ............................................................................................................. 21 Question 5 ............................................................................................................. 22 Question 6 ............................................................................................................. 22 Question 7 ............................................................................................................. 23 SAMPLING STRATEGY AND SELECTION OF RESPONDENTS ............................... 23 FIELD WORK................................................................................................................ 29 Selection of Firm and Recruitment of Enumerators ................................................... 29 PROGRAMMING INTO TABLETS AND TRAINING ..................................................... 30 DATA QUALITY CONTROL.......................................................................................... 31 OPERATIONAL PLAN FOR DATA COLLECTION ....................................................... 32 Evaluation and Reseach Design Report | II AMAA EVALUATION AND RESEARCH ETHICS CLEARANCE .................................................................................................. 33 LIMITATIONS OF THE PROPOSED APPROACH ....................................................... 33 REPORTING................................................................................................................. 34 DISSEMINATION PLAN................................................................................................ 34 YEAR 1 WORKPLAN.................................................................................................... 35 REFERENCES.............................................................................................................. 36 ANNEXES..................................................................................................................... 37 ANNEX I: EVALUATION AND RESEARCH STATEMENT OF WORK ......................... 38 ANNEX II: DATA COLLECTION INSTRUMENTS AND IRB MATERIALS.................... 52 Learner Instrument Baseline – May 2018.................................................................. 53 Learner Assent .......................................................................................................... 54 Caregiver Instrument Baseline – May 2018............................................................... 79 Community Leader Semi-Structured Interview- Baseline May 2018.......................... 91 ANNEX III: ESTIMATED LOE AND STAFFING PLAN ................................................ 100 ANNEX IV: ROLES AND RESPONSIBILITIES MATRIX ............................................. 103 ANNEX V: FINALIZED BUDGET ................................................................................ 105 Evaluation and Reseach Design Report | III AMAA EVALUATION AND RESEARCH LIST OF FIGURES AND TABLES Figure 1: Theory of Change ...................................................................................... 3 Table 1: Treatment and Comparison Communities in Machinga District ................. 9 Table 2: Treatment and Comparison Communities in Balaka District.................... 10 Table 3: Potential Progress for Learners ............................................................... 12 Table 4: Data Collection Activities ......................................................................... 25 Table 5: Summary of Approach to Addressing each Evaluation and Research Question .................................................................................................. 26 Evaluation and Reseach Design Report | IV AMAA EVALUATION AND RESEARCH ACRONYMS AMAA Apatseni Mwayi Atsikana Aphunzire CDSS Community Day Secondary Schools DEC Development Experience Clearinghouse DFID Department for International Development DID Difference-in-Difference E3/GenDev Office of Gender Equality and Women’s Empowerment FGD Focus Group Discussion GRS Grassroot Soccer IKI Invest in Knowledge JICA Japan International Cooperation Agency LGL Let Girls Learn MoEST Malawi’s Ministry of Education, Science and Technology OU Operating Units SCI Save the Children International SOW Scope of Work SRGBV School-Related Gender-Based Violence QED Quasi-Experimental Design USAID U.S. Agency for International Development Evaluation and Reseach Design Report | V AMAA EVALUATION AND RESEARCH EVALUATION AND RESEARCH PURPOSE, AUDIENCE, AND INTENDED USE INTRODUCTION While Malawi has seen a growth in primary school enrollment in recent decades, enrollment rates in secondary school remain relatively low. School fees, long distances to school, a lack of schools and trained teachers, and family responsibilities are all factors that can contribute to a pupil dropping out. According to the Ministry of Education’s 2016 school census, while transition rates from primary school to secondary school have improved in recent years (2011-2016), only 33 percent of boys and 36 percent of girls in primary school transitioned to secondary school in the 2015- 16 academic year. 1 Gender disparities in secondary school enrollment continue, despite the government’s efforts to maintain an equal selection of boys and girls for secondary school. In the 2015-16 academic year, more boys were enrolled (53 percent) compared to girls (47 percent), and more girls than boys dropped out in each grade of secondary school. 2 Girls were most likely to mention school fees as their reason for dropping out of secondary school, followed by pregnancy, marriage, and long distances to school. 3 Gender expectations of girls, combined with violence at school, can also make it more difficult for girls to remain in school. School-related gender-based violence is common in Malawi, with about 27 percent of children reporting knowing someone who experienced a problem while walking to school and about 20 percent of children reporting they suffered an incident themselves. 4 Societal norms for girls can also lead to early marriage, pregnancy, and placing less relative value in a girl’s education compared to a boy’s. These factors can influence a guardian’s decision to allocate scarce resources to support a girl’s education. PROJECT BACKGROUND Apatseni Mwayi Atsikana Aphunzire (AMAA) is a USAID-funded project, led by Save the Children (SC), and implemented in collaboration with Grassroot Soccer (GRS). AMAA is supported by USAID’s Let Girls Learn Challenge Fund awarded by the Office of Gender Equality and Women’s Empowerment (E3/GenDev). 1 Education Management Information System (EMIS) 2015/16 2 Ibid. 3 Ibid. 4 Burton, Patrick. "Suffering at School: Results of the Malawi Gender-Based Violence in Schools Survey." Institute for International Security Studies (2005) Evaluation and Reseach Design Report | 1 AMAA EVALUATION AND RESEARCH AMAA is a five-year project implemented in five districts across Malawi: Mzimba, Balaka, Machinga, Phalombe, and Chikwawa. The project is targeting over 60,000 girls aged 10-19 in both upper primary and secondary schools. AMAA has four main outcomes: (1) to improve the ability of girls to effect change in self and others; (2) to enhance the creation of an enabling environment for girls’ adolescent development and education achievement; (3) to strengthen the quality of teaching; and (4) improve access to secondary schools. AMAA is part of the global Let Girls Learn (LGL) initiative. LGL was launched by the United States Government in March 2015 to build upon existing efforts and invests in new programs to expand education for adolescent girls. LGL seeks to ensure that adolescent girls around the world are able to access quality education enabling them to reach their full potential. LGL activities adopt a “whole-of-girl” approach by recognizing that multiple solutions are needed to address the complex challenges faced by girls. The whole-of-girl approach is based on the recognition that discriminatory gender norms lead some parents to invest in educating their sons rather than their daughters. Girls risk harassment and violence on their way to school and in school. Girls drop out of school owing to pregnancy and early marriage. In some contexts, parents and teachers support boys more than girls in making headway in their studies and careers. Whole-of-girl interventions foster an enabling environment for adolescent girls’ education, safety and well-being, wherein girls increase self-efficacy and are valued by their families, communities, and other institutions. Using the whole-of-girl approach, LGL aims to mitigate the barriers to girls’ retention and survival in school through a package of interventions delivered in the home, at school, in the community, in the system, and with the girl herself. However, due to USAID and Malawi’s Office of Education (MoEST) priorities, as well as the range of girls’ education activities already existing at the local level, AMAA is delivering district￾specific activities in Malawi to mitigate particular barriers to girls’ enrollment and retention in school. In Mzimba, the project will extend an existing dormitory at a rural secondary school to enable more girls to reduce their travel distance and will provide scholarships for those girls who wish to board but whose families cannot pay tuition and boarding costs. In Balaka and Machinga, the project will construct 11 Community Day Secondary Schools (CDSS) – 5 in Machinga and 6 in Balaka – and provide scholarships to vulnerable boys and girls in each new school. Evaluation and Reseach Design Report | 2 AMAA EVALUATION AND RESEARCH In Phalombe and Chikwawa, Save the Children, together with GRS, will implement a full suite of activities focusing on improving a girl’s agency and self-efficacy, community support, teacher quality, and access to financial support. NORC’s evaluation and research of the AMAA program will be limited to the secondary school construction interventions implemented in Machinga and Balaka. The evaluation questions and design are informed by the Theory of Change model presented in Figure 1, created by NORC based on conversations with and documents received from SC and USAID. Figure 1 maps the causal links between AMAA’s construction of new Community Day Secondary Schools and impacts of interest for girls. The model illustrates the processes by which the schools’ construction leads to immediate changes in school capacity and proximity, which in turn leads to beneficial outcomes for girls in the surrounding areas. This theory of change considers and incorporates aspects of the whole-of-girl approach, which was an important framework for the design of the project and interventions. The model is intended to reflect a broad understanding of the overall program approach, which is then used to develop research hypotheses that are the subject of the evaluation. Figure 1: Theory of Change Evaluation and Reseach Design Report | 3 AMAA EVALUATION AND RESEARCH Outputs. Building new Community Day Secondary Schools in Balaka and Machinga districts will certainly lead to an increase in the capacity of the region to educate secondary school students. Additionally, introducing a secondary school into a community where there was not previously one reduces the distance students will need to travel from their homes for the opportunity to access secondary education. It also ensures that the school they attend is located within their community and is therefore more accessible to their caregivers and other members of their community. Outcomes. We expect that these outputs will have immediate effects on girls and their communities (Figure 1, Column 3). A secondary school closer to home eliminates travel time and cost for a girl to attend school, which can impact students in several ways. Students who may have needed to board near a school before can now live at home, making secondary education less costly for the family. A shorter commute frees up time for students—whether that time is used to study, interact with school staff and other students, participate in extracurricular activities, or help the family at home. Furthermore, a shorter commute to school reduces the risks of physical and emotional violence that students, particularly girls, encounter en route to school. Closer proximity to a secondary school may also have outcomes for the wider community. Reducing the distance between school and home brings teachers and school staff closer to the families and communities of their students. Secondly, a new school can place education in the forefront of community members’ minds—raising interest and expectations for the future of their children’s education and employment opportunities. Impacts. After we have determined the likely immediate changes to the realities of students seeking secondary education, we can envision the final impacts (Figure 1, Column 4). First, we expect to see more students attending secondary school. More students can afford to attend a day school versus a boarding school, and acceptance rates into secondary schools should increase as school capacity grows. Second, a reduction in costs, safety risks, and travel time to school can translate into better attendance and retention rates. Third, changes in the frequency of interactions between the school and the community could strengthen the web of support needed for a student to succeed in school. Given these impacts at the secondary school level, we can surmise there will also be impacts on younger students. If previously primary students failed to finish school because they did not see potential for them to attend secondary school, their motivation to complete primary school may improve and primary school retention and completion rates may also rise. Evaluation and Reseach Design Report | 4 AMAA EVALUATION AND RESEARCH We would be remiss to not mention that improved educational outcomes reverberate into many other aspects of students’, especially girls’, lives. Continuing education can delay marriage, delay pregnancy, and improve employment opportunities for a girl, and ultimately improve the lives of her children, including their own educational achievement. However, these long-term impacts are beyond the scope and time period of this study. AUDIENCES The findings, conclusions, and recommendations of this study can be applied to future school construction in Malawi and other places. The data generated will contribute to the growing body of evidence on the relationship between school construction and enrollment, informing new school construction programming for adolescent girls into the future. The audiences for this evaluation and research study comprise USAID Operating Units (OU), notably, USAID/Malawi, the Africa Bureau, E3/GenDev, the E3/Education Office; and the E3/ Energy and Infrastructure Office. Other important audiences are the Government of Malawi, primarily, Malawi’s Ministry of Education, Science and Technology (MoEST), and donors committed to building and/or supporting schools, such as Japan International Cooperation Agency (JICA), Department for International Development (DFID), the World Bank, and the World Food Program. Implementer Save the Children and its partners are also important audiences for the study. EVALUATION AND RESEARCH APPROACH AND DATA EVALUATION AND RESEARCH QUESTIONS AND INDICATORS In April 2017, representatives from USAID (US and the Malawi Mission), Save the Children AMAA team, and NORC met in Lilongwe to start planning the evaluation of AMAA secondary school construction. US/Malawi explained their objectives regarding the evaluation and research study and the importance of using results in the interactions with the MoEST. During January and February 2018, the AMAA team, USAID/Malawi, USAID Africa Bureau, E3/GenDev, the E3/Education Office, and E3/Energy and Infrastructure Office held conversations about the objectives of the study and possible research questions. The NORC evaluation team participated in some of those discussions. Based on those conversations, USAID E3/GenDev proposed the list of evaluation and research questions below. Evaluation and Reseach Design Report | 5 AMAA EVALUATION AND RESEARCH Question 1: What effect does embedding a new CDSS in a community have on (1) primary school completion and transition; and (2) secondary school enrollment, attendance, promotion and retention of girls and boys in the community? Question 2: (2a) What effect does embedding a new CDSS in a community have on the attitudes and expectations of girls, boys, parents and communities in the areas served by the new AMAA constructed new schools regarding: ■ Primary school retention and completion and transition to secondary schools; ■ Interest in continuing girls’ and boys’ education and/or vocational training; ■ Marriage and pregnancy; and ■ Future work/employment/career? (2b) What are the attitudes and expectations of (1) girls and boys who have dropped out of school; and (2) girls and boys who are in the new CDSSs regarding: ■ Perceived value of attending school and education as experienced thus-far; ■ Future education; ■ Marriage and pregnancy; and ■ Future work/employment/career? Question 3: What are the reasons for girls and boys to: (1) drop out of school; (2) repeat; and (3) be held back? Question 4: How are the CDSSs helping girls to overcome identified barriers, such as: education costs, early pregnancy, early marriage, lack of WASH facilities, distance to secondary school, and lack of space in secondary schools? Are there additional barriers to accessing schools faced by girls? Question 5: (5a) What effect do the new CDSSs have on the perceptions of learners regarding physical and socio-emotional aspects of safety? (5b) What are the perceptions of girls and boys dropping out of school regarding physical and socio-emotional aspects of safety? Question 6: What experience do new CDSSs have on attracting and retaining teachers? Question 7: Are girls, their parents/guardians, and community members satisfied with the building facilities of the new schools? Evaluation and Reseach Design Report | 6 AMAA EVALUATION AND RESEARCH METHODOLOGY AND ANALYSIS PLAN The proposed evaluation and research design includes different methods depending on the question to be addressed. Some questions will be answered using quantitative data only, but other questions will require the use of qualitative information. In some cases, quantitative and qualitative data will complement each other to gain details and understanding. While for some questions it will be possible to identify causal effects using a quasi-experimental approach, other questions are more suitable for descriptive data analyses. The information to be used will be gathered from learners, parents, teachers and others in communities where the secondary schools were built (Treatment) and in communities that do not have secondary schools but are similar otherwise (Comparison). SELECTION OF TREATMENT AND COMPARISON COMMUNITIES NORC took advantage of the site selection process that considered a group of similar communities as candidates for treatment. Treatment was only assigned to a subgroup of communities, allowing NORC to draw comparison groups from the rest of the list. The Treatment communities’ selection process is detailed below, followed by NORC’s process in selecting Comparison communities. The selection of Treatment communities that will receive the AMAA secondary schools was done by a committee made up of USAID/Malawi, Save the Children, and MoEST officials, and was supported by technical teams. USAID/Malawi selected the initial pool of candidate communities using the following criteria: ■ Primary school pass rate ■ Standard 8 enrollment ■ Located at least 10km from Community Day Secondary School (CDSS) The selection committee visited these communities to meet with the primary school head teacher, community/school structures and community leaders and conduct a community assessment. The committee then selected communities from this list based on their own preferences. Each selected community was then assessed on technical suitability with respect to the following criteria: ■ Large enough to cater for the size of the structures to be constructed ■ Land voluntarily donated by the community and land user ■ Not owned by the church ■ Soil suitable for construction ■ Nearby water source ■ Environmental impact of the construction Evaluation and Reseach Design Report | 7 AMAA EVALUATION AND RESEARCH Some selected communities were rejected on technical grounds and had to be replaced by others in the original list of candidates. This occurred several times, giving the selection process a high degree of (unintended) randomness and allowing the evaluation team to use the initial list as a source for selecting comparison communities. There are two important points to note: (1) none of these technical construction criteria are likely to have any direct impact on the outcomes of interest in this evaluation and therefore are unlikely to create any bias, and (2) treatment and comparison communities do not need to be identical, as the methodology uses statistical approaches to deal with differences if necessary. A. Machinga District During the initial selection process, the committee considered a total of 14 communities: Chisani, Kalambo, Kayuni, Matanda, Mkotamo, Msasa, Mwalasi, Nansato, Nkasaulo, Ntalala, Nyenyezi, Thobola, Msasa, Ntulira, and Nathendo. From this list, the committee initially selected the following communities in Machinga to receive the secondary school construction treatment: Kayuni, Matanda, Mkotamo, Mwalasi, and Ntalala. NORC then selected 4 comparison communities from the initial list of candidates, discarding several communities because their close proximity to treatment sites. The four communities selected to be used as comparison were: Nkasaulo, Nansato, Ntulira, and Nathendo. None of them are located within 10km from an existing or planned Community Day Secondary School (CDSS). To create an even number with the treatment groups, NORC needed to add one additional comparison site, so Chisese was selected using the same criteria USAID/Malawi used in creating the initial list of candidate communities (i.e., primary school enrollment, primary certification pass rate, and distance from Community Day Secondary Schools (CDSS)). Table 1 shows each community’s enrollment of boys and girls in standard 8, primary school pass rate in numbers and in percentages, and whether the community is in the treatment or comparison group. Evaluation and Reseach Design Report | 8 AMAA EVALUATION AND RESEARCH Table 1: Treatment and Comparison Communities in Machinga District Communities Considered for Treatment* Name of Community Standard 8 Enrollment Primary School Pass Rate Male Female Total Male Female Total % Group Chisani 44 24 68 39 12 51 75% Matanda 33 30 63 19 12 31 49% Treatment Mwalasi 30 24 54 20 15 35 65% Treatment Msasa 61 40 101 44 33 77 76% Ntulira 32 12 44 16 1 17 39% Comparison Kalambo 41 48 89 27 25 52 58% Mkotamo 41 61 102 24 17 41 40% Treatment Thobola 23 34 57 19 29 48 84% Nansato 15 15 30 13 12 25 83% Comparison Ntalala 19 19 38 17 11 28 74% Treatment Kayuni 39 29 68 25 13 38 56% Treatment Nyenyezi 21 20 41 17 8 25 61% Mkasaulo 37 35 72 30 27 57 79% Comparison Nathendo 22 30 52 16 15 31 60% Comparison Comparison Community Selected Using Matching Chisese 22 24 46 18 13 31 67% Comparison * Considered as potential treatment communities by USAID/Malawi, SC, and Malawi education officials. B. Balaka District The selection of treatment and control communities in the district of Balaka followed the same approach used in Machinga. The original communities under consideration were: Ng'onga, Chilanga, Makanjira, Ngwengwe, Nagulukitiche, Namikombe, Ntalikachao, Muthe, Thundu, Njerenje, Mitengwe, Namichere, and Chikoleka. From this list, the committee chose Chilanga, Makanjira, Namikombe, Nagulukitiche, Njerenje, and Namichere for treatment. However, on May 9, we were informed that Nagulukitiche could not be considered for treatment, as it was closer to Chilanga than desirable. The site was then replaced by Ngwengwe. However, this community was also rejected and then replaced by Muthe. NORC then selected the following comparison communities by the same method used in Machinga: Ng’onga, Ngwengwe, Nagulukitiche, Thundu, Mitengwe, and Chikoleka. On May 28, after the baseline data collection took place, NORC learned that Muthe was no longer deemed feasible for construction and thus the communities of Chawanda, Mzimundilinde, and Njese were under consideration as replacement treatment sites. On June 26, NORC learned that the replacement site would be Mzimundilinde. NORC Evaluation and Reseach Design Report | 9 AMAA EVALUATION AND RESEARCH worked with local partner Invest in Knowledge (IKI) to collect data in this new site and add it to the evaluation. NORC had already collected data in Muthe during the baseline and decided to include Muthe as a comparison community, increasing the number of comparison communities to seven. Table 2 shows each community’s enrollment of boys and girls in standard 8, primary school pass rate in numbers and in percentages, and whether the community is in the treatment or comparison group. Table 2: Treatment and Comparison Communities in Balaka District Communities Considered for Treatment* Name of Community Standard 8 Enrollment Primary School Pass Rate Male Female Total Male Female Total % Group Ng'onga 27 25 52 22 24 46 89% Comparison Chilanga 29 15 44 27 13 40 90% Treatment Makanjira 12 17 29 12 15 27 93% Treatment Ngwengwe 10 14 24 10 14 24 100% Comparison Namikombe 17 17 34 15 17 32 94% Treatment Nagulukitiche 18 22 40 15 18 33 82% Ntalikachao 30 35 65 26 26 52 80% Comparison Muthe 47 32 79 40 25 65 82% Comparison Thundu 17 33 50 10 23 33 66% Comparison Njerenje 21 24 45 18 20 38 84% Treatment Mitengwe 41 39 80 35 25 60 75% Comparison Namichere 24 22 46 24 21 45 97% Treatment Chikoleka 45 43 88 40 41 81 92% Comparison Mzimundilinde 19 20 39 19 20 39 100% Treatment * Considered as potential treatment communities by USAID/Malawi, SC and Malawi education officials. METHODOLOGY USED TO ADDRESS EVALUATION AND RESEARCH QUESTIONS Question 1 What effect does embedding a new CDSS in a community have on (1) primary school completion and transition; and (2) secondary school enrollment, attendance, promotion and retention of girls and boys in the community? We will answer the question by comparing primary school completion rates, rates of transition to secondary school, attendance, promotion and retention of learners between treatment communities and comparison communities. We propose to do this in two ways: Evaluation and Reseach Design Report | 10 AMAA EVALUATION AND RESEARCH A) Using existing school administrative data (or creating the system to collect the necessary data) we will track the all the learners (Census) attending standard 5, 6, 7, and 8 at the time of our baseline in May 2018. We will collect data yearly on these students, following those who transfer into the new secondary schools and create a panel to track them. These longitudinal data will allow us to calculate promotion and retention rates in secondary school, as well primary school completion rates. We could also learn which students take the primary school certification exam, and if they end up being admitted to secondary school. In addition, we will explore the feasibility of obtaining attendance data. This strategy allows us to assess the causal impact of school construction on the indicators of interest in question 1 and, given that administrative school data includes all learners, it allows us to maximize the precision of our estimates. However, administrative data has shortcomings. Firstly, while it will be possible to use primary and secondary school administrative data in treatment communities, comparison communities will lack secondary school data. Secondly, administrative data does not include information about the learners and their backgrounds other than very basic characteristics, such as sex. More detailed information will be needed to carry out our analysis. B) For a sample of learners (Sample), in treatment and control communities, we will complement the tracking enrollment data with a baseline survey to collect information on socio-demographic characteristics of the learner, disability status, socio-economic situation of his or her household, household contact information, and information on older siblings. The sample data will permit us to do several things: 1) use the household contact information to continue tracking learners in comparison communities after they finish standard 8, allowing us to see their transition into secondary school and their promotion and retention in the years to follow; 2) use household contact information to reach learners that drop out of schools (see Questions 3 and 4); and 3) understand how the educational path of boys and girls may differ depending on learners’ individual circumstances, such as their age or household financial situation, something that cannot be done with just school administrative data. As Table 3 shows below, boys and girls that are in standard 8 at baseline could remain in the school system until the end of our study in March 2021, and could potentially reach form 3. Similarly, those students in standard 7 and 6, could potentially reach form 2 and 1, respectively, at endline. Learners in standard 5 may be able to reach standard 8. Evaluation and Reseach Design Report | 11 AMAA EVALUATION AND RESEARCH Table 3: Potential Progress for Learners May-18 Mar-19* Mar-20* Mar-21* Std 8 Form 1 Form 2 Form 3 Std 7 Std 8 Form 1 Form 2 Std 6 Std 7 Std 8 Form 1 Std 5 Std 6 Std 7 Std 8 * Tentative date In order to minimize attrition, the NORC team proposes, tracking the status of the learners annually until the endline (planned for 2021). Analytically, to evaluate the impact of the program on, for example, enrollment for the average learner in a treatment community, we will compare enrollment rates between treatment and comparison communities at endline. The basic regression model we will use to estimate the causal impact of the program can be described by: (1) where Eci is a binary variable that is 1 if learner i from community c is enrolled in school at endline and 0 if not; Dc is a binary variable indicating treatment status of community c; Sci is the standard that learner i is attending at baseline; uci is an error term; and , , and are parameters to be estimated. The coefficient of interest is, which measures the effect of the school construction on enrollment at endline. The same model will be used to evaluate the impact of the program on other indicators. This model will, in principle, include all learners in the baseline roster in treatment and comparison groups, and will adapt to include the relevant subgroups depending on the indicator analyzed. For example, when estimating the impact of school construction on transition to secondary school, we will not include learners attending standard 5 at baseline because, in the best case, they would be attending standard 8 at endline. We expect to be able to identify each of the learners in the rosters as girls or boys, and therefore it will be possible to conduct the same analysis by sex. For a sample of learners, we will also have additional information that includes their socio-demographic characteristics and their household information. For them, the equation (1) can include that information and therefore changes to: (2) 𝐸𝐸𝑐𝑐𝑐𝑐 = 𝛽𝛽0 + 𝛽𝛽1𝐷𝐷𝑐𝑐 + 𝑋𝑋𝑐𝑐𝑐𝑐 ′ 𝛽𝛽2 + 𝑢𝑢𝑐𝑐𝑐𝑐 Evaluation and Reseach Design Report | 12 AMAA EVALUATION AND RESEARCH where Xci is a vector of characteristics of learner i, such as age, sex, standard level at baseline, and household level characteristics such as assets. Again, the coefficient of interest is , which measures the effect of the school construction on enrollment at endline. This model therefore will include all learners in the sample, in treatment and comparison groups, and will control for their characteristics. Alternatively, a matching approach can be used. In addition to estimating the overall impact of the construction of secondary schools, we will also carry out a sub-group analysis to examine whether the impacts were different for sub-groups of interest, such as girls or poorer households. We can estimate similar models to measure the effect on other outcomes of interest, such as dropout rates, transition to secondary school rates, repetition rates, etc. It will be possible to estimate the effect on outcomes such as enrollment for each year, and to understand how the trends in treatment and comparison schools have evolved. Finally, it is also possible to use other analytical approaches such as survival analysis to estimate the effect of the school construction on the outcomes of interest. Survival analysis measures the time between the beginning of observation until the occurrence of an event. In the present case, the event could be dropping out of school, for example. NORC will evaluate the possibility of using this approach to analyze and communicate the findings. Question 2 (2a) What effect does embedding a new CDSS in a community have on the attitudes and expectations of girls, boys, parents and communities in the areas served by the new AMAA constructed new schools regarding: ○ Primary school retention and completion and transition to secondary schools; ○ Interest in continuing girls’ and boys’ education and/or vocational training; ○ Marriage and pregnancy; and ○ Future work/employment/career? To answer this question, we will collect information from learners, parents and community leaders about their attitudes towards gender norms and girls’ education, as well as their expectations regarding future education, work and employment, marriage, and family. Attitudes. To estimate the effects of secondary school construction on attitudes, the NORC team proposes to collect data on opinions at baseline from learners attending Evaluation and Reseach Design Report | 13 AMAA EVALUATION AND RESEARCH standard 5 to standard 8 and their parents, in treatment and comparison communities. The exact same data collection will take place again at endline, returning to students attending standard 5 to standard 8 and their parents. These cross-section’ data will allow the use of a quasi-experimental designed to measure the impact of the secondary school construction activity on attitudes. We will estimate before-and-after changes in attitudes in communities that receive secondary schools and in communities that do not. The comparison of those changes is commonly known as the difference-in-difference (DiD) approach. The idea behind the DiD method is to eliminate the differences that the treatment and comparison groups may have (if any) and that are constant overtime. The methodology assumes that in the absence of the program, the two groups would display the same trends in attitudes. 5 We will compare attitudes towards gender norms and girls’ education, among cross sections of learners (or parents) and estimate the causal impact of the school construction activity using regression models of the following form: (3) where Acit is the attitude for student i in community c, at time t, Dc is a binary variable indicating treatment status of community c; Time is a dummy variable equal to 1 at endline and 0 otherwise; Xci is a vector of characteristics of learner i, such as age, sex, standard level at baseline, and household level characteristics; ucit is an error term and , , , , and the vector are parameters to be estimated. The difference-in￾difference estimator is , which measures the effect of the school construction on attitudes towards education, marriage and pregnancy, employment, etc. In addition, we will also conduct interviews with community leaders to collect information about their attitudes toward gender norms and girls’ education. The small number of interviews (12) will preclude us from an impact analysis; however, the information can be used to complete the understanding of the context. Expectations. Measuring the effect of the AMAA school construction on expectations only makes sense if we can measure expectations before the treatment group knows that a secondary school will be built in their community. Once it is known that a secondary school will be built, learners and parents would tend to adjust their expectations to the new reality that they will face, and this will tend to resemble expectations at endline in the treatment communities. 5 This assumption can be checked using some of a learner’s background data collected to respond to evaluation question 1 (e.g., information about their older siblings). Evaluation and Reseach Design Report | 14 AMAA EVALUATION AND RESEARCH The construction of schools in the district of Machinga is already taking place. Therefore, we will not try to measure the effect on expectations in that region. If the information about where school construction will take place in the Balaka district is still not public at the time of the baseline, we will measure changes in expectations there. We propose to collect data on expectations at baseline and endline from learners in standard 5 to standard 8 and their parents in the Balaka district and compute the DiD. However, if the information about school construction is already public in Balaka at the time of the baseline, we will use a qualitative approach to try to explore if and how expectations might have changed. In that case, we will collect information in Focus Groups Discussions (FGDs) with girl and boy learners, and parents. If quantitative data is collected to measure expectations, we will run an equivalent regression model to the one used for attitudes. In this case, the dependent variable is an indicator related to expectations, rather than attitudes. (4) where Ecit is the expectation for student i in community c, at time t; Dc is a binary variable indicating treatment status of community c; Time is a dummy variable equal to 1 at endline and 0 otherwise; Xci is a vector of characteristics of learner i, such as age, sex, standard level at baseline, and household level characteristics; ucit is an error term and , , , , and the vector are parameters to be estimated. The difference-in￾difference estimator is which measures the effect of the school construction on attitudes towards education, marriage and pregnancy, employment, etc. It should be noted that the number of treatment and comparison communities in Balaka is small (13 in total) and, unless the changes in expectations are large, the estimates will tend to be imprecise. If this is the case, it will be noted and findings will not be taken as conclusive but rather suggestive of a particular trend. If instead we use qualitative data to learn about changes in expectations, we will use inductive thematic analysis to identify key subjects and recurring items or ideas emerging from FGDs with learners and parents. The emerged themes will be organized and preliminarily coded. Then, an iterative process will take place to identify the most common concepts and categories. (2b) What are the attitudes and expectations of (1) girls and boys who have dropped out of school; and (2) girls and boys who are in the new CDSSs regarding: ○ Perceived value of attending school and education as experienced thus-far; Evaluation and Reseach Design Report | 15 AMAA EVALUATION AND RESEARCH ○ Future education; ○ Marriage and pregnancy; and ○ Future work/employment/career? To answer part b of the question, we will again collect information from learners, parents and community leaders on attitudes and expectations from all the learners at baseline. Over time, we will update these opinions and aspirations of learners during new interviews. We will track all the learners included in the baseline sample over time. Each time a learner included in the baseline sample drops out of school, an interview will be triggered. Among other questions, we will include an update on their opinions and expectations. Dropouts will be again interviewed at endline. In the case of learners that stay in school, we will collect new information at endline. These data will allow us to see how learners’ attitudes and expectations change over time and to compare girls and boys that dropped out of school vs. those still attending secondary schools. The analysis will be descriptive and will not attempt to make any causal claim related to AMAA secondary schools. Question 3 What are the reasons for girls and boys to: (1) drop out of school; (2) repeat; and (3) be held back? Question 3 looks to identify the reasons girls and boys drop out of school, repeat levels, or are held back.6 We will focus also on risk factors and aspects that girls and parents describe as challenging the ability of girls to attend school. We will compile and analyze these data at endline and compare between the treatment and comparison communities to study if there are differences in the reasons students drop out and to identify the main barriers girls face in attending secondary education. We will organize the data by reason for dropping out and we will also explore if there are differences in those reasons between boys and girls, and between the group of poorer learners and the rest. 6 By “held back,” we mean keeping the learners in standard 7, rather than promoting them to standard 8, in order to better prepare them for the final year of primary school and the certification test that takes place after completing the primary education level. Evaluation and Reseach Design Report | 16 AMAA EVALUATION AND RESEARCH Question 4 How are the CDSSs helping girls to overcome identified barriers, such as: education costs, early pregnancy, early marriage, lack of WASH facilities, distance to secondary school, and lack of space in secondary schools? Are there additional barriers to accessing schools faced by girls? The goal of question 4 is to understand whether constructed secondary schools helps girls to overcome barriers like education costs, early pregnancy, early marriage, lack of WASH facilities, distance to secondary school, and lack of space in secondary schools. We will focus also on risk factors and aspects that girls and parents describe as protecting girls’ capacities to attend school, persist from one grade to the next, and complete primary or secondary school. As in the case of question 2b, we will address this question by conducting interviews each time that one of the learners included in the sample created at baseline is found to have dropped out of school, repeats the standard, or is held back. In the cases of dropouts, a visit to the learner household would be necessary to find out the reasons behind the decision of leaving school and the current circumstances of the learner (e.g., marital status, activity, location, etc.). We will aim to document the main reason for dropping out and the surrounding circumstances, the current situation of the learner, and future plans. At endline, we will try to find these boys and girls, and ask again about their situation regarding education (e.g., if they went back to school), work and employment, marital status, children, and plans for the future. In cases of repetition or when the student is held back, we will request information from the teacher involved in the decision. These learners will continue to be followed over time until endline. In addition, in the case of treatment communties, we will organize FGDs with girls and parents to discuss the advantages of having access to the new AMAA secondary school and also the oportunities for and challenges of staying (keeping learners) in schools. The data will be transcribed verbatim and an inductive thematic approach to analyize it will be followed to identify the main problems that can keep learners from attending school and the main advantages of having a secondary school close to their homes. The qualitative data will be coded and used to complement the quantitative data and gain depth in understanding. Question 5 (5a) What effect do the new CDSSs have on the perceptions of learners regarding physical and socio-emotional aspects of safety? To address the question, we will compare perceptions of safety between those attending secondary schools in their communities vs. those that attend a secondary Evaluation and Reseach Design Report | 17 AMAA EVALUATION AND RESEARCH school away from their community and need to travel or to relocate to other communities to access secondary education. We will compare secondary students in treatment communities with secondary students in comparison communities. (5b) What are the perceptions of girls and boys dropping out of school regarding physical and socio-emotional aspects of safety? For girls and boys who drop out of school (in treatment and control groups), our approach to questions 3 and 4 partly answers this question since our interviews with this group will include questions of safety. We will also research how AMAA new secondary schools affect barriers such as distance to school. To learn more about the safety perceptions of dropouts, we will collect information during interviews triggered each time one of the learners in the baseline sample is found to have dropped out of school. In addition, at endline, we will revisit and re-interview dropouts and ask questions about safety from their counterparts who are still attending school. NORC will present a descriptive analysis of all these data about safety, making use of all the demographic and background data collected on these girls and boys at baseline. Question 6 What experience do new CDSSs have on attracting and retaining teachers? This question aims to document the experience that the new AMAA secondary schools have attracting and retaining teachers. NORC proposes to use administrative school data and teachers’ and head teachers’ interviews to construct longitudinal data on all new AMAA secondary school teachers. A descriptive analysis will be produced using that data. We will capture teachers’ basic demographic and education characteristics and document school vacancies. Over time, we will complete the information by adding data on teacher turnover and replacements, seniority, working conditions, and job satisfaction, etc. Teacher interviews will also include questions to document the main positive and negative aspects of teachers’ experiences in the new schools. Findings from research question 7, which relate to the satisfaction with the school building, will also be included. We will create the baseline for the teacher panel in the second visit to treatment communities (currently expected to take place in March 2019) when all secondary schools are supposed to be finished and running. The information will be updated annually in 2020 and 2021 during the same visits when we will update the panel of Evaluation and Reseach Design Report | 18 AMAA EVALUATION AND RESEARCH learners. Obviously, in this case, the data collection will only take place in treatment communities. It should be noted that the proposed approach consists of documenting and trying to understand the experiences of teachers in the rural locations where they will be deployed. We will not try to compare their retention or turnover rates with those of teachers in other schools. Question 7 Are girls, their parents/guardians, and community members satisfied with the construction design and facilities of the new schools? The construction design attributes should be further specified in collaboration with USAID. This question seeks to understand learners’, parents’, and community leaders’ satisfaction with the new AMAA secondary school buildings. The AMAA buildings follow the Malawi MoEST standards, therefore the aim is not to inquire about AMAA schools as a distinctive group but generally about the regular schools offered by the public sector. We will interview a random sample of AMAA secondary school learners, parents, and teachers at endline in 2021. During these interviews, we will inquire about their satisfaction with the building structure, including classrooms, assembly hall, laboratory, library, fences and gates, latrines, outdoor space, and water sources, etc. The analysis of the information will be descriptive, but we will use the findings from other questions, such as reasons for dropping out of school or discussions related to question 4, to complement it. DATA SOURCES AND COLLECTION METHODS In this section we explain in more detail the data that is required to respond each of the evaluation and research questions. As before, we proceed question by question. Question 1 What effect does embedding a new CDSS in a community have on (1) primary school completion and transition; and (2) secondary school enrollment, attendance, promotion and retention of girls and boys in the community? To respond to evaluation question 1, we will use school administrative data if available about all learners attending standard 5, 6, 7 and 8 at the time of our baseline, in May 2018. The data would come directly from the primary schools in comparison Evaluation and Reseach Design Report | 19 AMAA EVALUATION AND RESEARCH communities and from primary and AMAA new secondary schools in the case of treatment communities. If the data is not available or if it is incomplete, NORC will create the enrollment lists from scratch in every school. These data have the goal of tracking the cohorts of learners in standard 5, 6, 7, and 8 over time and for as long as possible, documenting the standard or form they are attending and if they remain enrolled in school, are promoted or repeat a year, and if they transfer to a new secondary school and continue their education, etc. The school administrative data is expected to only include very basic information about the learners (e.g., their sex). Therefore, we will supplement this information by collecting baseline data from a sample of students in standard 5 to 8 in May 2018. The baseline data will include detailed socio-demographic characteristics of each learner, as well as their disability status, socio-economic situation of his or her household, information on older siblings, and household contact information. In addition to answering question 1, this baseline sample will also serve as the basis for the data we will use to answer several other questions. We will update student enrollment status in both the school administrative data and the sample every year of the study – in 2019, 2020, and 2021 at endline (around March 2021). Question 2 (2a) What effect does embedding a new CDSS in a community have on the attitudes and expectations of girls, boys, parents, and communities in the areas served by the new AMAA constructed new schools regarding: ○ Primary school retention and completion and transition to secondary schools; ○ Interest in continuing girls’ and boys’ education and/or vocational training; ○ Marriage and pregnancy; and ○ Future work/employment/career? (2b) What are the attitudes and expectations of (1) girls and boys who have dropped out of school; and (2) girls and boys who are in the new CDSSs regarding: ○ Perceived value of attending school and education as experienced thus-far; ○ Future education; ○ Marriage and pregnancy; and ○ Future work/employment/career? Evaluation and Reseach Design Report | 20 AMAA EVALUATION AND RESEARCH The baseline sample of learners in standards 5 to 8 created to respond to question 1 will also answer evaluation question 2 about learners’ attitudes and expectations toward education, marriage, pregnancy, and employment/work. To fully address question 2a, we will also interview all of the parents/caregivers of learners in the sample in Balaka at baseline about their own attitudes and expectations towards they daughters’ and sons’ education, marriage, pregnancy and employment/work. Twelve leaders from 6 communities (3 treatment and 3 control) will be interviewed as well. At endline in March 2021, we will again interview parents and community leaders on these same questions. Answering question 2b requires examining the attitudes and expectations of the sampled learners over time. Therefore, we will include these questions in all interviews at endline and with each student who drops out. Question 3 What are the reasons for girls and boys to: (1) drop out of school; (2) repeat; and (3) be held back? We will update the student enrollment status of our baseline sample each year of the study – in 2019, 2020, and 2021 at endline (around March 2021). If a learner drops out of school, we will conduct a household interview to establish the cause for dropping out and the surrounding circumstances, current situation of the learner, and future plans. We will go back at endline to interview all learners who have dropped out and update their information. If a learner repeats the standard or is purposely held back, we will request a short interview with the teacher involved in the decision. These interviews will be done on a yearly basis as we learn about the learner enrollment status changes. The learners that repeat or are held back will continue to be tracked over time until endline or they drop out, whichever comes first. Question 4 How are the CDSSs helping girls to overcome identified barriers such as: education costs, early pregnancy, early marriage, lack of WASH facilities, distance to secondary school, and lack of space in secondary schools? Are there additional barriers to accessing schools faced by girls? In treatment communties we will organize FGDs with girls and parents to discuss the advantages of having access to the new secondary school and the challenges of staying (keeping learners) in schools. These discussions will probably take place in March 2020 and we will take into account the information collected for Question 3 in our dicussions.17 Evaluation and Reseach Design Report | 21 AMAA EVALUATION AND RESEARCH Question 5 (5a) What effect do the new CDSSs have on the perceptions of learners regarding physical and socio-emotional aspects of safety? (5b) What are the perceptions of girls and boys dropping out of school regarding physical and socio-emotional aspects of safety? To compare differences in safety among the learners that attend secondary school close to their homes and those that travel further or need to board, we will collect data at endline about safety perceptions (and experiences if possible). The relevant group of learners will be those in our original baseline sample that are still attending (secondary) school. They will be in forms 1 to 3 at the time of the endline. While secondary students can be found and interviewed at school in treatment communities, this will not be possible in comparison communities. The two options include finding the learner at home – which could be difficult – or finding the learner in the school that he or she is attending. Both cases present some practical challenges, which most likely could be solved with enough resources devoted to finding the learners. Learning about the safety situation of dropouts requires collecting this type of information during the interviews that will be triggered each time one of the learners in baseline sample is found to have dropped out of school. Question 6 What experience do new CDSSs have on attracting and retaining teachers? In March 2019, once the AMAA new schools are finished and running, we will use administrative school data and teachers’ and head teachers’ interviews to construct the baseline for a longitudinal panel of all new AMAA secondary school teachers. We will start by using the administrative school data to create a full list of teachers and document any vacancies. Our interviews with teacher interviews will allow us to gather basic demographic and education characteristics that can be added to this dataset. In each of the following years (2020 and 2021), we will data on turnover and replacements, seniority, working conditions, and job satisfaction, etc. Teacher interviews will also include questions to document the main positive and negative aspects of the teachers’ experience in the new schools. When new teachers are hired, we will interview them to collect their basic demographic and education information. Evaluation and Reseach Design Report | 22 AMAA EVALUATION AND RESEARCH Question 7 Are girls, their parents/guardians, and community members satisfied with the construction design and facilities of the new schools? The construction design attributes should be further specified in collaboration with USAID. At endline in 2021, we will interview a random sample of AMAA secondary school learners, parents, and teachers. The goal of the interviews is to collect information about their satisfaction with the school building structure. We will include questions about the different areas in the schools, including classrooms, latrines, outdoor space, water sources, etc. While teachers and learners will be very familiar with the building, it is possible that parents will not be. Therefore, we will also ask them how familiar they are with the school structure and infrastructure functioning. To have a point of reference, we can ask parents how the new buildings compare with the primary and secondary school buildings with which they are familiar. We do not recommend doing the same with learners because perceptions at different ages when young may be quite different. To the extent possible, we will try to interview learners and parents that were part of our baseline sample since we will have rich background information about them. SAMPLING STRATEGY AND SELECTION OF RESPONDENTS At baseline, we will create a sample of learners attending standard 5, 6, 7, and 8 in each of the treatment and comparison communities. We will randomly select 20 of the enrolled learners from each standard. The sample will include10 girls and 10 boys whenever possible. As with any quantitative analysis based on survey data, the required sample size is determined by a mathematical calculation that depends on a number of factors. These include features of the study design, properties of the data and outcome variables, and the desired precision of the analysis. For this exercise, we make the following assumptions: ■ α is the significance level of the test, or probability of Type I error. We use the standard value of 0.05. ■ β is the power of test, where (1-β) is the probability of Type II error. We use the standard value of 0.8. Evaluation and Reseach Design Report | 23 AMAA EVALUATION AND RESEARCH ■ ρ is the intracluster correlation coefficient, or ICC. The ICC in the present case is a measure of how much variability lies between schools and how much lies within schools. We assume an ICC of 0.1. ■ r 2 is the proportion of the variation in the outcome due to the covariates anticipated in the regression analysis. In our case, these covariates will include a range of household and individual characteristics. For this parameter, we assume an approximate value of 0.3 ■ Attrition is assumed to be 20% at endline. ■ MDES is the minimum detectable effect size. The MDES is the smallest impact of the activity on the outcome variable that the evaluation will be able to detect. The selected effect is a change in transition to secondary school from 30% to 50% when we do the analysis for each standard separately, and from 30% to 47% when we analyze standard 6 to 8 altogether. In addition, we will be able to detect changes from 50% to 70% in completion of primary school when analyzing standards separately, and from 50% to 65% when analyzing all standards together. Based on these parameters, the required sample size is 22 schools (includes treatment and comparison) with 80 learners in each school, for a total sample of 1,760 learners. In addition, in Balaka we will create a sample of the learners’ parents/caregivers to interview about their attitudes and expectations if the decision about secondary school location is not public in May 2018. We estimate that around 25-35 parents will actually respond our interview. We will also interview 12 community leaders to learn about their attitudes and expectations. 7 The teacher and head teacher information will be censual and therefore will include all AMAA new secondary school teachers and head teachers. We will collect qualitative data during focus group discussions. In March 2020, we will randomly select four treatment communities and conduct 2 FGDs with girl learners and 2 FGDs with parents in each of them. Girl learners and parents will be selected randomly from all those eligible. 8. Finally, in March 2021 we will interview a random sample of learners in form 1 to form 3 attending the new AMAA secondary schools and their parents. The sample will include 30 learners per school, 10 in each form (half of them girls whenever possible), for a total of 300 learners. We will also invite 30 parents to respond to the school building 7 According to a review conducted by Guest et al (2017) 12 interviews are enough to discover between 80 and 90 percent of the themes. 8 Previous research shows that conducting six FGD (by subpopulation of interest) allows discovering between 80 and 90 percent of the themes (Guest et al, 2017). Evaluation and Reseach Design Report | 24 AMAA EVALUATION AND RESEARCH satisfaction survey. These samples assume a confidence level of 95% and a margin of error of 5%. Table 4 presents a summary of data collection activities for this research and evaluation study. Table 4: Data Collection Activities Data collection activity Dates for data collection* Census or Sample size Learners – Administrative school data May 2018 Mar 2019 Mar 2020 Mar 2021 Census, of std. 5 to std. 8 learners at baseline; tracked annually Learners survey May 2018 Mar 2021 80 learners in std. 5 to std. 8 per school 40 learners in std. 5 to std. 8 per school (Balaka); Learners in sample that remain in school (approx. 800 learners**) Dropout (repetition, etc.) interviews Mar 2019 Mar 2020 Mar 2021 Approx. 500 learners** Approx. 250 learners** Approx. 1000 learners** Parents expectations and attitudes survey May 2018 Mar 2021 25-35 parents of pupils in std. 5 to std. 8 x 12 schools (Balaka) KIIs with community leaders May 2018 Mar 2021 12 KII with leaders (Balaka) Teacher roster Mar 2019 Mar 2020 Mar 2021 Census secondary teachers/head teacher in 11 schools; tracked annually Teachers interviews Mar 2019 Mar 2020 Mar 2021 Census secondary teachers/head teacher in 11 schools; updated annually FGD girls - Barriers Mar 2020 8 FGD with girl learners in treatment communities FGD parents - Barriers Mar 2020 8 FGD with parents in treatment communities Learners survey – school building satisfaction Mar 2021 30 learners in form 1 to 3 x 11 schools Parents survey – school building satisfaction Mar 2021 30 parents of form 1 to 3 learners x 11 schools * Dates are approximate. **Exact number depends on the number of students that drop out. Table 5 below presents a summary of our approach to addressing each evaluation and research question. Evaluation and Reseach Design Report | 25 AMAA EVALUATION AND RESEARCH Table 5: Summary of Approach to Addressing each Evaluation and Research Question Question Data Collection Activity Data Type/s Indicators Location and Group Date Analyses Q1. What effect does embedding a new CDSS in a community have on (1) primary school completion and transition; and (2) secondary school enrollment, attendance, promotion and retention of girls and boys in the community? - Learners – Administrative School Data - Learners Survey - Longitudinal data learners; tracking 2018 cohort Std5- Std8 CENSUS - Longitudinal data learners; tracking 80 learners per school - Enrollment, promotion, repetition, completion, and transition to secondary school - Socio￾demographic and economic background, older sibling information, HHLD contact information - Machinga and Balaka - Treatment and Comparison - May 2018 - Mar 2019 - Mar 2020 - Mar 2021 - Regression analysis with controls and/or propensity score matching - Survival analysis - Panel data analysis Q2a. What effect does embedding a new CDSS in a community have on the attitudes and expectations of girls, boys, parents and communities in the areas served by the new AMAA constructed new schools regarding: - Primary school retention and completion and transition to secondary schools; - Interest in continuing girls’ and boys’ education and/or vocational training; - Marriage and pregnancy; and - Future work/employment/career? Q2b. What are the attitudes and expectations of (1) girls and boys who have dropped out of school; and (2) girls and boys who are in the new CDSSs regarding: - Perceived value of attending school and education as experienced thus-far; - Future education; - Marriage and pregnancy; and - Future work/employment/career - Learners Survey - Parents expectations and attitudes survey - Community leaders - Learners survey and dropouts interviews - Repeated cross sections of learners, and parents - KII - Longitudinal data learners; tracking 80 learners per school - Gender & Education attitudes and expectations - Gender & Education attitudes and expectations - Balaka - Treatment and Comparison - Machinga and Balaka - Treatment and Comparison - May 2018 - Mar 2021 - May 2018 - After drop out event - Mar 2021 - Difference in Difference - Descriptive Analysis - Descriptive analysis Evaluation and Reseach Design Report | 26 AMAA EVALUATION AND RESEARCH Question Data Collection Activity Data Type/s Indicators Location and Group Date Analyses Q3. What are the reasons for girls and boys to: (1) drop out of school; (2) repeat; and (3) be held back? - Dropout (repetition, etc.) interview - Teacher interview - Longitudinal data learners; tracking 80 learners per school - Reasons for dropout, current situation - Reasons for repetition, being held back - Machinga and Balaka - Treatment and Comparison - Mar 2019 - Mar 2020 - Mar 2021 - Regression analysis with controls and/or matching Panel Data Analysis - Descriptive Analysis Q4. How are the CDSSs helping girls to overcome identified barriers, such as: education costs, early pregnancy, early marriage, lack of WASH facilities, distance to secondary school, and lack of space in secondary schools? Are there additional barriers to accessing schools faced by girls? - Household interview for dropout girl learners - Parents and girls FGDs - Longitudinal data learners; tracking - 80 learners per school - Qualitative Data - Reasons for dropout. - Current situation - Challenges and advantages faced by parents and girls - Machinga and Balaka - Treatment - Mar 2019 - Mar 2020 - Mar 2021 - Mar 2020 - Panel Data Descriptive Analysis - Phenomenological analysis of Qualitative Data Q5a. What effect do the new CDSSs have on the perceptions of learners regarding physical and socio-emotional aspects of safety? 5b. What are the perceptions of girls and boys dropping out of school regarding physical and socio-emotional aspects of safety? - Interviews with learners - Dropout interview - Learners in the baseline sample still attending schools (Form 1 to 3) - Learners in the baseline sample that dropped out of school - Learner perception of safety around school/ going to school - Perceptions of safety for those out of school - Machinga and Balaka treatment and comparison - Machinga and Balaka - Treatment and comparison - Mar 2021 - After dropout event and Mar 2021 - Descriptive Analysis - Descriptive Analysis Q6. What experience do new CDSSs have on attracting and retaining teachers? - Teacher Roster - Teacher Interviews - Longitudinal Data - Secondary School Teachers-ALL - Turnover, job satisfaction, vacancies, etc. - Machinga and Balaka - Treatment - Mar 2019 - Mar 2020 - Mar 2021 Descriptive analysis Evaluation and Reseach Design Report | 27 AMAA EVALUATION AND RESEARCH Question Data Collection Activity Data Type/s Indicators Location and Group Date Analyses Q7. Are girls, their parents/guardians, and community members satisfied with the building facilities of the new schools? Learners and parents surveys - School building satisfaction SUB-SAMPLE (30 learners per secondary school, 10 in each Form) and their parents (25 approx.) Satisfaction with the school building - Machinga and Balaka - Treatment Mar 2021 Descriptive Analysis Evaluation and Reseach Design Report | 28 AMAA EVALUATION AND RESEARCH FIELD WORK SELECTION OF FIRM AND RECRUITMENT OF ENUMERATORS NORC conducted a limited competition to secure proposals for the data collection component of the USAID AMAA School Construction Malawi program. In accordance with FAR 6.302-3(a)(ii), NORC limited its competition to firms in Malawi that we reasonably believe meet the high quality standards required for the data collection for this impact evaluation, which is not standard market research, but high-quality social science research. NORC understands that when working in the scientific field, limited competition may be employed to preserve the essential integrity of the work which extends to contracts for supplies and services as applied at FAR 6.302-3(b)(iii). Criteria we considered as we identified firms include: ■ Experience managing and conducting field work in primary and secondary schools, including obtaining permission to conduct surveys in schools from government and local officials ■ Experience conducting surveys of adolescent girls ■ Experience conducting surveys via tablets (computer assisted personal interviewing, or CAPI), including having the capability to program surveys ■ Experience conducting field work in Balaka and Machinga districts of Malawi, considering the language requirements needed ■ References NORC consulted with USAID, local research partners in Malawi, Save the Children, and other researchers that had conducted fieldwork in Malawi to identify firms with the necessary experience and who could conduct the data collection for this evaluation. Through these sources, NORC compiled the following list of candidate firms: ■ Centre for Development Management (CDM) ■ Invest in Knowledge (IKI) ■ Wadonda Consult (WACO) Limited In April 2017, Dr. Alicia Menendez, NORC’s Principal Investigator, met with representatives of these firms in person during a scoping trip for the project in order to further probe their staffing and technical capabilities. After determining that they would qualify for this assignment, NORC emailed a representative of each firm to gather capability statements. Following a review of the capability statements, NORC provided Evaluation and Reseach Design Report | 29 AMAA EVALUATION AND RESEARCH each firm with the Terms of Reference for the research and requested proposals and budgets from each. NORC then reviewed the technical approach and budgets of the three firms to analyze how the firms were planning to execute the tasks outlined in the TOR—especially in terms of data collection logistics and practical solutions for overcoming challenges in the field, such as how many interviews an enumerator could complete in a given school day, methods for efficiently collecting the administrative and caregiver data, and ability to track students over time. In addition, NORC conducted a cost analysis of each firm’s proposed budget. As mentioned above, NORC emailed each firm questions to clarify line items in the budget and assumptions that were made. Based on the comparative assessment of experience, qualifications, and technical approach to the terms of reference, we determined that IKI was most qualified to serve as our local subcontractor for the evaluation of AMAA School Construction. IKI meets all the requirements as stated in the SOW. They have worked extensively in Malawi and have strong quantitative and qualitative skills. They were able to demonstrate through multiple work examples their capabilities to conduct large-scale surveys in school settings, have field staff at their disposal who are familiar with the local context and language, and have experience interviewing adolescents. IKI was a pioneer in CAPI data collection in Malawi and has some of the most extensive experience with tablets in the country. In addition, we determined that IKI offered the most comprehensive budget and timeline to complete the necessary activities. Finally, IKI’s budget was more cost-competitive than WACO’s for the specified scope of work. PROGRAMMING INTO TABLETS AND TRAINING Programming. NORC has programmed all instruments to be used during the baseline in NField. NORC and IKI will adjust the programming of the instruments as necessary after testing all the tools and conducting the pilot work. The questionnaires used to interview learners, caregivers, and community leaders during baseline can be found in Annex II at the end of this report. Training. The baseline training starts on May 7 and will last a total of five days. The first two days of training will consist of classroom instruction, role playing exercises, exercises to familiarize enumerators with tablets, inter-rater reliability exercises, and several rounds of group and paired practice with each instrument. The first day of training will also include sessions on ethics, confidentiality, and important aspects of working with children. Evaluation and Reseach Design Report | 30 AMAA EVALUATION AND RESEARCH After the classroom portion of training is complete, all trainees will participate in two pilot exercises in out-of-sample local schools to gain practical experience and practice in conducting the interviews with actual learners and parents in Machinga communities and schools. These schools/communities closely resemble the communities and schools included in the study. In addition, to gain real experience and confidence, the main focus of this pilot exercise will be to evaluate each enumerator and allow the trainers to observe how well each of them is able to conduct interviews in an environment that closely mirrors the situations they will encounter during field work. Groups of trainees will be assigned a school for each pilot exercise, where each one will be required to conduct a minimum of three student interviews and one parent interview. After the first day of in-school piloting, the training that follows will consist of a feedback session with all trainees to further improve practices and correct errors that were observed during the pilot exercise. A second pilot will take place the following day. During the last day, the foci will be on logistics and further instructions for fieldwork processes. Supervisors will also be selected on the final day and will be taken through an additional training session to ensure they all understand the duties and responsibilities expected of them as supervisors. These duties and responsibilities will be discussed throughout training so that both enumerators and supervisors share the same expectations for every team member’s role and responsibilities. DATA QUALITY CONTROL All enumerators hired by IKI for this work have ample experience conducting fieldwork, school based surveys, and parental interviews. For the training, NORC and IKI will create a data quality control checklist that will be used by trainers and supervisors to assess the level of skills of enumerators. Twenty percent more trainees than necessary were invited to training so that only the most qualified will be selected for the actual field period. This selection will be based on assessments during the classroom training and the pilot tests. During the field work, supervisors will travel to each school with their team of enumerators. Supervisors will oversee the entire operation in the school and interviews throughout the day, ensuring survey instruments are administered correctly. Supervisors will also contribute to data collection activities on an as-needed basis to ensure all interviews are conducted before the end of the school day, and that caregivers and community leaders are reached. In addition, supervisors will also monitor the data upload every day and immediately communicate any problems to IKI’s data manager, who will further communicate with NORC as needed. Evaluation and Reseach Design Report | 31 AMAA EVALUATION AND RESEARCH Supervisors, the IKI data manager, and IKI project director will directly oversee interviews and will conduct spot-checks of interviews. Supervisors will directly observe at least 10% of all completed interviews conducted by their team throughout the field period and will also conduct spot checks on a minimum of 15% of the interviews conducted by their team each day. OPERATIONAL PLAN FOR DATA COLLECTION Eight staff members from IKI were deployed to the field prior to launch of the survey to contact all districts and schools and to ensure accuracy of the sample list. The four main objectives of these visits are to: (1) gain advance entry into the schools and explain the purpose of our data collection; (2) verify the eligibility of the schools as requested by USAID/Malawi; (3) create the sample of learners and parents; and (4) drop off the parental consent forms that we need from each learner in order for them to participate in the study. For the actual field period commencing on May 15, 2018, a total of four teams, each comprised of four enumerators and one supervisor, will administer all instruments over a period of three weeks. Each team will be assigned schools to visit throughout the field period in a manner that is as geographically compact as possible in an effort to minimize travel between schools during the field period. Each team member (all four enumerators and one supervisor) will be assigned a specific tablet and each team will have one extra tablet in the event of a technical failure while the team is in the field. Each tablet will be loaded with Nfield and will be fully tested prior to sending it to field. Each team member will also have protected show cards which will be used during the administration of the surveys Each team will be supplied with a vehicle throughout the field period so they will not have to rely on public transportation to access the schools. Teams will travel together to each school and will remain together until all work is completed in that school. Following enumeration, quantitative data will be exported into STATA and processed through two rounds of cleaning, first by IKI and then by NORC, who will house the server which stores all of the data as it streams in from the tablets. Data will be transferred between IKI and NORC, and ultimately sent to USAID, using a Secure File Transfer Protocol (SFTP) system. Evaluation and Reseach Design Report | 32 AMAA EVALUATION AND RESEARCH ETHICS CLEARANCE All human subjects related activities in this IE are subject to IRB ethics clearance and proper procedures are in place to obtain approval. All baseline plans were submitted to NORC at the University of Chicago IRB on April 3 for expedited review and approval was obtained on April 11. Local IRB approval to conduct research in Malawi is required. All materials for the Malawi National Committee on Research in the Social Sciences and Humanities (NCRSH) were submitted electronically and hard copies were delivered to the IRB by IKI on April 23. An informal approval was issue by the IRB on May 8 in order to go ahead with the pre-testing and piloting work. On May 15, the formal ethics clearance for the baseline fieldwork was received. The written authorization for our records was received on May 28. In addition, IKI has agreed to work with NORC throughout the process of training and field work to ensure they are meeting NORC’s child’s protection policies. Field teams will also be trained on standard ethical procedures for interviewing and personal research in general. This training includes detailed explanations of confidentiality and anonymity and all enumerators, supervisors, quality control officers, IKI field support staff, and data management staff will sign a pledge of confidentiality. This ethics training will also include proper handling of electronic data before, during, and after the interview, secure handling of all supplemental interviewing materials, and standards for proper and ethical treatment of respondents before, during, and after the interview. Finally, education district officials and school administrators were contacted to introduce and explain the work to be done, to explain that the work was ethically compliant with the Malawian and US human subjects’ principles, and to request collaboration with the data collection process. LIMITATIONS OF THE PROPOSED APPROACH The evaluation team has identified some limitations inherent to the design of this evaluation. We list some of the more relevant limitations below: Representativeness of the Sample. The sample is representative of the areas where AMAA school construction will take place, in the areas of Machinga and Balaka districts. Therefore, results are not directly generalizable at the national level, other geographical areas, or to other school levels. Despite this, we hope the study will be informative and Evaluation and Reseach Design Report | 33 AMAA EVALUATION AND RESEARCH contribute to the evidence regarding secondary school construction, secondary school attendance, and dropouts in developing countries. Longitudinal Attrition and Absenteeism. The evaluation employs a longitudinal design where we will interview the same learners and teachers at baseline, follow-ups, and endline. Some learners will be still attending school, while some other will drop out. Absenteeism rates are, however, high and could present a problem. For dropouts, we plan to visit them at home. All measures available to keep track of learners are in place to reduce attrition. School Sample Size. It should be noted that the nature of the activity necessarily limits our sample size. The number of treatment and comparison communities in Balaka is small (13 in total) and unless the changes are large, estimates will tend to be imprecise. If this is the case, it will be noted and findings will not be taken as conclusive but rather suggestive of a particular trend. In general, however, we expect the sample to be large enough to estimate the expected effects. We expect that the number of schools to be large enough to identify changes in secondary school enrolment. REPORTING NORC will produce a baseline report in 2018 and a final evaluation and research report in the last year of the project in 2021, after all data has been collected and analyzed. If findings between baseline and endline are deemed as important to be communicated to the USAID/Malawi mission, USAID Gen/Dev program, or other stakeholders, they will be communicated in a timely manner by producing short reports, briefs, or presentations. NORC plans to hold presentations in Washington, DC, and in Malawi to share evaluation results. DISSEMINATION PLAN The primary audiences for dissemination are the Government of Malawi, USAID, and Save the Children-AMMA. USAID comprises different Operating Units that include USAID/Malawi, the Africa Bureau, E3/GenDev, E3/Education, and E3/Energy and Infrastructure. Evaluation and Reseach Design Report | 34 AMAA EVALUATION AND RESEARCH The results can generate input and evidence that can be used by the Government of Malawi, USAID, and other aid agencies and funders to make informed decisions and guide public policy. More broadly, the study may contribute to the literature on the construction of schools and its effects on school enrollment and education attainment. In particular, this study may generate useful the evidence given its focus on the construction of rural secondary education schools that require the payment of schools fees, in a poor country. This is a research area far less explored than primary school construction where evidence is scarce. Dissemination will be tailored to the different audience and purposes. The main dissemination materials that will be developed are the baseline and the evaluation and research reports. The evaluation and research reports will include a clear and comprehensive description of the evaluation goals, methodology, findings, and recommendations. In addition, NORC will also produce a policy brief to summarize findings in a more accessible way. Infographics will be used to show the most relevant results and key lessons learned will be highlighted. NORC will hold dissemination workshops with stakeholders to share the main results of the research and evaluation so participants can provide feedback and ask outstanding questions. One workshop will present the final baseline report; a second workshop will present the evaluation and research findings. The final locations where these workshops will take place will be decided jointly with USAID, but we envision at least one in Malawi and another in Washington, DC. Finally, NORC will hold a webinar to discuss evaluation and research findings, conclusions, and recommendations. Datasets produced for the evaluation and research will be made publically available after being anonymized. YEAR 1 WORKPLAN ACTIVITY PERIOD Preparation for baseline fieldwork, instruments, IRB, pilots March 25 – May 1, 2018 Meetings with Save the Children and USAID/Malawi in Lilongwe May 1 – 3, 2018 Training of enumerators, school pre-visits, and pilots May 5 – 12, 2018 Baseline data collection May 15 – June 8, 2018 Data quality review, data cleaning and preparation June 11 –21, 2018 Baseline data analysis and report writing June 21 – August 8, 2018 Report Review by USAID and Save the Children August 8 –17, 2018 Final Baseline Report September 2018 Evaluation and Reseach Design Report | 35 AMAA EVALUATION AND RESEARCH Update July 5, 2018: Given that data is being collected during the first week in July in the last site identified for school construction, the data analysis and report writing will be postponed. We estimate that the draft baseline report will be finalized by approximately September 8 and the final report will be complete approximately one month later. REFERENCES Burton, Patrick. "Suffering at School: Results of the Malawi Gender-Based Violence in Schools Survey." Institute for International Security Studies (2005) Guest, Greg, Emily Namey and Kevin McKenna (2017). How Many Focus Groups Are Enough? Building an Evidence Base for Nonprobability Sample Sizes. Field Methods 2017, Vol. 29(1) 3-22. Directorate of Education Planning (2017). Malawi Education Statistics Final Report 2015/16, Education Management Information System (EMIS), Ministry of Education, Science and Technology, Malawi. Evaluation and Reseach Design Report | 36 AMAA EVALUATION AND RESEARCH ANNEXES Evaluation and Reseach Design Report | 37 AMAA EVALUATION AND RESEARCH ANNEX I: EVALUATION AND RESEARCH STATEMENT OF WORK STATEMENT OF WORK (DRAFT 3) FOR The Apatseni Mwayi Atsikana Aphunzire (AMAA) Evaluation, Malawi February 22, 2018, updated September 14, 2018 I. PURPOSE OF THE EVALUATION: OBJECTIVES, AUDIENCES AND USES Apatseni Mwayi Atsikana Aphunzire (AMAA) is supported by USAID’s Let Girls Learn Challenge Funds awarded by the Office of Gender Equality and Women’s Empowerment (E3/GenDev) to Save the Children, Inc. in December 2016. AMAA is implemented by Save the Children Inc. in collaboration with several partners through Cooperative Agreement AID-OAA-A-17-00001. AMAA is currently managed by the E3/Office of Gender Equality and Women’s Empowerment. Plans are underway to transfer AMAA management to USAID/Malawi in FY2018. The evaluation of AMAA’s school construction effects is funded by E3/GenDev. The AMAA evaluation operations will be carried out by NORC, through the Reading and Access Evaluations contract (Time and Material): GS-10F-0033M/AID-OAA-M-13- 00010. The Reading and Access Evaluations contract is managed by the E3/Office of Education (E3/ED). GenDev’s Advisor for M&E will serve as the Activity Manager of the AMAA evaluation operations. The audiences for this evaluation comprise USAID Operating Units (OU), notably, USAID/Malawi, the Africa Bureau, E3/GenDev, the E3/Education Office; and the E3/ Energy and Infrastructure Office. Other important audiences are the Government of Malawi, primarily, Malawi’s Ministry of Education, Science and Technology (MoEST), and donors committed to building and/or supporting schools, such as, Japan International Cooperation Agency (JICA), Department for International Development (DFID), the World Bank and the World Food Program. Save the Children and its implementing partners are key audiences too. AMAA’s Whole-of-Girl programming package is formed of multiple and district specific interventions in five districts. However, this AMAA evaluation focuses only on the set of CDSS construction interventions implemented in two districts, Machinga and Balaka. Evaluation and Reseach Design Report | 38 AMAA EVALUATION AND RESEARCH The objectives of the AMAA evaluation are to: • Review existing evidence on the relationship between new school construction and enrollment as relevant to Malawi, and interpret the findings of the AMAA evaluation in the light of this evidence; • Building on the overall AMAA logical framework and theory of change, refine the theory of change for AMAA new Community Day Secondary School (CDSS) construction in collaboration with implementing partners and other stakeholders including USAID/Malawi; • Evaluate and report on the effects of the AMAA new CDSS construction relating to completion of primary education by girls and boys, transition of girls and boys from upper primary to the new AMAA CDSSs, and attendance and retention of girls and boys in the new AMAA CDSSs. • Examine the effects of AMAA new CDSS construction on perceptions of safety by girls and boys. The evaluation will also assess whether girls, boys, teachers, parents and community members are satisfied with the school construction design and facilities. • Assess the USAID and AMAA approach to constructing new CDSSs. The findings, conclusions and recommendations of this evaluation can be applied to future school construction in Malawi and other places. The data generated by this evaluation will contribute toward building the growing body of evidence on the relationship between school construction and enrollment, informing new school construction programming for adolescent girls. 2. SUMMARY INFORMATION Strategy, Project or Activity Name Apatseni Mwayi Atsikana Aphunzire (AMAA) in Malawi Implementer Save the Children, Inc. Cooperative Agreement/Contract # Cooperative Agreement NoAID-OAA-A-17-00003 Total Estimated Ceiling of the Evaluated Project/Activity (TEC) $ 10.5 million Life of Strategy, Project, or Activity December 15, 2016 to September 30, 2020 Active Geographic Regions AMAA will be implemented in five districts in Malawi: Balaka, Machinga, Phalombe, Chikwawa and Mzimba districts. Eleven new schools will be built in the Machinga and Balaka districts. Evaluation and Reseach Design Report | 39 AMAA EVALUATION AND RESEARCH Development Objective(s) (DOs) E3/Office of Gender Equality and Women’s Empowerment PAD DO2: Gender-informed approaches scaled in USAID programs and processes. USAID/Malawi, CDCS DO1: Social Development Improved USAID Office E3/Office of Gender Equality and Women’s Empowerment in partnership with USAID/Malawi 3. BACKGROUND 3A. Description of the Problem, Development Hypothesis, and Theory of Change 3A.1. The Problem: Since Malawi introduced free primary education in 1994, the numbers of boys and girls enrolling in the first four years of primary school has grown exponentially. Commendable progress has been made towards gender parity at the lower primary level, however, from Standard 5, the number of girls dropping out far surpasses that of boys. For example, while the survival rates to Standard 5 have remained similar for boys and girls (58% and 59% respectively in 2012; 66% and 62% in 2013), the survival rates to Standard 8 have consistently been higher for boys than for girls (41% and 35% respectively in 2012; 35% and 27% in 2013). Girls also tend to fare worse than boys in examinations. As an example, in 2014 73% of boys who sat the Primary School Leaving Certificate Examination (PSLCE) passed compared with 61% for girls9. Only a small percentage of girls and boys who pass are able to go to CDSS due to lack of schools and trained teachers. Although the government tries to maintain an equal selection of girls and boys for CDSS, fewer girls than boys actually complete CDSS due to long distances, lack of school fees, family responsibilities, marriage and pregnancy. In 2015, there were 46,829 boys in Form 4, the last year of CDSS, versus just 38,654 girls10. Many girls are leaving the education system without having attained basic literacy, numeracy or essential life skills, making them more dependent on marriage and a husband’s support. Societal norms that lead to gender inequalities make girls responsible for the majority of household chores, resulting in them arriving late to school, or reducing time for homework, causing them to fall behind. This gender bias and society’s perceptions of what a girl’s role can and should be and what she is capable of, also influences whether a guardian will allocate scarce resources to support a girl’s education. 9 Education Management Information System (EMIS) Malawi, 2014 10 EMIS, 2015 Evaluation and Reseach Design Report | 40 AMAA EVALUATION AND RESEARCH Girls are also more likely to be victims of violence at school, or on the way to and from school. Long distances can prevent both boys and girls from attending school. However, this affects girls differently as the farther a girl has to travel, the greater the concerns for her safety. Parents may be reluctant to allow their daughters to travel long distances, or the girl herself might think school is not worth the additional risk. At school, sexual assault and rape by teachers and classmates is commonplace11. In the Government’s Violence Against Children Survey, 2013, 22.8% of girls (age 13-17) said they had experienced sexual violence in the previous 12 months. Yet victims of physical and sexual violence rarely seek help. Only 14% of girls aged 13-17 said they had sought help after a sexual or physical attack12. Sexual violence and abuse, the lack of sexual and reproductive health knowledge and youth friendly services, and girls’ inability to negotiate safe sex due to restrictive gender norms can mean many girls are forced to drop out of school due to pregnancy. Nearly 29% of girls under 19 have started childbearing, an increase of 4% on the previous year13, and a trend that heavily impacts access to upper primary and CDSS, because, as soon as a girl is pregnant she must drop out of school for at least a year14. Additionally, a portion of these girls are married or soon become married; 46% of girls are married before they reach the age of 1915. Girls who do return to school are often quite far behind, and older than peers in their class making it likely they will drop out again. In fact, 75% of all new HIV infections are among adolescent girls and young women16 Due to the prevailing cultural and societal norms17, young girls are the least able to negotiate safe sex, or resist unwanted sexual advances from men, including their teachers. Unprotected sex also puts adolescent girls at risk of contracting HIV. 3A.2. AMAA’s Theory of Change: To address these multiple interlinked barriers that limit a girl’s ability to develop, Save the Children Federation, Inc. (SC), together with its partners Grassroot Soccer (GRS), and Forum for African Women Educationalists – Malawi Chapter (FAWEMA), are implementing Apatseni Mwayi Atsikana Aphunzire (AMAA), which translates in Chichewa to Give Girls a Chance to Learn. The project adopts a “whole of girl approach” in it’s design and interventions. AMAA’s Whole of Girl conceptual approach (Figure 1) recognizes that girls are actors in their own personal development, but contends girls’ development cannot be considered outside a social web of relationships, influential actors, norms and systems: The AMAA 11 Malawi Gender Assessment: Gender Based Violence and Girls’ education Outcomes, 2016, US State Department 12 Violence Against Children and Youth survey, 2013 13 Malawi Demographic and Health Survey, 2015-2016 14 Violence Against Children (VAC) Survey, Government of Malawi, 2014 15 UNICEF, State of the World’s Children, 2016 16 DHS 2015 17 Malawi ranks 145 out of 149 countries on the UN’s Gender Inequality Index, according to the HDI 2016 Evaluation and Reseach Design Report | 41 AMAA EVALUATION AND RESEARCH theory of change aims at addressing the five main common barriers to girls’ education; school fees, early pregnancy, early marriage, distance to CDSS and lack of space in CDSS. AMAA intends to address these common barriers by delivering three outcomes for improving the girl’s agency and self-efficacy, improving access to education and performance for girls and improving the community support for girls’ education and safety. This will be at the girl as an individual, the family, the community, school and the system level to ensure context specific interventions that fully addresses the different structural problems facing girl’s education in Malawi. Figure 1: AMAA Theory of Change Diagram: The Whole of Girl Approach 3B. Summary Strategy/Project/Activity/Intervention to be evaluated 3B.1. The Let Girls Learn Whole-of-Girl Approach: In March 2015, the U.S. Government launched Let Girls Learn (LGL), a global initiative which builds upon existing efforts and invests in new programs to expand education for adolescent girls. LGL seeks to ensure that adolescent girls around the world are able to access quality education enabling them to reach their full potential. LGL activities adopt a Whole-of-Girl approach by recognizing that multiple solutions are needed to address the complex challenges faced by girls. Barriers to girls’ education pose formidable challenges. Evaluation and Reseach Design Report | 42 AMAA EVALUATION AND RESEARCH The Whole-of-Girl approach is based on the recognition that discriminatory gender norms motivate some parents to invest in educating their sons rather than their daughters. Girls attending school risk harassment and violence on their way to school and in school. Girls drop out of school owing to pregnancy and early marriage. In some contexts, parents and teachers tend to support boys rather than girls to make headway in their studies and careers. Whole-of-Girl interventions foster an enabling environment for adolescent girls’ education, safety and well-being, wherein girls increase self-efficacy and are valued by their families, communities and other institutions. 3B.2. AMAA: AMAA is delivering a range of district specific activities in Malawi to mitigate the barriers to girls’ enrollment and retention in school. The project works in five priority districts targeting over 60,000 girls aged 10-19 in both upper primary and CDSSs . Led by Save the Children, AMAA works with local and international NGOs, as well as Peace Corps, the World Food Program and the Ministry of Education, Science and Technology (MoEST) in Malawi to mitigate the barriers preventing girls’ enrollment and retention in school. AMAA developed a package that includes targeted yet competitive bursaries, construction of CDSSs, extracurricular and in school activities to empower girls and strategies to transform gender norms and reduce school-related gender based violence (SRGBV). The AMAA project focuses on the reduction of early and child marriage and teen pregnancy for the purpose of increased school access, performance and retention. For girls’ to achieve academic success, they must enter school, stay in school, be learning while in school, and be healthy and supported by their community at all times. AMAA implements a coordinated set of interventions with the following activities: ● increasing access to schools and safety for girls by constructing of one 50-bed girls’ dormitory and eleven new CDSSs; protecting girls from gender-based violence by improving reporting and referral systems; and involving fathers from the community in AMAA “Real Fathers” program on building healthy father￾daughter relationships; ● offering additional learning opportunities and support for all adolescent girls to continue their education by establishing school buddies for girls who are caring for babies; and training male and female teachers to inspire and support girls to remain in school; and ● ensuring that girls are receiving a practical, high quality education by improving the academic and teaching performance of teachers in English, science and math; and inviting girls to yearly science and math camps. Evaluation and Reseach Design Report | 43 AMAA EVALUATION AND RESEARCH 4. EVALUATION QUESTIONS The evaluation questions broadly fall into two categories. The Part 1 evaluation questions focus on the effects of AMAA new CDSS construction on girls, boys, parents, teachers and communities. It’s essential for the evaluation team to define key terms in consultation with USAID and Save the Children. These key terms include, school “retention”, “transition”, “completion, “drop-out”, “repetition”,and “held-back” etc. The evaluation team should also specify the physical and socio-emotional aspects of safety (Q.5) and the attributes of school design (Q.7) that will be assessed by beneficiaries. The Part 2 evaluation question focuses on assessing the AMAA approach to new school construction. The areas of inquiry for this question should be determined in consultation with USAID and Save the Children. 4A. PART 1: These evaluation questions18 will capture the effects of AMAA’s new CDSS construction interventions on beneficiary girls, teachers, parents, and communities. Question 1: What effect does embedding a new CDSS in a community have on primary school completion and transition; and CDSS enrollment, attendance, promotion and retention of girls and boys in the community? Question 2: (a) What effect does embedding a new CDSS in a community have on the attitudes and expectations of girls, boys, parents and communities in the areas served by the new AMAA constructed new schools regarding: ● Primary school retention and completion and transition to CDSSs? ● Interest in continuing girls’ and boys’ education and/or vocational training ● Marriage and pregnancy ● Future work/employment/career. (b) What are the attitudes and expectations of (1) girls and boys who have dropped out of school; and (2) girls and boys who are in the new CDSSs regarding: ● Perceived value of attending school and education as experienced thus- far ● Future education ● Marriage and pregnancy ● Future work/employment/career Question 3: What are the reasons for girls and boys to: (a) drop out of school; (b) 18 The AMAA evaluation questions are focused on the AMAA new secondary school construction package. This package supports AMAA IR 1.1 Improved secondary school enrollment. In 2018-19, AMAA is building 11 new secondary schools, five in the Machinga district and 6 in the Balaka district. The school buildings will be contructed by an engineering company. The community will be involved in building the latrines and the teachers’ quarters. Evaluation and Reseach Design Report | 44 AMAA EVALUATION AND RESEARCH repeat, and (c) be held back? Question 4: How are the CDSSs helping girls to overcome identified barriers, such as: education costs, early pregnancy, early marriage, lack of WASH facilities, distance to CDSS, and lack of space in CDSSs? Are there additional barriers to accessing schools faced by girls? Question 5: (a) What effect do the new CDSSs have on the perceptions of learners regarding physical and socio-emotional aspects of safety? (b) What are the perceptions of girls and boys dropping out of school regarding physical and socio-emotional aspects of safety? Question 6: What experience do new CDSSs have on attracting and retaining teachers? Question 7: Are girls, their parents/guardians, and community members satisfied with the building facilities of the new schools? 4B. PART 2: This evaluation question will assess the AMAA approach to constructing schools in Malawi Evaluation.Q.8: Assess the approach adopted by AMAA to construct schools in Malawi. How were challenges overcome? What are the strengths and limitations of the school construction experience? What are the promising practices and lessons for USAID and its partners?19 5. EVALUATION DESIGN AND METHODOLOGY The evaluation team is tasked with designing and implementing an evaluation of AMAA new CDSS construction in Malawi’s Machinga and Balaka districts. The evaluation designs for Part 1 and Part 2 of the evaluation should be developed in consultation with USAID and the AMAA team. 5A. AMAA Evaluation Part 1 The evaluation team will design Part 1 of this evaluation and also conduct Part 1 baseline data collection and reporting in 2018. To answer the Part 1 evaluation questions, the AMAA evaluation team will adopt a mixed methods approach tailored to answering each evaluation question. This will involve collecting qualitative and quantitative data through literature review, document analysis including existing administrative records, surveys, key informant interviews, and focus groups. Malawi’s Education Management Information System (EMIS) is an important source of administrative data on school enrollment and attendance. The evaluation design should 19 The Part 2, Evaluation Question 8, will be further elaborated when the Part 2 evaluation design works starts based on the schedule for completing school construction in the Machinga and Balaka districs. Evaluation and Reseach Design Report | 45 AMAA EVALUATION AND RESEARCH leverage EMIS and attendance information maintained by schools if and when appropriate and feasible. The literature review should draw upon existing research on secondary school attendance in Malawi, Africa and even globally as relevant to the problems and opportunities faced by the students in Machinga and Phalombe, Malawi. The evaluation team could apply a quasi-experimental design using treatment and comparison groups to answer some of the Part 1 evaluation questions. If a quasi￾experimental design is used, the treatment group(s) would be drawn from the communities living in areas adjacent to the new schools. The treatment groups should comprise girls and boys likely to transfer to the new CDSSs and those who have transitioned successfully to the new CDSSs. The control or comparison groups would be drawn from areas that are similar to the treatment areas on key attributes, but currently do not have CDSSs close to primary schools. The analysis and presentation of data gathered in response to the Part 1 evaluation questions should be informed by the literature and available evidence on school enrollment and retention, if the studies are relevant to Malawi. 5B. AMAA Evaluation Part 2 Part 2 AMAA evaluation planning work will begin after the CDSSs are built in the Machinga and Balaka districts. This SOW will be updated to include more details regarding the terms of reference for Part 2. To answer the Part 2 evaluation question, the evaluation team will analyze relevant documents, such as, AMAA school construction proposal(s), work-plan(s) and budgets, Initial Environmental Examination (IEE) records and site specific Environmental Monitoring and Mitigation Plans (EMMP), the AMAA progress reports to USAID, documentation relating to procurement, school construction designs (drawings and specifications) and contractor/community plans. For Part 2 of the AMAA evaluation, the evaluation team will also gather information from USAID staff and non-USAID stakeholders. The USAID stakeholders include the Malawi Mission, the Africa Bureau, E3/GenDev, E3/Education, E3/Environment offices, E3/Energy and Infrastructure, E3/Water. The non-USAID stakeholders include Save the Children and AMAA staff, key personnel from Malawi’s Ministry for Education, Science and Technology (MoEST) and other Malawi government agencies supporting school construction; community members and donors engaged in secondary school construction in Malawi. 5C. Research Context To optimize learning from this evaluation, in the Part 1 and Part 2 evaluation reports, the evaluation team will cogently summarize relevant research findings on adolescent girls relating to, for example: • The effects of secondary school education on girls’ wellbeing, aspects of empowerment, future livelihoods, and earnings; and • School construction designs that facilitate positive learning environments and safety for girls. Evaluation and Reseach Design Report | 46 AMAA EVALUATION AND RESEARCH The research findings could be as applicable, sourced by wide-ranging studies that address girls’ secondary education effects and issues at the global, regional or country levels. The audiences for this evaluation should be able to understand the AMAA evaluation findings in the context of these research findings. 5D. Evaluation Tasks In alignment with the requirements and standards in Section C: Scope of Work of the Reading and Access contract, the evaluation team will undertake and complete the following tasks, including: • Finalize the Part 1 and Part 2 evaluation questions drafted to-date in consultation with USAID and AMAA team; • Prepare the Evaluation Design Report and the corresponding evaluation work￾plan and detailed evaluation budget using the templates recommended by the E3/Education R&A Contracting Officer’s Representative (COR). The Evaluation Design Report should: o Describe the evaluation methodology to collect and analyze quantitative and qualitative data, and disaggregate data by sex, age and other relevant variables; o Provide information on the sampling frame, power calculations, control and treatment groups; o Discuss internal and external validity and how generalizable the results will be; o Identify the strengths and limitations of the proposed methodology as well as evaluation implementation challenges and proposed solutions. o Discuss the approach to optimizing cost-efficiency; o Describe the plan to leverage local knowledge to benefit the evaluation team; o Describe the data collection tools for the surveys, focus group discussions and key informant interviews; o Use the evaluation report template(s) recommended by the R&A COR. • Prepare the data collection tools; • Recruit and deploy an evaluation implementation team with the recommended qualifications, including Malawian nationals; • Implement the AMAA evaluation (Part 1 and Part 2) in close consultation with USAID and the AMAA team; • Report the findings, conclusions and findings of the AMAA evaluation in the context of the needs of adolescent girls in Malawi and the AMAA school construction experience; and • Implement the activities to learn from the AMAA evaluation’s findings, conclusions and recommendations during the baseline, midline and end-line stages. Evaluation and Reseach Design Report | 47 AMAA EVALUATION AND RESEARCH 6. DELIVERABLES AND REPORTING REQUIREMENTS 6A. Deliverables With reference to the requirements for deliverables in the Reading and Access Evaluations contract, the evaluation team will deliver the following products: • In 2018 by March 31st, an Evaluation Design Report with a work-plan and detailed budget for the AMAA evaluation with an in-house presentation on the evaluation design to USAID and the AMAA team using audio-visual media to connect audiences; • The set of data collection tools; • Briefings for USAID/Malawi and AMAA team during data collection visits in Malawi; • In 2018, by November 30, a draft Part 1 baseline report, the final baseline report and a presentation to USAID and the AMAA team, using audio-visual media to connect audiences; • In 2019 by September 30, a draft Part 2 evaluation report, the final Part 2 evaluation report and presentation to USAID and the AMAA team, using audio￾visual media to connect audiences; • In 2021, byJune 30, a draft Part 1 end-line report and the final Part 1 end-line evaluation report with an in-house presentation to USAID and the AMAA team, using audio-visual media to connect audiences; • In 2021 by August 27, a 2 page summary Briefing Note • In 2021 by September 8, hold a webinar on the evaluation’s findings, conclusions and recommendation for Agency-wide stakeholder audiences and post the webinar recording on Agency web sites recommended by USAID. 6B. Reporting The evaluation reports (Part 1 baseline, Part 2 final report, and Part 1 end-line) will discuss the AMAA evaluation’s findings, conclusions and recommendations in the context of the needs of girls in Malawi and the AMAA school construction experience. Each evaluation report (Part 1 baseline, Part 2 final report, and Part 1 end-line) should contain the following components: ● Executive Summary; ● Introduction (including evaluation purpose, audience, anticipated uses, and questions); ● Program Background (overview of the needs of adolescent girls in Malawi, the AMAA Project and the AMAA new school construction activities); ● Methodology (overview of data collection and analysis methods, including limitations, anticipated challenges and solutions); ● Findings, conclusions and recommendations for each evaluation question in the context of research findings, the relevant needs of adolescent girls in Malawi, and the AMAA school construction experienceimplementation experience; and ● Annexes (including evaluation SOW, references, list of respondents, and data collection instruments). Evaluation and Reseach Design Report | 48 AMAA EVALUATION AND RESEARCH The AMAA evaluation reports will meet the evaluation quality criteria described in the USAID Evaluation Policy (see text box below): USAID EVALUATION POLICY, APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Evaluation and Reseach Design Report | 49 • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. 7. EVALUATION TEAM COMPOSITION Within the Labor Categories described in the Reading and Access Evaluations contract, NORC will select an evaluation team that includes a Principal Investigator, statistician(s), gender in education specialist(s), architect/engineer, administrative associate(s), and in-country data collectors. In alignment with the requirements and standards in Section C: Scope of Work of the Reading and Access Evaluations contract, the AMAA evaluation team is tasked with designing and implementing an the evaluation to answer the Part 1 and Part 2 in consultation with USAID and the AMAA team. The WW evaluation team will collectively reflect the following qualifications scaled appropriately to senior, mid-level and junior levels: • Graduate degree(s) in the social sciences; • Expertise in applying a variety of methods to gather and analyze quantitative and qualitative data, as well as designing and implementing impact evaluations and performance evaluations; • Knowledge of USAID’s performance monitoring and evaluation guidance; • International development programming experience; • Demonstrated experience in gender equality and women’s empowerment and/or programming; AMAA EVALUATION AND RESEARCH • Specialist knowledge on the needs of adolescent girls and education systems in Malawi; • Professional architect/engineer qualifications with knowledge of USAID’s construction standards and extensive school construction experience; • Experience in managing projects/activities funded by USAID; • Specialist expertise in working with databases, and software to collect and analyze data and to disseminate findings; • Data collection, data analysis, report writing and communication skills, including languages spoken in the WW implementation areas; and • Excellent ability to work collaboratively with evaluation team members, USAID, WEI/B and its partners. The Evaluation Design Report will propose the staffing plan for this evaluation, including, specific positions and CVs for proposed individuals to serve in the evaluation team positions. Each evaluation team member will sign USAID’s conflict of interest statement before conducting any documentary or field research. To leverage local knowledge, the contractor is encouraged to use host-country evaluation specialists with the requisite qualifications. 8. PART 1 EVALUATION SCHEDULE20 DELIVERABLES PERIOD 1. Submit draft Part 1 Evaluation Design Report March 23, 2018 2. Submit final Part 1 Evaluation Design Report, incorporating USAID’s feedback on the draft 10 business days after receiving written feedback from all USAID stakeholders and implementing partner on the draft EDR 3. Prepare Part 1 baseline data collection tools and gather baseline data from learners, parents and community leaders May, 2018 4. Submit draft Part 1 Baseline Report September 5, 2018 5. Hold presentation on the Part 1 Baseline Report to USAID and IP audiences November 13, 2018 6. Submit Part 1 final Baseline Report 10 business days after receiving written feedback from all reviewers on the draft report 7. Collect administrative data from learners May 2018, March 2019, March 2020, March 2021 8. Conduct mid-line and endline surveys on learners May 2018, March 2021, May 2021 20 This SOW will be updated in the future based on the schedule for completing school construction in the Machinga and Balaka districts. The updated SOW will include the deliverables and evaluation schedule of Part 2 of the evaluation. Evaluation and Reseach Design Report | 50 AMAA EVALUATION AND RESEARCH DELIVERABLES PERIOD 9. Conduct interviews with school drop-outs March 2019, March 2020, March 2021 10. Conduct survey on endline survey on parents March 2021 11. Collect data from community leaders March 2021 12. Gather information from teachers and administrative data on teachers March 2019, March 2020, March 2021 13. Hold focus-group discussions with girls and parents March 2020 14. Submit draft Part 1 endline evaluation report June 30, 2021 15. Hold presentation(s) on the draft Part 1 evaluation report’s findings, conclusions and recommendations to USAID audiences and the IPs July 2021 16. Submit final Evaluation Report 10 business days after receiving written feedback from all USAID stakeholders on the draft report 17. Prepare 2-page briefing note August 27, 2021 18. Hold webinar on the final evaluation report’s findings, conclusions and recommendations September 8, 2021 19.OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided in machine￾readable, non-proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the required elements of the SOW of the contract/agreement, technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, need to be agreed upon in writing by the COR. Any revisions should be updated in the SOW that is included as an annex to the Evaluation Report. Evaluation and Reseach Design Report | 51 AMAA EVALUATION AND RESEARCH ANNEX II: DATA COLLECTION INSTRUMENTS AND IRB MATERIALS Evaluation and Reseach Design Report | 52 AMAA EVALUATION AND RESEARCH NORC – AMAA SCHOOL CONSTRUCTION EVALUATION LEARNER INSTRUMENT BASELINE – MAY 2018 Field control District Name of School School EMIS number Name of Learner Gender Boy Girl Standard and class Standard Class Main teacher’s name Date of Assessment Day Month Year 2018 ID/Name of evaluator [PROGRAMMER: Capture the date, the start time and the end time of the interview, as well as the GPS location] Par_consent: Signed parental/caregiver consent (only for interviewers) Yes No [PROG: GO TO END_TEXT] April 2018 Evaluation and Reseach Design Report | 53 AMAA EVALUATION AND RESEARCH LEARNER ASSENT Chilolezo cha ophunzira Hello, we are working with Investment in Knowledge and NORC, a research organization, and we want to learn about students like you in this area of Malawi. Zikomo, tikugwira ntchito ndi Invest in Knowledge ndi NORC. Bungwe la kafukufuku, ndipo tikufuna tiphunzire zokhudza ophunzira ngati iweyo mu dera lino la Malawi. Today, I would like to ask you some questions that will help us to understand students in Malawi. You were picked by chance, like in a raffle or lottery. Lero, ndikufuna ndikufunse mafunso amene atithandize kumvetsa za ana asukulu kuno ku Malawi .. Unasankhidwa mwamwayi, ngati mwa wachiwona ndani. This survey is voluntary. It is completely your choice to answer the questions and if you don’t want to participate, it is fine and nothing will happen to you. Kafukufukuyu ngongodzipereka. Chisankho chonse chili mmanja mwako kuyankha mafunso, ndipo ngati sukufuna kutenga nawo mbali, zabwinobwino ndipo palibe chilichonse chikuchitikire iweyo. If you agree to answer the questions, all your answers will be strictly confidential and we will not share them with anyone. I will NOT show your answers to your teacher, friends or anyone. Your identity will always be kept confidential and your name will never be used. Ngati uvomere kuyankha mafunso, mayankho ako onse adzakhala a chinsinsi ndipo sitidzanena kwa wina aliyense. Sindionetsa mayankh ako kwa aphunzitsi, anzako kapena wina aliyense. Za iwe zones zizasungidwa mwachinsinsi nthawizonse ndipo dzina lako siloizagwiritsidwa ntchito. This is NOT a test and it will NOT affect your grade/standing at school. Awa si mayeso ndipo sidzikhudzana ndi malikisi kapena kakhonzedwe kako pasukulu. If any of the questions make you uncomfortable, we can skip them. You can also choose to stop answering questions at any time. Ngati usowe mtendere ndi funso lirilonse, titha kulidumpha. Ndiponso utha kusankha kusiya kuyankha mafunso nthawi yiliyonse. Do you want to answer the questions today? Kodi ukufuna kuyankha mafunso lero? Yes No [Go to END_TEXT] Evaluation and Reseach Design Report | 54 AMAA EVALUATION AND RESEARCH Now, we are going to ask you some questions so we can learn more about you. Tsopano tikufunsani mafunso ena pofuna kuphunzira zambiri za iwe SEX Observe: (unless not clear, then ask) Girl Boy AGE How old are you? Uli ndi zaka zingati? _ _ years Don’t know (-98) Refused (-99) GRD In what standard are you currently? Padakali pano uki kalasi yanji? Standard 5 Standard 6 Standard 7 Standard 8 Don’t know (-98) PROG: GO TO END_TEXT Refused (-99) PROG: GO TO END_TEXT PRVGRD In which standard where you enrolled last year? Kodi chaka chatha unali kalasi yanji? Standard 4 Standard 5 Standard 6 Standard 7 Standard 8 Refused (-99) TRAVEL Generally, how do you travel to school? MARK ALL THAT APPLY Kodi umayenda pachani popita kusukulu? By foot (1) Pansi By bicycle (2) Pa njinga yakapalasa By motorcycle (3) Pa njinga ya moto By car (4) Pa galimoto By public transportation (eg.Kabaza (of either bicycle or motorcycle), taxi, bus) (5) pa thirasipoti zolipira Other, specify______(96) zina TRAVELWITH Generally, do you travel to school… MARK ALL THAT APPLY Kodi umayenda ….. popita kusukulu? … alone (1) Evaluation and Reseach Design Report | 55 AMAA EVALUATION AND RESEARCH Wekha … with an adult (2) ndi munthu wa mkulu …with older children (3) ndi ana anzako okulirapo? …with children of your age or younger (4) ana koma ochepreko msinkhu kapena ofanana nawo msinkhu …other, specify ________(96) Zina TIME How long does it take you to travel from home to school, in minutes? Zimakutengera nthawi yayitali bwanji kuyenda kuchokera kunyumba kukafika ku sukulu? _____ minutes [PROG: min 0; max 200] mphindi LANG What is the primary language that you speak at home? Kodi kukhomo kwanu mumalankhulara chilankhulo chiti pafupipafupi? Chichewa Chiyao Chinyanja Chinsena Chilomwe Chitumbuka English Other ETHNIC What do you consider to be your ethnic group or community? [PROG: Select all that apply] Kodi umazitchula kuti ndiwe wa mtundu wanji ? (Select all that applies) Chewa Lomwe Yao Ngoni Tumbuka Nyanja Sena Tonga Ngonde Other Don’t know (-98) Refused (-99) EAT Did you eat anything today? Kodi unadya chili chonse lero? Yes (1) No (2) Don’t know (-98) Refused (-99) MAXSCH How far in school do you expect to go? Evaluation and Reseach Design Report | 56 AMAA EVALUATION AND RESEARCH Kodi ukuyembekezera kuti sukulu uzapita nayo patali bwanji? Finish Standard 5 Finish Standard 6 Finish Standard 7 Finish Standard 8 Finish Form 1 Finish Form 2 Finish Form 3 Finish Form 4 Some vocational or technical school after secondary school Finish vocational or technical school after secondary school Some college or university Finish college or university Don’t know (-98) Refused (-99) NOPRIM Why won’t you complete primary school? [PROG: If MAXSCH is less than 4] [ENUMERATOR: Probe: Are there any other reasons?] [PROG: Select all that apply]. Ndi chifukwa chiyani suzamaliza ma phuniziro ako a pulayimale? I am not interested in school (1) I will be helping at home /home farm/ family business (2) I will be learning a trade (3) I will be working for pay (4) It is unsafe for me to walk to school (5) I am concerned with safety at school (6) School is too far from my home (7) School costs too much (e.g. books, uniforms, school supplies, transport, fees) (8) I am not doing well in school (9) Members of my household do not want me to attend school (10) Other, specify____________________________(96) Don’t know (-98) Refused (-99) NOSEC Why won’t you complete secondary school? [PROG: If MAXCHL is 4 5 6 or 7] [ENUMERATOR: Probe: Are there any other reasons?] [PROG: Select all that apply]. Ndi chifukwa chiyani suzamaliza ma phuniziro ako a sekondale? I am not interested in school (1) I will be helping at home/ home farm/ family business (2) I will be learning a trade (3) I will be working for pay (4) It is unsafe for me to walk to school (5) I am concerned with safety at school (6) School is too far from my home (7) School costs too much (e.g. books, uniforms, school supplies, transport, fees) (8) Members of my household do not want me to attend school (9) Other, specify____________________________(96) Don’t know (-98) Refused (-99) MISSSCHL How many days did you miss school last week? Evaluation and Reseach Design Report | 57 AMAA EVALUATION AND RESEARCH Kodi musabata yatha unajomba kusukulu masiku angati? _________Days [PROG: Min is 0; Max is 5] Don’t know (-98) Refused (-99) MISSREASON What was the main reason you missed school last week? [PROG: If MISSCHHL is greater than 0] Kodi chifukwa chachikulu chomwe unajombera kusukulu chinali chiyani? I was sick (1) I had housework (2) I had farmwork/work to do (3) I had to take care of a child or sick relative (4) I had to go to work for money (5) I was too tired (6) I had plans with friends (7) I did not want to go to school (8) I was menstruating (9) I had outstanding school-related costs (10) Other, specify____________________________ (96) Don’t know (-98) Refused (-99) SEE Do you have difficulty seeing? If you wear glasses, do you have difficulty seeing when you have the glasses on? Kodi uli ndi vuto la maso? Ukavara magalasi umakhalabe ndi vuto lakuowna? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) HEAR Do you have difficulty hearing? If you use a hearing aid, do you have difficulty hearing when using the hearing aid? Kodi uli ndi vuto lakumva? Ukavara zothandizira kumva, kodi umakhalabe ndi vuto lakumva? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) WALK Do you have difficulty getting around, such as walking or climbing steps? Kodi uli ndi vuto lakuyenda kapena kukwera ma sitepe? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) Evaluation and Reseach Design Report | 58 AMAA EVALUATION AND RESEARCH THINK Do you have difficulty thinking, such as remembering or concentrating? Kodi uli ndi vuto loganiza monga kukumbukira zinthu kapena kumvetsetsa zinthu? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) WASH Do you have difficulty taking care of yourself, for example washing all over or dressing yourself? Kodi uli ndi vuto lozisamlira wekha monga kuzisambitsa kapena kuziveka wekha? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) SPEAK Do you have difficulty communicating, for example understanding or being understood when you speak [prim_lang]? Kodi uli ndi vuto kuti umvetsetse chinthu chomwe chanendwa kapena kuti anthu avetse chomwe ukunena? No - no difficulty (1) Yes – some difficulty (2) Yes – a lot of difficulty (3) Cannot do at all (4) Don’t know (-98) Refused (-99) RATE Academically, what type of student are you…. Ungadziyike pa nambala yanji kumaphunziro… ….an excellent student? / Wophunzira wochita bwinono zedi? …a very good student? / Wophunzira wochita bwino kwambiri? …a good student? / Wophunzira wochita bwino? …a not so good student? / Wophunzira wochita bwino kweni kweni? Don’t know (-98) Refused (-99) Evaluation and Reseach Design Report | 59 AMAA EVALUATION AND RESEARCH Now, I would like to ask you some questions about the people who live with you. Pano ndikufuna ndi kufunse mafunso a wanthu omwe umakhala nawo LIVE_MOTHER Do you live with your biological mother? Kodi umakhala ndi mayi ako okubereka? Yes (1) No (2) Refused (-99) READ_MOTHER Does your mother know how to write and read ? [PROG: If 20=Yes] Kodi amayi akowa amatha kulemba ndi kuwerenga? Yes (1) No (2) Don’t know (-98) Refused (-99) EDU_MOTHER What is the highest level of education that your mother completed? [PROG: If LIVE_MOTHER=Yes] Amayi ako sukulu analekezera kalasi yanji? No schooling (1) Some primary (2) Complete primary (3) Some secondary (4) Complete secondary (5) Higher than secondary (6) Other, specify_______________________________(96) Don’t know (-98) Refused (-99) LIVE_FATHER Do you live with your father? Kodi umakhala ndi bamboo ako okubereka? Yes (1) No (2) Refused (-99) READ_FATHER Does your father know how to write and read? [PROG: If LIVE_FATHER=Yes] Kodi abambo akowa amatha kulemba ndi kuwerenga? Yes (1) No (2) Don’t know (-98) Refused (-99) EDU_FATHER What is the highest level of education that your father completed? [PROG: If 23=Yes] Kodi abambo ako sukulu adalekezera kalasi yanji? No schooling (1) Some primary (2) Complete primary (3) Some secondary (4) Complete secondary (5) Higher than secondary (6) Evaluation and Reseach Design Report | 60 AMAA EVALUATION AND RESEARCH Other, specify_______________________________(96) Don’t know (-98) Refused (-99) GUARDIAN Who is your guardian? Your guardian is the adult who takes care of you the most. [PROG: If LIVE_MOTHER=No AND LIVE_FATHER=No] Kodi amakuyang’anira iwe ndi ndani? Uyu ndi munthu yemwe amakuyang’anira nthawi zambiri. Step-mother (8) Step-father (9) Grandmother (1) Grandfather (2) Aunt (3) Uncle (4) Sister (5) Brother (6) Cousin (7) Neighbor (10) Other, specify ________(96) Don’t know (-98) Refused (-99) READ_GUARDIAN Does your guardian know how to write and read? [PROG: If LIVE_MOTHER=No AND LIVE_FATHER=No] Kodi okuyanganirawa amatha kulemba ndi kuwerenga? Yes (1) No (2) Don’t know (-98) Refused (-99) EDU_GUARDIAN What is the highest level of education that your guardian completed? [PROG: If LIVE_MOTHER=No AND LIVE_FATHER=No] Kodi okuyang’anirawa sukulu adalekezera kalasi yanji? No schooling (1) Some primary (2) Complete primary (3) Some secondary (4) Complete secondary (5) Higher than secondary (6) Other, specify_______________________________(96) Don’t know (-98) Refused (-99) SIBS Do you have older siblings? Kodi uli ndi azimbale ako obadwa nawo bele limodzi omwe ali aakulu kwa iwe? Yes (1) No (2) Don’t know (-98) Refused (-99) Evaluation and Reseach Design Report | 61 AMAA EVALUATION AND RESEARCH NUM_SIBS How many older siblings do you have? [PROG: If SIBS=Yes] Uli ndi azibale ako angati obadwa nawo bele limodzi omwe ali akulu kwa iwe? One (1) Two (2) Three (3) Four (4) Five (5) More than five (6) Don’t know (-98) Refused (-99) I would like to ask you a few simple questions about your older sibling/s [[PROG: If SIBS=Yes] Uli ndi azibale ako angati obadwa nawo bele limodzi omwe ali akulu kwa iwe? SIB_1 Think about your oldest sibling, is a brother or a sister? Taganizira yemwe unabadwa naye bele limodzi yemwe ali wa mkulu kwa nonse. Kodi ndi mchimwene kapena mchemwali? Brother (1) Sister (2) Don’t know (-98) Refused (-99) SIB_1_AGE How old is he/she? Ali ndi zaka zingati? _________years old Don’t know (-98) Refused (-99) SIB_1_STUD Is he/she a student (any level, primary, secondary, vocational, university, etc.)? Kodi padakali pano ali pa sukulu? Yes (1) No (2) [PROG: Skip to SIB_1_MAXSCHL] Don’t know (-98) Refused (-99) SIB_1_STUD_LVL If yes, what level is he/she attending? [PROG: If SIB_1_STUD=Yes] Ngati eya ali kalasi yanji ya sukulu? Less than Standard 5 Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Evaluation and Reseach Design Report | 62 AMAA EVALUATION AND RESEARCH Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ SIB_1_MAXSCHL If no, what was the highest level of education that he/she completed? [PROG: If SIB_1_STUD=NO or Don’t know] Ngati ayi, kodi sukulu adalekezera kalasi yanji yomwe adakhonza? Less than Standard 5 Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ Don’t know but less than Primary complete Don’t know but more than Primary and less than Secondary complete Don’t know but more than Secondary complete Don’t know and has no idea SIB_1_MARR Is he/she married? Kodi ali pa banja? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_1_CHLD Does he/she have children? Kodi ali ndi ana? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_2_SEX Think about your next oldest sibling, is a brother or a sister? [PROG: If NUM_SIBS>2 & <7] Pano taganizira yemwe anapondana ndi yemwe tamalizira kukambayu, kodi ndi mnyamata kapena mtsikana? Brother (1) Sister (2) Don’t know (-98) Refused (-99) SIB_2_AGE How old is he/she? Evaluation and Reseach Design Report | 63 AMAA EVALUATION AND RESEARCH Ali ndi zaka zingati? _________years old Don’t know (-98) Refused (-99) SIB_2_STUD Is he/she a student (any level, primary, secondary, vocational, university, etc.)? Kodi ali pa Sukulu? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_2_STUD_LVL If yes, what level is he/she attending? [PROG: If SIB_2_STUD=Yes] Ngati eya ali kalasi yanji ya sukulu? Less than Standard 5 Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ SIB_2_MAXSCHL If no, what level is the highest level of education that he/she completed? [PROG: If SIB_2_STUD=NO or Don’t know] Ngati ayi, kodi sukulu adalekezera kalasi yanji? Less than Standard 5 Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ Don’t know but less than Primary complete Don’t know but more than Primary and less than Secondary complete Don’t know but more than Secondary complete Evaluation and Reseach Design Report | 64 AMAA EVALUATION AND RESEARCH Don’t know and has no idea SIB_2_MARR Is he/she married? Kodi ali pa banja? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_2_CHLD Does he/she have children? Kodi ali ndi ana? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_3_SEX Finally, think about your next oldest sibling, is a brother or a sister? [PROG: If NUM_SIBS >3 & <7] Pano taganizira yemwe anapondana ndi yemwe tamalizira kukambayu, kodi ndi mnyamata kapena mtsikana? Brother (1) Sister (2) Don’t know (-98) Refused (-99) SIB_3_AGE How old is he/she? Ali ndi zaka zingati _________years old Don’t know (-98) Refused (-99) SIB_3_STUD Is he/she a student (any level, primary, secondary, vocational, university, etc.)? Kodi ali pasukulu? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_3_STUD_LVL If yes, what level is he/she attending? [PROG: If SIB_3_STUD=Yes] Ngati eya ali kalasi yanji ya sukulu? Less than Standard 5 Standard 5 Standard 6 Standard 7 Evaluation and Reseach Design Report | 65 AMAA EVALUATION AND RESEARCH Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ SIB_3_MAXSCHL If no, what level is the highest level of education that he/she completed? [PROG: If SIB_3_STUD=NO or Don’t know] Ngati ayi, kodi sukulu adalekezera kalasi yanji? Less than Standard 5 Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Finish Form 4 Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other. 13.a. Specify__________________ Don’t know but less than Primary complete Don’t know but more than Primary and less than Secondary complete Don’t know but more than Secondary complete Don’t know and has no idea SIB_3_MARR Is he/she married? Kodi ali pa banja? Yes (1) No (2) Don’t know (-98) Refused (-99) SIB_3_CHLD Does he/she have children? Kodi ali ndi ana? Yes (1) No (2) Don’t know (-98) Refused (-99) Evaluation and Reseach Design Report | 66 AMAA EVALUATION AND RESEARCH The next questions are about things in your house. Mafunso otsatira ndi akatundu yemwe muli naye pakhomo panu Does your household have… Kodi kunyumba kwanu muli ndi Q Yes (1) No (2) Don’t know (-98) No response (-99) CHAIR a chair in good condition? Mpando wogwira ntchito wabwinobwino? BED a bed in good condition? Kama/bedi logwira ntchito wabwinobwino? CLOCK a clock in good working condition? Watchi yogwira ntchito wabwinobwino? RADIO a radio in good working condition? Wayalesi yomvera yogwira ntchito wabwinobwino? TELEVISION a television in good working condition? Wayalesi yakanema yogwira ntchito wabwinobwino? COMPUTER a computer in good working condition? Kopyuta yogwira ntchito wabwinobwino? BIKE a bicycle in good working condition? Njinga yakapalasa yogwira ntchito wabwinobwino? MOTOR a motorcycle in good working condition? Njinga ya moto yogwira ntchito wabwinobwino? CAR a car, truck, or boat with engine in good working condition? Galimoto thiraki kapena boti la injini logwira ntchito wabwinobwino? REFRI a refrigerator in good working condition? Filiji yogwira ntchito wabwinobwino? STOVE a stove in good working condition? (excluding firewood) Sitovu yogwira ntchito wabwinobwino (kupatula nkhuni)? Evaluation and Reseach Design Report | 67 AMAA EVALUATION AND RESEARCH ELECTRICITY Does your household have electricity (any type: solar panels, grid, hydro)? Kodi nyumba yakwnau ili ndi magesi? Yes (1) No (2) Don’t know (-98) Refused (-99) PHONE Do you or someone in your household have a working cellphone? Kodi iwe kapena wina wapakhomo panu ali ndi telefoni ya m’manja yogwira ntchito? Yes (1) No (2) Don’t know (-98) Refused (-99) BOOKS Apart from schoolbooks, do you have any other books that you can read at home? Kupataula ma buku asukulu, kodi muli ndi ma buku ena oti mungamawerenge kunyumba? Yes (1) No (2) Don’t know (-98) Refused (-99) SCHLSHIP Do you have any type of scholarship to help with school fees or other school related expenditures? Kodi uli ndi thandizo lilironse loti likuthandize ndi kulipira sukulu kapena zinthu zina ndi zina zofunika kusukulu? Yes (1) No (2) Don’t know (-98) Refused (-99) OPINIONS ABOUT GENDER NORMS: I am going to read a number of statements about how people think that girls, boys, men and women should behave. Please consider each statement carefully and decide which of the following options best describes your view: If I read out the statement you may answer 1 = Strongly disagree 2 = Disagree 3 = Neutral 4 = Agree 5 = Strongly agree, depending on you own view Pano ndikuwerengera ziganizo zingapo za momwe anthu amaganizira kuti atsikana, anyamata, amuna ndi azimayi ayenera kukhalira. Ganizra ziganizo izi bwinobwino ndipo undiwuze kuti ndi ziti mwa izi zikuyimira momwe iwe umaziwonera zinthuzi. Untha kundiyanka kuti ukusutsana nazo kwambiri, ukusutsana nazo, uli pakatikati, ukugwirizana nazo, ukugwirizana nazo kwambiri. 1 = Strongly disagree (Kusutsana nazo kwambiri) 2 = Disagree (Kusutsana nazo) 3 = Neutral (Pakatikati) 4 = Agree (Kugwirizana nazo) 5 = Strongly agree (Kugwirizana nazo kwambiri). (Agree SHOWCARD) Evaluation and Reseach Design Report | 68 AMAA EVALUATION AND RESEARCH Ok, are you ready? Let’s start. Kodi wakonzeka? (1) (2) (3) (4) (5) Gen1 Boys are usually more intelligent than girls. Nthawi zambiri anyamata ndi anzeru kuposa atsikana Gen2 Boys are naturally better at math and science than girls Mwachibadwa anyamata amachita bwino pa maphunziro a masamu ndi a sayansi kuposa atsikana Gen3 It is more important for boys to do well in school than it is for girls. Ndi chabwino kwambiri kuti anyamata azichita bwino mukalasi kusiyana ndi atsikana Gen4 Since girls have to get married, they should not be sent for higher education. Chifukwa atsikana ayenera kukwatiwa, sayenera kutumizidwa ku maphunziro apamwamba Gen5 Girls like it when boys tease and make fun of them. Atsikana amasangalara ngati anyamata akuwaselewula iwo? Gen6 Girls provoke boys by wearing short dresses. Atsikana amakopa anyamata povala ma diresi aafupi. Gen7 It is a girl’s fault if a teacher sexually harasses her. Ndi vuto lamtsikana ngati mphunzitsi aseweretsa thupi lake. Gen8 It is acceptable for a teacher to get a learner pregnant if he marries her. Ndizovomerezeka mphunzitsi kupereka mimba kwa mwana wasukulu ngati atamukwatire Gen9 It is unacceptable for a woman to disagree with her husband. Sizovomerezeka kuti mzimayi asutsane ndi mwamuna wake Gen10 Men need more care as they work harder than women. Azibambo amfunika chisamaliro chambiri chifukwa magwira ntchito zamphamvu kusiyana ndi azimayi Evaluation and Reseach Design Report | 69 AMAA EVALUATION AND RESEARCH Gen11 Bathing and feeding the children are the mother’s responsibility. Kasambitsa ndi kudyetsa ana ndi ntchito ya azimayi Gen12 There are times when a man needs to beat his wife. Pali nthawi zina zoti mzibambo ayenera kumenya mkazi wake Gen13 A mother should tolerate violence from the father in order to keep the family together. Mayi ayenera kupirira nkhanza kuchokera kwa mwamuna wake kuti banja lisapasuke Gen14 A woman’s most important role is to take care of the family and cook Ntchito ya mzimayi yeni yeni ndi yosamala banja ndi ku phika Gen15 A woman should always obey her husband Mkazi ayenera kumvera mwamuna wake nthawi zones Gen16 A woman should have the right to divorce Mkazi ayenera kukhala ndi ufulu wothetsa banja. Evaluation and Reseach Design Report | 70 AMAA EVALUATION AND RESEARCH Aspirations and Expectations I am now going to ask you your opinion on various issues. There are no right and wrong answers. I just want to know what you think. Tsopano ndikufunsa za mmene umaganizira pa zinthu zosiyanasiyana. Palibe yankho lokhoza ndi lolakwa. Ndingofuna ndidziwe mmene ukuganizira. EXPSCHL Imagine you had no constraints and could study for as long as you liked. What is the highest level of formal education that you would like to complete? Taganiza kuti unalibe zolepheretsa ndipo ukanaphunzira mmene ukufunira. Kodi ukhoza kufuna utalekezera pati maphunziro? Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Form 4 Some Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other (specify)_____________________________ Don’t know (-98) Refused (-99) EXREACH Think about your situation now. Do you think you will be able to reach that level of education? Ganiza za zomwe ukudutsamo pano. Ukuganiza kuti utha kufikira mlingo umenewo wa maphunziro? Yes (1) No (2) Don’t know (-98) Refused (-99) OLDEDU How old do you think you will be when you finish all of your education? Kodi ukuganiza kuti uzakhala ndi zaka zingati kuti uzamalize maphunziro ako? _________ years Don’t know (-98) Refused (-99) JOB Think about yourself at age 25, would you like to have job or be working at that age? Uziganizire utafika zaka 25, utha kufuna utadzakhala pa ntchito kapena utamadzagwira ntchito pa msinkhu umenewo? Yes (1) No (2) [PROG: Skip to ACTIV] Don’t know (-98) [PROG: Skip to ACTIV] Refused (-99)[PROG: Skip to ACTIV] JOBTYPE What type of job or work would you like to do? Kodi ukufuna utamadzagwira ntchito yanji? Evaluation and Reseach Design Report | 71 AMAA EVALUATION AND RESEARCH (Do not prompt. Enter code from provided list) OTHER Specify _________ 01=Accountant 02=Actor/actress 03=Administrative Assistant/Secretary/Clerical 04=Artist 05=Civil servant 06=Computer operator 07=Construction worker 08=Cook 09=Dentist 11=Doctor/Physician 12=Domestic Worker 13=Driver/Conductor 14=Electrician 15=Engineer 16=Factory worker 16=Farmer 17=Firemean/Firewoman 18=Fisherman/woman 19=Fulltime parent/Housewife 20=Journalist 20=Labourer/Unskilled 21=Lawyer 22=Lecturer/Professor 23=Manager 23=Market Trader/shop assistant 24=Mason/Builder 25=Mechanic 26=Nurse 27=Painter/decorator 28=Pilot 29=Policeman/woman 30=Politician 31=President/leader of country 32=Religious leader/priest/imam/shaik/pastor Evaluation and Reseach Design Report | 72 AMAA EVALUATION AND RESEARCH 32=Security Guard 33=Scientist 34=Singer 35=Soldier 36=Sportsman/woman 37=Tailor/Seamstress 38=Taxi Driver 39=Teacher 40=Trader/businessman/woman 41=Veterinarian 42=Other -98=Don’t know -99=Refuse EXJOB Given your current situation do you expect that you will be able to have that kind of job at 25? Kutengera za mmene ulili pano, kodi ukuyembekezera kuzakhala pa ntchito ngati imeneyo podzafika zaka 25? Yes (1) No (2) Don’t know (-98) Refused (-99) ACTIV If you don’t want to have a job or be working at 25, what would you like to do? Ngati sukufuna kudzakhala pa ntchito kapena kugwira ntchito pa zaka 25, kodi ukufuna kumadzatani? [PROG: Ask only if JOB=No] Study Take care of my household/children Nothing Volunteer Travel Other. Specify_______________ Don’t know (-98) Refused (-99) MARR Would you like to get married at some point in your life? Kodi utha kufuna kudzakhala pa banja nthawi yina? Yes (1) No (2) [PROG: SKIP to EXCHILDEVER] Don’t know (-98) ) [PROG: SKIP to EXCHILDEVER] Refused (-99) ) [PROG: SKIP to EXCHILDEVER] AGEMARR At what approximate age would you like to get married? Kodi ukufuna utadzakwatiwa/tira ndi zaka zingati? _________ years Evaluation and Reseach Design Report | 73 AMAA EVALUATION AND RESEARCH Don’t know (-98) Refused (-99) EXCHILDEVER Would you like to have children at some point in your life? Kodi ukufuna udzakhale ndi ana pa nthawi yina? Yes (1) No (2) [PROG: Skip to LAD1] I already have a child of my own (3) [PROG: Skip to LAD1] Don’t know (-98) [PROG: Skip to LAD1] Refused (-99) [PROG: Skip to LAD1] EXPCHILDAGE At what approximate age would you like to have your first child? Kodi ukufuna udzakhale ndi mwana utafika zaka zingati? ___30______ years Don’t know (-98) Refused (-99) EXCHILDNUM And, how many children would you like to have? Ndipo kodi ukufuna uzakhale ndi ana angati? One (1) Two (2) Three (3) Four (4) Five (5) Six (6) Seven (7) Eight (8) More than eight (9) Don’t know (-98) [PROG: Skip to LAD1] Refused (-99) [PROG: Skip to LAD1] LADDER EXERCISE (Ladder SHOWCARD) There are nine steps on this ladder. Suppose we say that the ninth step, at the very top, represents the best possible life for you and the bottom represents the worst possible life for you. Pali ma sitepe asanu ndi anayi pa makwerero awa. Nde titanena kuti sitepe ya chisanu ndi chinayi, pamwamba penipeni, ikuyimirira moyo wabwino kwambiri wako ndipo ya pansi moyo wovutikitsitsa wako. LAD 1 Where on the ladder do you feel you personally stand at the present time? Kodi pamakwererowa ukuona kuti uli pati panopa? _________________ Record step number: 1 - 9 Don’t know (-98) Refused (-99) LAD 2 Where on the ladder do you think you will be at age 25? Evaluation and Reseach Design Report | 74 AMAA EVALUATION AND RESEARCH Kodi udzakhala uli pati pamakwererowa ukadzafika zaka 25. _________________ Record step number: 1 - 9 Don’t know (-98) Refused (-99) CONTACT INFORMATION Now I would like to ask you some information that could help us remember who you are. The reason I am asking for this information is that we may want to ask you a few more questions in the future. Panopa ndifuna ndikufunse zokhudza zomwe zingatithandize kukumbukira kuti ndiwe ndani. chifukwa chomwe ndikukufunsira izi ndihoti titha kudzafuna kudzakufunsa mafunso ena oonjezera mtsogolo muno. HASPHONE Do you have a phone number/s where we could reach you or leave a message? (We will capture up to 3 numbers) Kodi uli ndi nambala ya lamya yoti titha kukuyimbira kapena kutumiza uthenga? Yes (1) No (2) SKIP to PHONE1 What is your phone number? Nambala ya lamya yanu ndi chani? (For Interviewers: +265 (0) - DO NOT WRITE THE LEADING 0 -+265--------------------------------------------- (Phone Number) Cannot provide a number PHONE1_OWNER Who is the owner of this phone number? Kodi mwini wake wa nambala ya lamyayi ndi ndani? Self (1) Mother/Female caregiver (2) Father/Male caregiver (3) Grandparents (4) Brother/Sister (5) Neighbor (6) Aunt/Uncle (7) Cousin (8) Village leader (9) Friend/Schoolmate (10) Teacher / School head teacher (11) Other, specify (-96) PHONE1_NAME: What's the full name of the owner of the phone number you gave? Kodi dzina lonse la mwini wake wa nambala ya lamya waperekayi ndi ndani? ___________________________________________ (SECOND PHONE NUMBER) HASPHONE2 Is there another phone number you could give me? Yes No PROG: Skip to EMAIL PHONE2 What is your phone number? Kodi nambala ya lamya yako ndi chani?(For Interviewers: +265 (0) - DO NOT WRITE THE LEADING -+265--------------------------------------------- (Phone Number) Evaluation and Reseach Design Report | 75 AMAA EVALUATION AND RESEARCH Cannot provide a number PHONE2_OWNER who is the owner of this phone number? Kodi mwini wake wa nambala ya lamyayi ndi ndani? Self (1) Mother/Female caregiver (2) Father/Male caregiver (3) Grandparents (4) Brother/Sister (5) Neighbor (6) Aunt/Uncle (7) Cousin (8) Village leader (9) Friend/Schoolmate (10) Teacher / School head teacher (11) Other, specify (-96) PHONE2_NAME What's the full name of the owner of the phone number you gave? Kodi dzina lonse la mwini wake wa nambala ya lamya waperekayi ndi ndani? ______________________________ (THIRD PHONE NUMBER) HASPHONE3 Is there another phone number you could give me? Yes No PROG: Skip to EMAIL PHONE3 What is your phone number? Kodi nambala ya lamya yako ndi chani? (For Interviewers: +265 (0) - DO NOT WRITE THE LEADING 0 -+265--------------------------------------------- (Phone Number) Cannot provide a number PHONE3_OWNER Who is the owner of this phone number? Kodi mwini wake wa nambala ya lamyayi ndi ndani? Self (1) Mother/Female caregiver (2) Father/Male caregiver (3) Grandparents (4) Brother/Sister (5) Neighbor (6) Aunt/Uncle (7) Cousin (8) Village leader (9) Friend/Schoolmate (10) Teacher / School head teacher (11) Other, specify (-96) PHONE3_NAME What's the full name of the owner of the phone number you gave? Kodi dzina lonse la mwini wake wa nambala ya lamya waperekayi ndi ndani? Evaluation and Reseach Design Report | 76 AMAA EVALUATION AND RESEARCH __________________________________ EMAIL Can you give me an electronic email address? Kodi mungandigayileko imelo adiresi? Yes (1) No (2) [Skip to Electronic email address (if email=YES) ______________________________ (PLEASE, RECORD HERE THE INFORMATION ON THE LOCATION OF THE RESPONDENT'S FAMILY HOUSE) DISTRICT_HOME What is the District? Kodi ndi boma lanji? Machinga Balaka TOWN What is the Village/Town ? Kodi ndi mudzi/tawoni wanji List of Villages/Towns OTHER_TOWN: Other Village, specify ______________________________ DIRECTIONSHOME Direction to reach the house (for interviewers) Please write directions on how to find the home of the respondent. Start with a location that is well known in the area as a reference point. In the directions, include landmarks, roads, and any other detailed information where relevant. References to specific businesses or homes "(ask for the home of...”) should also be included where possible. ________________________________ POSTAL Does your household have an address where you collect mails? Kodi banja lanu lili ndi keyala yomwe mumalindirilako makalata? Yes (1) No (2) If yes, Postal address number: (if mail home=yes) address If yes, Road name / Neighborhood: __________________ Mseu END_TEXT: END OF SURVEY, GREAT WORK! DISPOSITION Please enter the final status of this interview Completed (1) Refused (2) Absent from school (3) No parental consent (4) Parent refused (5) Other reason, specify (96) Enumerator Questions PROG: If disposition=1 EUNDERSTAND Did the respondent appear to understand the questions? Never (1) Seldom (2) Some of the time (3) Most of the time (4) Evaluation and Reseach Design Report | 77 AMAA EVALUATION AND RESEARCH EINTEREST Overall, how was the respondent’s interest in the interview? Very low (1) Below average (2) Average (3) Above average (4) Very high (5) COMMENTS: (complete only if something was unusual and merits to known) _______________________________________________________________ Evaluation and Reseach Design Report | 78 AMAA EVALUATION AND RESEARCH NORC – AMAA SCHOOL CONSTRUCTION EVALUATION CAREGIVER INSTRUMENT BASELINE – MAY 2018 Field control Enumerator: Complete the following Information EDIS District boma Machinga Balaka ESCH School Name: dzina la sukulu ___________________________________________ ELEAR Learner’s Name: dzina la ophunzira_____________________________________ ESEX Learner’s Sex: Boy Girl ESTD Learner’s Standard during current academic year (2017/18) kalasi la ophunzira mu chaka chasukuluchi (2017/18) Standard 5 Standard 6 Standard 7 Standard 8 LOC Interview Location malo ofunsirana mafunso School (1) [PROG: go to School] Household (2) [PROG: go to Household] School [PROG: if LOC=1] SPARENT Are you the parent or guardian of [CHILD NAME]? Kodi ndinu kholo kapena mlezi wa….(dzina la mwana) Yes No [PROG: Skip to END] SGRDE What standard is [CHILD NAME] attending currently? Kodi (dzina la mwana) akuphunzira mu kalasi lanji panopa? Standard 5 [PROG: Skip to Consent] Standard 6 [PROG: Skip to Consent] Standard 7 [PROG: Skip to Consent] Standard 8 [PROG: Skip to Consent] Enumerator needs to confirm the identification of correct learner. If correct, continue with consent. If not correct end interview and find the correct parent/caregiver Household [PROG: if LOC=2] Evaluation and Reseach Design Report | 79 AMAA EVALUATION AND RESEARCH No. visit Date Time Disposition Code Appointment Date and Time 1 2 3 Disposition codes: 1: Completed interview 2: No one at home or no adult at home 3: Head of household or adult caregiver is not home 4: Rescheduled (interview postponed and appointment was made) 5: Learner’s household could not be located 6: Partial complete/interview finished 7: Temporary refusal (supervisor to visit) 8: Final refusal 9: Other, specify Replace a household for codes 5 (learner’s household could not be located) and 9 (final refusal); or if a household has been visited 3 times HOUSE Is this the home of [CHILD NAME]? Kodi ili ndi khomo la …(dzina la mwana) Yes [PROG: Skip to CARE] No FIND Do you know where I can find [CHILD NAME]’s household? Kodi mukudziwako komwe ndingathe kupeza khomo la kwawo kwa (dzina la mwana) Yes [Enumerator: Collect information to find household] No [PROG: Go to end] CARE Is the parent or adult caregiver of [CHILD NAME] available? Kodi kholo kapena mlezi wa (dzina la mwana) aliko? Yes [PROG: Skip to SCHL] No APPT When could I find the parent or adult caregiver of [CHILD NAME]? Kodi ndingalipeze liti kholo kapena mlezi wa (dzina la mwana)? Date: Time: [PROG: Skip to END] SCHL What was [CHILD NAME’S] school this academic year? Kodi (dzina la mwana) amaphuynzira sukulu yanji mu chaka chasukuluchi? GRDE What standard is [CHILD NAME] attending currently? Kodi (dzina la mwana) akuphunzira kalasi yanji panopa? Standard 5 Standard 6 Standard 7 Standard 8 Enumerator needs to confirm the identification of correct learner. If correct, continue with consent. If not correct end interview and find the correct parent/caregiver Evaluation and Reseach Design Report | 80 AMAA EVALUATION AND RESEARCH Consent chilolezo Hello, we are working with Invest in Knowledge and NORC, a research organization, to learn about how students in Malawi. The Ministry of Education Science and Technology has approved this research work. Zikomo, ife tikugwira ntchito ndi Invest in Knowledge ndi NORC, bungwe la kafukufuku, kufuna kuphunzira za ana asukulu kuno ku Malawi. A unduna wa zamaphunziro sayansi ndi luso avomereza kafukufukuyu. You might remember that we sent a letter for you to sign in order to interview your child. Now we would like to ask about your own opinions regarding education, marriage, work, and employment. There are no right or wrong answers; I would like to know what you think. Pano ndikufuna ndikufunseni maganizo anu pa nkhani zokhudza maphunziro, ma ukwati, komanso nkhani za ntchito.. Palibe mayankho olondola kapena olakwa; ndikufuna kudziwa zomwe mumaganiza. This survey is voluntary. If you choose not to participate, there will be no consequences of any type for you or your child. The choice not to participate will not affect your child’s grade/standing at school and does not entail any penalty or loss of benefits. Kafukufukuyu ngongodzipereka chabe. Ngati musankhe kusatengapo mbali, palibe chilango chomwe chidzaperekedwe kwa inu kapena mwana wanu. Chisankho chosatengapo mbali sichikhudza malikisi kapena makhozedwe a mwana wanu ku sukulu ndipo sizikutanthauza chilango kapena kuluza zolowa. We will keep your responses confidential and nothing that you say will be shared with anyone outside of the research team. Your identity will always be kept confidential and your name will never be used. Tidzasunga mayankho anu mwachinsinsi ndipo chilichonse chomwe muyankhule sichidzanenedwa kwa wina aliyense oti si wamukafukufukuyu. Zokhudza inu zidzasungidwa mwachinsinsi ndipo dzina lanu silidzagwiritsidwa ntchito. You are free to not answer any questions you are not comfortable with or to stop the interview at any time. Muli ndi ufulu osayankha funso lilironse lomwe musali omasuka nalo kapena kusiya kuyankha mafunso nthawi yiliyonse. This survey will take about 15 minutes. Kafukufukuyu atenga pafupi fupi mphindi khumi ndi zisanu. Ngati muli ndi mafunso ena aliwonse, mutha kufunsa panopa kapena nthawi yina, kapenanso kufunsana mafunso kukatha. Ngati mukufuna kufunsa mafunso nthawi yina, mutha kuyankhulana ndi a James Mkandawire omwe akuyang’anira kafukufukuyu kuno ku Malawi pa nambala iyi 0888370081 kapena mutha kulemba kalata ku P.O. Box 506 Zomba. If you have questions, complaints, or get sick or injured as a result of being in this study. Call or contact the National Committee on Research in Social Sciences and Humanities (NCRSH) if you have questions about your rights as a study participant. You may also contact the NCRSH if you feel you have not been treated fairly or if you have other concerns. Their contact information is: Evaluation and Reseach Design Report | 81 AMAA EVALUATION AND RESEARCH NCST, 1st Floor Lingadzi House, Robert Mugabe Crescent, Private Bag B303, Lilongwe 3, Malawi.Email: directorgeneral@ncst.mw Phone: +265 1 771 550 Ngati muli ndi mafunso, madandaulo, kapena mudwala kapena kuvulala chifukwa chotenga nawo mbali mukafukufukuyu. Imbani kapena kumanani ndi a National Committee on Research in Social Sciences and Humanities (NCRSH)Ngati muli ndi mafunso okhudza ufulu wanu monga wotenga nawo mbali, mukhonza kufunsa a NCRSH. Ngati mukuona kuti mwachitidwa nkhanaza kapena muli ndi nkhawa zina adilesi yawo CONSENT Do you agree to participate in this study? Kodi mukuvomera kutenga nawo mbali mu kafukufukuyu? Yes No SEX Observe: What is the respondent’s gender? Kodi mtengambali ndi wamkazi kapena wamwamuna? Male Female AGE What is your age? Kodi zaka zako ndi zingati? _ _ years EDU What was the highest level of education that you completed? Kodi sukulu unamalizira pati? No schooling (1) Some primary (2) Complete primary (3) [PROG: GO TO RLT] Some secondary (4) [PROG: GO TO RLT] Complete secondary (5) [PROG: GO TO RLT] Higher than secondary (6) [PROG: GO TO RLT] Other, specify_______________________________(96) [PROG: GO TO RLT] Don’t know (-98) Refused (-99) RW Do you know how to write and read? Kodi mumadziwa kulemba ndi kuwerenga ? Yes (1) No (2) Refuse (-99) Evaluation and Reseach Design Report | 82 AMAA EVALUATION AND RESEARCH RLT What is your relationship to [CHILD NAME]? Kodi pali ubale wanji ndi (dzina la mwana) Mother (1) Father (2) Step-mother (11) Step-father (12) Grandmother (3) Grandfather (4) Aunt (5) Uncle (6) Sister (7) Brother (8) Cousin (9) Friend (10) Other: ________(96) Don’t know (-98) Refused (-99) OPINIONS ABOUT GENDER NORMS I am going to read a number of statements about how people think that girls, boys, men and women should behave. Please consider each statement carefully and decide which of the following options best describes your view: If I read out the statement you may answer 1 = Strongly disagree 2 = Disagree 3 = Neutral 4 = Agree 5 = Strongly agree, depending on you own view Pano ndikuwerengera ziganizo zingapo za momwe anthu amaganizira kuti atsikana, anyamata, amuna ndi azimayi ayenera kukhalira. Ganizra ziganizo izi bwinobwino ndipo undiwuze kuti ndi ziti mwa izi zikuyimira momwe iwe umaziwonera zinthuzi. Untha kundiyanka kuti ukusutsana nazo kwambiri, ukusutsana nazo, uli pakatikati, ukugwirizana nazo, ukugwirizana nazo kwambiri. 1 = Strongly disagree (Kusutsana nazo kwambiri) 2 = Disagree (Kusutsana nazo) 3 = Neutral (Pakatikati) 4 = Agree (Kugwirizana nazo) 5 = Strongly agree (Kugwirizana nazo kwambiri). (Agree SHOWCARD) Ok, are you ready? Let’s start. Kodi wakonzeka? (1) (2) (3) (4) (5) Gen1 Boys are usually more intelligent than girls Nthawi zambiri anyamata ndi anzeru kuposa atsikana Gen2 Boys are naturally better at math and science than girls Mwachibadwa anyamata amachita bwino pa maphunziro a masamu ndi a sayansi kuposa atsikana Evaluation and Reseach Design Report | 83 AMAA EVALUATION AND RESEARCH Gen3 It is more important for boys to do well in school than it is for girls. Ndi chabwino kwambiri kuti anyamata azichita bwino mukalasi kusiyana ndi atsikana Gen4 Since girls have to get married, they should not be sent for higher education. Chifukwa atsikana ayenera kukwatiwa, sayenera kutumizidwa ku maphunziro apamwamba Gen5 Girls like it when boys tease and make fun of them. Atsikana amasangalara ngati anyamata akuwaselewula iwo? Gen6 Girls provoke boys by wearing short dresses. Atsikana amakopa anyamata povala ma diresi aafupi. Gen7 It is a girl’s fault if a teacher sexually harasses her. Ndi vuto lamtsikana ngati mphunzitsi aseweretsa thupi lake. Gen8 It is acceptable for a teacher to get a learner pregnant if he marries her. Ndizovomerezeka mphunzitsi kupereka mimba ngati atamukwatire Gen9 It is unacceptable for a woman to disagree with her husband. Sizovomerezeka kuti mzimayi asutsane ndi mwamuna wake Gen10 Men need more care as they work harder than women Azibambo amfunika chisamaliro chambiri chifukwa magwira ntchito zamphamvu kusiyana ndi azimayi Gen11 Bathing and feeding the children are the mother’s responsibility. Kasambitsa ndi kudyetsa ana ndi ntchito ya azimayi Gen12 There are times when a man needs to beat his wife. Pali nthawi zina zoti mzibambo ayenera kumenya mkazi wake Evaluation and Reseach Design Report | 84 AMAA EVALUATION AND RESEARCH Gen13 A mother should tolerate violence from the father in order to keep the family together. Mayi ayenera kupirira nkhanza kuchokera kwa mwamuna wake kuti banja lisapasuke Gen14 A woman’s most important role is to take care of the family and cook Udindo ofunikira kwambiri kwa mzimayi ndi kusamala banja ndi kuphika Gen15 A woman should always obey her husband Mkazi ayenera kumvera mwamuna wake nthawi zonse Gen16 A woman should have the right to divorce Mkazi ayenera kukhala ndi ufulu wothetsa banja. Evaluation and Reseach Design Report | 85 AMAA EVALUATION AND RESEARCH Parent/Guardian’s Aspirations and Expectations I am now going to ask you your opinion on various issues. There are no right and wrong answers. I just want to know what you think. Panopa ndikufunsani za maganizo anu pa zinthu zosiyana siyana. Palibe mayankho okhoza ndi olondola. Ndingofuna kudziwa mmene mumaganizira. LEVELSCH Ideally what is the highest level of formal education would you like [Child Name] to complete? Mukuganiza kwanu mukufuna kuti (dzina la mwana) ataphunzira kumalizira pati sukulu Standard 5 Standard 6 Standard 7 Standard 8 Form 1 Form 2 Form 3 Form 4 Some Vocational or technical school after secondary school Finish vocational or technical school after secondary school College or university Other (specify)_____________________________ Don’t know (-98) SKIP TO JOB2 Refused (-99) SKIP TO JOB2 CMPTSCHL Do you expect [Child Name] to complete that level of education? Mukuyembekezera kuti (dzina la mwana) adzamaliza maphunziro ake pa mulingo umenewo? Yes (1) No (2) Don’t know (-98) No response (-99) JOB2 Think about [ChildName] at 25, would you like her/him to hold a job or work? Ganizani za (dzina la mwana) pa zaka 25, kodi mukufuna atadzakhala pa ntchito kapena kugwira ntchito? Yes (1) No (2) [PROG: Skip to ACTIV] Don’t know (-98) [PROG: Skip to ACTIV] No response (-99) [PROG: Skip to ACTIV] JOBTYPE What kind of job or work would you most like [ChildName] to do at that age? Kodi mungafune kwambiri kuti (dzina la mwana) atadzagwira ntchito yanji pa zaka zimenezo? (Do not prompt. Enter code from provided list) OTHER Specify _________ 01=Accountant 02=Actor/actress 03=Administrative Assistant/Secretary/Clerical 04=Artist 05=Civil servant 06=Computer operator Evaluation and Reseach Design Report | 86 AMAA EVALUATION AND RESEARCH 07=Construction worker 08=Cook 09=Dentist 11=Doctor/Physician 12=Domestic Worker 13=Driver/Conductor 14=Electrician 15=Engineer 16=Factory worker 16=Farmer 17=Firemean/Firewoman 18=Fisherman/woman 19=Fulltime parent/Housewife 20=Journalist 20=Labourer/Unskilled 21=Lawyer 22=Lecturer/Professor 23=Manager 23=Market Trader/shop assistant 24=Mason/Builder 25=Mechanic 26=Nurse 27=Painter/decorator 28=Pilot 29=Policeman/woman 30=Politician 31=President/leader of country 32=Religious leader/priest/imam/shaik/pastor 32=Security Guard 33=Scientist 34=Singer 35=Soldier 36=Sportsman/woman 37=Tailor/Seamstress 38=Taxi Driver 39=Teacher Evaluation and Reseach Design Report | 87 AMAA EVALUATION AND RESEARCH 40=Trader/businessman/woman 41=Veterinarian 42=Other -98=Don’t know -99=Refuse JOB When [Child Name] is about 25 years old, what job or activity do you think s/he will actually be doing? [Enumerator: Do not prompt. Enter code in provided list] Pamene (dzina la mwana) adzakwanitse zaka 25, kodi ndi ntchito yanji yomwe azidzagwira kapena azidzapanga chani? ______________ OTHER Specify ___________________ ACTIV If you don’t want [ChildName] to have a job or be working at 25, what would you like him/her to be doing? Ngati simukufuna (dzina la mwana) atadzakhala pantchito kapena kugwira ntchito pa zaka 25, mukufuna iyeyo azidzapanga chani? Studying Taking care of his/her household/children Nothing Volunteer Travel Other. Specify_______________ Don’t know (-98) Refused (-99 Ladder Exercise (Ladder SHOWCARD) There are nine steps on this ladder. Suppose we say that the ninth step, at the very top, represents the best possible life for you and the bottom represents the worst possible life for you. Pali ma sitepe asanu ndi anayi pa makwerero awa. Nde titanena kuti sitepe ya chisanu ndi chinayi, pamwamba penipeni, ikuyimirira moyo wabwino kwambiri wanu ndipo ya pansi moyo wovutikitsitsa wanu. LAD1 Where on the ladder do you feel you personally stand at the present time? Kodi mukuganiza kuti muli pati pa makwerero awa panopa? Record step number: 1-9 [ __ ] LAD2 Where do you think you will be on the ladder in four years from now? Kodi mukuganiza kuti muzakhala uli pati pa makwerero pa zaka zinayi zikubwerazi? Record step number: 1-9 [ __ ] LAD3 Where do you think that [ChildName] will be when he/she is your age? Kodi mukuganiza kuti (dzina la mwana) adzakhala ali pati pa msinkhu wanu? Record step number: 1-9 [ __ ] Evaluation and Reseach Design Report | 88 AMAA EVALUATION AND RESEARCH Child expectations Now, I am going to ask you about some of your expectations for [Child Name]’s life specifically. Panopa ndikufunsani za zina mwa ziyembekezo pa moyo wa (dzina la mwana) maka maka. At what age do you expect [ChildName] to: Kodi ndi pa zaka zingati pamene mukuyembekezerta kuti (dzina la mwana) adza: EXP1 Start earning money to support your household? [ __ __ ] Not able ( -97) DKN(-98) Yambe kupeza ndalama zothandizira nyumba yanu? EXP2 Leave full-time education? [ __ __ ] Not able ( -97) DKN(-98) Kusiyiya shool yomapita tsiku ndi tsiku? EXP3 Be financially independent of HIS/HER parents? [ __ __ ] Not able ( -97) DKN(-98) Khale odziyimira payekha pa zachuma osadalira makolo? EXP4 Leave this household? [ __ __ ] Not able ( -97) DKN(-98) chokepo pa khomo lino? EXP5 Get married and start living together with a husband/wife? [ __ __ ] Not able ( - 97) DKN(-98) kwatiwe nkuyamba kukhala limodzi ndi mwamuna/mkazi? EXP6 Have a child? [ __ __ ] Not able ( -97) DKN(-98) Khale ndi mwana? CONTACT INFORMATION Now I would like to ask you some information that could help us remember who you are. The reason I am asking for this information is that we may want to ask you a few more questions in the future. Panopa ndifuna ndikufunse zokhudza zomwe zingatithandize kukumbukira kuti ndiwe ndani. chifukwa chomwe ndikukufunsira izi ndihoti titha kudzafuna kudzakufunsa mafunso ena oonjezera mtsogolo muno. HASPHONE Do you have a phone number/s where we could reach you or leave a message? Kodi uli ndi nambala ya lamya yoti titha kukuyimbira kapena kutumiza uthenga? Yes (1) No (2) SKIP to PHONE1 What is your phone number? Nambala ya lamya yanu ndi chani? (For Interviewers: +265 (0) - DO NOT WRITE THE LEADING 0
Enter -95 if no phone number can be provided.) -+265--------------------------------------------- (Phone Number) Cannot provide a number PHONE1_OWNER Who is the owner of this phone number? Kodi mwini wake wa nambala ya lamyayi ndi ndani? Mother/Female caregiver (2) Father/Male caregiver (3) Grandparents (4) Evaluation and Reseach Design Report | 89 AMAA EVALUATION AND RESEARCH Brother/Sister (5) Neighbor (6) Aunt/Uncle (7) Cousin (8) Village leader (9) Friend/Schoolmate (10) Teacher / School head teacher (11) Other, specify (-96) PHONE1_NAME: What's the full name of the owner of the phone number you gave? Kodi dzina lonse la mwini wake wa nambala ya lamya waperekayi ndi ndani? EMAIL. Can you give me an electronic email address? Yes No Don’t Know (-98) Refused (-99) EMAIL_ADDRESS ____________________________________________ DIRECTIONSHOME Direction to reach the house (for interviewers) Please write directions on how to find the home of the respondent. Start with a location that is well known in the area as a reference point. In the directions, include landmarks, roads, and any other detailed information where relevant. References to specific businesses or homes "(ask for the home of...”) should also be included where possible. ________________________________ END_TEXT END OF SURVEY, THANK YOU VERY MUCH! Evaluation and Reseach Design Report | 90 AMAA EVALUATION AND RESEARCH NORC – AMAA SCHOOL CONSTRUCTION EVALUATION COMMUNITY LEADER SEMI-STRUCTURED INTERVIEW- BASELINE MAY 2018 Introduction and Consent chiyambi ndi chilolezo Hello, we are working with Invest in Knowledge and NORC, a research organization, to learn about students in Malawi. This research has been approved by the NORC Ethics Board, the National Committee on Research in the Social Sciences (NCRSH) and the Ministry of Education, Science and Technology. Zikomo, ife tikugwira ntchito ndi Invest in Knowledge ndi NORC, bungwe la kafukufuku, kufuna kuphunzira za ana asukulu kuno ku Malawi. Kafukufukuyu wavomerezedwa ndi ounika malamulo a NORC, komiti ya kafukufuku ya dziko ya social science ndi aunduna wa zamaphunziro, sayansi ndi luso. The main purpose of this interview is to learn about education decisions and ideals. You might know that we are interviewing students and parents in this area. Now we would like to ask about your own opinions and ideas regarding education, marriage, and work of youth. There are no right or wrong answers; I would like to know what you think. Cholinga cheni cheni cha mafunsowa ndikufuna kuphunzira za zisankho ndi ziganizo zokhudza maphunziro. Mukuyenera kudziwa kuti tikufunsa ophunzira ndi makolo a mudera lino. Panopa tikufuna tifunse za maganizo anu zokhudza maphunziro, banja, ndi ntchito za achinyamata. Palibe mayankho olondola ndi okhoza; ndingofuna kudziwa mmene mumaganizira. This survey is voluntary. If you choose not to participate, there will be no consequences of any type for you or your child. The choice not to participate will not affect your child’s grade/standing at school and does not entail any penalty or loss of benefits. Kafukufukuyu ngongodzipereka chabe. Ngati musankhe kusatengapo mbali, palibe chilango chomwe chidzaperekedwe kwa inu kapena mwana wanu. Chisankho chosatengapo mbali sichikhudza malikisi kapena makhozedwe a mwana wanu ku sukulu ndipo sizikutanthauza chilango kapena kuluza zolowa. We will keep your responses confidential and nothing that you say will be shared with anyone outside of the research team. Your identity will always be kept confidential and your name will never be used. Tidzasunga mayankho anu mwachinsinsi ndipo chilichonse chomwe muyankhule sichidzanenedwa kwa wina aliyense oti si wamukafukufukuyu. Zokhudza inu zidzasungidwa mwachinsinsi ndipo dzina lanu silidzagwiritsidwa ntchito. You are free to not answer any questions you are not comfortable with or to stop the interview at any time. Muli ndi ufulu osayankha funso lilironse lomwe musali omasuka nalo kapena kusiya kuyankha mafunso nthawi yiliyonse. This survey will take about 15 minutes. Kafukufukuyu atenga pafupifupi mphindi khumi ndi zisanu. Evaluation and Reseach Design Report | 91 AMAA EVALUATION AND RESEARCH If you have any questions, you may ask them now or later, even after the interview has been completed. If you wish to ask questions later, you may contact: Ngati muli ndi mafunso ena aliwonse, mutha kufunsa panopa kapena nthawi yina, kapenanso kufunsana mafunso kukatha. Ngati mukufuna kufunsa mafunso nthawi yina, mutha kuyankhulana ndi: a James Mkandawire omwe akuyang’anira kafukufukuyu kuno ku Malawi pa nambala iyi 0888370081 kapena mutha kulemba kalata ku P.O. Box 506 Zomba. If you have questions, complaints, or get sick or injured as a result of being in this study. Call or contact the National Committee on Research in Social Sciences and Humanities (NCRSH) if you have questions about your rights as a study participant. You may also contact the NCRSH if you feel you have not been treated fairly or if you have other concerns. Their contact information is: NCST, 1st Floor Lingadzi House, Robert Mugabe Crescent, Private Bag B303, Lilongwe 3, Malawi.Email: directorgeneral@ncst.mw Phone: +265 1 771 550 Ngati muli ndi mafunso, madandaulo, kapena mudwala kapena kuvulala chifukwa chotenga nawo mbali mukafukufukuyu. Imbani kapena kumanani ndi a National Committee on Research in Social Sciences and Humanities (NCRSH)Ngati muli ndi mafunso okhudza ufulu wanu monga wotenga nawo mbali, mukhonza kufunsa a NCRSH. Ngati mukuona kuti mwachitidwa nkhanaza kapena muli ndi nkhawa zina adilesi yawo NCST 1st Floor Lingadzi House, Robert Mugabe Crescent, Private Bag B303, Lilongwe 3, Malawi. Email: directorgeneral@ncst.mw Phone: +265 1 771 550 CONSENT Do you agree to participate in this study? Kodi mukuvomera kutenga nawo mbali mu kafukufukuyu? Yes No DISTRICT boma Machinga Balaka TOWN Town Name dzina la mudzi_________________________ SEX Observe: What is the respondent’s gender? Kodi oyankha mafunso ndi wamkazi kapena wamwamuna Male Female (ask if not clear) Evaluation and Reseach Design Report | 92 AMAA EVALUATION AND RESEARCH AGE What is your age? Muli ndi zaka zingati? _ _ years Refuse (-99) EDU What was the highest level of education that you completed? Kodi sukulu munamalizira pati kuphunzira No schooling (1) Some primary (2) Complete primary (3) Some secondary (4) Complete secondary (5) Higher than secondary (6) Other, specify_______________________________(96) Don’t know (-98) Refused (-99) Enumerator: If response to EDU is 1 or 2 then ask the following question RW Do you know how to read and write? Kodi mumadziwa kuwerenga ndi kulemba? Yes (1) No (2) Refuse (-99) POSITION What is your current position in this community? Kodi muli ndi udindo wanji panopa mudera lanu lino? WORKPLACE What is your current place of work or institution? Kodi panopa mukugwira ntchito kuti? I am going to read a number of statements about how people think that girls, boys, men and women should behave. Please consider each statement carefully and decide which of the following options best describes your view: Ndiwerenga ziganizo zingapo za momwe anthu amaganizira za momwe atsikana, anyamata, amuna ndi akazi azikhalira. Chonde mulingalire chiganizo hilichonse mosamala ndi kupanga chisankho kuti ndi chisankho chiti chomwe chikulongosola maganizidwe anu. 1 = Strongly disagree 2 = Disagree 3 = Neutral 4 = Agree 5 = Strongly agree. (Agree SHOWCARD) Ok, are you ready? Let’s start. Kodi wakonzeka? (1) (2) (3) (4) (5) Gen1 Boys are usually more intelligent than girls. Nthawi zambiri anyamata ndi anzeru kuposa atsikana Evaluation and Reseach Design Report | 93 AMAA EVALUATION AND RESEARCH (1) (2) (3) (4) (5) Gen2 Boys are naturally better at math and science than girls Mwachibadwa anyamata amachita bwino pa maphunziro a masamu ndi a sayansi kuposa atsikana Gen3 It is more important for boys to do well in school than it is for girls. Ndi chabwino kwambiri kuti anyamata azichita bwino mukalasi kusiyana ndi atsikana Gen4 Since girls have to get married, they should not be sent for higher education. Chifukwa atsikana ayenera kukwatiwa, sayenera kutumizidwa ku maphunziro apamwamba Gen5 Girls like it when boys tease and make fun of them. Atsikana amasangalara ngati anyamata akuwaselewula iwo? Gen6 Girls provoke boys by wearing short dresses. Atsikana amakopa anyamata povala ma diresi aafupi. Gen7 It is a girl’s fault if a teacher sexually harasses her. Ndi vuto lamtsikana ngati mphunzitsi aseweretsa thupi lake. Gen8 It is acceptable for a teacher to get a learner pregnant if he marries her. Ndizovomerezeka mphunzitsi kupereka mimba kwa mwana wasukulu ngati atamukwatire Gen9 It is unacceptable for a woman to disagree with her husband. Sizovomerezeka kuti mzimayi asutsane ndi mwamuna wake Gen10 Men need more care as they work harder than women. Azibambo amfunika chisamaliro chambiri chifukwa magwira ntchito zamphamvu kusiyana ndi azimayi Gen11 Bathing and feeding the children are the mother’s responsibility. Kasambitsa ndi kudyetsa ana ndi ntchito ya azimayi Evaluation and Reseach Design Report | 94 AMAA EVALUATION AND RESEARCH (1) (2) (3) (4) (5) Gen12 There are times when a man needs to beat his wife. Pali nthawi zina zoti mzibambo ayenera kumenya mkazi wake Gen13 A mother should tolerate violence from the father in order to keep the family together. Mayi ayenera kupirira nkhanza kuchokera kwa mwamuna wake kuti banja lisapasuke Gen14 A woman’s most important role is to take care of the family and cook Ntchito ya mzimayi yeni yeni ndi yosamala banja ndi ku phika Gen15 A woman should always obey her husband Mkazi ayenera kumvera mwamuna wake nthawi zones Gen16 A woman should have the right to divorce Mkazi ayenera kukhala ndi ufulu wothetsa banja. EDU_LEVEL B Ideally, what level of education do you think a boy should try to reach? Mmene mukuganizira, kodi mnyamata ayenera kuyesetsa kufika pati wanji pa maphunziro. Why? Chifukwa chani? EDU_LEVEL_G And how about girls? Ideally what level of education should they aim for? Nanga atsikana? Mukuganiza kuti azilaka laka kufikira mlingo wanji pa maphunziro? Why? Chifukwa chani? If less than boys, ask why there is a difference Ngati wachepa kwa amuna, funsani chifukwa chani zikusiyana CMPTSCHL_B Do you expect boys in this community to complete that level of education? Kodi mukyembekezera amuna kumaliza mlingo wa maphunziro umenewu? Yes (1) No (2) Don’t know (-98) No response (-99) What percentage of boys will complete Secondary school? Kodi ndi anyamata angati pa makumi khumi angamalize mlingo umenewu? (use more less than half/less than one quarter ,etc. if percentage is a problem) Do you think some of those boys will go to the University/College? How many? A handful? More? Evaluation and Reseach Design Report | 95 AMAA EVALUATION AND RESEARCH Kodi mukuganiza kuti ena mwa anyamata amenewo adzapita ku sukulu ya ukachenjede? Angati? Ochepa? Ambiri? BARRIERS_B If no, what are the main problems/challenges for boys? Ngati ayi, kodi ndi mavuto/zikhomo zikulu zikulu za a anyamata? CMPTSCHL_G Do you expect girls in this community to complete that level of education? Kodi mukiuyembekezera kuti atsikana a mdera lino atha kumaliza mlingo umenewo wamaphunziro? Yes (1) No (2) Don’t know (-98) No response (-99) What percentage of girls will complete Secondary school? Kodi ndi atsikana angati pa makumi khumi angamalize sekondale sukulu? (use more less than half/less than one quarter ,etc. if percentage is a problem) Do you think some of those girls will go to the University/College? How many? A handful? More? Kodi mukuganiza kuti ena mwa atsikana amenewo adzapita ku sukulu ya ukachenjede? Angati? Ochepa? Ambiri? BARRIERS_G If no, what are the main problems/challenges for girls? Ngati ayi, kodi ndi mavuto/zikhomo zikulu zikulu za a atsikana? COMM_EDU_OPINION Do you think that most people in this community would agree with your opinion about ideal levels of education for boys and girls? Would they aspire to higher levels of education? Or lower? kodi mukuganiza kuti anthu ambiri mu dera lino atha kuvomerezana nanu pa maganizo a mlingo wa maphunziro a anyamata ndi atsikana?kodi atha kukhumbira mlingo wapamwamba wamaphunziro? Kapena ocheperapo? JOB_B What types of job or activity do you expect that an average boy of this community who is 12-14 years of age today would be doing at age 25? kodi ndi mtundu wanji wa ntchito kapena zochita zomwe mumayembekezera kuti mnyamata wapakatikati mdera lino wa zaka khumi ndi ziwiri mpaka khumi ndi zinayi panopa, atha kuchita pa zaka 25? JOB_G What types of job or activity do you expect that an average girl of this community who is 12-14 years of age today would be doing at age 25? kodi ndi mtundu wanji wa ntchito kapena zochita zomwe mumayembekezera kuti mtsikana wapakatikati mdera lino wa zaka khumi ndi ziwiri mpaka khumi ndi zinayi panopa, atha kuchita pa zaka 25? MARRIED_G What is an ideal age for a girl to get married? Kodi mtsikana ayenera kukwatiwa ali ndi zaka zingati? MARRIED_B And for a boy? Nanga mnyamata? CHILD Do you think everyone should have children? Kodi mukuganiza kuti aliyense ayenera kukhala ndi mwana? Why? Evaluation and Reseach Design Report | 96 AMAA EVALUATION AND RESEARCH Chifukwa chani? CHILD_M How about the ideal age to have children? For a Man? Nanga zaka zoyenerera kukhala ndi mwana? Kwa mwamuna? Why? Chifukwa chani? CHILD_W And, for a Woman? Nanga kwa mkazi? Why? Chifukwa chani? FAM_SIZE What is the ideal family size? Kodi kukula kwa banja koyenerera ndi chani? Is there anything you would like to add? Muli ndi choonjezera chilichonse? Thank you! Zikomo? Evaluation and Reseach Design Report | 97 AMAA EVALUATION AND RESEARCH Evaluation and Reseach Design Report | 98 AMAA EVALUATION AND RESEARCH Evaluation and Reseach Design Report | 99 AMAA EVALUATION AND RESEARCH ANNEX III: ESTIMATED LOE AND STAFFING PLAN The NORC team will be led by Dr. Alicia Menendez who will serve as the Principal Investigator and Evaluation Economist. Dr. Menendez will provide technical leadership across the entire evaluation and oversee the technical team. Varuni Dayaratna will serve as the Project Director and will be responsible for maintaining clear communication channels between USAID and NORC, as well as ensure that the team follows the project work plan. The rest of the technical team includes Jennifer Schulte, Dr. Alejandro Ome, Letitia Onyango, Gregory Haugan, and Ilse Paniagua. Jennifer Schulte will work with Dr. Menendez on the design of tools, protocols and analyses in particularly on Gender and Gender Based Violence areas. Dr. Ome will assist Dr. Menendez in the project design and advise on analysis phases when necessary. Ms. Onyango will serve as the team’s qualitative analyist and oversight qualitative data collection. Gregory Haugan will assist in the quantitative data analysis. Ilse Paniagua will support fieldwork process and quantitative data collection, including preparation of enumerator training materials and supporting the enumerator training. Team Summary Name of: Staff Consultant Sub￾Contractor Title (MOBIS) Roles and Responsibilities Qualifications (related to responsibilities) Estimated LOE Alicia Menendez Principal Investigator/ Evaluation Economist (RA/Writing and analysis) Provide technical leadership, across the impact evaluation Develop the Design Evaluation Plan Development of questionnaires Supervise data collection activities including data quality monitoring Conduct econometric analysis Preparation of baseline final evaluation reports Presentation of findings to client and stakeholders. Ph.D. in Economics with 20 years of professional experience, including: experience designing and implementing RCTs and quasi￾experimental design impact evaluations of education projects; evaluations including adolescents; SRGBV evaluations; advanced econometric analysis; qualitative methods; survey development; oversight of large data collection efforts. 126 Evaluation and Reseach Design Report | 100 AMAA EVALUATION AND RESEARCH Name of: Staff Consultant Sub￾Contractor Title (MOBIS) Roles and Responsibilities Qualifications (related to responsibilities) Estimated LOE Varuni Dayaratna Project Director (Program Manager) Manage communications between USAID and NORC Manage evaluation timeline and deliverables Advise and provide input on report writing. M.A. in Public Affairs and 20 years of professional experience, including 15 years of experience managing research, evaluation, and technical assistance projects. 11.5 Alejandro Ome Senior Evaluation Advisor (SESPI) Provide guidance and input in the evaluation design development Advise and provide input on analysis and report writing. Ph.D. in Public Policy and 7 years of professional experience, including: experience designing and implementing RCTs and quasi￾experimental design impact evaluations of education projects; advanced econometric analysis including the use of innovative methods to detect impact with small sample sizes; mixed method evaluation design; and strong leadership of education evaluations of various sizes. 2 Evaluation and Reseach Design Report | 101 AMAA EVALUATION AND RESEARCH Name of: Staff Consultant Sub￾Contractor Title (MOBIS) Roles and Responsibilities Qualifications (related to responsibilities) Estimated LOE Letitia Onyango Senior Research Analyst - Qualitative Expert (RA/Writing and analysis) Support qualitative research, including design of the protocols for FGDs Supervise all qualitative data cleaning and analysis including using NVivo software to undertake systematic coding of qualitative data transcripts. Support preparation of final evaluation reports M.S. in International Public Service and 7 years of experience in social science research. 32 Greg Haugan Research Analyst – Quantitative (RA/Data) Lead DQA process Assist in analysis of survey data. Assist in the preparation of final reports. MA in International Affairs, 2 years of professional experience in policy research. 74 Ilse Paniagua Research Assistant (RA/Data) Assist with IRB submission and approval process Assist in the design of quantitative instruments. Prepare enumerator training materials. Lead enumerator training prior to data collection. Oversee quantitative data collection. B.S. in Policy Analysis and Management and two years professional experience in policy research. 43 Jennifer Schulte GBV expert Work on GBV approach, SBV questionnaire and other tools development, child protection protocols. Analysis. 74 ***Please Attach Updated Resumes of all staff (not including names of local data collectors) Evaluation and Reseach Design Report | 102 AMAA EVALUATION AND RESEARCH ANNEX IV: ROLES AND RESPONSIBILITIES MATRIX High Level Functions/Tasks MoE USAIDMalawi USAID DC NORC AMAA EVALUATION DESIGN Draft Evaluation Design Report & Timeline X Review/Input on Evaluation Design Report x x x BASELINE - 2018 Secure local data collection partner X Draft instruments and associated protocols X Review instruments x x x Pretest instruments X Modify instruments as necessary X Obtain necessary IRB/ethics board approvals X Develop and implement enumerator training X Assist with gov’t approvals for data collection x x x Assist with school/community introductions for data collection x x x Field baseline X Field QA for baseline data collection X Analysis of baseline data X Draft Baseline Report X Review/Input on Baseline Report x x x x Finalize Baseline Report X FOLLOW UP 1 – 2019 and 2 - 2020 Draft Instruments and Associated protocols as needed X Review instruments x x x Obtain necessary IRB/ethics board approvals X Modify instruments and pilot as need X Evaluation and Reseach Design Report | 103 AMAA EVALUATION AND RESEARCH High Level Functions/Tasks MoE USAIDMalawi USAID DC NORC AMAA Develop and implement enumerator training X Assist with gov’t approvals for data collection x x x Assist with school/community introductions for data collection x x x Field Follow UP 1 / Follow UP 2 X Field QA for Year 1 / Year 2 data collection X Cleaning and compilation of FU1 and 2 data X ENDLINE – 2021 Draft Instruments and associated protocols as needed X Review instruments x x x Obtain necessary IRB/ethics board approvals X Modify instruments and pilot as need X Develop and implement enumerator training X Assist with gov’t approvals for data collection x x x Assist with school/community introductions for data collection x x x Field Endline x Field QA for Endline data collection x Analysis of Baseline/FU1/FU2 and Endline data x Draft Evaluation Report x Review/Input Evaluation Report x x x x Finalize Evaluation Report x Engage in dissemination and discussion x x x x x Evaluation and Reseach Design Report | 104 AMAA EVALUATION AND RESEARCH ANNEX V: FINALIZED BUDGET See separate document Evaluation and Reseach Design Report | 105 AMAA EVALUATION AND RESEARCH U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Evaluation and Reseach Design Report | 106