MIDTERM IMPACT EVALUATION OF THE REPRODUCTIVE HEALTH IN THE COMMUNITY (REPROSALUD) PROJECT By: Delicia Ferrando Nery Serrano Carlos Pure Submitted by: LTG Associates, Inc. and TvT Associates, Inc. Submitted to: The United States Agency for International Development in Peru Under Contract No. HRN-I-00-99-00002-00, Task Order Number 812 February 2002 Midterm Evaluation of the Reproductive Health in the Community (ReproSalud) Project, was prepared under the auspices of the U.S. Agency for International Development (USAID) under the terms of the Monitoring, Evaluation and Design Support (MEDS) project, Contract No. HRN-I-00-99-00002-00, Task Order No. 812. The opinions expressed herein are those of the authors and do not necessarily reflect the views of LTG Associates, TvT Associates, or USAID. Information about this and other MEDS publications may be obtained from: Monitoring, Evaluation and Design Support (MEDS) Project 1101 Vermont Avenue, N.W., Suite 900 Washington, DC 20005 Phone: (202) 898-0980 Fax: (202) 898-9397 cbillingsley@medsproject.com www.medsproject.com ACRONYMS AND FOREIGN TERMS CBOs Community-based organizations DFID British Department for International Development DHS Demographic and Health Survey (ENDES in Peru) FP Family planning IE&C Information, education and capacity building INEI Instituto Nacional de Estiadísticas e Informática (National Institute for Statistics, Peru) IR Intermediate Results MNISA Ministerio de Salud (Ministry of Health, Peru) OR Odds Ratio ReproSalud Reproductive Health in the Community project SO Strategic Objective USAID United States Agency for International Development CONTENTS ACRONYMS AND FOREIGN TERMS I. Introduction....................................................................................................................1 II. Methodology..................................................................................................................3 A. Evaluation Methodology....................................................................................3 Objectives of the Evaluation........................................................................3 Data Sources ................................................................................................3 Evaluated Indicators.....................................................................................4 Evaluation Sample .......................................................................................5 Population Surveyed ....................................................................................5 B. Methodology of the Base Line and Midterm Evaluation Surveys.....................6 Tools Used for Data Collection...................................................................6 Sample..........................................................................................................7 III. Results..........................................................................................................................17 A. Project Scope and Capacity Building Activities..............................................17 B. Project Impact: Achievement of the Strategic Objective.................................23 C. Intermediate Results.........................................................................................31 D. Summary of the Changes in the Indicators of the Results Framework............57 IV. Project Impact and Results: Communities with Education Activities and a Component of Community Banks Compared to Communities with Education Activities Only............................................................65 A. Project Impact: Achievement of the Strategic Objective.................................66 B. Intermediate Results.........................................................................................72 V. Conclusions..................................................................................................................79 VI. Recommendations........................................................................................................83 ANNEXES A. Documents Reviewed ..................................................................................................87 B. Results Framework ......................................................................................................89 C: Counterpart Communities Versus Control Communities............................................93 D: Indicators of the Base Line and Midterm Evaluation Surveys ....................................95 E: Sample of Communities with Community Banks and with Education Activities Only....................................................................................99 F: Indicator Tables..........................................................................................................101 1 I. INTRODUCTION Towards the middle of 1995, the non governmental organization Manuela Ramos received support from the United States Agency for International Development (USAID) to develop a new, long-term project, Reproductive Health in the Community (ReproSalud), the goal of which is to improve the reproductive health of rural and urban women of limited means and to empower them through adult education strategies and a small income generation component through community banks (with the loan of money) or product development (by providing materials). In the first phase, which ran from 1995 to 2001, the project focused on the provision of information, education and capacity building (IE&C) to women (and a smaller number of men) in matters relating to reproductive health which they themselves prioritized through participatory appraisal and which essentially related to family planning, care during pregnancy and birth, and vaginal discharge and infections of the genital tract, amongst others. Through education, the project aims for women to achieve greater equity in their relationships with men, to have a greater awareness of their anatomy and of that of men, and to value and take care of themselves by making use of the health services. In the second phase (from 2001), with a more political focus, the main objectives are the promotion and defense of the rights of empowered women, such that they will be able to discuss and negotiate - on equal terms - with the ministry of health (MINSA)1 . ReproSalud works with poor women living in isolated rural areas and peripheral urban areas in the departments of Huancavelica, Ancash, La Libertad, Puno Quechua and Aymara and Ayacucho. The project's activities in San Martín, Ucayali and Lima Este came to an end in the year 2000. In each department, following a careful selection process, community-based organizations (CBOs) fulfilling the requirements set out by the project (see section A1 of chapter two of this report) were chosen to become counterparts in the project with a 'winning' CBO. Given that the aim of ReproSalud was to reach as many beneficiaries as possible, the winning CBOs in turn selected 'sister' CBOs in what are termed 'associated' communities. The originality of ReproSalud derives from the strategy of involving the community from the very outset: prioritizing the reproductive health problems which are of greatest concern to them, taking part in capacity building sessions and participating in 1 Moreover, ReproSalud fulfils the CIPD (1994) mandate in at least two ways: Firstly, it aims to reduce the unmet need for family planning and other reproductive health services, using a different approach to that of traditional family planning programs. ReproSalud does not deliver services. It builds the capacity of and empowers women in CBOs such that they themselves may defend improvements in available health services. Secondly, the project, through various strategies including adult education, encourages women to overcome socio-economic, cultural and gender barriers which prevent them from taking decisions and actions to benefit their reproductive health, including an equitable relationship with their partners and a better use of available reproductive health services. Anna-Britt Coe, (2001). MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 2 negotiations with local and health authorities, amongst others. In this way the populations feel and treat the project as their own2 . In its six years of field activities, until December 2001, the project had provided IE&C on reproductive health to some 123,000 women beneficiaries (amongst counterpart and associated communities) and to an additional number of 66,000 men living in the project's area of influence. This is a total of approximately 200,000 beneficiaries, either direct (those living in communities with winning CBOs) or indirect (those living in communities with associated CBOs). Of the total number of beneficiaries, 70 percent live in rural, mainly mountainous, areas, and 23 percent live in urban areas, in the periphery of cities of the departments where ReproSalud is working. Several documents (see annex A) record the project results. This impact evaluation is centered on the analysis of quantitative achievements, by studying the indicators of the Results Framework for the Strategic Objective (SO) and for the Intermediate Results (IR) separated into the various issues (see annex B) on which an impact is expected. It is based on data collected by the project’s Monitoring and Evaluation Unit prior to the start of the project and after a minimum of two years of work. Complementary information from the project’s own monitoring system and from external sources was also added to put the findings into context. Only part of the information available was used in this evaluation, specifically that relating to 25 counterpart communities with a control community to compare changes in the indicators and link them to the project. Reasons relating to the design of the sample and the development of the instruments used for data collection (both done in a gradual manner) resulted in the number of indicators analyzed being reduced from 46 to 33, although a further 6 were defined on this occasion from available data and added to the other 33. This evaluation is part of a more detailed evaluation that includes the project's process evaluation and its cost-effectiveness. 2 For the Process Evaluation, which forms a part of the Midterm Global Evaluation of ReproSalud which also includes this Impact Evaluation, approximately 50 women from 12 communities in 5 departments were interviewed and expressed positive views on the project and the usefulness of what they had learned through capacity building for living their lives and for their relationships with their partners and families. (Reports were collected personally during fieldwork carried out between October and November 2001 in Ancash, Huancavelica and La Libertad). 3 II. METHODOLOGY A. EVALUATION METHODOLOGY Objective of the Evaluation General Objective To determine, pursuant to the Results Framework, the impact of ReproSalud on the target communities after a minimum period of two years from the start of IE&C activities in reproductive health and gender, and of income generating projects in the form of community banks. Specific Objectives ß To compare the change in the indicators selected from the Results Framework in a sample of target communities and a sample of control communities. ß To analyze possible additional project impact on the indicators of the Results Framework if an income generation component, in this case community banks, is implemented in addition to the IE&C component. Data Sources Base Line and Midterm Evaluation Surveys The Technical Monitoring and Evaluation Unit of the project carried out surveys (a base line survey at the beginning of the project, between 1997 and 1999; and a midterm evaluation survey after a minimum of two years, between 2000 and 2001) in a sample of women of reproductive age and of men aged between 15 and 59 from communities with counterpart (or winning) CBOs, a sample from communities with an associated CBO (where activities are carried out indirectly through the winning CBO) and a sample from comparison or control CBOs. The information gathered is varied and abundant, but for the purposes of this evaluation only that relating to communities with CBO counterparts, for which a control community was also surveyed has been used. Project Monitoring Information The project keeps track of its activities by means of a data information system that is managed at the project's head office in Lima and allows progress in each department to be monitored through the information sent from the branch offices. This information was used without evaluation, on the assumption that the information provided by the system is MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 4 of a good quality. The total number of communities in the project, their location, total population, ranking of reproductive health issues by the women, as well as number of beneficiaries, both women and men, amongst others, were taken into account. Project Related Documents Additional information to that provided by the mentioned surveys was obtained from other project related documents, such as annual reports, evaluation reports, reports of field visits and supervisory reports. Information was also obtained from interviews with various actors taking part in the project both within the communities and at the project head office in Lima, as well as with persons carrying out the surveys and processing the information. Official Documents on Reproductive Health and Population Official documents prepared by the National Institute for Statistics and Information were used to place the findings of this evaluation into context; amongst them, the reports in the DHS (in all four of versions), country and district population projections, and the national census of 1993. Evaluated Indicators In accordance with the project objectives, the education component should result in an improvement in preventive health practices by women, including a greater readiness to attend health centers. The advocacy component should result in an increase in the negotiating capacity of women with health authorities, which it is expected will in turn result in an improved quality of attention and therefore in greater use of the services by women. The above is represented in the Results Framework, which contains the SO of the project and the IRs with their respective indicators (see annex B). These are the key elements for evaluating achievements. Thus, achievement of the project objective is measured by the SO indicators which also measure the impact of the project; the achievements of the process are measured by the IR indicators. Whilst reviewing the information collected and prior to its analysis, six new indicators were developed (see table 1) and tentatively numbered following the classification in the Results Framework, bearing in mind whether they related to the SO or to the IRs. METHODOLOGY 5 Table 1: Evaluation Indicators Number Indicator SO: Women increase the use of reproductive health check-ups 1.a percent of women who have had 4 or more check-ups by health personnel during the last pregnancy occurring two years prior to the survey IR 1: Women have more equitable gender relations with their partners and their families 11.a percent of women in a relationship who have spoken with their partner regarding the number of children they wish to have IR 1.1: Women strengthen their capacity to bring about changes in their gender relations 16.a percent of women who know where to turn for help or advice in the event of physical aggression IR 2: Women have a greater capacity to access reproductive health services 24. percent of women who have attended a health center for issues of reproductive health or family planning IR 2.6: Women increase their knowledge regarding their reproductive health needs 43.a percent of women who have heard of Papanicolau or breast examination 43.b percent of women who believe that it is difficult to get pregnant while breastfeeding Evaluation Sample The Monitoring and Evaluation unit of ReproSalud started its activities towards the end of 1997 and decided to carry out a base line survey in all (rather than in a sample of) the communities targeted to that date. Some of these communities were therefore surveyed when the project was already ongoing. As new communities were added to the project, it was decided that, given the costs involved, it was better to carry out surveys in a sample of 30 communities with counterpart CBOs for which control communities were set. Out of these 30 communities, 5 were dropped after the first subproject3 , leaving only 25 counterpart communities with their respective control communities. This evaluation is based on the data from such communities. All the departments, with the exception of Lima, are represented in the sample. Prior to analysis of the indicators, the database was purged and weighted (to represent the universe of target communities bearing in mind their distribution by department, by urban and rural areas, and by age) for the purpose of obtaining consistent indicators. The process used is explained in section II.B of this report. Population Surveyed Prior to the start of the project, information was gathered relating to women of reproductive age and of men aged between 15 and 59 from a sample of households in communities with counterpart CBOs and in control communities. Surveys were also carried out in associated communities, but they are not analyzed in this report. The same households were surveyed in the midterm evaluation, and all the women of reproductive 3 a subproject is a set of capacity-building sessions on an issue of reproductive health, with an approximate duration of 8 to 10 months. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 6 age and men in the mentioned age group who were in the house at the time were interviewed, rather than only those who had been interviewed two or three years earlier in the base line survey. This was done on the principle that the project should have an impact not only on the beneficiary population, but also on the community as a whole. There appears to be a great territorial mobility in the population, affecting mainly rural areas, and the communities in which ReproSalud works are no exception. Indeed, of the 3858 women of reproductive age interviewed in the midterm evaluation, 2612 had also been interviewed in the base line survey. This means that 32.3 percent of the original sample was replaced by new residents of the households and, obviously, of the community. This factor must be considered when analyzing the change in indicators between the two surveys. B. METHODOLOGY OF THE BASE LINE AND MIDTERM EVALUATION SURVEYS Tools Used for Data Collection ß Household questionnaire to collect general information regarding the persons living in the household and the main characteristics of the home. ß Individual questionnaire for women of reproductive age (between 15 and 49 years) living in the selected household, to collect information on their demographic characteristics, attitude and behavior in relation to reproductive health, gender and family relationship issues, empowerment, participation in community based organizations and health care issues (e.g., awareness of the fertile period, awareness of symptoms of risk during pregnancy, number of prenatal check-ups, attendance at health centers, expenditure on health care, etc.). ß Individual questionnaire for men between the ages of 15 and 59 to collect information regarding their knowledge, attitude and practices on reproductive health, gender relationships, family relationships, work, etc. ß Area questionnaire to collect data regarding the population center and the area of influence of the community based organization participating in the project. ß Questionnaire for the head of the health center. ß Questionnaire for a member of the community who is also a member of the Local Health Administration Committee. ß Questionnaire for a community leader not a member of the Local Health Administration Committee. METHODOLOGY 7 This evaluation, the specific objective of which is to analyze project impact and the change in the indicators of the Results Framework before and after project implementation, used only the information collected in the household questionnaire and in the individual questionnaires for men and women4 . A few comments should be made regarding the development of the individual questionnaires for women and men. As the definition of the sample framework was gradual, so was the development of the final version of these two tools. Indeed, nine questions were added to the first version, as the need for new indicators emerged, and so on thereafter. Some indicators were in fact defined at the start, but for the most part they were developed after the first version as a natural consequence of the gradual definition of the project5 . New implementation issues and new target communities emerged, and with these the need to measure them by means of indicators. This resulted in the questionna ires being modified up to 14 times throughout the field work, such that one same questionnaire was not used for all the communities in the 2 surveys. Questions added at a later stage were applied to few of the communities in the midterm evaluation, and thus corresponding information for the base line survey is not available and cannot be compared6 . As a result, it was decided in this impact evaluation to: ß Limit data collection to 25 communities with counterpart CBOs, which also have control communities, where the most complete version of the individual questionnaires was used. This greatly reduced the number of cases, and it was therefore not possible to obtain reliable data by department. ß Exclude associated communities from the evaluation, as they were very few in number and different versions of the individual questionnaires had been used. ß Reduce the number of indicators of the Results Framework from 46 to 33, eliminating those for which no base line information is available. Sample The communities making up the sample have in common the existence of one or more CBOs—groups which bring together women to carry out activities beneficial both to the community and to themselves. 4 The information collected with the remaining tools described in section II.B of this report is valuable and varied. It should be analyzed and published in research reports to share the various aspects of the work and achievements of ReproSalud within the community with other institutions and development agencies. 5 It took some time for ReproSalud to be consolidated, given that it is a new experience not only as regards methodology used and working strategies, but also given the fact that it works with remote rural communities. 6 However, such data could be of use in the project’s final evaluation. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 8 The sample communities were classified in the following categories: a) 70 communities taking part in ReproSalud in which one CBO implements a reproductive health education sub-project and/or an income generation (community bank or product development) subproject after having been selected through tender by the project. They receive funding for their activities and technical support directly from ReproSalud; b) 17 associated communities, selected by the winning CBOs to also benefit from the project, but through an intermediary community; and c) 25 control communities where the project was not active and which are used to measure the impact and the results of the project. Although details of these communities are given further on (section III.A), it should be pointed out that those working with ReproSalud were not selected randomly, but pursuant to certain criteria which set them apart from the other communities in the district. They were communities with better-organized CBOs, and probably with experience in development projects, although, judging from their poverty indicators, poorer than the control communities. This should be borne in mind when analyzing the results. Sample Framework This is formed by the communities, which were selected for participation in the project in 1997, 1998, and 1999. As CBOs were selected to take part, the communities in which they were based became part of the sample framework. This means that there was no predefined sample framework, but rather that it was gradually established as the project progressed, starting in 1996 as shown in table 2. Most of the communities (55 percent) were selected between the years 1998 and 1999. Table 2: Communities Selected to Participate in ReproSalud Year of Selection Department Total 1996 1997 1998 1999 2000 Total 247 20 43 52 82 50 Percent 100.0 8.1 17.4 21.1 33.2 20.2 Ancash 31 2 7 9 12 1 Ayacucho 35 2 8 6 14 5 Huancavelica 39 2 6 7 17 7 La Libertad 36 2 5 6 13 10 Lima Este 8 1 3 3 1 0 Puno Quechua 33 2 5 4 10 12 Puno Aymara 35 2 3 7 8 15 San Martín 15 2 4 7 2 0 Ucayali 15 5 2 3 5 0 Sample Design The sample has the following characteristics: ß Gradual selection of the analysis unit for the sample (communities) in accordance with the gradual definition of the sample framework. METHODOLOGY 9 ß Given the gradual selection of the communities, the base line survey was carried out in a period spanning years rather than months; the same is true for the midterm evaluation. ß Within the community, households of CBO members and households of nonmembers were selected randomly following a selection technique that is explained in section II.B of this report. ß Within selected households, all women of reproductive age and all men aged between 15 and 59 who were regular residents of the household were interviewed. ß In the midterm evaluation only communities which had received capacity building on at least two issues of reproductive health, i.e. which had carried out two sub-projects in a period of two or three years, were interviewed. ß No new sample was defined for the midterm evaluation: the same households as in the base line survey were visited and all women of reproductive age and men aged between 15 and 59 who were living in such households were interviewed. Sample Selection As has already been mentioned, the communities were included in the sample in a gradual manner, reaching a total of 70 communities with counterpart CBOs in a period of three years. The greater part of the sample (94.3 percent) was selected between 1997 and 1998 (although selection covered the period 1997 to 19997 ). This bears no relation to the years in which a larger number of communities were included in the project, and which, as shown in table 3, was between 1998 and 1999. Only 4 communities were included in the sample in 1999 (5.7 percent), whereas 82 (33 percent) were added to the 115, which were already part of the project. Table 3: Communities Selected for the Project and for the Sample Per Year Communities Total 1996 1997 1998 1999 2000 Total counterpart communities 247 20 43 52 82 50 Percent 100.0 8.1 17.4 21.1 33.2 20.2 Communities in the total sample 70 40 26 4 Percent 100.0 57.2 37.1 5.7 Communities in the sample for this evaluation 25 21 4 Percent 100.0 86.7 13.3 7 The communities included in the project in 1996 and surveyed in the base line after intervention (a total of 36 communities) were excluded from the sample. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 10 Collection of information in control and associated communities began in 1998. In this three-year period in which the sample was selected, both the criteria for selecting counterpart communities and the strategies for project implementation may have changed, such that the communities surveyed more recently have the benefit of the experience accumulated in previous years, i.e., since 1996. This could affect the results of the project, which might be better for such communities, and therefore alter the comparability of the data. It was therefore decided to carry out this evaluation with data from a sub-sample of communities interviewed for the base line in 1998 and 1999 (see table 4). Given that intervention was gradual, all departments have different numbers of communities selected for the sample (see table 4). This means that the n sample by department (the number of communities in the sample per department) is not proportional to the N of intervention (number of communities with winning CBOs in the ReproSalud project area). This lack of a relationship, which is vital for the data to be truly representative, was solved by weighting the number of communities interviewed by department with the corresponding proportion of total counterpart communities in the project to December 1999. None of communities selected for the project since 2000 are part of the sample framework and thus neither are they part of the sample. Table 4: Counterpart Communities Selected Between 1997 and 1999 Versus Communities in the Sample in 1998 and 1999 Counterpart Communities Department Sample Framework* Sample* Number Percent Number Percent Total 170 100.0 25 100.0 Ancash 28 16.5 4 16.0 Ayacucho 28 16.5 3 12.0 Huancavelica 30 17.6 5 20.0 La Libertad 24 14.1 2 8.0 Puno Quechua 19 11.2 2 8.0 Puno Aymara 18 10.6 4 16.0 San Martín 13 7.6 2 8.0 Ucayali 10 5.9 3 12.0 *excludes Lima East Selection of Households This comprised two groups: 1. Households of (women) members of CBOs In this case the size of the sample was determined with the formula of finite populations, since the number of members was known. In general, where CBOs had less than 40 members, all of them were interviewed. Where the CBOs were very small, there were few interviews: between 5 and 20 members, the only ones of METHODOLOGY 11 reproductive age 8 . By contrast, in large CBOs there were more the 80 interviews9 . On average, members interviewed represented 50 percent of the total number of women interviewed, with the exception of La Libertad, where the average was 70 percent, and Ancash, where it was 58 percent. At this stage, households were selected as follows: ß The number of active members10 of the CBO was determined on the basis of information provided by the president of the organization. ß Members who were under 15 and over 49 years of age were not interviewed. ß If, after discarding members outside the 15 to 49 age group, the number remaining was 40 or less, all of them were interviewed. If there were more than 40, the following formula11 was used to calculate the size of the sample: ( / 1) 1 1 / 2 2 2 2 + - = t PQ d N t PQ d n , where n = estimated size of the sample t = distribution point related to a reliability level of 95 percent (1.96) d = reliability interval (.10 ) N = size of the population (160) p = value of the indicator to be calculated ß Once the size of the sample was known, a random selection of member households was carried until the number to be interviewed according to the preceding formula had been reached. The following were not interviewed: ß households of members who were absent during the interview period, and ß households of members living in another community, who do not participate in the organization and only receive foodstuffs. 8 For example, in the Mothers' Group Antonieta Chu of CCPP Poloponta in the Zapatero district, Lamas province in the department of San Martín. Similarly, in the Mothers' Group Virgen de las Mercedes, of CCPP Huancha, San Marcos district in the province of Huari, department of Ancash, where 13 members were interviewed in the base line and nine in the midterm evaluation, this being the number of members of reproductive age. Likewise in the Mothers' Group Virgen del Rosario, of CCPP Ccachaccara, María Parado district of Bellido, Cangallo province in Ayacucho, where 21 members were interviewed in the base line and 16 in the midterm evaluation, this being the number of members of reproductive age in the CBO. 9 For example in Vaso de Leche San Bernardo of the Acopalca community, Huari district, department of Ancash, where the number of interviews was 83 both in the base line and in the midterm evaluation. In the same department, in the Comité del Vaso de Leche Marcará of the CCPP Marcará, Marcará district, in the province of Carhuaz, where 72 members were interviewed in both surveys. 10 those who attend meetings and take part in the activities of the CBO. 11 Cochran, W. (1963). Sampling Techniques. NY: Wiley p. 75. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 12 2. Households of non-CBO members This was done in two different ways. In rural communities where it was possible to count the number of households the formula of finite populations was applied. Households were randomly selected. In urban communities a neighbor household to the member household was interviewed, and the sample was also randomly selected. The procedure was as follows: In small communities all households were counted and registered, taking care not to count any twice or to leave any out. From the register obtained sample households were selected as follows: 1. Occupation of the house was verified. If it was not occupied it was eliminated from the register. 2. In occupied households the persons living in it were identified. If a member of a CBO was living in the household it was also discarded, given that member households were selected separately. 3. The remaining households were numbered and those that were to form part of the sample were then randomly selected. The steps and criteria followed were the same as for selection of member households. The same formula was used. 4. In large urban communities 30 households located close to the CBO member household selected for interview were chosen. There was no set criteria for selection, and this varied according to the location of the member households. In some cases the household next to the member household was interviewed, in others the interviews were carried out in neighboring areas to that of the member household, and in others again the households of relatives of members were interviewed, since there was a greater probability that they would talk about what they had learned with their relatives. The sample of men was taken from households of CBO members and non-members of the community. In each selected household all men between the ages of 15 and 59 were interviewed. Weighting The data collected were weighed or weighted following three criteria: ß The distribution by urban and rural areas of counterpart communities within each department. METHODOLOGY 13 ß The proportion of counterpart communities in the department in relation to the to total number of counterpart communities within the project area. ß The distribution of the population by age in order for the sample to represent the structure of such communities as per the census of 1993, confirmed by the monitoring information of ReproSalud on the total beneficiaries in counterpart and associated communities. T he sample, which initially did not have the same ‘shape’ (see figure 1) as such sources, was matched to these, giving the structure shown in figure 2, and thereby correcting the age bias, which had apparently been produced on selecting the sample. The same procedure was used for women and men. Figure 1: Age Structure in Sample Communities Figure 2: Age Structure in Sample Communities: Matched Sample Process Used for Sample Weighting Weight for each case in any urban area region: [ ( NurbRegint * Ntotsample ) / ( NurbRegsample * Ntotint ) ] * PAi -30 -20 -10 0 10 20 30 15 - 19 20 -24 25 -29 30 -34 35 -39 40 - 44 45 - 49 Control Counterpart 30 20 10 0 10 20 30 Sample of communities with counterpart CBOs and control communities -30 -20 -10 0 10 20 30 15 - 19 20 -24 25 -29 30 -34 35 -39 40 - 44 45 - 49 Census Beneficiaries 30 20 10 0 10 20 30 1993 Census and total Reprosalud beneficiaries MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 14 Weight for each case in any rural area region: [ ( NrurRegint * Ntotsample) / ( NrurRegsample * Ntotint ) ] *PAi Where: NurbRegint: Number of urban communities participating in ReproSalud in the department Ntotsample: Number of communities selected for the sample NurbRegsample: Number of urban area communities selected for the sample in the department Ntotint: Total number of communities participating in ReproSalud NrurRegint: Number of rural communities participating in ReproSalud in the region NrurRegsample: Number of rural communities selected for the sample in the department PAi: Population age index for correcting the structure Table 5 shows the number of cases of women interviewed, both weighted and unweighted, by department after applying the above formulas. Table 5: Counterpart Communities: Weighted and Unweighted Sample of Women in the Base Line and Midterm Evaluation Surveys Distribution of Cases by Percentage Departments Base Line Midterm evaluation Unweighted Weighted Unweighted Weighted Total 100.0 100.0 100.0 100.0 Ancash 17.3 16.4 19.8 18.2 Ayacucho 11.9 15.4 11.3 14.0 Huancavelica 16.3 12.3 15.6 12.3 La Libertad 5.9 8.8 5.9 8.9 Puno Aymara 20.4 17.0 18.7 15.7 San Martín 6.7 6.7 7.9 8.0 Ucayali 17.2 16.2 16.6 15.9 Puno Quechua 4.5 7.1 4.2 6.9 Control Communities The decision to use controls was taken in mid-1998, and from then onwards a control community in which base line and midterm evaluation surveys were also carried out was selected for each of the communities participating in the project. A control community is a community with no project intervention. It is a similar community selected in a somewhat experimental manner. As far as possible, it should have the same characteristics as participating communities: same height above sea level, literacy rate, population size, degree of urbanization, poverty level and an active community based organization. It appeared to be difficult to find similar communities, as shown in annex C. Indicators derived from the population census of 1993 (lacking a METHODOLOGY 15 more recent source) for the districts (for it is likewise not possible to obtain information by community) where ReproSalud communities and control communities were selected reveal differences, which are sometimes in favor of the counterpart communities and other times in favor of control communities. On the whole, it seems that the latter are less poor, less rural, and have a higher literacy rate than counterpart communities. For this reason, initial values for the Results Framework indicators are at times different from the values for the counterpart communities. These differences should be borne in mind when interpreting the results, since they could act to increase or decrease net gains in the indicators for each community, or affect the interpretation of the findings. Table 6 shows examples of counterpart communities and their respective control community as a way of illustrating the differences between the two. Table 6: Examples of Counterpart Communities and Corresponding Control Communities COMMUNITIES COUNTERPART CONTROL Department Province District Community Population Illiteracy Percent Rural Population Percent Percent with an Unmet Basic Need Department Province District Community Population Illiteracy Percent Rural Population Percent Percent with an Unmet Basic Need Ancash/Carhuaz Ancash/Carhuaz Marcará Marcará 7,704 64.3 87.1 60.9 Tinco Tinco 2,514 38.3 68.3 68.3 Ayacucho/Cangallo Ayacucho/Huanta Cangallo Incaraccay 6,193 52.4 67.2 84.9 Santillana Arahuay 6,395 73.4 88.2 80.3 Huancavelica/Huancavelica Huancavelica/Tayacaja Huancavelica Barrio Santa Ana 36,826 24.3 15.9 75.5 Pampas Pampas 9,649 36.1 49.4 61.7 La Libertad/Otuzco La Libertad/Sanchez Carrión Usquil Chuquizon￾guillo 24,203 38.9 91.7 75.1 Huamachuco Chuquizon￾guillo 37,708 47.0 53.1 77.2 Puno Quechua/Azangaro Puno Quechua/Puno Arapa Pucamoco 10,757 41.5 93.0 87.4 Coata Sucasco￾Tarizani 6,301 39.7 93.3 73.1 Puno Aymara/Yunguyo Puno Aymara/Puno Yunguyo Machacmarca 30,360 43.6 70.3 56.8 Acora Chanchilla 29,420 36.9 93.0 67.5 San Martin/El Dorado San Martin/Picota Shatoja Shatoja 1,653 33.0 52.1 79.8 Tingo de Ponasa Leoncio Prado 2,605 11.8 87.0 89.1 Ucayali/Coronel Portillo Ucayali/Atalaya Masisea Masisea 12,083 19.5 84.0 92.2 Sepahua Sepahua 3,698 26.5 59.8 87.0 MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 16 Sample Size The number of cases interviewed, unweighted, and used in the analysis of the results presented in section III of this report are shown in table 7. Table 7: Sample Size Base Line Midterm evaluation Community Community Counterpart Control Counterpart Control Women Men Women Men Women Men Women Men 2,132 1,649 1,967 1,543 1,852 1,691 1,598 1,502 The characteristics of the sample by level of education, marital status, age and number of children, in the case of women, is shown in table 8. Table 8: Sample Characteristics Women Men Variable Counterpart Control Counterpart Control Education 100.0 100.0 100.0 100.0 Uneducated 16.8 14.0 4.1 3.4 Primary 44.8 44.6 33.1 35.7 Secondary 33.1 36.4 53.6 51.9 Higher 5.3 5.0 9.2 9.0 Marital status 100.0 100.0 100.0 100.0 Single 33.2 26.7 45.1 44.2 Married 61.3 65.9 53.0 53.5 Divorcee/widow 5.5 7.4 1.9 2.3 Number of children 100.0 100.0 na na None 28.2 23.9 1 child 15.0 15.4 2 children 13.5 13.2 3 children 10.5 13.9 4 or more 32.8 33.6 na=not applicable 17 III. RESULTS This part of the report presents the project results measured by means of indicators from the ReproSalud information system at the head office in Lima and indicators from the Results Framework, calculated pursuant to information collected in the two household surveys carried out prior to the start of project activities (base line) and after two or three years (midterm evaluation). A. PROJECT SCOPE AND CAPACITY-BUILDING ACTIVITIES: INFORMATION FROM THE REPROSALUD MACRO INFORMATION SYSTEM Project Scope Pursuant to its design, ReproSalud began its activities by selecting districts in 9 departments of the country12 based on the following criteria: ß unmet basic needs, ß overall birth rate, ß population size and percentage of rural population, ß accessibility of the area, ß existence of community based organizations driven by and made up of women, and ß degree of security for implementing project activities, namely the absence of terrorism and/or drugs-related activities. In each of the districts a general call for participating communities was made; however, selection was not random, since required criteria introduced from the start a certain degree of intentional selection. To participate in ReproSalud it was not sufficient for the communities to have a high level of unmet basic needs or for the average number of children per family to be high: they also had to be accessible, since the project aimed to maximize resources (achieve a balanced cost/benefit) and reach a larger number of beneficiaries (women and men), which automatically excluded the more remote communities. Moreover, the communities which applied in response to the call were possibly those with better established community organizations and those which had very probably 12 Lima East, Ayacucho, Ancash, Puno Quechua, Puno Aymara, Huancavelica, La Libertad, San Martín and Ucayali MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 18 taken part in other development projects (whether of other non-governmental organizations or of the Peruvian government). These factors differentiated them from other communities in the district and resulted in a process of self-selection totally outside the control of the project management. Although poverty and population indicators date back to 1993, it is true to state that the project works with very poor and remote communities, thereby meeting the objective of reaching the most deprived groups. Counterpart communities, i.e, direct beneficiaries that were to receive funding for education and/or income generation activities, were selected following a detailed evaluation carried out by a Committee comprised not only by the women themselves but also by local authorities, personnel from the Health Directorates and personnel from ReproSalud. Given that the project aimed to reach through its activities a larger population than merely the women of communities with a winning CBO, each of them in turn selected a certain number of 'sister' communities, called associated communities, which would indirectly benefit from the education and information activities on reproductive health and gender. From the start of the project and up to the end of the year 2000, ReproSalud has worked with a total of 247 counterpart communities, 80 percent of which are located in a rural area. This same percentage of distribution by area is true for the 2,528 associated communities, as shown in table 9. Table 9: CBOs Participating in ReproSalud, by Type of Participation: 1996–2000 Communities Department Total Counterpart Associated Total Urban Rural Total Urban Rural Total Urban Rural Total 2,775 554 2,221 247 67 180 2,528 487 2,041 Percent 100.0 20.0 80.0 100.0 27.1 72.9 100.0 19.3 80.7 Puno Aymara 524 30 494 35 4 31 489 26 463 La Libertad 484 97 387 36 9 27 448 88 360 Ayacucho 440 99 341 35 7 28 405 92 313 Huancavelica 422 62 360 39 8 31 383 54 329 Puno Quechua 249 52 197 33 6 27 216 46 170 Ancash 222 16 206 31 6 25 191 10 181 San Martin 183 73 110 15 12 3 168 61 107 Ucayali 174 55 119 15 8 7 159 47 112 Lima East 77 70 7 8 7 1 69 63 6 By department, although Huancavelica has the largest number of communities with counterpart CBOs (39), it does not have the largest total number of communities when associated communities are included. Puno, in the Aymara area, has 524 communities (versus 422 in Huancavelica), because 480 associated communities were added to its 35 counterpart communities. In Puno, each counterpart community has an average of 12 associated communities; whereas in the rest of the departments the average is 10. RESULTS 19 Ayacucho and La Libertad are other departments with a total of more than 400 communities participating in the project, and, with the exception of Lima East where there were only 77 participating communities, the jungle departments San Martín and Ucayali have a smaller number of participating communities, whether counterpart or associated. The number of beneficiaries, women and men, also varies by department, as shown in table 10. In its six years of activity, ReproSalud has offered capacity building to some 200,000 persons, both women and men, of which 124,000 are women. Table 10: Beneficiary Population of the Project and Total Population of Participating Districts 2001 population of districts participating in ReproSalud Women beneficiaries Total beneficiaries 1 2 3 4 5 6 7 8 9 Department Total Urban Rural Total Urban Rural Total Urban Rural Total 100.0 67.0 33.0 100.0 31.1 68.9 100.0 23.0 77.0 ( population ) 2,667,474 123,917 190,287 Ayacucho 100.0 41.3 58.7 100.0 26.9 73.1 100.0 23.3 76.7 141,154 17,470 26,293 Huancavelica 100.0 53.7 46.3 100.0 19.9 80.1 100.0 18.5 81.5 177,315 18,580 30,187 Ancash 100.0 38.2 61.8 100.0 14.4 85.6 100.0 13.4 86.6 86,822 15,273 23,131 Puno-Quechua 100.0 76.2 23.8 100.0 22.2 77.8 100.0 22.0 78.0 130,929 17,962 31,221 Lima East 100.0 99.4 0.6 100.0 93.3 6.7 100.0 93.0 7.0 1,309,158 3,266 3,707 Ucayali 100.0 91.5 8.5 100.0 46.7 53.3 100.0 45.4 54.6 354,321 6,429 8,180 Puno-Aymara 100.0 17.0 83.0 100.0 8.0 92.0 100.0 7.0 93.0 212,910 19,071 29,111 San Martín 100.0 91.8 8.2 100.0 54.4 45.6 100.0 54.0 46.0 73,218 7,541 12,105 La Libertad 100.0 42.7 57.3 100.0 23.2 76.8 100.0 24.4 75.6 181,287 18,325 26,352 Considering participating districts by department to December 2000, ReproSalud has benefited 18 percent of the population of participating districts in Ancash, 14 percent in Puno Quechua, 12 percent in Ayacucho, and 10 percent in Huancavelica and San Martín, as shown in figure 2. The distribution of beneficiaries by area was provided by the project itself. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 20 Figure 2: Percentage of ReproSalud Beneficiary Population as a Proportion of Total Population of the Participating Districts Of total beneficiaries, more than 190,000, most are in Puno Quechua, Huancavelica and Puno Aymara, in descending order, each having just over 16 percent; Lima East has the smallest number of beneficiaries, as do the jungle departments as shown in figure 3. Figure 3: Total Project Beneficiaries In order to determine the potential beneficiary population of the project if in an expansion phase it were to cover the total jurisdiction of participating districts, a sample exercise 17.6% 13.7% 12.4% 10.5% 10.3% 10.1% 9.0% 1.8% 0.2% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Ancash Puno Quechua Ayacucho Huancavelica San Martin La Libertad Puno Aymara Ucayali Lima East Beneficiary population Ayacucho 14% Huancavelica 16% Ancash 12% Puno Quechua 16% Puno Aymara 15% Lima East 2% Ucayali 4% San Martin 7% La Libertad 14% RESULTS 21 was carried out. The projected population as calculated by the INEI13 for these districts was taken and distributed into urban and rural areas, in accordance with the Population Census of 1993 (Table 10, columns 2 and 3), as a more recent source giving such data is not available. In any case, the bias in applying this to projected data for 2001 would underestimate the percentage of urban population, given that in the period 1993-2001 the urban population grew from 70 to 72 percent (INEI, 2001) and many small villages and settlements became part of the urban population. In the event that the project were to expand to the total jurisdiction of all participating districts, ReproSalud would reach 2.7 million people, i.e. 10 percent of the national population (estimated at 26.3 million). If this were the case, 67 percent of all beneficiaries would be urban beneficiaries. This would not be in accordance with the target population of the project, which is rural population of limited means, as shown by the results to date. From the total number of current women beneficiaries (123,917) only 31 percent are from urban areas, the great majority being from rural areas. Similarly, of the total number of beneficiaries (almost 200,000 including men) just over one fifth are from urban areas (see figure 4). Figure 4: Distribution of Beneficiary Population in Urban and Rural Areas Participatory Appraisal Versus Capacity-Building Activities ReproSalud worked through the modality of sub-projects consisting of IE&C (or income￾generating) activities on an issue or issues of reproductive health which had been ranked in order of importance by participatory appraisal and which were to be given priority by the project. Capacity building was strengthened by the work of facilitators (a total of 8,424: 5,580 women and 2,844 men) with health workers from the health centers, who were requested to provide better care for women, whilst women were encouraged to attend the health center for receiving professional care. 13 National Institute for Statistics, Technical Directorate for Population. Special Bulletin 14. Population projection by departments, provinces and districts: 1990-2020. July 2001. Internal document, not published. Urban Rural 69% 31% Women beneficiaries Urban Rural 77% 23% Total beneficiaries MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 22 Various reproductive health problems were mentioned in the participatory appraisal sessions, the most important being vaginal discharges, or ‘white periods’ as it is called by women. This was ranked as the most serious problem by 37 percent of all the communities participating in the project. This was especially so in Ucayali, where it was ranked first in 80 percent of appraisals, and in La Libertad, where it was also placed top of the list by 62 percent of CBOs (see table 11). Table 11: Issues Mentioned in Participatory Appraisals Department Total Discharge Many children Pain during delivery Prolapse Inflammation of the ovaries Critical age Ovarian and uterine cancer Physical abuse Retention of the placenta Bowel inflammation Others Total Percent 100.0 36.6 31.9 12.8 3.1 2.7 2.3 1.9 1.2 1.2 1.2 5.1 Total 257 94 82 33 8 7 6 5 3 3 3 13 Ayacucho 35 14 12 4 3 1 1 Huancavelica 46 11 11 11 2 1 4 1 1 2 2 0 Ancash 27 12 7 4 1 1 2 Puno-Quechua 36 9 20 5 2 Lima East 9 2 4 1 2 Ucayali 15 12 1 1 1 Puno-Aymara 36 6 19 7 3 1 San Martin 19 7 2 4 1 5 La Libertad 34 21 6 1 2 2 2 The second most important problem of reproductive health as ranked by women in counterpart communities was the high birth rate, expressed as ‘many children.’ Almost a third of the women (32 percent) mentioned this. Indeed, it is the most important problem in the department of Puno in the two Quechua areas (in 56 percent of the communities) and in Aymara (53 percent), as well as in Ayacucho and Huancavelica for more than a third of the communities carrying out a participatory appraisal. Pain during childbirth, complications in childbirth and problems relating to childbirth are the third most important issues of reproductive health affecting the population of reproductive age, according to 13 percent of the communities participating in the project. This is particularly so in Huancavelica, where it is considered serious by a quarter of the communities, which place it top of the list of all the problems affecting women's health, as well as in Puno Aymara and Ancash. In this ranking of reproductive health problems, other issues were also classified as pressing, although by much smaller percentages than the three problems mentioned above. Amongst these are, in descending order of importance ß prolapse or rupturing of the uterus, ß inflammation of the ovaries, ß citical age, ß ovarian cancer and cancer of the neck of the uterus, ß physical abuse, RESULTS 23 ß retention of the placenta, ß bowel inflammation, ß miscarriage, ß death in childbirth, ß infidelity, ß abandonment, ß problems in the use of contraceptive methods, ß teenage pregnancies, ß problems following childbirth, ß complications in pregnancy, and ß vaginitis. Educational material was developed for training, and as was evidenced on comparing the ranking of issues during participatory appraisal with those touched in the training, there was a perfect match between the needs of the women and the training afforded in most cases. There was only one exception in a community in Ucayali where, through the decision of the women, training was given on discharges instead of on physical abuse (the issue chosen during participatory appraisal) to avoid conflicts with partners. B. PROJECT IMPACT: MEASURING ACHIEVEMENT OF THE STRATEGIC OBJECTIVE (SO) The SO: "Women increase the use of reproductive health services" summarizes the project impact measured through practices and habits of reproductive health which will contribute to a better quality of life for women, and which are, essentially: care during pregnancy and childbirth by trained personnel, and the use of contraceptives. Annex D shows the indicators of the Results Framework with their relevant measurements. The number of cases for each indicator is recorded, as is the statistical significance and relevant value for z; furthermore, an estimation of the Odds Ratio (OR)14 is also given. Care During Pregnancy and Childbirth by Trained Personnel As is known, the aim of ReproSalud is to increase the use of reproductive health services, particularly of formal health services. This is the perception of the people working in the centers of MINSA, as evidenced in interviews carried out during visits to hospitals, health centers and health posts in project areas for the evaluation process carried out in the last quarter of 2001 (Shepard, 2002). Service providers stated that ReproSalud had built a bridge between the community and the centers with the community facilitators, allaying the fears of the population and their apprehension at 'being seen' by an 'unknown person'. It seems that training which emphasizes the need for women to attend 14 The OR is the disparity value. It measures the influence of an activity within a group compared with a control group. An OR value greater than 1 shows an impact for the activity, and the size of same. A value below 1 means that the project had no impact. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 24 professional health services had positive effects. Two key indicators measure this result: care during pregnancy and childbirth by trained personnel. Care During Pregnancy by Trained Personnel Prior to the start of the project, the percentage of pregnant women who went to trained health personnel for check-ups in counterpart communities was 55.6 percent; after two or three years this increased to 83.1 percent. The percentage also increased in control communities, but to a lesser extent (from 62.2 percent to 81 percent). Net gain in the former was 27.5 percentage points (significant to 5 percent, z = -7.240), while in the latter it was 18.8 percentage points (also significant to 5 percent, z = -5.114). See figure 5 indicator 1.a in Table 12. Figure 5: Prenatal Care by Trained Personnel During the Last Pregnancy Table 12: Care by Trained Personnel during Pregnancy and Childbirth Counterpart communities Control Communities N° Indicator Base Line MTE* Difference z Stat. Sig. Base Line MTE* Difference z Stat. Sig. 1.a Percent of women who had 4 or more controls by healthcare personnel during the last pregnancy occurring two years prior to the survey 55.6 83.1 27.5 -7.240 Sig 0.05 62.2 81.0 18.8 -5.114 Sig 0.05 2 Percent of women who had their last delivery, occurring two years prior to the survey, with trained personnel 36.1 48.0 11.9 -3.200 Sig 0.05 40.9 42.7 1.8 -0.494 *Midterm Evaluation Care During Childbirth by Trained Personnel Professional care in childbirth underwent the same change. At the start of the project, a little over one third (36.1 percent) of women in counterpart communities gave birth with the aid of trained personnel, and after two or three years this increased to 48 percent. In control communities, however, this change was smaller (from 40.9 percent to 42.7 50 55 60 65 70 75 80 85 Base Line Midterm Evaluation Percentage Counterpart Control Net Gain 27.5 18.8 Counterpart Control RESULTS 25 percent). Net gain in the former was therefore 12 percentage points; i.e., 6 times higher than in the latter (1.8 points). In the former, the change is significant to 5 percent, while in the latter it is significant to 10 percent (see figure 6 and indicator 2 of table 12). Figure 6: Last Birth Attended by Trained Personnel The improvements shown give a clear indication of the important impact of ReproSalud in two crucial aspects of care in reproductive health: pregnancy and childbirth. Lack of such care is an important cause of death in childbirth, especially in poor, rural areas. Given that initial values for the indicators measured were different in the two types of communities, and particularly due to the fact that net gains in both are statistically significant, the OR (table 24, see footnote 14) was used to verify the probability that the dependent variable (the indicator) would exist in the presence of the activity. The value of the OR shows that as a result of the work of ReproSalud, care during pregnancy by trained personnel increased 15 percent more in counterpart communities, care in childbirth by trained personnel increased by 27 percent more. Prevalence of the Use of Methods of Contraception The use of family planning methods is an important indicator for evaluating the impact of ReproSalud for, on the one hand, it reveals the level at which information and access to services is available, and, on the other, it is the key to improving reproductive health, especially in remote communities. Training and working strategies implemented by the project aim to: a) increase knowledge on the ways and means available to prevent unwanted pregnancies; b) inform potential users on the benefits and side effects of each method; and c) help the woman and/or partner to voluntarily choose the method which best suits their needs. Data collected from the base line survey reveal that the use of methods of contraception increased by 13.4 percentage points (from 58.4 percent to 71.8 percent) in counterpart communities and by only 4 percentage points in control communities (from 60.6 percent to 64.6 percent). Although net gain was significant to 5 percent in both types of 25 30 35 40 45 50 55 60 Base Line Midterm Evaluation Counterpart Control Net Gain 11.9 1.8 Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 26 communities, it is obvious that the intensity of change was greater in the former (see figure 7 and indicator 3 in table 13). Figure 7: Percentage of Women in Relationships Who Use Some Method of Contraception In this case, the relative change (which is obtained by dividing the value of the indicator in the second survey by the value in the first, table 24) in counterpart communities was 23 percent as opposed to 6.6 percent in controlled communities. These percentages show that use of contraception increased by 15 percent more in counterpart communities, due to project intervention. Unmet Family Planning Needs This variable was measured with two indicators: 1) unmet family planning needs as defined in the demographic and health surveys (DHS)15 (ENDES in Peru), and 2) unmet needs including women in a relationship who do not wish to have more children or do not wish to have them soon and use the rhythm method but are unaware of the fertile days of the cycle. This last indicator has been defined and used in other studies16 and is known as 'insufficient protection'. It affects many women who do not wish to have a (or another) child but are not adequately protected against becoming pregnant. It is estimated that in the year 2000 around 850,000 women in Peru were insufficiently protected (Ferrando 2001). These figures show that the project has had a significant impact on this issue also, having reduced unmet family planning needs from 27 percent to 18 percent in counterpart communities (i.e., a drop of 9 percentage points), while in control communities the drop was from 26.1 percent to 21.1 percent (i.e. only 5 percentage points). Although in both communities the change observed is statistically significant to 5 percent, change is 15 Includes women in a relationship who do not wish to have (more) children or wish to have them later (women who are not pregnant) but do not use any modern methods; or whose last pregnancy was unplanned or unwanted (pregnant women). 16 The Alan Guttmacher Institute, 1994; Ferrando, D. 2001. 55 60 65 70 75 Base Line Midterm evaluation Counterpart Control Net Gain 13.4 4.0 Counterpart Control RESULTS 27 greater in the former (indicator 4.1 in table 13), with a relative drop of 34 percent in counterpart communities as compared to control communities. Table 13: Use of Contraception and Unmet Family Planning Needs Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. 3 Percent of women in a relationship who use a method of contraception 58.4 71.8 13.4 -6.071 Sig 0.05 60.6 64.6 4.0 -1.783 Sig 0.05 4.1 Percent of women in a relationship with unmet FP needs 27.1 17.9 -9.2 4.741 Sig 0.05 26.1 21.1 -5.0 2.525 Sig 0.05 4 Perc ent of women in a relationship with unmet FP needs. Includes users of the rhythm method who are unaware of the fertile period 48.4 37.3 -11.1 4.834 Sig 0.05 50.1 46.9 -3.2 1.379 Sig 0.10 *Midterm evaluation As regards a wider definition of unmet needs (indicator 4 in table 13) the case is similar to that described above. The difference in improvement by communities is greater in counterpart communities (see figure 8). In counterpart communities insufficient protection was reduced by 11 percentage points (from 48.4 percent to 37.3 percent), while in control communities it was reduced by only 3 points (from 50.1 percent to 46.9 percent). This means that changes are significant to 5 percent in the former (z = 4.834) and to 10 percent in the latter (z = 1.379). In this case, relative reduction was 23 percent in counterpart communities, compared to 6.4 percent in control communities. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 28 Figure 8: Women in a Relationship with Unmet Family Planning Needs The balance of the relative reduction in unmet needs by type of community shows that project intervention resulted in an additional decrease of 18 percent for both indicators in counterpart communities (OR = 0.817 for indicator 4.1 and OR = 0.823 for indicator 4). Awareness of the Different Aspects in the Use of Contraception Within the Results Framework there are other indicators relating to the use of contraception which are analyzed below although they were not developed to measure the achievements of the Strategic Objective, but rather those of Intermediate Result 2.6: Women increase awareness of their reproductive health needs. Awareness of Modern Methods Given that this is basically a project for IE&C on reproductive health, another of the important aspects for ReproSalud is an improvement regarding knowledge and awareness of women regarding family planning methods, basically in order to reduce (and hopefully eliminate) their fears and prejudices surrounding certain methods, as a first step to increasing their use. Indeed, figure 9 shows an extraordinary improvement in awareness regarding methods, measured by the percentage of women in a relationship who know how at least one modern method works. This percentage increased fourfold in counterpart communities, from 13.5 percent to 55.5 percent between the base line survey and the midterm evaluation, resulting in a net gain of 42 percentage points, as compared with a gain of 17 percentage points in control communities, where the percentage increased from 14.9 percent to 31.9 percent. 5 15 25 35 45 Base Line Midterm Evaluation Counterpart Percentage 25 35 45 55 65 Base Line Midterm Evaluation Control Percentage Net Change -9.2 -5.0 Counterpart Control Net Change -11.1 -3.2 Counterpart Control RESULTS 29 Figure 9: Percentage of Women in a Relationship Who Are Aware of at Least One Modern Method of Contraception The changes in this indicator are statistically significant in both types of communities. However, net gain in counterpart communities is almost two and a half times greater than in control communities (indicator 38 in table 14). In the ranking of indicators for the project, from that of greater effect to that of least effect (table 25), this indicator is in first place as showing greatest gain. Table 14: Awareness of Methods of Contraception Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base Line MTE* Difference z Stat. Sig. 38 percent of women in a relationship who are aware of at least one modern method 13.5 55.5 42.0 -24.410 Sig 0.05 14.9 31.9 17.0 -10.713 Sig 0.05 39 percent of users of rhythm method who are aware of the fertile days of the cycle 22.5 28.3 5.8 -1419 Sig 0.10 24.0 24.1 0.1 -0.027 43b percent of women who believe that it is more difficult to get pregnant if they are breastfeeding. 34.0 56.3 22.3 -10.477 Sig 0.05 30.9 46.9 16.0 -7.594 Sig 0.05 *Midterm evaluation Awareness of the Fertile Period Awareness of the fertile days of the cycle amongst users of the rhythm method showed a net increase of 5.8 percentage points (significant to 10 percent) in counterpart communities, rising from 22.5 percent to 28.3 percent, whereas in control communities it remained unchanged at around 24 percent (see figure 10 and indicator 39 in table 14). In the ranking of indicators for the project (see table 25) this indicator is fifth in order of impact. Awareness on this issue must be strengthened, as it appears to be quite fragile. During the training sessions, participants (women and men) seem to assimilate the theory but forget 10 20 30 40 50 60 Base line Midterm evaluation Percentage Counterpart Control Net Gain 42.0 17.0 Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 30 it after a short period of time if they do not have an opportunity to talk about the issue. In interviews with women beneficiaries and facilitators (both women and men) in five departments during the process evaluation (between October and November 2001) questions were asked regarding the fertile days of the cycle. Some women said that they knew about it but had forgotten what it was, others gave incorrect answers, and very few gave a correct answer17. Although the mentioned interviews are not representative, they do show that people are aware of the issue after a training session, but become confused later on and cannot remember it. Figure 10: Percentage of Women Using the Rhythm Method Who Are Aware of the Fertile Days of the Cycle Perception Regarding Breastfeeding Although this is more an opinion than actual knowledge, breastfeeding has been included in this section due to its association with the issue of contraception. The idea that while breastfeeding a child conception is difficult has experienced a significant positive change in counterpart communities (from 34 percent to 56.3 percent), but this was also the case in control communities, though to a lesser degree (from 30.9 percent to 46.9 percent). Thus, net gain was 22.3 percentage points in the former and 16 points in the latter (see figure 11 and table 14). 17 As an anecdote, the wife of a ReproSalud facilitator, herself a facilitator, stated that she was expecting their second child because "the calendar failed". This fact does not necessarily show unawareness of the fertile period, but perhaps lack of willpower to avoid sexual intercourse on days when pregnancy is possible, or perhaps, male imposition. Net gain 5.8 0.1 Counterpart Control 15 20 25 30 35 Base line Midterm evaluation Percentage Counterpart Control RESULTS 31 Figure 11: Percentage of Women Who Believe It Is Difficult to Become Pregnant While Breastfeeding A comparison of the value of this indicator in the midterm evaluation with that in the base line survey shows a relative change of 66 percent in counterpart communities as opposed to 52 percent in control communities (see table 24). Thus, the project achieved a 9 percent greater increase in awareness of this specific issue of reproductive health in the communities in which it was working. On comparing the value of the OR (see tables 24 and 25) for the three indicators of table 14, it would appear that the possibility of the indicator existing because of the project decreases as the issues in consideration become more complex. As an example, the possibility of increasing the knowledge of how at least one modern method works is very high (92 percent), that of increasing awareness of the fertile days of the cycle is 25 percent and that of increasing the knowledge of the benefits of breastfeeding as a method of contraception is of only 9 percent. C. INTERMEDIATE RESULTS This section analyzes the indicators of the Results Framework for the Intermediate Results (IR) that the project must achieve if it is to attain its Strategic Objective18 . Although there are three of these19, only two will be evaluated: those relating to gender and to the use of health services. The third indicator, relating to empowerment of women and their participation in policy decisions, will not be evaluated20, although some 18 SO: Women increase their use of reproductive health services 19 RI 1: More equitable gender relationships of women with their partner and with their families RI 2: Greater capacity and readiness of women in rural and peripheral urban areas to access reproductive health services in the formal sector RI 3: Effective participation of women from CBOs in the process for developing policy proposals, adjustment and monitoring of reproductive health programs 20 since such issues are being given greater emphasis in Phase II of the project, which has been running for less than a year and has so far only an 'advocacy' base line. Net gain 22.3 16.0 25 Counterpart Control 30 35 40 45 50 55 60 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 32 empowerment indicators were identified as part of RI 1 and 2, which are analyzed in this section. The IRs cover sub-results with indicators (see annex B) to measure progress and are duly numbered. However, to facilitate analysis and interpretation of the figures, they have been grouped in accordance with their relevance to issues of gender and family relationships, empowerment and use of services. Within this general classification they have been subdivided into indicators, which measure perception, knowledge, attitudes or practices. For this reason, the indicators are not shown in numerical order in the tables which follow, the original number of the indicator as per the Results Framework having been maintained. Annex 3 shows the indicators in numerical order. It also shows the number of cases per indicator, the statistical significance, value of z in each case and the value of the Odds Ratio. Gender Relations Perceptions Perception of women and men regarding the importance of the work of women as compared to the work of men Perceptions are ideas, opinions and beliefs held by people regarding things and events in life. Due to their very nature, they are dynamic and change according to social events and circumstances. They are therefore subject to change depending on external factors. As regards perceptions on gender, the project would appear to have made no difference in counterpart communities as compared to control communities; if it did, this is not clearly reflected by the figures (see table 15). As an example, when it comes to assigning the same value to women's work as to men's work, the percentage of women who were of this opinion in counterpart communities increased by 6.4 percentage points (from 51.4 percent to 57.8 percent) and by 6.8 points (from 48.4 percent to 55.2 percent) in control communities. Net gain was practically the same in the two communities and statistical significance was also to 5 percent in both. Figure 12: Percentage of Women and Men Who Believe that Work Done by Women Is as Important as that of Their Partners Net Gain 6.4 6.8 17.7 12.4 Counterpart Control Counterpart Control 45 Women Men 50 55 60 65 70 Base Line Midterm Evaluation Percentage Women Counterpart Women Control Men Counterpart Men Control Men Women RESULTS 33 However, amongst men this increase was slightly higher in counterpart communities as compared to control communities (17.7 percentage points as compared to 12.4 points), which implies a greater value attached by men to women's work21 (see indicators 19 and 19.1 in table 15). As regards the value attached to work in the home compared to the value attached to the work of men outside the home22, both by men and by women, the increase has been greater in control communities, where it went from 30.2 percent to 42.5 percent amongst women and from 33.3 percent to 47.9 percent amongst men. In counterpart communities, however, it increased from 33.7 percent to 44.1 percent amongst women and from 36.25 percent to 47.4 percent amongst men. These percentages give a net gain in women's opinion of 12.5 percentage points in control communities and 10.4 percentage points in counterpart communities; figures for men’s opinion are 14.6 percentage points and 11.2 percentage points respectively. 21 Some non-exclusive explanations can be put forward on the difference in perception by men and women of women's work: a) men have, in general, a higher level of education than women. In the sample being analyzed, the percentage of illiterate women is four times higher than that of men (16.8 percent as compared to 4.1 percent), whilst the proportion of those with secondary education is 33 percent and 54 percent respectively; b) this higher level of education gives men a greater opportunity than women to access information and thus to be more in touch with the modern world. In such circumstances, they are better able to evaluate the importance of the work done by women. They have a more modern appreciation than women as regards their own work. c) men have perhaps given an answer which does not reflect their real feelings in order to please the interviewer, whereas women have perhaps been more sincere in their replies. d) women still have difficulty in realizing the value of their domestic chores and, generally, their own worth in society. e) the work of ReproSalud with men, although involving a much smaller number than women, has had a positive effect. Of the total number of men interviewed in counterpart communities (1,773), 33 percent (581) had attended training sessions. 22 The definition of this indicator is not clear: ' percent of women who believe that domestic work is as important as the work of their husband'. It should therefore be assumed that it refers to the value attached to the work of women in the home as compared to the work of men outside the home. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 34 Table 15: Perception by Women and Men on the Importance of the Work of Women and Men and on the Level of Education to Be Received by Daughters and Sons Counterpart Communities Control Communities N° Indicator Base Line MTE* Difference z Stat. Sig. Base Line MTE* Difference z Stat. Sig. Women's perceptions 19 percent of women who believe that their work is as important as that of their partner 51.4 57.8 6.4 -2.522 Sig 0.05 48.4 55.2 6.8 -2.852 Sig 0.05 20 percent of women who believe that work in the home is as important as their husband's work 33.7 44.1 10.4 -4.339 Sig 0.05 30.2 42.7 12.5 -5.631 Sig 0.05 21 percent of women who believe that daughters and sons should study to the same level 86.7 91.9 5.2 -2.745 Sig 0.05 94.5 89.6 -4.9 2.957 Sig 0.05 Men's perceptions 19. 1 percent of men who believe that their work is as important as that of their partner 52.1 69.8 17.7 -5.987 Sig 0.05 55.9 68.3 12.4 -4.388 Sig 0.05 20. 1 percent of men who believe that the work of their wife in the home is as important as the husband's work 36.2 47.4 11.2 -3.997 Sig 0.05 33.3 47.9 14.6 -5.393 Sig 0.05 21. 1 percent of men who believe that daughters and sons should study to the same level 89.1 91.4 2.3 -1.077 91.6 90.9 -0.7 0.325 *Midterm evaluation The percentages and differences for each type of community in the two measuring surveys are shown in table 15 (see indicators 20 and 20.1), whilst the opinion trend in men and women by community type is shown in figure 13. Figure 13: Percentage of Women and Men Who Believe that Housework Is as Important as the Work of the Husband Net Gain 10.4 12.5 11.2 14.6 Counterpart Control Counterpart Control Women Men 25 30 35 40 45 50 Base Line Midterm evaluation Percentage Women Counterpart Women Control Men Counterpart Men Control Men Women RESULTS 35 Perception of Women and Men on the Rights of Daughters and Sons to Be Educated to the Same Level The positive opinion on the same rights to education of daughters and sons increases slightly in counterpart communities (see figure 14) and decreases slightly in control communities. Net gain of women's opinion in favor of the same rights to education of girls and boys was of 5.2 percentage points in the former and a drop of -4.9 points in the latter. As regards men's opinion, figures were 2.3 points and -0.7 points respectively (see indicator 21 and 21.1 in table 15). Figure 14: Percentage of Women and Men Who Believe that Daughters and Sons Should Study to the Same Level According to these figures, it also appears that women attach greater importance to the education of their daughters than do men. The percentage of women who believe that daughters and sons have the same right to education is 5.6 percent as opposed to 2.3 percent of men, probably because they wish better social and financial opportunities for their daughters than they have themselves23. The project has had a positive impact (albeit as yet a small one in men) on an important aspect relating to the lives of their daughters: the same rights to education as sons. However, the decline observed in control communities would warrant an explanation that does not fall within the scope of this evaluation. The balance in partial changes by type of community summarized in the OR shows that as regards the value attached to women's work the project does not improve women's perception of this and affords only a slight improvement in men's perception. As regards the rights to education of daughters and sons, the project has had a slight impact on the favorable perception amongst men (3.4 percent) and a much greater impact amongst women (11.8 percent). 23 the majority of women in counterpart communities are either illiterate (17 percent) or have only received primary education (45 percent). Net Gain 5.2 -4.9 2.3 -0.7 Counterpart Control Counterpart Control Women Men 80 85 90 95 100 Base Line Midterm Evaluation Percentage Women Counterpart Women Control Men Counterpart Men Control Men Women MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 36 Perception of Men Regarding Physical Abuse and Forced Sexual Relations Changes in the opinion of men regarding physical abuse of women and sexual violence took place in both counterpart and control communities, although changes were slightly greater in ReproSalud counterpart communities, where there is a greater awareness by men that women should be respected and should not be forced into having sexual relations against their will. Prior to the start of the project, the percentage of men in counterpart communities who believed that women should not by physically abused under any circumstances was 57.4 percent, and this increased to 64 percent after two or three years. In control communities the change was from 58.8 percent to 63 percent. These changes give a net gain of 6.6 percentage points in the former (statistically significant to 5 percent) and 4.2 percentage points in the latter (significant to 10 percent) (see indicator 17 of table 16 and figure 15a). Table 16: Men’s Perceptions Regarding Physical Abuse of Women and Forced Sexual Relations Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. 17 percent of men who believe that women should not be physically abused under any circumstances 57.4 64.0 6.6 - 2.53 1 Sig 0.05 58.8 63.0 4.2 -1.557 Sig 0.10 18 percent of men who believe that men have no right to demand sexual relations against the wife's will 84.6 91.5 6.9 - 3.98 1 Sig 0.05 88.8 91.8 3.0 -1.831 Sig 0.05 *Midterm evaluation The proportion of men who believe that men have no right to demand sexual relations against a wife’s will also increased in both counterpart and control communities, to a somewhat greater degree in counterpart communities. Thus, net gain in counterpart communities is 6.9 percent, more than double that in control communities (3 percent). See figure 15.b and indicator 18 in table 16. RESULTS 37 Figure 15: Men’s Perceptions Regarding Physical Abuse of Women and Forced Sexual Relations 15a 15b However, a comparison of the partial changes (see table 24) by type of community on the basis of the base line and midterm evaluation values of the indicators shows that the project had little impact on the positive practices of men in their relationship with women. With regard to control communities, the project only achieved an increase of 4 percent in the percentage of men who believe that women should not be physically abused under any circumstances, and of 4.6 percent in the percentage of men who believe that men have no right to demand sexual relations against their wife's will. This modest impact of the project on gender opinions and attitudes should be analyzed in the light of the social changes that took place throughout the country in the second half of the 1990s. Subsequent to the International Conference on Population and Development (UNFPA, 1994) at the end of 1994, discussions on gender equity became a fashion, this being the main theme of the Conference. Given the wide-reaching effects of the said Conference, which emphasized gender equity, especially in matters of sexual and reproductive health, there emerged a current of opinion which has created a favorable environment not only for openly discussing the issue but also to endeavor to overcome obstacles and modify old customs which go against the right of women to be respected and appreciated. Issues of sexual and reproductive rights, as well as gender issues, are currently high on the agenda of social and political leaders, and are also widely discussed in the media and in mass productions which are broadcast on local radio and television, thereby contributing to a positive change in public opinion, even in the more remote communities, regarding a more equitable perception of gender relations, without the need for specific intervention, although it does appear that changes on certain issues are slightly greater when such intervention takes place. Practices Given the foregoing external variable which tends to put perceptions regarding gender issues on an equal footing amongst counterpart and control communities, from a project Percentage of men who believe that women physically abused under any circumstances 55 57.5 60 62.5 65 Base line Midterm evaluation Percentage Counterpart Control Percentage of men who believe men have no demand sexual relations against a woman's right to 82.5 85 87.5 90 92.5 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 38 point of view it is of greater interest to measure practices in the relations between women and men, and the way in which these are consolidated in counterpart communities. In general, some changes can be seen in the behavior of women within the home and in their communication with their partner in communities where the project is working, whereas in others these changes have not taken place. One example is that some decisions are being made jointly by the couple. Of the six indicators selected for measuring this issue (see table 17), three bear relation to the main focus of the project: that women should have the same decision-making capacity as their husband or partner within the home. Table 17: Gender Practices of Women and Men Counterpart Communities Control Communities N° Indicator Base line MTE* Differ￾ence z Stat. Sig. Base line MTE* Differ￾ence z Stat. Sig. Women's practices 7 percent of women in a relationship who decide together with their partner on sexual relations, methods of contraception and number of children 23.7 32.9 9.2 -4.431 Sig 0.05 22.6 27.8 5.2 -2.588 Sig 0.05 9 percent of women who carry out one or more tasks in the home together with their partner 23.2 46.7 23.5 -10.775 Sig 0.05 18.6 37.9 19.3 -9.392 Sig 0.05 11 percent of women in a relationship who decide jointly with their partner on the level to which daughters and sons should be educated 57.4 64.0 6.6 -2.798 Sig 0.05 57.8 58.0 0.2 -0.084 11.a percent of women in a relationship who have talked with their partner regarding the number of children they wish to have 67.8 72.6 4.8 -2.273 Sig 0.05 69.3 74.0 4.7 -2.241 Sig 0.05 13 percent of women in a relationship who have talked with their partner on FP in the last 12 months 15.2 20.3 5.1 -2.908 Sig 0.05 11.4 14.1 2.7 -1.757 Sig 0.05 Men's practices 5 percent of women in a relationship whose partner helps when children are ill 38.7 44.7 6.0 -2.527 Sig 0.05 30.0 35.3 5.3 -2.338 Sig 0.05 *Midterm evaluation As an example, the percentage of women in a relationship who decide together with their partner on sexual relations, methods of family planning and number of children has increased more in counterpart communities (from 23.7 percent to 32.9 percent) than in control communities (from 22.6 percent to 27.8 percent) (indicator 7 in table 17). This resulted in a net gain of 9.2 percentage points in the former as opposed to 5.2 points in the latter. These changes are significant to 5 percent in both communities. RESULTS 39 The relative change in counterpart communities was 39 percent and 23 percent in control communities. The balance of these changes shows that the project resulted in an improvement of 13 percent more in counterpart communities. Figure 16: Percentage of Women Who With Their Partner Jointly Decide Regarding Sexual Relations, Contraception, and Number of Children A positive change can also be seen in counterpart communities regarding joint decisions on the education of children (a positive change was also seen regarding perceptions on this same issue). Prior to the start of the project, the situation was the same in both types of communities, but the project has made a difference in counterpart communities, where the percentage of women who decide together with their partner on the education of their children increased from 57.4 percent to 64.0 percent (a net gain of 6.6 percentage points), this difference being statistically significant to 5 percent (z = -2.798). However, in control communities this percentage did not change, staying at around 58 percent (a net gain of 0.2 points) (figure 17 and indicator 11 in table 17). Thus, project activities resulted in an additional increase of 11 percent in the proportion of women in counterpart communities who take decisions together with their partner regarding the level of education for their daughters and sons. Figure 17: Percentage of Women Who With Their Partners Jointly Decide on the Level of Education for Their Daughters and Sons Furthermore, a greater increase can be seen in counterpart communities in the percentage of women in a relationship who have talked with their partner on family planning in the Net gain 5.2 9.2 Counterpart Control 15 20 25 30 35 Base Line Midterm Evaluation Percentage Counterpart Control Net Gain 6.6 0.2 Counterpart Control 50 55 60 65 70 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 40 last 12 months (from 15.2 percent to 20.3 percent) as compared to control communities (where the percentage changed from 11.4 percent to 14.1 percent) (indicator 13 in table 17). This difference in intensity of change (see figure 18) resulted in a net gain in the former (5.1 percentage points), which was almost double that of the latter (2.7 points), although in both types of communities these increases are significant to 5 percent. On this issue, the increase due to the project in counterpart communities is 8 percent. Figure 18: Percentage of Women Who Discussed Family Planning with Their Partner in the Past 12 Months As opposed to these results, which show a positive impact of the project, there are others that are not as encouraging. As an example, the proportion of women who carry out one or more household chores together with their partner (figure 19), which, though showing a slightly greater increase in counterpart communities (23.5 percentage points) than in control communities (19.3 points), the intensity of change was essentially the same for both types of communities (indicator 9 in table 17). Thus, the project does not appear to have had a significant impact in counterpart communities regarding the percentage of women who have the help of their partner for the household chores (OR = 0.988). Figure 19: Percentage of Women Who Carry Out One or More Household Chores Jointly with Their Partner The same is true for the percentage of women who have talked with their partner on the number of children they wish to have. As can be seen in table 17 (indicator 11.a), given that increases were the same for both types of communities (4.8 percentage points), the Net gain 2.7 5.1 Counterpart Control 5 10 15 20 25 Base Line Midterm Evaluation Percentage Counterpart Control Net gain 23.5 19.3 Counterpart Control 15 25 35 45 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 41 difference prior to and after project activities remained the same in each, resulting in parallel lines for the indicator trend, as can be seen in figure 20. Figure 20: Percentage of Women in a Relationship Who Have Discussed with Their Partner the Number of Children They Wish to Have Table 17 has three indicators that measure the same issues in a different way, and their results appear to be contradictory. According to indicator 7, the percentage of women who decide together with their partner on sexual relations, use of contraception and number of children has increased by 9.2 percent between the base line and the midterm evaluation. According to indicator 11.a, the percentage of women who have discussed with their partner the number of children they wish to have has increased by only 4.8 percent and, finally, the percentage of women who have talked with their partner on family planning has increased by scarcely 5 percent. It seems to be a contradiction that a larger number of women decide on these issues when the percentage of those who talk about them has not increased in the same proportion. This confusion would be due to the manner in which indicator 11 is made up, since it summarizes three indicators (women who decide on sexual relations, or on contraception, or on number of children), whereas the other two (indicators 11.a and 13) each measure a single issue. For a more correct interpretation of the results, and in order to avoid apparent contradictions, indicator 11 should be divided into three separate indicators. The behavior of men in the household as regards caring for the children when they are ill has not changed. The percentage of husbands or partners who help their wife in looking after the children when they are ill has increased in both types of communities, slightly more in counterpart communities (by 6 percentage points) than in control communities (5.3 points) (indicator 5 in table 17), but this does not differentiate the communities, as is confirmed by an OR value = 0.982, which indicates that project activities have not changed the behavior of men on this particular issue. This similar degree of change is shown in figure 21, which shows the parallel trend in both lines. Net gain 4.8 4.7 Counterpart Control 60 65 70 75 80 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 42 Figure 21: Percentage of Women in a Relationship Whose Partner Helps When the Children Are Ill This similarity in the change in behavior of parents in the communities surveyed could be due to the work done by MINSA, and project activities cannot be isolated from this. One of the strategies of public policies on healthcare is the involvement of the family in the treatment and care of ill persons. The erratic results of the indicators in table 17, some of which show better results in counterpart communities whereas others remain at the same level and are similar to control communities, could be due, as has been said, to the fact that these were anyway different from counterpart communities. Furthermore, a third of the population interviewed in the midterm evaluation was new to the community and, lastly, control communities may have been subjected to activities which have not been reported and which could influence the opinions and practices of the population living in such communities. Family Relationships There is only one indicator in the Results Framework that can be used to measure this issue, and that is the one shown in figure 22 and table 18 relating to the percentage of women who have talked with their children over 12 regarding relationships and family planning in the 12 months prior to the interview. This percentage more than doubled in counterpart communities (from 7.1 percent to 16.0 percent), but then so did it in control communities (from 3.9 percent to 10.4 percent). The net gain of 8.9 percentage points in the former and 6.5 points in the latter is of no real significance, since the intensity of change was greater in control communities, where the percentage of women who have talked with their children regarding relationships and family planning multiplied by 1.6 percent, whereas in counterpart communities this was only 1.25 percent. The value of OR = 0.845 shows that the project has not contributed in this respect. Net gain 5.3 6.0 Counterpart Control 25 30 35 40 45 50 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 43 Figure 22: Percentage of Women in a Relationship Who Have Discussed Relationships and Family Planning with Their Children in the Past 12 Months Table 18: Family Relations Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. 16 percent of women in a relationship who have talked with their children about relationships and family planning in the last 12 months 7.1 16.0 8.9 -3.893 Sig 0.05 3.9 10.4 6.5 -3.352 Sig 0.05 Empowerment As regards empowerment practices, counterpart communities show interesting achievements, although some of these could contain some of the initial bias of self￾selection of the communities. Although counterpart communities were quite poor, they were nevertheless better organized and, to a certain extent, had better information and knowledge on issues of gender and empowerment (which is the reason why they had stronger CBOs) than their controls in the area where control sample was selected. Knowledge In accordance with the figures ava ilable, the project has improved the information available to women regarding where to go to seek help if they are physically abused (see figure 23). In counterpart communities this figure rose from 45.6 percent to 65.6 percent, while in control communities it rose from 44.8 percent to 59.5 percent (indicator 16.a of table 19). Thus, net gain for knowledge was 20 percentage points in the former and 14.7 points in the latter, both significant to 5 percent. The balance of relative changes in each type of community shows that where project activities were carried out the knowledge of women in the counterpart communities who know where to seek help if they are physically abused increased by 8.3 percent more than in control communities (table 24). Net gain 8.9 6.5 Counterpart Control 0 5 10 15 20 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 44 Figure 23: Percentage of Women Who Know Where to Seek Help of Advice in the Event of Physical Abuse Attitudes This increase in knowledge results in an increase in readiness to act accordingly. Following project activities, more than double the number of women in the project counterpart communities would be prepared to go to the police or to the authorities in the event of physical abuse by their husband (from 14.3 percent to 38.4 percent). This percentage also doubled in control communities (from 15.7 percent to 37.2 percent) and, in both, the change in the indicator is significant to 5 percent. Net gain is greater (24.1 percentage points) in counterpart communities than in control communities (21.5 points), as is the intensity of change, although the almost parallel lines in figure 24 seem to indicate differently (see the base line and midterm evaluation percentages in table 19, indicator 14). In counterpart communities, the initial percentage of women who would go to the police to seek help multiplied by 1.69, while in control communities it increased by multiplied by 1.37 (table 24). The difference in these figures shows that as a result of project activities, the proactive attitude of women to defend themselves increased 13 percent more in counterpart communities. Figure 24: Percentage of Women Who Would Seek Help from the Police/Authorities in the Event of Physical Abuse by Their Husband Net gain 20.0 14.7 Counterpart Control 40 45 50 55 60 65 70 Base Line Midterm Evaluation Percentage Counterpart Control Net Gain 24.1 21.5 Counterpart Control 10 15 20 25 30 35 40 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 45 Perceptions As regards perceptions on the issue of empowerment, the percentage of women who agree that women should take precautions in order not to have children even if their partner opposes this has decreased, as shown in figure 25, both in counterpart communities (from 68.9 percent to 64 percent) and in control communities (from 65 percent to 58.7 percent), net loss being the same in both types of community, approximately -4.5 percentage points (indicator 12, table 19). Figure 25: Percentage of Women Who Agree that Women Should Take Precautions in Order Not to Have Children Even if the Husband Disagrees This opinion could be attributed to the fact that during training sessions issues regarding communication with the partner and the importance of taking decisions jointly were emphasized. But this decrease also occurred in control communities where ReproSalud did not carry out any training, perhaps as a result of the work carried out by MINSA through its Family Planning Program, which emphasizes the fact that the decision on the use of contraception should be taken by the couple. The value of OR = 1.006 shows that project intervention has had no impact on this issue in counterpart communities as compared to control communities. Table 19: Empowerment Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. Knowledge 16.a percent of women who know where to seek help or advice in the event of physical abuse 45.6 65.6 20.0 - 10.9 83 Sig 0.05 44.8 59.5 14.7 -7.737 Sig 0.05 Perceptions 12 percent of women who agree that women should take precautions in order not to have children even if the husband disagrees 68.9 64.4 -4.5 2.07 1 Sig 0.05 65.0 60.4 -4.6 2.056 Sig 0.05 Net Loss -4.5 -4.6 Counterpart Control 57.5 60 62.5 65 67.5 70 Base Line Midterm Evaluation Percentage Counterapart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 46 Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. Attitude 14 percent of women who would go to the police/authorities for help in the event of physical abuse by their husband 14.3 38.4 24.1 -12.015 Sig 0.05 15.7 37.2 21.5 -10.691 Sig 0.05 Practices 6 percent of women in a relat ionship who decide what to spend the money they themselves earn on 31.6 36.1 4.5 -2.596 Sig 0.05 29.8 27.8 -2.0 1.158 12.1 percent of women who do not have sexual relations by force or persuasion 50.6 64.0 13.4 -5.852 Sig 0.05 55.4 58.7 3.3 -1.435 Sig 0.10 *Midterm evaluation Practice By contrast, as regards practices on issues of empowerment, the project has had a significant impact. As an example, the percentage of women who decide what to spend the money they themselves earn on increased in counterpart communities from 31.6 percent to 36.1 percent and decreased in control communities from 29.8 percent to 27.8 percent) (figure 26 and indicator 6 in table 19). Net gain in the former was 4.5 percentage points, and net loss in the latter was 2 percentage points. The project contribution is ratified by the value of OR = 1.225, which gives the project an increase of 23 percent more in the percentage of women in counterpart communities who decide what to spend the money they earn on. Figure 26: Percentage of Women Who Decide How to Spend the Money They Earn The project seems to have had the same impact as regards understanding of the right of women not to be sexually abused by their partner, and this is shown by the increase in the percentage of women who do not have sexual relations by force or persuasion. In project communities this figure increased from 50.6 percent to 64 percent, while in control communities it increased only from 55.4 percent to 58.7 percent (indicator 12a in table Net Change 4.5 -2.0 Counterpart Control 25 27.5 30 32.5 35 37.5 40 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 47 17). Net gain of 13.4 percentage points and 3.3 percentage points respectively is eloquent and reveals an important aspect of the self-respect that women in communities where ReproSalud is working have for themselves. These results are probably indicative of a capacity for negotiation with their partners on issues of sexual relations. This is confirmed by the balance in partial changes (table 24) in the two types of communities, which attributes to the project activities an additional positive increase of 19.4 percent for this indicator in counterpart communities. Figure 27: Percentage of Women Who Do Not Have Sexual Relations by Force or Persuasion Use of Health Services Some indicators which reveal a greater use of health services were discussed in Section B.1 of Part Two of this report when analyzing achievements of the Strategic Objective. There now follows an analysis of additional indicators on the opinions and attitudes of women as well as on their practices in relation to the use of health services. Perceptions Regarding the Quality of Health Services Perceptions regarding health services were inquired about by asking women whether they thought that the service provided by the nearest health center, assuming that this is the one they attend, is good (or very good). The figures show that in general women from control communities have a more ‘generous’ perception of the quality of the health services when replying regarding the attention they themselves or somebody they were accompanying had received (see figure 28). I ndeed, this percentage grew from 52.8 percent to 69.7 percent, while in counterpart communities it grew from 45.5 percent to 56.2 percent (indicator 26 in table 20). As a result, net gain in control communities was 16.9 percentage points, while in counterpart communities it was 10.7 points. Net Gain 13.4 3.3 45 Counterpart Control 50 55 60 65 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 48 Table 20: Perception Of Women of Reproductive Age Regarding Quality of the Health Services Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. Perceptions 26 percent of women who believe that the services afforded by the nearest health center are good (or very good) 45.5 56.2 10.7 -5.107 Sig 0.05 52.8 69.7 16.9 -8.397 Sig 0.05 27 percent of women who have attended a health center for pre and post natal care and believe that the services afforded by the nearest center are good (or very good) 55.4 61.2 5.8 -1.450 Sig 0.10 67.8 69.3 1.5 -0.389 Attitudes 29 percent of women in a relationship who would attend a health center in the event of potentially dangerous problems 42.6 71.6 29.0 -11.111 Sig 0.05 36.8 62.2 25.4 -10.678 Sig 0.05 *Midterm evaluation When the same question is made to women who attended a hospital, health center or health post for their own care for either pre or post natal check-up, it is observed that the positive change in the percentage of women with a favorable opinion of the health services they received was greater in counterpart communities (from 55.4 percent to 61.2 percent) than in control communities (from 67.8 percent to 69.3 percent). These increases resulted in a net gain of 5.8 percentage points in counterpart communities and 1.5 points in control communities (indicator 27 in table 20). Figure 28: Women’s Use of Health Services Percentage of women who utilized services for any problem and who believe that the services offered by the nearest health center are good 40 45 50 55 60 65 70 75 Base Line Midterm Evaluation Percentage Control % of women who attended for pre or post natal care and who believe that the services offered by the nearest health center are good 40 45 50 55 60 65 70 75 Base line Mid-term evaluation % Counterpart Net Gain 10.7 16.9 Counterpart Control Net Gain 1.5 5.8 Counterpart Control RESULTS 49 These findings could be valued for their aspect of empowerment coupled with a sophistication in the critical judgment of women on the services they receive. Since they were trained on what to expect from the service, it is possible that on replying regarding the quality of the services afforded by health centers they may have given a more objective evaluation of what they received (indicator 26, table 20). Furthermore, the greater increase in counterpart communities of the positive perception of health services amongst women who attend for pre or post natal care (indicator 27 in table 20) would imply an effective improvement in the health services as a result of the work carried out by the Health Ministry to afford quality care in mother-and-child services24, and this may have been strengthened by the facilitators of ReproSalud, who approached the service providers in an attempt to improve client relations. The OR value for indicator 26 (0.936) shows that the project had no impact, although if the theory of sophistication of the critical judgment of women is to be used, this measurement might not necessarily be negative for the project. On the other hand, OR = 1.081 for indicator 27 shows an additional increase, attributable to the project, of 8 percent in counterpart communities regarding the positive perception on the quality of service amongst women who attended for pre or post natal care. Attitudes: Readiness of Women to Attend Health Centers In general, whether a women attends a health center for her own treatment or for accompanying another person, after a minimum of two years of project activities the figures show that almost 60 percent of women in counterpart communities and almost 70 percent in control communities are of the opinion that the services are of a good quality. Accordingly, their readiness to attend a health center is positive and has increased considerably, from 42.6 percent to 71.6 percent amongst the former and from 36.8 percent to 62.2 percent amongst the latter. In both, the increase is significant to 5 percent. Net gain is 29 percentage points in counterpart communities and 25.4 percent in control communities (see table 20, indicator 29, and figure 29). The intensity of change was, however, greater amongst women in control communities, where in the midterm evaluation the percentage of those who would attend a health center increased by 76.4 percent as compared to the base line, while in counterpart communities it increased by 60.5 percent. The OR value of 0.936 shows that the project had no impact on this indicator. 24 MINSA is committed to increasing coverage and improving the quality of reproductive health and family planning services, and for this it has the support of international aid agencies such as the US Agency for International Development USAID, the United Nations, the British Department for International Development DfID, the World Bank and the Interamerican Development Bank, amongst others. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 50 Figure 29: Percentage of Women in a Relationship Who Would Visit a Health Center for High Risk Problems Practices Relating to Health The purpose of ReproSalud is to instill in the population positive practices relating to healthcare and to boost women’s appreciation of themselves and their self-confidence. The aim is to encourage them to seek professional help for their physical healthcare needs and thereby achieve the necessary balance between physical and mental health, which will contribute to an increased quality of life. It is hoped that women will care not only for their children, their partner or their parents, as they have been doing for centuries, but that they will also care for themselves. Spending on Health and Visits to Health Centers The project selected various indictors to measure these practices in the population and monitor their consolidation as project activities are implemented. The results likewise show that the project had no impact on this issue. As an example, both in counterpart communities and in control communities the percentage of women spending on healthcare in the last 12 months has remained virtually unchanged, at about 50 percent in counterpart communities and 45 percent in control communities (indicator 33 in table 21 and figure 30). The difference between the two surveys is not significant in either community, and although the increase in counterpart communities and the decrease in control communities might perhaps suggest a better situation in the former, the change is minimal and makes little difference between the two types of communities. This is confirmed by the OR value = 1.021, which attributes to the project a positive impact of only 2 percent for this indicator. Net Gain 25.4 29.0 30 Counterpart Control 40 50 60 70 80 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 51 Table 21: Professional Health Care Counterpart Communities Control Communities N° Indicator Base Line MTE* Difference z Stat. Sig. Base Line MTE* Difference z Stat. Sig. Practices 33 percent of women who spent on healthcare in the last 12 months 49.5 50.0 0.5 -0.273 45.7 45.2 -0.5 0.241 24 percent of women who attended a health center for issues of reproductive health or family planning 26.3 42.2 15.9 -9.187 Sig 0.05 22.5 39.7 17.2 -9.851 Sig 0.05 *Midterm evaluation Figure 30: Percentage of Women Who Spent Money on Healthcare in the Past 12 Months The percentage of women who have attended a health center for reproductive health or family planning issues has increased slightly more in control communities (from 22.5 percent to 39.7 percent) than in counterpart communities (from 26.3 percent to 42.2 percent), and thus net gain is 17.2 percentage points in the former and 15.9 percentage points in the latter (figure 31). The change observed in the two types of communities is statistically significant to 5 percent (indicator 24 in table 21). The value of OR = 0.909 shows that the situation both before and after project implementation was the same in the two types of communities. It appears that the work of the MINSA through its community workers, mobile clinics and regional fairs, events which gather a large number of clients for the centers, has been more effective in control communities than in counterpart communities, and this has neutralized the work of ReproSalud, the impact of which it has not been possible to isolate with the indicators provided. Net Change -0.5 0.5 Counterpart Control 42.5 45 47.5 50 52.5 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 52 Figure 31: Percentage of Women Who Have Visited a Health Center for Family Planning or Reproductive Health Services Differences could also be due to the fact that women did not receive a good response when they sought professional attention for genital tract infections or other reproductive health problems, as was noted during the interviews in the process evaluation. At the time, many women complained of the waiting time, of the high cost of the service, of the price of medicines, and especially of not receiving the results of the Papanicolau examination. These findings are in stark contrast with the notable increase in the use of health services for pre and postnatal check-ups, as discussed in section III.B of this report. Treatment of Vaginal Discharge (or ‘White Period’) As has been mentioned, discharges were reported as the main reproductive health problem for women in 36.6 percent of the participatory appraisal exercises. This is just over a third of the communities taking part in the project; in Ucayali it was considered to be the most important problem faced by women in 80 percent of the communities, in 62 percent of the communities in Ayacucho and in 54 percent of the communities in Huancavelica. Given these findings and the suffering expressed by the women themselves, the project aimed to target this problem by explaining how and why it occurs, how it can be prevented, how it should be treated and especially by trying to persuade those who suffer from it to attend a health center in order to seek professional help. The figures show the percentage of women who were attended by trained personnel and those who were attended by ‘traditional’ doctors. Project results on this issue are significant and reflect the efforts made to concentrate activities on it. Indeed, the percentage of women who sought help for vaginal discharge 25 in counterpart communities rose from 63.2 percent to 73.4 percent while in control communities it fell 25 Amongst those women who said they suffered from this problem. Net Gain 15.9 17.2 Counterpart Control 20 25 30 35 40 45 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 53 from 71.5 percent to 70.5 percent. Thus, net gain was 10.2 percentage points in the former and -1.0 points in the latter (figure 32). The value of OR = 1.178 shows that as a result of project activities the percentage of women who sought help for discharges increased by 18 percent more in counterpart communities. Figure 32: Women’s Health Seeking Behavior for Vaginal Discharge The percentage of women who sought the help of trained personnel experienced a smaller increase, from 50.6 percent to 61.4 percent in counterpart communities (a net gain of 10.8 points) and from 51.4 percent to 58.4 percent in control communities (a net gain of 7 points) (indicator 30.a in table 22). The balance of the partial changes by type of community shows that in counterpart communities the project increased the number of women who sought the help of trained personnel for vaginal discharge by almost 7 percent more, OR = 1.068. These figures show that women in counterpart communities are more worried if they have vaginal discharge and deal with this by seeking help either through formal or informal health services. For these women, it is also valid to attend a traditional doctor or herbalist, who can help improve external symptoms, although not necessarily cure the problem. It is a legitimate wish to resort to traditional medicine, although probably not effective in the case of abnormal (i.e. non-physiological) discharges caused by bacteria, fungi or virus which require specific treatment through antibacterial, antifungal or antiviral preparations. Home remedies or traditional medicine may be effective in the case of normal physiological discharges such as occur, for example, after menstruation and during pregnancy. Percentage of women who sought help for vaginal discharge 45 50 55 60 65 70 75 Base Line Midterm Evaluation Percentage Control Percentage of women who sought help from trained personnel for vaginal discharge 45 50 55 60 65 70 75 Base Line Midterm Evaluation Percentage Counterpart Net Change 10. 2 - Counterpart Control 1.0 Net Gain 7. 10. Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 54 Table 22: Help for Problems of Vaginal Discharge Counterpart Communities Control Communities N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. 30 percent of women who sought help for vaginal discharge 63.2 73.4 10.2 -2.673 Sig 0.05 71.5 70.5 -1.0 0.257 30.a percent of women who sought help from trained personnel for vaginal discharge 50.6 61.4 10.8 -2.669 Sig 0.05 51.4 58.4 7.0 -1.637 Sig 0.10 Self Care The objective of ReproSalud was not only to encourage women to attend formal health services, but also to use other means of improving health, including self-care. The assumption is that even by simply confiding a problem to a neighbor or family member this might eventually lead to professional help after a chain of events or persons which would ultimately lead to a health center. Prior to a specific practice of self-care as indicative of change towards improvement of the quality of life of women, the project aims to increase the knowledge of women regarding health-related issues. On the basis of this assumption the indicators of table 23 were developed, the values of which show that the project has improved the information available to women in counterpart communities on issues relating to reproductive health. In particular, the percentage of those who know how discharges are transmitted has increased from 4.9 percent to 19 percent, a net gain of 14.1 percentage points. By contrast, in control communities this increase was smaller, from 4.7 percent to 11.4 percent, a net gain of scarcely 6.7 points. As can be seen in figure 33 and in table 23, the two types of community start from a similar percentage of around 4.7 percent, but after project activities the lines diverge to give a significant difference on the second survey in favor of counterpart communities. As a result, the gain attributable to the project regarding knowledge on the way in which discharges are transmitted is quite high, namely 60 percent. Figure 33: Percentage of Women Who Know How ‘White Period’ Is Transmitted Net Gain 14.1 6.7 0 Counterpart Control 5 10 15 20 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 55 There is also an increase in knowledge of the Papanicolau exam or of breast examination amongst women in counterpart communities of 18.2 percentage points (from 63.2 percent to 81.4 percent), while in control communities there was an increase of 12 points (from 59.6 percent to 71.8 percent) (indicator 43.a in table 23 and figure 34). Project contribution to the increase of this indicator was 7 percent. Figure 34: Percentage of Women Who Have Knowlege of the Papanicolau or Breast Examination The percentage of women capable of identifying risk symptoms during pregnancy and after childbirth which would warrant seeking medical attention has shown a similar increase (net gain of 26.6 percent) in both types of communities, as shown by the parallel lines in figure 35. However, on analyzing the base line and midterm evaluation values for each type of community, it can be seen that partial change was much greater in control communities (203 percent) than in counterpart communities (137 percent). Thus, the balance of partial changes gives an OR value = 0.782, showing that the project had no impact on this issue. Indeed, this indicator is the one of least impact for the project as per the ranking shown in table 25. Figure 35: Percentage of Women in a Relationship Who Are Able to Recognize Risk Symptoms During Pregnancy and After Childbirth Net Gain 12.2 18.2 Counterpart Control 55 60 65 70 75 80 85 Base Line Midterm Evaluation Percentage Counterpart Control Net Gain 26.6 26.8 Counterpart Control 10 20 30 40 50 Base Line Midterm Evaluation Percentage Counterpart Control MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 56 Table 23: Self Care Counterpart Communities Control Communities Nª Indicator Base line MTE* Difference Z Stat. Sig. Base line MTE* Difference z Stat. Sig. Knowledge 40 percent of women in a relationship who can recognize symptoms of risk during pregnancy and after childbirth 19.4 46.0 26.6 -12.428 Sig 0.05 13.2 40.0 26.8 -13.40 Sig 0.05 43 percent of women who know how 'white period' is transmitted 4.9 19.0 14.1 -10.568 Sig 0.05 4.7 11.4 6.7 -5.448 Sig 0.05 43.a percent of women who have heard of Papanicolau or breast examination 63.2 81.4 18.2 -11.037 Sig 0.05 59.6 71.8 12.2 -6.732 Sig 0.05 Practice 32 percent of women who look after their health to feel good 56.8 65.1 8.3 -4.368 Sig 0.05 65.2 56.4 -8.8 4.741 Sig 0.05 *Midterm evaluation There is a project indicator that has also been developed to measure the increase in positive appreciation of women's health within the household and within the community (RI 2.2). This relates to healthcare as a way to ‘feel good.’ It is difficult to explain the figures, which show an increase in counterpart communities (from 56.8 percent to 65.1 percent) and a drop in control communities (from 65.2 percent to 56.4 percent) (see figure 36). Figure 36: Percentage of Women Who Maintain Their Health to Feel Good It is possible that the women interviewed did not understand the question, although it could also be that women in counterpart communities learnt to appreciate themselves through their training and do indeed look after their health to ‘feel good’ or to ‘be healthy’. However, the decrease in control communities would require an explanation. Net Change 8.3 -8.8 Counterpart Control 55 57.5 60 62.5 65 67.5 Base Line Midterm Evaluation Percentage Counterpart Control RESULTS 57 D. SUMMARY OF CHANGES IN THE INDICATORS OF THE RESULTS FRAMEWORK This section aims to summarize the findings in respect of achievement of the SO and the IRs, which are shown in table 24, where the indictors are numbered as in the original version. The six indicators created for this evaluation have been included, as have the base line and midterm evaluation values of each indicator and the value of the OR. Net gains, as described and shown in section III.B and C of this report, measure the change in percentage points between the base line and the midterm evaluation and give a good indication of the benefits to counterpart communities of project activities. However, they do not show how different such gains are in respect of what they might have been in control communities, particularly given the fact that such communities are different from project communities. For this reason, this section attempts to give a more sophisticated analysis, using relative changes/increments (obtained by dividing the value of the indicator in the midterm evaluation by the value of the indicator in the base line) and explaining the strength of the relationship between such changes and the project by the value of the Odds Ratio, or disparity factor (table 24). According to these figures, the SO of the project, namely that women increase their use of reproductive health services, has been achieved. Indeed, the indicators defined for measuring its achievement showed a favorable change in the direction of the project's mission in the project communities. The relative increase in professional care during pregnancy was 50 percent in project communities (as compared to 30 percent in control communities), that of professional care during childbirth was 33 percent (as compared to only 4 percent in control communities) and prevalence in the use of contraception was 23 percent (as compared to 7 percent). The reduction in unmet family planning needs was 34 percent (as opposed to 19 percent). The values of the OR confirm this. Under equal conditions, the OR values would be 1, but in the presence of project activities anything over one can be attributed to such activities. Thus, the project has achieved that professional care during childbirth and the use of contraception have each increased by 15 percent more in counterpart communities, and professional care during childbirth has done so by 27 percent more. The unmet demand for family planning has also been reduced by 18 percent more in counterpart communities than in control communities. As regards IR 1, which aims for women to have more equitable gender relations with their partner and with their family, the project achievements can be clearly seen in three of the six indicators. In the communities where ReproSalud is working, 23 percent more women than in control communities decide what to spend the money they earn on, 13 percent more decide together with their partner on sexual relations, methods of contraception and number of children, and 11 percent more decide together with their partner on the level of education for their daughters and sons. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 58 In the light of these results, the project did not produce any changes in indicators, which measure the participation of men in household chores. Thus, project communities and control communities are equal in terms of the percentage of women whose partner helps when the children are ill, in the percentage of women who carry out one or more household tasks together with their partner, and in the percentage of women who have talked with their partner regarding the number of children they wish to have. This last factor is not consistent with the increase in the percentage of women who decide together with their partner on, amongst other things, the number of children (indicator 7 in the Results Framework). Table 24: Relative Change Between The Base Line and the Midterm Evaluation for Each Indicator of the Results Framework, and Calculation of the Odds Ratio Values Communities Counterpart Control Indicators Base Line MTE Relative Change Base Line MTE Relative Change Odds Ratio SO: Women increase their use of reproductive health services 1.a percent of women who had 4 or more check-ups by trained personnel during the last pregnancy occurring two years before the survey 55.6 83.1 1.495 62.2 81.0 1.302 1.148 2. percent of women whose last delivery, two years before the survey, was under the care of trained personnel 36.1 48.0 1.330 40.9 42.7 1.044 1.274 3. percent of women in a relationship who use some method of contraception 58.4 71.8 1.229 60.6 64.6 1.066 1.153 4. percent of women in a relationship with unmet family planning needs (includes users of the rhythm method who are unaware of the fertile days of the cycle) 48.4 37.3 0.771 50.1 46.9 0.936 0.823 4.1 percent of women in a relationship with unmet family planning needs 27.1 17.9 0.661 26.1 21.1 0.808 0.817 IR 1: Women have more equitable gender relati ons with their partner and with their family 5. percent of women in a relationship whose partner helps when the children are ill 38.7 44.7 1.155 30.0 35.3 1.177 0.982 6. percent of women who decide what to spend the money they earn on 31.6 36.1 1.142 29.8 27.8 0.933 1.225 7. percent of women who decide together with their partner on sexual relations, methods of contraception and number of children 23.7 32.9 1.388 22.6 27.8 1.230 1.129 9. percent of women who carry out one or more household tasks together with their partner 23.2 46.7 2.013 18.6 37.9 2.038 0.988 11. percent of women who decide together with their partner on the level of education for daughters and sons 57.4 64.0 1.115 57.8 58.0 1.003 1.111 11.a percent of women in a relationship who have talked with their partner regarding the number of children they wish to have 67.8 72.6 1.071 69.3 74.0 1.068 1.003 IR 1.1: Women strengthen their capacity to bring about changes in their gender relations 12. percent of women who agree that women should take precautions even if their partner disagrees 68.9 64.4 0.935 65.0 60.4 0.929 1.006 12.1 percent of women who do not have sexual relations by force or persuasion 50.6 64.0 1.265 55.4 58.7 1.060 1.194 13. percent of women who have frequently talked with their partner regarding FP in the last 12 months 15.2 20.3 1.336 11.4 14.1 1.237 1.080 14. percent of women who would seek help from the police/ authorities in the event of physical abuse by their husband 14.3 38.4 2.685 15.7 37.2 2.369 1.133 16. percent of women who have frequently talked with their children regarding relationships and family planning in the last 12 months 7.1 16.0 2.254 3.9 10.4 2.667 0.845 RESULTS 59 Communities Counterpart Control Indicators Base Line MTE Relative Change Base Line MTE Relative Change Odds Ratio 16.a percent of women who know where to seek help or advice in the event of physical abuse 45.6 65.6 1.439 44.8 59.5 1.328 1.083 IR 1.2: Increase in the positive attitudes of men in their relati onship with their wife and their family 17. percent of men who believe that women should not be physically abused under any circumstances 57.4 64.0 1.115 58.8 63.0 1.071 1.041 18. percent of men who believe men have no right to demand sexual relations against their wife's will 84.6 91.5 1.082 88.8 91.8 1.034 1.046 IR 1.3: Women increase their knowledge on gender issue 19. percent of women who believe their work is as important as that of their partner 51.4 57.8 1.125 48.4 55.2 1.140 0.986 19.1 percent of men who believe that their work is as important as that of their partner 52.1 69.8 1.340 55.9 68.3 1.222 1.097 20. percent of women who believe that work in the home is as important as the work of their husband 33.7 44.1 1.309 30.2 42.7 1.414 0.926 20.1 percent of men who believe that the work done by their wife in the home is as important as the work of the husband 36.2 47.4 1.309 33.3 47.9 1.438 0.910 21. percent of women who believe that daughters and sons should have the same level of education 86.7 91.9 1.060 94.5 89.6 0.948 1.118 21.1 percent of men who believe that daughters and sons should have the same level of education 89.1 91.4 1.026 91.6 90.9 0.992 1.034 IR 2: Women have a greater capacity to access reproductive health services 24. percent of women who have attended a health center for matters of reproductive health or family planning 26.3 42.2 1.605 22.5 39.7 1.764 0.909 RI 2.1: Women increase their capacity as end users of formal health services 26. percent of women who believe that the services afforded by the nearest health center are good (or very good) 45.5 56.2 1.235 52.8 69.7 1.320 0.936 27. percent of women who have attended a health center for pre and postnatal check-up and believe that the services of the nearest center are good (or very good). 55.4 61.2 1.105 67.8 69.3 1.022 1.081 IR 2.2: Increase in the positive appreciation of women's health in the home and in the community 29. percent of women in a relationship who would attend a health center in the event of risk symptoms 42.6 71.6 1.681 36.8 62.2 1.690 0.994 30. percent of women who sought help for vaginal discharges 63.2 73.4 1.161 71.5 70.5 0.986 1.178 30.a percent of women who sought help for vaginal discharges from trained health personnel 50.6 61.4 1.213 51.4 58.4 1.136 1.068 32. percent of women who care for their health to feel good 56.8 65.1 1.146 65.2 56.4 0.865 1.325 33. percent of women who spent on their health in the last 12 months 49.5 50.0 1.010 45.7 45.2 0.989 1.021 IR 2.6: Women increase their knowledge on their reproductive health needs 38. percent of women in a relationship who know how al least one modern method works 13.5 55.5 4.111 14.9 31.9 2.141 1.920 39. percent of women using the rhythm method who are aware of the fertile days of the cycle 22.5 28.3 1.258 24.0 24.1 1.004 1.253 40. percent of women in a relationship who can recognize pre and post natal symptoms which imply risk 19.4 46.0 2.371 13.2 40.0 3.030 0.782 43. percent of women who know how 'white period' (vaginal discharge) is transmitted 4.9 19.0 3.878 4.7 11.4 2.426 1.599 43.a percent of women who have heard about Papanicolau or breast examination 63.2 81.4 1.288 59.6 71.8 1.205 1.069 43.b percent of women who believe that it is difficult to get pregnant while breastfeeding 34.0 56.3 1.656 30.9 46.9 1.518 1.091 MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 60 As regards IR 1.1, which aims for women to strengthen their capacity to bring about changes in their gender relations, the figures show important achievements in some indicators, and a modest or no impact in others. As regards the former, in counterpart communities project activities have increased the percentage of women who do not have sexual relations by force or persuasion by 19 percent more, and the percentage of women who would seek help from the authorities in the event of physical abuse by their husband by 13 percent more. A modest impact can be seen in the increase in the percentage of women who have talked with their partner about family planning26 (8 percent more than in control communities) and in the percentage of women who know where to seek help or advice in the event of abuse, which also increased by 8 percent more. The project had no impact on the percentage of women who agree that women should take precautions even if their partner disagrees, nor in the percentage of women who have talked with their children on relationships and family planning. The achievements in IR 1.2, which aims to increase the positive attitudes of men in their relationship with women and with the family, are small. Project activities have increased the percentage of men who believe that women should not be physically abused under any circumstances and the percentage of those who believe that men have no right to demand sexual relations against their wife's will by scarcely 4 percent. Achievements in IR 1.3, which aims to increase the knowledge of women on gender relations, have been modest for two indicators: the percentage of women who believe that daughters and sons should study to the same level, which due to project activities increased by 12 percent more in counterpart communities, and the percentage of men who believe that the work they carry out is as important as that of their partner, which increased by 10 percent more in counterpart communities. Achievements were small (3 percent) in the increase in the percentage of men who believe that daughters and sons should be educated to the same level. The project has had no impact on the increase in appreciation of the work of women by women nor, and this seems contradictory27, on the percentage of men who believe that the work of their wife in the home is as important as the work of the husband. It appears that men value the work of women if this takes place outside the home. It is not possible to reach a conclusion on IR 2, which aims to increase the capacity of women to access health services, since no indicators have been identified in the Results Framework to measure this. This evaluation developed one indicator, but it may perhaps not be the most appropriate. According to this indicator, the project has had no impact in counterpart communities as compared to control communities in the percentage of women who have attended a health center for issues of reproductive health or family planning. These findings are not consistent with the significant increases in prenatal care and care during childbirth by trained personnel, nor with the likewise significant increase in the use of contraception, which were discussed in relation to the SO. 26 although this figure is not consistent with the better achievement in indicator 7 which measures, among other things, having talked with their partner regarding methods of contraception. 27 since the project had a moderate impact on the percentage of men who believe that their work is as important as that of their partner RESULTS 61 As regards IR 2.1, which aims to increase the capacity of women as end users of formal health services, the results of the indicators which measure the valuation of the quality of the health services in the nearest health center are difficult to interpret, since they appear to be more suited to measuring empowerment, in which case they show a positive impact of the project. IR 2.2, which aims to increase the positive appreciation of the health of women within the home and within the community, shows significant achievements in two cases. Thanks to project activities, the percentage of women who look after their health to feel good increased by 32 percent more in project communities than in control communities, and the percentage of those who sought help for vaginal discharge problems increased by 18 percent more. However, the percentage of women who sought help for this reason from trained health personnel increased by only 6.8 percent more. The project has not affected the percentage of women who would attend a health center in the event of symptoms implying risk, and has had very little effect on the percentage of women who have spent on healthcare in the last 12 months. Lastly, achievements for IR 2.6 which aims to increase the knowledge of women regarding their reproductive health needs, have been generally quite significant, with the exception of the percentage of women in a relationship who recognize pre and post natal symptoms implying risk, which has not been affected by the project. Results for other indicators are, however, quite surprising. Particularly so is the percentage of women who know how at least one modern method of contraception works, which increased due to project activities by 92 percent more in counterpart communities than in control communities; the percentage of women who know how 'white period' is transmitted, which increased by 60 percent more; and the percentage of women who are aware of the fertile days of the cycle, which increased by 25 percent more. In the ranking of indicators, from greater to smallest impact of the project, these three are first, second and fifth, respectively (Table 25). The project has had a modest impact on the percentage of women who have heard about the Papanicolau examination or breast examination, which increased by 7 percent more through project activities, and in the percentage of women who believe that while a mother is breastfeeding it is difficult for her to get pregnant. As has been seen, the SO of ReproSalud has been fulfilled, since the indicators created for measuring it have increased significantly. Achievements in the IRs are varied, with a significant increase in some indicators, a moderate impact in others, and a modest impact in a few. There are also certain indicators on which the project has had no impact. As regards fulfillment of the SO (which is measured essentially with the indicators on use of health services), the increase in the use of contraception and the reduction in unmet family planning needs are consistent with the achievements in the relevant indicators: for example, the enormous increase in the percentage of women who know how at least one modern method works, in the percentage of women who are aware of the fertile days of the cycle, in the percentage of women who do not have sexual relations by force or persuasion, and the increase, to a slightly lesser degree than for the foregoing indicators, MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 62 of the percentage of women who believe that it is difficult for a women to become pregnant while breastfeeding, among others. However, the indicators proposed for measuring this do not easily explain the significant increase in care of pregnant women and during pregnancy and childbirth by trained personnel. Indeed, pursuant to the ranking of indicators, from greater to smallest project impact, as shown in table 25, two indicators on which the project has had no impact are: the percentage of women who have attended a health center for reproductive health or family planning issues, and the percentage of women who would attend a health center in the event of symptoms of risk. Furthermore, project impact is minimal on the percentage of women who have spent on healthcare in the last 12 months, although this could be explained by the fact that care during pregnancy and childbirth is free in public health centers. Table 25: Indicators from the Results Framework Ranked in Order of Impact (Greatest First) as Measured by the Value of the Odds Ratio Net gain, in percentage points, between the base line and the midterm evaluation Indicators Counterpart Control Odds Ratio Odds Ratio from 1.199 to 1.999 38. percent of women in a relationship who know how at least one modern method works 42.0 17.0 1.920 43. percent of women who know how 'white period' (vaginal discharge) is transmitted 14.1 6.7 1.599 32. percent of women who look after their health to feel good 8.3 -8.8 1.325 2. percent of women whose last delivery, occurring two years before the survey, was under the care of trained personnel 11.9 1.8 1.274 39. percent of users o f the rhythm method who are aware of the fertile days of the cycle 5.8 0.1 1.253 6. percent of women who decide what to spend the money they earn on 4.5 -2.0 1.225 4.1 percent of women in a relationship with unmet family planning needs -9.2 -5.0 0.817 4. percent of women in a relationship with unmet family planning needs (includes users of the rhythm method who are unaware of the fertile days of the cycle) -11.2 -3.2 0.823 Odds Ratio from 1.100 to 1.198 12.1 percent of women who do not have sexual relations by force or persuasion 13.4 3.3 1.194 30. percent of women who sought help for vaginal discharge problems 10.2 -1.0 1.178 3. percent of women in a relationship who use some method of contraception 13.4 4.0 1.153 1.a percent of wo men who had 4 or more check-ups with trained health personnel during their last pregnancy occurring two years before the survey 27.5 18.8 1.148 14. percent of women who would seek help from the police / authorities in the event of physical abuse by their husband 24.1 21.5 1.133 RESULTS 63 Net gain, in percentage points, between the base line and the midterm evaluation Indicators Counterpart Control Odds Ratio 7. percent of women who decide together with their partner on sexual relations, methods of contraception and number of children 9.2 5.2 1.129 21. percent of women who believe that daughters and sons should have the same level of education 5.2 -4.9 1.118 11. percent of women who decide together with their partner on the level of education for their daughters and sons 6.6 0.2 1.111 Odds Ratio from 1.050 to 1.099 19.1 percent of men who believe that their work is as important as that of their partner 17.7 12.4 1.097 43.b percent of women who believe that it is difficult to get pregnant while bre astfeeding 22.3 16.0 1.091 16.a percent of women who know where to seek help or advice in the event of physical abuse 20.0 14.7 1.083 27. percent of women who have attended a health center for pre and post natal check-up and believe that the services afforded by the nearest center are good or very good 5.8 1.5 1.081 13. percent of women who have frequently talked with their partner regarding family planning in the last 12 months 5.1 2.7 1.080 43.a percent of women who have heard of Papanicolau or breast examination 18.2 12.2 1.069 30.a percent of women who sought help from trained health personnel for vaginal discharge ('white period') 10.8 7.0 1.068 Odds Ratio from 1 to 1.049 18. percent of men who believe that men have no right to demand sexual relations against their wife's will 6.9 3.0 1.046 17. percent of men who believe that women should not be physically abused under any circumstances 6.6 4.2 1.041 21.1 percent of men who believe that daughters and sons should have the same level of education 2.3 -0.7 1.034 33. percent of women who spent on healthcare in the last 12 months 0.5 -0.5 1.021 12. percent of women who agree that women should take precautions even if the partner disagrees -4.5 -4.6 1.006 11.a percent of women in a relationship who have talked with their partner regarding the number of children they wish to have 4.8 4.7 1.003 Odds Ratio less than 1 29. percent of women who would attend a health center in the event of symptoms implying risk 29.0 25.4 0.994 9. percent of women who carry out one or more household tasks together with their partner 23.5 19.3 0.988 19. percent of women who believe that their work is as important as that of their partner 6.4 6.8 0.986 5. percent of women in a relationship whose partner helps when the children are ill 6.0 5.3 0.982 26. percent of women who believe that the services afforded by the nearest health center are good (or very good) 10.7 16.9 0.936 20. percent of women who believe that work in the home is as important as that of their husband 10.4 12.5 0.926 20.1 percent of men who believe that work done by women in the home is as important as the work of the husband 11.2 14.6 0.910 24. percent of women who attended a health center for issues of reproductive health or family planning 15.9 17.2 0.909 16. percent of women who have frequently talked with their children on relationships and family planning in the last 12 months 8.9 6.5 0.845 40. percent of women in a relationship who recognize some pre and post natal symptoms which indicate risk 26.6 26.8 0.782 It is very likely that the results are affected by variables which are difficult to control, given the lack of equivalence between counterpart and control communities, by the internal migration which affected both types of communities, and by other factors which have not been detected and which should be investigated and considered in future evaluations of the project. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 64 65 IV. PROJECT IMPACT AND RESULTS: COMPARING COMMUNITIES WITH EDUCATION ACTIVITIES AND COMMUNITY BANKS WITH COMMUNITIES WITH EDUCATION ACTIVITIES ONLY As explained, ReproSalud also aims to strengthen the economic capacity of women through an income generation component working in two areas: community banks and product development (the former entails a small financial loan, and the latter the provision of materials). This is done on the assumption that if women have the financial means for paying for health services, they will have a greater readiness to attend health centers to seek professional help. The income generation component also aims to empower women by improving their decision-making capacity and their ability to negotiate issues relating to their own lives, such as reproductive health, both on a domestic level (within the family) and a social level (with local authorities and health personnel). The empowerment of women sought by the project is a key element in the sustainability of its results and for its ultimate expansion. The benefits of the microcredit component mainly reached communities in Ucayali, San Martín, Puno and La Libertad, given that the CBOs and the women of these departments who applied for loans met the requirements set down for them. Beneficiary communities were initially among the 247 counterpart communities of ReproSalud, and had the same IE&C activities as the rest. The community bank component subsequently became a special project independent from ReproSalud. Given that these are two similar experiences, with economic activities in one of the cases to differentiate them, it is of interest to measure the impact in each in order to determine whether provision of financial support does in fact add to the achievements of the education activities. There are several problems of a diverse nature which restrict the possibilities for making such a comparison and reaching categorical conclusions, amongst them: ß The analysis is centered on communities having CBOs with community banks (hereafter referred to as banks), since there are very few (36 in total) with product development activities: Puno Aymara (9), Ayacucho (9), Ancash (5), Ucayali (4), Puno Quechua (5), San Martín (2) y Lima East (2). ß There are a total of 487 CBOs with banks, located in 53 districts (it was not possible to obtain information at community level). The base line and midterm evaluation surveys were carried out only in 8 communities having CBOs with banks, but control communities where only education activities MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 66 were carried out were found only for four of these: two in Puno Aymara and 2 in La Libertad. ß Given the foregoing, it was decided to carry out a trial study where the sample was reduced to 4 communities with banks28 and 4 control communities29 (see Annex 4). The results are therefore not representative of what might have occurred in the total number of communities with a component of community banks and should be considered to give an indication of rather than a categorical conclusion. ß Lastly, the impact evaluation of ReproSalud using household surveys was not designed to measure any additional results of the project deriving from income generation activities to complement education activities. Figures might contain biases that are difficult to identify and, what is worse, difficult to control. It could well be, for example, that women beneficiaries of the banks component are in a better socioeconomic position and have greater empowerment. ß The findings and trends observed in the indicators which are analyzed hereafter could serve to develop a specific evaluation for this issue. The format of this section is the same as that of section III.B and C of this report. It includes an analysis of the project impact and of the IRs. The three tables in annex F give a list of the indicators that were evaluated listed as per the Results Framework. The figures included the values of each in the base line and in the midterm evaluation, the number of cases, the statistical significance, including the value of z, and the value of theOR. A. PROJECT IMPACT. ACHIEVEMENT OF THE STRATEGIC OBJECTIVE Professional Care During Pregnancy and Childbirth Care during pregnancy by trained personnel has had a very similar increase in communities with banks and in those with only the education (IE&C) component, as shown by the two parallel lines in figure 37. Net gain between the base line and the midterm evaluation was 25.9 percentage points and 24.1 percentage points respectively. In the former, the percentage rose from 41.7 percent to 67.6 percent, while in the latter it rose from 49.1 percent to 73.2 percent. For both groups the differences between the two measurements is statistically significant to 5 percent (indicator 1.a of table 26). 28 with a sample of 186 women in the base line and 146 in the midterm evaluation 29 with a sample of 258 women in the base line and 212 in the midterm evaluation PROJECT IMPACT AND RESULTS 67 Figure 37: Prenatal Check-ups by Trained Personnel During the Last Pregnancy Table 26: Care in Pregnancy and Childbirth by Trained Personnel Communities with Banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig Base line MTE* Difference z Stat. Sig. 1.a percent of women who had 4 or more check-ups by trained health personnel during the last pregnancy occurring two years prior to the survey 41.7 67.6 25.9 -2.174 Sig 0.05 49.1 73.2 24.1 -2.362 Sig 0.05 2 percent of women whose last delivery, occurring two years prior to the survey, was attended to by trained health personnel 15.6 45.7 30.1 -2.950 Sig 0.05 30.6 46.7 16.1 -1.699 Sig 0.05 On the issue of professional care in childbirth a greater achievement can in fact be seen in communities with banks as compared to those with only the IE&C component (see figure 38). Net gain in the former was 30.1 percentage points, just under double the amount in the latter (16.1 points) (indicator 2 in table 26). The percentage of women whose last delivery was attended to by trained health personnel in communities with banks almost trebled (a net gain of 30 percentage points) in the period between the base line and the midterm evaluation (from 15.6 percent to 45.7 percent), while in control communities the increase was only 16 points (from 30.6 percent to 46.7 percent). 40 50 60 70 80 Base Line Midterm Evaluation Percentage Banks IE&C Net Gain 25.9 24.1 Banks IE&C MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 68 Figure 38: Last Delivery Attended by Trained Health Personnel Prevalence in the Use of Methods of Contraception and Unmet Family Planning Needs The indicators relating to family planning vary greatly. On the one hand, prevalence of use increases in both types of community (figure 39), but net gain between the base line and the midterm evaluation is only slightly higher (5.5 percentage points) in communities with banks than in communities with IE&C (4.1 points). Table 27 shows the percentages for the base line and the midterm evaluation measurements of indicator 3. Statistically, both communities have remained unchanged after two or three years of project activities. In this case, the project only increased the possibility that women in communities with banks may use contraceptives by 2.5 percent more (OR = 1.025) (see table 1 in annex F). Figure 39: Percentage of Women in a Relationship Using Some Method of Contraception 10 20 30 40 50 Base Line Midterm Evaluation Percentage Banks IE&C Net Gain 30.1 16.1 Banks IE&C Net Gain 5.5 4.1 Banks IE&C 60 65 70 75 80 Base Line Midterm Evaluation Percentage Banks IE&C PROJECT IMPACT AND RESULTS 69 Analysis of unmet family planning needs, which is a way of measuring the use of family planning methods as compared to the demand for them, shows a favorable change in communities with banks. The reduction was much higher in these communities than in communities with only IE&C, as shown in figure 40 and table 27 (indicators 4 and 4.1). Indeed, unmet family planning needs measured in the traditional way, as is done by DHS, decreased by 7.6 percentage points in the former communities as compared to 0.5 in the latter. This decrease from 23.5 percent to 15.9 percent in communities with banks is significant to 10 percent. Table 27: Use of Contraception and Unmet Family Planning Needs Communities with Banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE * Difference z Stat. Sig. 3 percentage of women in a relationship who use some method of contraception 62.7 68.2 5.5 -0.819 67.5 71.6 4.1 -0.726 4.1 percentage of women in a relationship with unmet family planning needs 23.5 15.9 -7.6 1.336 Sig 0.10 21.0 20.5 -0.5 0.101 4 percentage of women in a relationship with unmet family planning needs (includes users of the rhythm method who are unaware of the fertile period) 48.3 42.0 -6.3 0.895 42.6 42.1 -0.5 0.082 *Midterm evaluation Insufficient protection (which includes women who do not wish to have more children or who do not wish to have them soon and use the rhythm method but are not aware of the fertile period) dropped by 6.3 percentage points in communities with banks and by 0.5 points in communities with IE&C, again showing a more favorable situation in the former. The values of OR = 0.697 for indicator 4.1 and OR = 0.888 for indicator 4 show a greater impact of the project in communities with banks. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 70 Figure 40: Women in a Relationship with Unmet Family Planning Needs Awareness of Certain Aspects in the Use of Methods of Contraception The knowledge of how at least one modern method of contraception works has increased dramatically in communities with banks (48.9 percentage points) in the period between the base line and the midterm evaluation, from 6.1 percent to 55.0 percent. It should be noted that the initial value (6.1 percent) increased by a factor of 8 to reach 55 percent at the time of the midterm evaluation. In communities with IE&C increase was also significant (net gain of 31.5 percentage points), though less dramatic (from 25.8 percent to 57.3 percent) (indicator 38 in table 28). The value of OR = 4.060 shows an excellent impact of the banks project. Table 28: Awareness of Methods of Contraception Communities with banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE * Difference z Stat. Sig. 38 percentage of women in a relationship who know how at least one method of contraception works 6.1 55.0 48.9 -9.893 Sig 0.05 25.8 57.3 31.5 -6.936 Sig 0.05 39 percentage of users of the rhythm method who are aware of the fertile period 31.4 34.9 3.5 -0.311 32.4 37.4 5.0 -0.475 43.b percentage of women who believe that it is difficult to get pregnant while breastfeeding 39.0 57.2 18.2 -2.751 Sig 0.05 39.2 60.7 21.5 -3.910 Sig 0.05 *Midterm evaluation Women in a relationship with unsatisfied FP needs Includes users of rhythm who are unaware of the fertile period 35 40 45 50 55 60 Base Line Base Line Midterm Evaluation Percentage Women in a relationship with unmet family planning needs 10 15 20 25 30 35 Base Line Midterm Evaluation Percentage IE&C Net Change -7.6 -0.5 Banks IE&C Net Change -6.3 -0.5 Banks IE&C PROJECT IMPACT AND RESULTS 71 By contrast, the percentage of women using the rhythm method who are aware of the fertile period and the percentage of women who believe that it is difficult to get pregnant while breastfeeding, has increased slightly less in communities with banks than in those with only IE&C (see table 28, indicators 39 and 43.b and figure 41). The OR values for these indicators are less than one, which shows that the project had no additional impact in communities with banks. Figure 41 Analysis of the indicators for project impact which measure achievement of the SO seem to suggest, albeit in an inconclusive manner given the previously mentioned limitations with regard to representativity of the figures (and probably because women of communities with banks enjoy a better social position), that the microcredit component provides additional improvements to some indicators relating to the use of formal health services. In particular, there is a greater increase in communities with banks in the percentage of care in childbirth by trained health personnel, and a greater decrease in unmet family planning needs, but this does not seem to bear much relationship to the increase in the use of contraception, which is only slightly higher than in communities with IE&C. As regards indicators for measuring awareness of methods, the banks component appears to add greatly to the increase in knowledge of how at least one modern method works, but it does not contribute to increased awareness regarding the fertile days of the cycle, nor regarding the benefits of breastfeeding as a method of contraception. Percentage of women in a relationship who know how at least one modern method works 0 10 20 30 40 50 60 Base Line Midterm Evaluation Percentage IE&C Percentage of users of the rhythm method who are aware of the fertile period 20 25 30 35 40 45 Base Line Midterm Evaluation Percentage Banks Net Gain 31.5 48.9 Banks IE&C Net Gain 5.0 3.5 Banks IE&C MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 72 B. INTERMEDIATE RESULTS (IRS) Gender Relations Perceptions A somewhat odd result appears in communities with banks, in that on gender issues they seem to systematically show a decrease as compared to the same opinions in communities where only education activities were carried out. This worsening of opinion on issues, which shows awareness of gender equity concepts, could be due merely to chance or be brought about by the mentioned limitation of the information available, despite the figures shown in table 29. As an example, the percentage of women who believe that daughters and sons should be educated to the same level decreased in communities with banks from 96.7 percent to 78.7 percent (a net loss of -18 percentage points), and that of men dropped from 95.7 percent to 93.5 percent (a net loss of -2.2 points) (see figure 42). It seems surprising that the favorable opinion of men on the same rights of education for daughters and sons does not drop as much as does that of women. This would merit a more in-depth analysis of the differences of opinion between the sexes, complementing the figures obtained in household surveys with information from other sources, such as, for example, an ethnographic analysis. Figure 42: Men’s and Women’s Belief About Children’s Education Level Percentage of women who believe that daughters and sons should be educated to the same level 75 80 85 90 95 100 Base Line Midterm Evaluation Percentage IE&C Percentage of men who believe that daughters and sons should be educated to the same level 75 80 85 90 95 10 0 Base Line Midterm Evaluation Percentage Banks Net Change -18.0 5.3 Banks IE&C Net Change -2.2 3.0 Banks IE&C PROJECT IMPACT AND RESULTS 73 Table 29: Perceptions of Women and Men on Various Aspects of Gender Relations Communities with Banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Differenc e z Stat. Sig. Perceptions of women 21 percentage of women who believe that daughters and sons should be educated to the same level 96.7 78.7 -18.0 2.929 Sig 0.05 86.5 91.8 5.3 -1.096 Perceptions of men 21.1 percentage of men who believe that daughters and sons should be educated to the same level 95.7 93.5 -2.2 0.429 84.5 87.5 3.0 -0.441 17 percentage of men who believe that women should not be physically abused under any circumstances 75.5 73.3 -2.2 0.338 53.1 79.7 26.6 -3.688 Sig 0.05 18 percentage of men who believe that men have no right to demand sexual relations against their wife's will 76.8 93.0 16.2 -3.007 Sig 0.05 76.0 92.4 16.4 -2.619 Sig 0.05 *Midterm evaluation On the perception of men regarding respect for women, the situation deteriorates further. In communities with banks the percentage of men who believe that women should not be physically abused under any circumstances dropped by 2.2 percentage points, while in communities with IE&C it increased by 26.6 percentage points (indicator 17 in table 29). The percentage of men who believe that men have no right to demand sexual relations against their wife's will increased by practically the same amount (16 percentage points) in the two types of communities (indicator 18 in table 29 and figure 43). Figure 43: Women’s and Men’s Beliefs about Forced Sexual Relations Percentage of men who believe that women should not be physically abused under any circumstances 50 55 60 65 70 75 80 85 Base Line Midterm Evaluation Percentage Banks Percentage of men who believe that men have no right to demand sexual relations against their wife's will 65 70 75 80 85 90 95 100 Base Line Midterm Evaluation Percentage IE&C Net Change 16.4 16.2 Banks IE&C Net Change 26.6 -2.2 Banks IE&C MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 74 From the results regarding perceptions on gender issues, it could be concluded that the banks component makes no additional contribution to the education component. Indeed, in some indicators it seems to have a negative impact, canceling out any capacity building activities. Once again, it should be pointed out that the trends described could be due to chance or to the lack of representative figures. Practices A similar result to that obtained for perceptions can be seen in the indicators, which measure practices. In communities with banks, there is either a smaller improvement than in IE&C communities, or a drop in the period between the two surveys, as shown in table 30. This is the case, for example, of the percentage of women in a relationship who carry out one or more household tasks together with their partner: an increase from 26.3 percent to 38.6 percent in communities with banks, compared to an increase from 33.1 percent to 67.2 percent in IE&C communities. Thus, net gain in the latter was more than double (34.1 percentage points) that in the former (12.3 percentage points). The same is true for the percentage of women who have talked with their partner regarding the number of children they wish to have: net increase in communities with banks was 2.5 points as compared to 10.4 percentage points in communities with IE&C only, almost five times greater. This change in some of the mentioned indicators is shown in figure 44. Table 30: Practices of Women and Men Regarding Gender Issues Communities with Banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. Practices of women 7 percentage of women in a relationships who decide together with their partner on sexual relations, methods of contraception and number of children 22.2 30.6 8.4 -1.348 Sig 0.10 28.2 38.7 10.5 -1.804 Sig 0.05 9 percentage of women in a relationship who carry out one or more household tasks together with their partner 26.3 38.6 12.3 -1.879 Sig 0.05 33.1 67.2 34.1 -5.577 Sig 0.05 11 percentage of women in a relationship who decide together with their partner on the level of education for daughters and sons 75.2 72.7 -2.5 0.377 55.4 51.9 -3.5 0.549 11.a percentage of women in a relationship who have talked with their partner on the number of children they wish to have 75.8 78.3 2.5 ? 73.5 83.9 10.4 -2.036 Sig 0.05 Practices of men 5 percentage of women in a relationship whose partner helps when children are ill 16.7 47.4 30.7 -4.454 Sig 0.05 30.3 32.7 2.4 -0.338 The care of children who are ill by men has improved significantly in communities with banks, the percentage increasing from 16.7 percent to 47.4 percent, while in communities with IE&C it increased from 30.3 percent to 32.7 percent. Thus, net gain was 30.7 percentage points in the former and 2.5 points in the latter (see figure 44). For the group PROJECT IMPACT AND RESULTS 75 of indicators in table 30, only indicator 5 shows a greater impact of communities with banks. Figure 44 In general, the changes discussed here, and the figures themselves, would require a more in-depth analysis to explain unexpected behaviors, particularly on gender issues which are the basis of ReproSalud and which showed significant achievements when comparing counterpart communities with control communities in section III.C of this report. Empowerment: Knowledge, Attitudes, and Practices The indicators for empowerment also show erratic results in communities with banks: a) some increase significantly, such as for example the percentage of women who would seek help in the event of physical abuse by their husband, which goes from 19.5 percent to 43.2 percent (a net gain of 23.7 percentage points) compared to an increase from 18.5 percent to 35.7 percent (a net gain of 17.2 points) in communities with IE&C; b) others increase less than in control communities. This is the case in the percentage of women Percentage of women who carry out one or more household tasks together with their husband 20 30 40 50 60 70 Base Line Midterm Evaluation Banks IE&C % Net gain 34.1 12.3 Banks IE&C Percentage of women in a relationship who have talked with their partner regarding the number of children they wish to have 50 60 70 80 90 100 Base Line Midterm Evaluation Banks IE&C % Net gain 2. 5 10. 4 Banks IE&C Percentage of women who decide together with their partner on the level of education for daughters and sons 40 50 60 70 80 Base Line Midterm Evaluation Banks IE&C % Net loss -3.5 -2.5 Banks IE&C Percentage of women in a relationship whose partner helps when the children are ill 10 20 30 40 50 Base Line Midterm Evaluation Banks IE&C Net gain 30.7 2.4 Banks IE&C MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 76 who do not have sexual relations by force or persuasion, which increased by only 5.2 points, while in communities with IE&C it increased by almost 20 points, and c) some remain unchanged, although it could be said that they decrease slightly in the period between the base line and the midterm evaluation. As an example, the percentage of women in a relationship who decide what to spend the money they earn on, which remained unchanged at a value of around 36 percent at both surveys (see figure 45). However, it should be pointed out that in communities with IE&C the decrease was greater (6.5 percentage points). Figure 45: Women’s Empowerment The figures for each indicator and the changes between the base line and the midterm evaluation are shown in table 31. The values of the OR (table 1 of annex F) for the first three indicators of the table show that the banks component results in an improvement of 26 percent for indicator 12, of 15 percent for indicator 14 and of 18 percent for indicator 6. Only indicator 12.1 shows a better improvement for communities with the education component only. Percentage of women who decide what to spend the the money they earn on 25 30 35 40 45 50 55 Base line Mid-term evaluation Banks os IE&C % Net loss -6.5 -0.3 Banks IE&C Percentage of women who agree they should take care not to have more children even if the husband disagrees 50 55 60 65 70 75 80 Base line Mid-term evaluation Banks IE&C % Net change -10.9 4.0 Banks IE&C Percentage of women who do not have sexual relations by force or by persuasion 40 45 50 55 60 65 70 Base Line Midterm Evaluation Banks IE&C % Net gain 19.4 5.2 Banks IE&C Percentage of women who do not have sexual relations by force or by persuasion 40 45 50 55 60 65 70 Base Line Midterm evaluation Banks IE&C % Net gain 19.4 5.2 Banks IE&C PROJECT IMPACT AND RESULTS 77 Table 31: Some Indicators of Perceptions, Attitudes, and Practices Relating to Empowerment Communities with banks Communities with IE&C N° Indicator Base line MTE* Difference z Stat. Sig. Base line MTE* Difference z Stat. Sig. Perceptions 12 percentage of women who agree that women should take precautions even if the partner disagrees 51.7 55.7 4.0 -0.569 76.4 65.5 -10.9 1.978 Sig 0.05 Attitude 14 percentage of women who would seek help from police/authorities in the event of abuse by their husband 19.5 43.2 23.7 -3.685 Sig 0.05 18.5 35.7 17.2 -2.624 Sig 0.05 Practice 6 percentage of women in a relationship who decide what to spend the money they earn on 36.2 35.9 -0.3 0.056 41.5 35.0 -6.5 1.434 Sig 0.10 12.1 percentage of women who do not have sexual relations by force or persuasion 63.4 68.6 5.2 -0.777 41.3 60.7 19.4 -3.149 Sig 0.05 *Midterm evaluation Use of Health Services The assumption that women would tend to use health services more often if they had the financial means to cover the cost of treatment and medicines, which was taken into account for including a community bank component in the project, would appear to be true. Indeed, the two indicators selected for analyzing this aspect show significant positive changes between the base line and the midterm evaluation, as shown in table 32. As an example, the percentage of women who spent on their health in the last 12 months increased in communities with banks from 39.2 percent to 57.9 percent (a net gain of 18.7 percentage points), while in communities which had only the educational component this percentage decreased by 2.0 points (from 46.8 percent to 44.8 percent) (see figure 46). Table 32: Indicators for the Use of Services Banks IE&C N° Indicator Base line MTE* Difference z Sig. Stat. Base line MTE* Difference z Stat. Sig. 33 percentage of women who spent on health in the past 12 months 39.2 57.9 18.7 -3.387 Sig 0.05 46.8 44.8 -2.0 0.433 25 percemtage of women who attended a health center for an issue of reproductive health or family planning 20.8 32.8 12.0 -2.472 Sig 0.05 28.8 32.2 3.4 -0.798 In the same way, the percentage of women who attended a health center for an issue of reproductive health or in order to receive information and/or services on family planning, also increased four times more in communities with banks (from 20.8 percent to 32.8 percent, a net gain of 12 percentage points) than in communities which had only the IE&C component (from 28.8 percent to 32.2 percent, a net gain of 3.4 points). MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 78 Figure 46 In general, the indicators on the use of health services, both those just described and those for care in childbirth by trained personnel, show that adding to the education component a component to financially benefit women improves their good practices towards health and their readiness to attend a health center for professional care in the event of problems related to reproductive health, and possibly also in other cases. These findings are, however, not conclusive and would warrant a more in-depth analysis and an explanation for unexpected and at times contradictory results. Percentage of women who have spent on health in the last twelve months 35 40 45 50 55 60 Base Line Midterm Evaluation Percentage Banks Percentage of women who have attended a health center for issues of RH or FP 15 20 25 30 35 40 Base Line Midterm Evaluation Percentage IE&C Net Change -2.0 18.7 Banks IE&C Net Gain 3.4 12.0 Banks IE&C 79 V. CONCLUSIONS The information provided by a sample of women of reproductive age reveals that in the communities where ReproSalud is working there is a general tendency towards improved health practices as compared to communities where there were no project activities. According to the information collected, the SO of the project has been met because women are making greater use of the health services. This results in a significant increase in prenatal care and childbirth attended by trained personnel and in the prevalence of contraception, as well as in a reduction of the unmet demand for family planning in the project's counterpart communities as compared to those where there was no project intervention. Net gain/variation prior to project intervention and after an average of three years in counterpart communities is significant for the mentioned indicators and is evidence of the contribution that ReproSalud has made to reproductive and maternal health to complement the work carried out by the Peruvian government. When ReproSalud began, Peru was undergoing important social changes, especially as regards health issues, and this is still the case, though to a lesser extent. MINSA, with the support of external aid organizations30, concentrates its efforts on improving the quality of mother-and-child healthcare and its coverage by the government institutions; as well as on increasing dissemination of information and services on contraceptive methods within the framework of the National Family Planning Program. It is possible that because of this several indicators also show significant improvement in control communities, which is graphically represented by parallel lines in the trend for such indicators in both types of communities. As regards gender issues, empowerment and the use of health services the achievements of the project vary and separate comments are provided for each IR. But it can generally be stated that women beneficiaries31 in remote communities or in peripheral areas in some cities of the project's area of influence have absorbed the knowledge they have obtained and are transforming it into positive attitudes and assertive behavior which contribute to improve their negotiating capacity on a domestic and social level. As regards IR 1, the aim of which is for women to achieve gender-equitable relationships with their partners and families, the project has made significant inroads on women's empowerment. The percentage of women deciding on the use of the money they earn and taking decisions jointly with their partners regarding sexual relations, methods of contraception, number of children and the level of education for daughters and sons has 30 The United Nations, the United States Agency for International Development USAID, the Inter￾American Development Bank, the World Bank, the Japanese Development Agency, the British Department for International Development (DFID), the Spanish International Development Agency, amongst others. 31 Women for whom this was the first opportunity to hear about such matters went from an initial stage of uneasiness (given that these are issues which, in their culture and in their minds, were 'taboo') to one of satisfaction and subsequent gratitude (and there is ample verbal evidence of this) for having been given knowledge and information which has had a positive effect on the way they live their lives. MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 80 increased. However, the project has failed to increase male participation in household tasks. As regards IR 1.1, which aims to strengthen the capacity of women to bring about change in their gender relations, the percentage of women who do not have sexual relations by force or persuasion has increased significantly, as has that of women who are willing to go to the police or to the authorities if they are assaulted by their partners. Moderate success has been achieved in improving communications between women and their partners on family planning, and in raising awareness of where to go for help or advice if they are physically abused. The project has made no impact on its target communities as compared to control communities regarding favorable attitudes to women exercising the right to protecting themselves against an unwanted pregnancy even when their partner is not in agreement, nor in the percentage of women who have talked with their children on relationships and family planning. The project has achieved modest success as regards IR 1.2, the aim of which is to increase positive attitudes of men in their relationships with women and with the family. The percentage of men who believe that women can under no circumstances be physically abused, and of those who believe that men have no right to demand sexual relations with a women if she does not wish it, changed little in project communities as compared to control communities. As regards IR 1.3, the aim of which is to increase women’s knowledge on gender equity, the project had a significant positive impact on the percentage of women who believe that daughters and sons should be educated to the same level and on the percentage of men who believe that the work they carry out is as important as that of their partner; there was a low impact on the percentage of men who believe that daughters and sons have the same rights to education; but there has been no change in the value attached by women to women's work nor in the positive appreciation by men of women’s work within the home. The latter seems to suggest that men appreciate women’s work more when this is done outside the home. These low achievements of the project in respect of gender perceptions should be analyzed in the context of the changes that are taking place on such issues generally within society. The Cairo Conference has given rise to a current of public opinion, which favors discussion and positioning on equity relations among the sexes, the rights of women and empowerment, and these issues have become frequently discussed topics, even on local radio and television programs. This would explain why perceptions on gender amongst project communities and control communities are similar, implying that no specific intervention is necessary to achieve positive change. However, the project has speeded up these changes to a considerable extent. It is difficult to reach a conclusion on IR 2, which aims to build women’s capacity to access health services, given the lack of indicators within the Results Framework to measure such results. The only available indicator, which was developed for this evaluation (and which may not be the most suitable), shows that in project communities CONCLUSIONS 81 the percentage of women attending a health center for matters of reproductive health or family planning has not been any higher than in control communities. This seems to contradict the significant achievement reported under the indicators for the SO. As regards IR 2.1, which aims to increase the capacity of women as end users of health services, the project has moderately increased positive opinion on the quality of the service of women who attended health centers for prenatal and post-delivery care, but it has not had the same effect on those attending for any other complaint or those who went to accompany somebody else. As regards intermediate Result 2.2, which aims to increase the positive appreciation of women's health within the home and within the community, the project has had a significant impact on the percentage of women who take care of their health in order to feel good and of those who sought help complaining of vaginal discharge. It has had a low impact on the percentage of women who have invested in their health, and has had no impact on the willingness of women to attend a health center for alarm signals that might be indicative of risk. The project’s success is clear on studying the indicators for IR 2.6, the aim of which is to increase women's knowledge of their reproductive health needs. In fact, there has been a surprising increase in the knowledge of how at least one of the modern methods works (this being the most important indicator out of the 39 analyzed) and how vaginal discharge is transmitted (second in importance), and a moderate impact on the percentage of women who use the rhythm method and are aware of the fertile days of the cycle (fifth in order of importance). There is a slight impact on the percentage of women who have heard of Papanicolau or breast examination and on the percentage of those with an opinion on the contraceptive advantages of breastfeeding. The project has had no effect on the level of knowledge on complications in pregnancy and birth that are indicative of risk. Figures in relation to achievement of the SO of the project are conclusive. The increase in the use of contraception and the reduction in the unmet demand for family planning are consistent with the very significant increase in the knowledge of how at least one modern method works, with the moderate increase in the knowledge of the fertile days of the cycle and with the percentage of women who do not have sexual relations by force or persuasion. But the significant increase in prenatal and birth care by trained personnel is not so well supported, since the project has had no impact on indicators such as the percentage of women attending a reproductive health or family planning center, or in the percentage of women attending a health center due to conditions which might be indicative of risk. It is very possible that other indicators not included in the Results Framework could support such achievements, and the project must therefore focus on identifying these. As regards the additional impact of the project if an income generation component is added to the education component, results are not conclusive in that the analysis was based on limited data. Apparently, the activities of microcredit would provide an MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 82 improvement in the outcome of the project in matters relating to health care, as well as in the willingness to use formal care services. A larger increase in prenatal and birth care by trained personnel is observed in communities where a community bank exists, as compared to those where only educational activities are carried out. A greater use of contraception and a greater reduction in unmet family planning needs is also observed. However, as regards perceptions on gender and empowerment, not only does the microcredit component not have an added impact on the educational component, but rather it seems to have a negative effect on this. The positive and negative results of this section would merit a more in-depth study with more representative and better quality data. 83 VI. RECOMMENDATIONS 1. Evaluate the relevance of the indicators as regards the IRs they were developed to measure. Some examples: ß IR 1.2 aims to measure the increase in positive attitudes of men in their relationships with women and with the family; however, three proposed indicators (17, 18 and 18.1) measure opinions and perceptions, rather than attitudes. ß The same is true for IR 1.3, which aims to increase women's knowledge on gender issues; with the exception of indicator 23, all indicators measure opinions and ideas, rather than knowledge. ß As opposed to this, IR 2.2, which aims to increase the positive appreciation of women's health within the home and within the community, which should be measured with an indicator on perceptions, has three indicators which measure practices. ß The indicators measuring opinions on quality of the health services do not seem to be the most relevant for measuring IR 2.1: improving the capacity of women as end users of formal health services. To a certain extent they measure empowerment, but even this is not clear. 2. Create new indicators to measure IRs and to complement those which already exist (some have been suggested in Table 1), taking into account the information obtained in the last version of the individual questionnaire given to women of reproductive age and to men between the ages of 15 and 59. 3. Review the phrasing and definition of some of the indicators in the IRs of the Results Framework, such as: ß Define the number of times which an event must take place for it to be considered 'very frequent' in indicators 13, 13.1 and 16. ß Fix the period for care by trained personnel in the last pregnancy and/or birth (indicators 1, 1a and 2) at two years prior to interview, rather than three years. ß Re-phrase indicator 20 to compare work in the home with that carried out outside the home, or women's work in the home compared to men's work outside the home. At present the indicator compares 'work in the home to work done by your husband'. ß Rephrase or change indicator 32. It is possible that the question on which it is based may not be understood by women. The indicator can be developed with MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 84 a set of questions, asking in what way or ways women look after their health; then asking whether they attend a center for a medical examination Papanicolaou test or breast examination), how often they go, and thereby finding out why women look after their health. ß Divide indicator 7 into three sub-indicators to learn on what issues (sexual relations, family planning, or number of children) women have been able to take decisions jointly with their partner and thus achieve a more equitable gender balance. ß Re-phrase indicators 26 and 27 such that women’s opinion on the service refers to the center the woman actually attends for help rather than to the nearest center. It is very possible that women do attend the nearest center, but this should be specified in the indicator. 4. In the eventuality that USAID should request that the project provides the indicators specified in the Project Paper (Number 527–0355, August 1995, page vi) for measuring achievement of the objectives and intermediate results of ReproSalud, review the document and structure a reply. In fact, only a few of the indicators specified in that document can be obtained, the rest being very difficult if not impossible to obtain. The Paper quotes: "At the goal level, project success will be measured by such indicators as total fertility, maternal mortality, infant mortality, chronic malnutrition and sexually transmitted disease prevalence. At the subgoal level, success will be measured by percentage of women participating in decision-making at the local level; percent increase in women-controlled organizations at the community level and percent expansion of economic opportunities for women. At the purpose level, project success will be measured by contraceptive prevalence, contraceptive failure, contraceptive discontinuation, length of birth intervals, duration of exclusive breastfeeding, use of prenatal care, births attended by trained personnel, prevalence of genital tract infections and prevalence of iron-deficiency anemia." 5. The information available on ReproSalud, collected by means of the instruments described in Part One of this report, as well as through project monitoring, is plentiful, varied and diverse. It should be analyzed globally and by regions and published in investigative reports such that the various aspects of the work of ReproSalud and its achievements within the communities may be shared with other organizations and development agencies. Even the interviews carried out in the households, on the basis of which this evaluation was carried out, have scope for detailed analysis at department level if the 70 counterpart communities are included in the database, and at a global level by comparing counterpart communities with associated and control communities in order to study the aggregate effect of the activities in accordance with the type of community participation in the project. ANNEXES A: Documents Reviewed ..................................................................................................87 B: Results Framework ......................................................................................................89 C: Counterpart Communities Versus Control Communities............................................93 D: Indicators of Base Line and Midterm Evaluation Surveys ..........................................95 E: Sample of Communities with Community Banks and with Education Activities Only....................................................................................99 F: Indicator Tables..........................................................................................................101 ANNEX A DOCUMENTS REVIEWED MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 88 The Alan Guttmacher Institute. Clandestine Abortion: a Latin American Reality (Aborto Clandestino: Una realidad latinoamericana). New York: 1994. Anderson, J. Building Bridges (Tendiendo Puentes). Quality of service from the perspective of rural women and health service providers. Manuela Ramos. 2001. Coe, Anna-Britt. Health, Rights and Realities: An analysis of the ReproSalud Project in Peru. Center for Health and Gender Equity. Working Papers. 2001. Feringa, B. Two years in the field: ReproSalud seen through the eyes of women. (Dos años en el campo: ReproSalud visto a través de los ojos de las mujeres). 1999. Ferrando D. Prevalence of induced abortion in Peru. (Prevalencia del aborto inducido en el Perú.) Flora Tristán Center for the Peruvian Woman and Pathfinder International, (in print). 2001 INEI-CEPAL. Peru: Population Estimates and Projections (Perú: Estimaciones y Proyecciones de Población), 1950-2050. Urbana-Rural 1970-2025. Bulletin of Demographic Analysis N° 35. Population Investigations N° 1. 2001 Macro International, Inc. Survey on Demography and Family Health (Encuesta demográfica y de salud Familiar). General Report: 1986, 1991-1992, 1996 and 2000. United Nations Population Fund. International Conference on Population and Development. Cairo: 1994. United States Agency for International Development (USAID)/Peru. Office of Health, Population & Nutrition. Project Paper. Reproductive Health in the Community (ReproSalud). A bottom-up approach to promoting reproductive health & women´s empowerment. Project Number 527-0355. 1995. Yon, C. Reproductive Preferences and Contraception: Andean Women Talk (Preferencias Reproductivas y Anticoncepción. Hablan las Mujeres Andinas). Manuela Ramos. 2000. ANNEX B RESULTS FRAMEWORK MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 90 1 2 3 2.1 1.1 3.1 2.2 1.2 2.3 3.2 1.3 2.4 3.3 2.5 2.6 MARCO DE RESULTADOS Incremento de las mujeres en la identificación y desarrollo de actividades de salud reproductiva Mayor participación de las mujeres en la indentificación y desarrollo de actividades de acuerdo a sus prioridades Las OCBs y la población organizada incrementan sus habilidades para organizar servicios comunitarios de salud reproductiva Las mujeres en áreas rurales y peri￾urbanas tienen mayor acceso a las fuentes de generación de ingresos, crédito y a los mercados Aumenta la valoración positiva de la salud reproductiva de la mujer dentro de los hogares y comunidades Mujeres mejoran sus capacidades como consumidoras finales de servicios formales de salud Mujeres y varones incrementan sus expectativas de igualdad de género Autoridades sensibilizadas y con dispocición a realizar cambios basados en valores democráticos que incorporen derechos sexuales y reproductivos desde la perspectiva de las mujeres Participación efectiva de mujeres de OCBs en el proceso de formulación de propuestas de políticas, adecuación y fiscalización de programas de salud reproductiva Las mujeres de las OCBs aumentan sus habilidades para representar defender y negociar los intereses y derechos sexuales y reproductivos de las mujeres Alianzas con grupos establecidos de la sociedad civil, conociendo, apoyando y defendiendo las demandas y derechos sexuales y reproductivos de las mujeres de las OCBs Mayor capacidad y disposición de las mujeres rurales y peri-urbanas para buscar atención de SR en el sector formal Relaciones de género más equitativas de las mujeres con sus parejas y familias MUJERES INCREMENTAN LA UTILIZACION DE INTERVENCIONES EN SALUD REPRODUCTIVA La mujer busca cambiar la comunicación en sus relaciones de género Incremento de actitudes positivas de los varones en su relación con las mujeres y la familia Objetivo Estratégico Oferta de servicios que responden a las demandas desde la perspectivas de género Ambiente político favorable para políticas normas y programas relacionadas con salud reproductiva desde la perspectiva de la mujer de escasos recursos con énfasis en zonas rurales ANNEXES 91 Net increase, in percentage points, between the baseline and the midterm evaluation Indicators Intervention Control Odds Ratio Odds Ratio from 1.199 to 1.999 VERY SIGNIFICANT 38. percent of women who know how at least one modern contraceptive method works 42.0 17.0 1.920 43. percent of women who know how RTIs are spread 14.1 6.7 1.599 32. percent of women who take care of their health in order to feel well 8.3 -8.8 1.325 2. percent of women whose last delivery, occurring within the last two years, was performed by a healthcare professional 11.9 1.8 1.274 39. percent of women using the rhythm method who know the fertile days of their cycle 5.8 0.1 1.253 6. percent of women who decide how to spend the money they earn 4.5 -2.0 1.225 4.1 percent of women with an unmet need for family planning -9.2 -5.0 0.817 4. percent of women with an unmet need for family planning (including those who use the rhythm method and are unsure of their own fertile days) -11.2 -3.2 0.823 Odds Ratio from 1.100 to 1.198 SOMEWHAT HIGH 12.1 percent of women who state their unwillingness to be forced or convinced to have sex 13.4 3.3 1.194 30. percent of women who have sought treatment for symptoms of RTIs (or vaginal discharge) 10.2 -1.0 1.178 3. percent of women who use some method of birth control 13.4 4.0 1.153 1.a percent of women who had at least 4 prenatal visits to a healthcare professional during a pregnancy occurring within the last two years 27.5 18.8 1.148 14. percent of women would go to the police/authorities if their partner abused them 24.1 21.5 1.133 7. percent of women who make joint decisions with their partner about sexual relations, birth control methods, and number of children 9.2 5.2 1.129 21. percent of women who believe that their sons and daughters should reach the same level of education 5.2 -4.9 1.118 11. percent of women who make a joint decision with their partner about the educational level their children should reach 6.6 0.2 1.111 Odds Ratio from 1.050 to 1.099 MODERATE 19.1 percent of men who believe that the work they do outside the home is as important as the work their partner does outside the home 17.7 12.4 1.097 43.b percent of women who believe that while a women is breastfeeding it is dificult for her to become pregnant 22.3 16.0 1.091 16.a percent of women who know where to go for help or advice if they are abused 20.0 14.7 1.083 27. percent of women who have gone to a healthcare facility for prenatal and postnatal care, and who believe that the services provided by the nearest facility are good (or very good) 5.8 1.5 1.081 MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 92 Net increase, in percentage points, between the baseline and the midterm evaluation Indicators Intervention Control Odds Ratio 13. percent of women who have spoken with their partner about family planning more than 2 times in the last 12 months 5.1 2.7 1.080 43.a percent of women who have heard about Pap and breast exams 18.2 12.2 1.069 30.a percent of women who have gone to a healthcare professional due to discomfort related to RTIs (or vaginal discharge) 10.8 7.0 1.068 Odds Ratio from 1 to 1.049 LOW 18. percent of men who believe that a man does not have the right to force an unwilling woman to have sex 6.9 3.0 1.046 17. percent of men who believe that it is never right to hit a woman 6.6 4.2 1.041 21.1 percent of men who believe that their sons and daughters should reach the same level of education 2.3 -0.7 1.034 33. percent of women who incurred a health expense in the last 12 months 0.5 -0.5 1.021 12. percent of women who agree that a woman should be able to decide to take care of herself even if her partner objects -4.5 -4.6 1.006 11.a percent of women who have spoken with their partner about the number of children they want to have 4.8 4.7 1.003 Odds Ratio less than 1 PROGRAM HAD NO EFFECT 29. percent of women who would go to a healthcare facility in the event of symptoms that are warning signs of risk 29.0 25.4 0.994 9. percent of women who share one or more household chore with their partner 23.5 19.3 0.988 19. percent of women who believe that the work they do outside the home is as important as the work done outside the home by their partner 6.4 6.8 0.986 5. percent of women whose partner helps care for the children if they become ill 6.0 5.3 0.982 26. percent of women who believe that the services provided by the nearest healthcare facility are good (or very good) 10.7 16.9 0.936 20. percent of women who believe that housework is as important as the work done outside the home by their partner 10.4 12.5 0.926 20.1 percent of men who believe that the housework done by their partner is as important as his own work done outside the home 11.2 14.6 0.910 24. percent of women who have gone to a healthcare facility for consultation regarding reproductive health or family planning 15.9 17.2 0.909 16. percent of women who have spoken with their children age 12 years and older about male-female relationships and family planning more than two times in the last 12 months 8.9 6.5 0.845 40. percent of women who can recognize some symptom of pregnancy or postpartum warning signs that indicate risk 26.6 26.8 0.782 ANNEX C COUNTERPART COMMUNITIES VERSUS CONTROL COMMUNITIES MIDTERM EVALUATION OF THE REPROSALUD PROJECT 94 Distrito Comunidad *Estab. Nº de Msnm % de % de Con Provincia Distrito Comunidad *Estab. Nº de Msnm % de de Area habi- (altitud) analfa- población una de Area habi- analfa￾Salud tantes betismo rural NBI Salud tantes (altitud) betismo ANCASH 1 Carhuaz Marcará Marcará C.S U 7704 2726 64.3 87.1 60.9 Carhuaz Tinco Tinco P.S. U 2514 2588 38.3 2 Carhuaz Shilla Llipta NT R 3307 3910 84.2 76.8 77.6 Carhuaz Amashca Shapashmarca NT R 1816 2850 62.2 3 Huaraz Tarica Tarica P.S U 4743 2802 55.6 80.4 89.0 Carhuaz Amashca Amashca P.S. U 1816 2850 62.2 4 Carhuaz Marcará Paltash NT R 7704 2726 64.3 87.1 60.9 Carhuaz Ataquero Huellap NT R 1792 2719 64.7 AYACUCHO 5 Cangallo Los Morochucos Pampacangallo C.S. U 6909 3330 61.4 88.8 97.8 Víctor Fajardo Colca Colca P.S U 1513 2972 54.9 6 Huamanga S Juan Bautista Barrio las Américas NT U 20558 2800 26.1 2.4 57.9 Huamanga Ayacucho Barrio Covadonga NT U 82131 2746 21.3 7 Cangallo Cangallo Incaraccay P.S. R 6193 2577 52.4 67.2 84.9 Huanta Santillana Arahuay P.S. R 6395 3262 73.4 HUANCAVELICA 8 Huancavelica Huancavelica Barrio Santa Ana H.A U 36826 3660 24.3 15.9 75.5 Tayacaja Pampas Pampas C.S. U 9649 3276 36.1 9 Acobamba Paucará Paucará C.S. U 9270 3806 56.5 79.7 98.7 Tayacaja Pazos Pazos C.S. U 7501 3840 45.7 10 Huancavelica Acoria Pallalla P.S. R 22656 3167 57.2 95.7 97.1 Huancavelica Vilca Vilca P.S R 3305 3275 30.5 11 Angaraes Lircay Ocopa NT R 20045 3278 60.0 76.9 92.5 Tayacaja Ahuaycha Purohuay NT R 4393 3280 44.2 12 Angaraes Anchonga Anchonga P.S. U 5573 3298 74.6 94.5 99.9 Tayacaja Salcabamba Salcabamba P.S. U 5765 3073 53.0 LA LIBERTAD 13 Otuzco Usquil Ex hacienda Chuq P.S. R 24203 3018 38.9 91.7 75.1 Sanchez CarriónHuamachuco Chuquizonguillo P.S. R 37708 3169 47.0 14 Otuzco Agallpampa Yamobamba NT R 9656 3117 34.4 92.6 89.6 Julcan Calamarca Sicchal P.S. R 8057 3150 39.2 PUNO QUECHUA 15 Melgar Ayaviri Ayaviri Barrio Puno H.A. U 23281 4400 26.2 26.5 53.9 Canchis Sicuani Barrio San Andres H.A. U 51083 3554 33.5 16 Azangaro Arapa Pucamoco NT R 10757 3838 41.5 93.0 87.4 Puno Coata Sucasco-Tarizani NT R 6301 3814 39.7 PUNO AYMARA 17 El Collao Ilave Ancoamaya P.S. R 48054 3847 34.6 70.5 71.0 Puno Acora Marca-Esqueña NT R 29420 3867 36.9 18 Yunguyo Yunguyo Machacmarca NT R 30360 3826 43.6 70.3 56.8 Puno Acora Chanchilla NT R 29420 3867 36.9 19 Chucuito Pomata Barrio Pueblo Libre C.S. U 18891 3863 38.0 91.7 85.7 Moho Moho San Pedro de Moho C.S. U 20120 3882 35.6 20 Chucuito Zepita Izani Central Zona P.S. U 19085 3814 35.3 94.3 81.1 Puno Acora Santa Rosa de Yan P.S. U 29420 3867 36.9 SAN MARTIN 21 Lamas Tabalosos Barrio Partido Alto C.S. U 11086 1050 31.9 16.7 71.9 Lamas Lamas Lamas ( Ancoallo) H.A. U 13651 809 28.0 22 El Dorado Shatoja Shatoja P.S. U 1653 700 33.0 52.1 79.8 Picota Tingo de PonasaLeoncio Prado C.S. U 2605 400 11.8 UCAYALI 23 Coronel Portillo Masisea Vista Alegre de PachitP.S. R 12083 225 19.5 84.0 92.2 Coronel PortilloMasisea Caimito P.S. R 12083 225 19.5 24 Coronel Portillo Masisea Masisea C.S. U 12083 225 19.5 84.0 92.2 Atalaya Sepahua Sepahua P.S U 3698 276 26.5 25 Padre Abad Irazola Monte Alegre P.S. U 13280 212 18.7 85.2 89.5 Padre Abad Curimaná Curimaná P.S. U S/I S/I S/I * H.A. =Hospital; C.S.= Centro de salud; P.S. = Puesto de salud; NT = No tiene S/I = Sin información en el censo de población y vivienda 1993 UTEM 5/2/2002 Departamento y Provincia COMUNIDADES CONTRAPARTE CONTROL CUADRO COMPARATIVO DE COMUNIDADES CONTRAPARTE Y COMUNIDADES DE CONTROL ANNEX D COMPARISON OF THE BASE LINE (BL) AND MIDTERM EVALUATION (IE) RESULTS FRAMEWORK INDICATORS ANNEXES 96 Counterpart Community Control Community Indicators BL IE Dif. n BL n IE z Signif BL IE Dif. n BL n IE z Signif Odds Ratio OR SO: Increased use by women of interventions in reproductive health 1.a percent of women who had at least 4 prenatal visits to a healthcare professional during a pregnancy occuring within the last two years 55.6 83.1 27.5 372 261 -7.240 Sig (0.05) 62.2 81.0 18.8 312.0 294.0 -5.114 Sig (0.05) 1.148 2. percent of women whose last delivery, occurring within the last two years, was performed by a healthcare professional 36.1 48.0 11.9 465 279 -3.200 Sig (0.05) 40.9 42.7 1.8 425 321 -0.494 1.274 3. percent of women who use some method of birth control 58.4 71.8 13.4 1046 855 -6.071 Sig (0.05) 60.6 64.6 4.0 1076 823 -1.783 Sig (0.05) 1.153 4. percent of women with an unmet need for family planning (including those who use the rhythm method and are unsure of their own fertile days) 48.4 37.3 -11.1 1034 848 4.834 Sig (0.05) 50.1 46.9 -3.2 1071 818 1.379 Sig (0.10) 0.823 4.1 percent of women with an unmet need for family planning 27.1 17.9 -9.2 1042 855 4.741 Sig (0.05) 26.1 21.1 -5.0 1069 821 2.525 Sig (0.05) 0.817 RI 1: More equitable gender relations between women and their partners/families 5. percent of women whose partner helps care for the children if they become ill 38.7 44.7 6.0 955 783 -2.527 Sig (0.05) 30.0 35.3 5.3 979 753 -2.338 Sig (0.05) 0.982 6. percent of women who decide how to spend the money they earn 31.6 36.1 4.5 1599 1389 -2.596 Sig (0.05) 29.8 27.8 -2.0 1540 1246 1.158 1.225 7. percent of women who make joint decisions with their partner about sexual relations, birth control methods, and number of children 23.7 32.9 9.2 1035 848 -4.431 Sig (0.05) 22.6 27.8 5.2 1067 817 -2.588 Sig (0.05) 1.129 9. percent of women who share one or more household chore with their partner 23.2 46.7 23.5 1045 855 -10.77 Sig (0.05) 18.6 37.9 19.3 1075 824 -9.392 Sig (0.05) 0.998 11. percent of women who make a joint decision with their partner about the educational level their children should reach 57.4 64.0 6.6 965 776 -2.798 Sig (0.05) 57.8 58.0 0.2 971 767 -0.084 1.111 11.a percent of women who have spoken with their partner about the number of children they want to have 67.8 72.6 4.8 1054 853 -2.273 Sig (0.05) 69.3 74.0 4.7 1074.0 820.0 -2.241 Sig (0.05) 1.003 RI 1.1: Strengthened ability of women to achieve equality in gender relations 12. percent of women who agree that a woman should be able to decide use family planning even if her partner objects 68.9 64.4 -4.5 1039 855 2.071 Sig (0.05) 65.0 60.4 -4.6 1073 823 2.056 Sig (0.05) 1.006 12.1 percent of women who state their unwillingness to be forced or convinced to have sex 50.6 64.0 13.4 1040 852 -5.852 Sig (0.05) 55.4 58.7 3.3 1067 820 -1.435 Sig (0.10) 1.194 13. percent of women who have spoken with their partner about family planning more than 2 times in the last 12 months 15.2 20.3 5.1 1044 852 -2.908 Sig (0.05) 11.4 14.1 2.7 1074 822 -1.757 Sig (0.05) 1.080 14. percent of women would go to the police/authorities if their partner abused them 14.3 38.4 24.1 1037 854 -12.01 Sig (0.05) 15.7 37.2 21.5 1067 822 -10.691 Sig (0.05) 1.133 ANNEXES 97 Counterpart Community Control Community Indicators BL IE Dif. n BL n IE z Signif Bl IE Dif. n BL n IE z Signif Odds Ratio OR 16. percent of women who have spoken with their children age 12 years and older about male-female relationships and family planning more than two times in the last 12 months 7.1 16.0 8.9 396 381 -3.893 Sig (0.05) 3.9 10.4 6.5 388 289 -3.352 Sig (0.05) 0.845 16.a percent of women who know where to go for help or advice if they are abused 45.6 65.6 20.0 1607 1395 -10.98 Sig (0.05) 44.8 59.5 14.7 1549.0 1251.0 -7.737 Sig (0.05) 1.083 RI 1.2: Increase in positive attitudes of men towards equitable relationships with women and family 17. percent of men who believe that it is never right to hit a woman 57.4 64.0 6.6 707 697 -2.531 Sig (0.05) 58.8 63.0 4.2 679 633 -1.557 Sig (0.10) 1.041 18. percent of men who believe that a man does not have the right to force an unwilling woman to have sex 84.6 91.5 6.9 706 697 -3.981 Sig (0.05) 88.8 91.8 3.0 681 633 -1.831 Sig (0.05) 1.046 RI 1.3: Increase in women's knowledge about gender equality 19. percent of women who believe that the work they do outside the home is as important as the work done outside the home by their partner 51.4 57.8 6.4 730 813 -2.522 Sig (0.05) 48.4 55.2 6.8 997 786 -2.852 Sig (0.05) 0.986 19.1 percent of men who believe that the work they do outside the home is as important as the work done outside the home by their partner 52.1 69.8 17.7 476 620 -5.987 Sig (0.05) 55.9 68.3 12.4 612 571 -4.388 Sig (0.05) 1.097 20. percent of women who believe that housework is as important as the work done outside the home by their partner 33.7 44.1 10.4 804 855 -4.339 Sig (0.05) 30.2 42.7 12.5 1073 822 -5.631 Sig (0.05) 0.926 20.1 percent of men who believe that the housework done by their partner is as important as his own work done outside the home 36.2 47.4 11.2 561 698 -3.997 Sig (0.05) 33.3 47.9 14.6 682 633 -5.393 Sig (0.05) 0.910 21. percent of women who believe that their sons and daughters should reach the same level of education 86.7 91.9 5.2 610 492 -2.745 Sig (0.05) 94.5 89.6 -4.9 602 442 2.957 Sig (0.05) 1.118 21.1 percent of males who believe that their sons and daughters should reach the same level of education 89.1 91.4 2.3 385 386 -1.077 91.6 90.9 -0.7 370 320 0.325 1.034 RI 2: Increased capacity of women in use of reproductive health services 24. percent of women who have gone to a healthcare facility for consultation regarding reproductive health or family planning 26.3 42.2 15.9 1606 1392 -9.187 Sig (0.05) 22.5 39.7 17.2 1549 1248 -9.851 Sig (0.05) 0.909 RI 2.1: Improved capacity of women as end users of formal health services 26. percent of women who believe that the services provided by the nearest healthcare facility are good (or very good) 45.5 56.2 10.7 1260 1039 -5.107 Sig (0.05) 52.8 69.7 16.9 1550 968 -8.397 Sig (0.05) 0.936 27. percent of women who have gone to a healthcare facility for prenatal and postnatal care, and who believe that the services provided by the nearest facilityt are good (or very good) 55.4 61.2 5.8 271 344 -1.450 Sig (0.10) 67.8 69.3 1.5 314 271 -0.389 1.081 RI 2.2: Increase in women's health as a priority within the home and community 29. percent of women who would go to a healthcare facility in the event of symptoms that are warning signs of risk 42.6 71.6 29.0 655 783 -11.11 Sig (0.05) 36.8 62.2 25.4 1005 786 -10.678 Sig (0.05) 0.994 30. percent of women who have sought treatment for symptoms of RTIs (or vaginal discharge) 63.2 73.4 10.2 392 247 -2.673 Sig (0.05) 71.5 70.5 -1.0 358 219 0.257 1.178 MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 98 Counterpart Community Control Community Indicators BL IE Dif. n BL n IE z Signif BL IE Dif. n BL n IE z Signif Odds Ratio OR 30a. percent of women who have gone to a healthcare professional due to discomfort related to RTIs 50.6 61.4 10.8 393 246 -2.669 Sig (0.05) 51.4 58.4 7.0 358 219 -1.637 Sig (0.10) 1.068 32. percent of women who take care of their health in order to feel well 56.8 65.1 8.3 1246 1393 -4.368 Sig (0.05) 65.2 56.4 -8.8 1535 1250 4.741 Sig (0.05) 1.325 33. percent of women who have incurred a health expense in the last 12 months 49.5 50.0 0.5 1608 1395 -0.273 45.7 45.2 -0.5 1549 921 0.241 1.021 RI 2.6: Increase in women's knowledge about their reproductive health needs 38. percent of women who know how at least one modern contraceptive method works 13.5 55.5 42.0 1608 1395 -24.41 Sig (0.05) 14.9 31.9 17.0 1550 1251 -10.713 Sig (0.05) 1.920 39. percent of women using the rhythm method who know the fertile days of their cycle 22.5 28.3 5.8 239 214 -1.419 Sig (0.10) 24.0 24.1 0.1 306 233 -0.027 0.782 40. percent of women who can recognize some symptom of pregnancy or postpartum warning signs that indicate risk 19.4 46.0 26.6 1044 855 -12.43 Sig (0.05) 13.2 40.0 26.8 1076 823 -13.399 Sig (0.05) 0.782 43. percent of women who know how RTIs are spread 4.9 19.0 14.1 1193 1150 -10.57 Sig (0.05) 4.7 11.4 6.7 1007 921 -5.448 Sig (0.05) 1.599 43.a percent of women who have heard about Pap and breast exams 63.2 81.4 18.2 1608 1395 -11.04 Sig (0.05) 59.6 71.8 12.2 1550.0 1251.0 -6.732 Sig (0.05) 1.069 43.b percent of women who believe that while a women is breastfeeding it is dificult for her to become pregnant 34.0 56.3 22.3 1193 1001 -10.48 Sig (0.05) 30.9 46.9 16.0 1200.0 951.0 -7.594 Sig (0.05) 1.091 ANNEX E SAMPLE OF COMMUNITIES WITH COMMUNITY BANKS AND WITH EDUCATION ACTIVITIES ONLY MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 100 Muestra de cominidades con bancos comunales y con intervención educativa únicamente Nombre de la comunidad Nombre del con Banco MUJ Socias HOMB TOTAL Banco MUJ Socias HOMB TOTAL MUJ Socias HOMB TOTAL LINEA DE BC Cielo Salpino 46 13 33 79 Bc 14 de Setiembre 69 22 63 132 115 35 96 211 BASE BC Carmen del Rosari 26 11 31 57 Santa Barbara 45 12 43 88 71 23 74 145 TOTAL 72 24 64 136 114 34 106 220 186 58 170 356 EVALUACION BC Cielo Salpino 38 11 32 70 Bc 14 de Setiembre 45 13 50 95 83 24 82 165 INTERMEDIA BC Carmen del Rosari 29 14 32 61 Santa Barbara 34 10 39 73 63 24 71 134 TOTAL 67 25 64 131 79 23 89 168 146 48 153 299 Nombre de la comunidad Nombre del con IE&C MUJ Socias HOMB TOTAL Control MUJ Socias HOMB TOTAL MUJ Socias HOMB TOTAL LINEA DE Santa Rosa 71 30 40 111 Tomas Vargas 79 62 63 142 150 92 103 253 BASE Santa Rosa de Lima 48 19 26 74 Micaela Bastidas 60 40 47 107 108 59 73 181 TOTAL 119 49 66 185 139 102 110 249 258 151 176 434 EVALUACION Santa Rosa 52 22 36 88 Tomas Vargas 69 49 51 120 121 71 87 208 INTERMEDIA Santa Rosa de Lima 46 13 27 73 Micaela Bastidas 45 30 34 79 91 43 61 152 TOTAL 98 35 63 161 114 79 85 199 212 114 148 360 CUADRO COMPARATIVO CONTROL DE BANCOS LINEA DE BASE Vs EVALUACION INTERMEDIA LA LIBERTAD PUNO AYMARA TOTAL CUADRO COMPARATIVO BANCOS LINEA DE BASE Vs EVALUACION INTERMEDIA LA LIBERTAD PUNO AYMARA TOTAL ANNEX F INDICATOR TABLES MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 102 Tabla 1: Comparison of the Indicators of the Base Line (LB) and the Midterm Evaluation (EI): Communities with Community Banks and with Education Activities Only Bancos IE&C Indicadores LB EI Dif. n LB n EI z signif LB EI Dif. n LB n EI z signif Odds Ratio OE: MUJERES INCREMENTAN LA UTILIZACIÓN DE INTERVENCIONES EN SALUD REPRODUCTIVA 1a. percent de mujeres que ha tenido 4 o más controles con personal de salud durante el último embarazo ocurrido dos años antes de la encuesta 15.6 45.7 30.1 45 35 -2.950 Sig (0.05) 30.6 46.7 16.1 62 45 -1.699 Sig (0.05) 1.087 2. percent de mujeres que atendió su último parto, ocurrido dos años de la encuesta, con personal de salud 41.7 67.6 25.9 36 34 -2.174 Sig (0.05) 49.1 73.2 24.1 53 41 -2.362 Sig (0.05) 1.920 3. percent de mujeres unidas que usa algún método anticonceptivo 62.7 68.2 5.5 118 88 -0.819 67.5 71.6 4.1 157 116 -0.726 1.025 4. percent de mujeres unidas con necesidad insatisfecha de planificación familiar (incluye a usuarias de ritmo que no conocen su período fértil) 48.3 42.0 -6.3 116 88 0.895 42.6 42.1 -0.5 155 114 0.082 0.880 4.1 percent de mujeres unidas con necesidad insatisfecha de planificación familiar 23.5 15.9 -7.6 115 88 1.336 Sig (0.10) 21.0 20.5 -0.5 157 117 0.101 0.693 RI 1: MUJERES TIENEN RELACIONES DE GÉNERO MÁS EQUITATIVAS CON SUS PAREJAS Y SUS FAMILIAS 5. percent de mujeres unidas cuya pareja ayuda cuando los niños se enferman 16.7 47.4 30.7 102 78 -4.454 Sig (0.05) 30.3 32.7 2.4 76 98 -0.338 2.630 6. percent de mujeres que decide en qué se gasta el dinero que gana ella. 36.2 35.9 -0.3 186 145 0.056 41.5 35.0 -6.5 256 210 1.434 Sig (0.10) 1.176 7. percent de mujeres que decide conjuntamente con su pareja sobre relaciones sexuales, métodos anticonceptivos y número de hijos 22.2 30.6 8.4 117 85 -1.348 Sig (0.10) 28.2 38.7 10.5 156 111 -1.804 Sig (0.05) 1.004 9. percent de mujeres que hace una o más tareas en su casa conjuntamente con su pareja 26.3 38.6 12.3 118 88 -1.879 Sig (0.05) 33.1 67.2 34.1 157 116 -5.577 Sig (0.05) 0.723 11. percent de mujeres que decide conjuntamente con su pareja hasta qué nivel de educación deben estudiar hijas e hijos 75.2 72.7 -2.5 101 77 0.377 55.4 51.9 -3.5 148 104 0.549 1.032 11a percent de mujeres unidas que ha hablado con su pareja sobre el número de hijos que desean tener 75.8 78.3 2.5 118 88 73.5 83.9 10.4 157 114 -2.036 Sig (0.05) 0.905 RI 1.1: Mujeres fortalecen sus habilidades para conseguir cambios en sus relaciones de género 12. percent de mujeres que está de acuerdo que la mujer decida cuidarse aún cuando la pareja se oponga 51.7 55.7 4.0 118 88 -0.569 76.4 65.5 -10.9 157 116 1.978 Sig (0.05) 1.257 12.1 percent de mujeres que no tiene relaciones sexuales obligada o convencida 63.4 68.6 5.2 118 88 -0.777 41.3 60.7 19.4 156 114 -3.149 Sig (0.05) 0.736 ANNEXES 103 14. percent de mujeres que acudiría a pedir ayuda a la policía-autoridades si su marido la golpease 19.5 43.2 23.7 118 88 -3.685 Sig (0.05) 18.5 35.7 17.2 81 115 -2.624 Sig (0.05) 1.148 Bancos IE&C Indicadores LB EI Dif. n LB n EI z signif LB EI Dif. n LB n EI z signif Odds Ratio 16a. percent de mujeres que sabe a dónde acudir a pedir ayuda o consejo en caso de ser golpeadas 65.6 71.8 6.2 186 146 -1.205 43.6 73.4 29.8 258 212 -6.494 Sig (0.05) 0.650 RI 1.2: AUMENTO DE ACTITUDES POSITIVAS DE LOS VARONES EN SU RELACIÓN CON LAS MUJERES Y LA FAMILIA 17. percent de varones que cree que a la mujer no se le puede golpear en ningún caso 75.5 73.3 -2.2 94 86 0.338 53.1 79.7 26.6 98 79 -3.688 Sig (0.05) 0.647 18. percent de varones que cree que el hombre no tiene derecho a exigir relaciones sexuales cuando la mujer no quiere 76.8 93.0 16.2 95 86 -3.007 Sig (0.05) 76.0 92.4 16.4 50 79 -2.619 Sig (0.05) 0.996 RI 1.3: MUJERES INCREMENTAN EL CONOCIMIENTO SOBRE LA SITUACIÓN DE GÉNERO 21. percent de mujeres que cree que las hijas y los hijos deben estudiar hasta el mismo nivel 96.7 78.7 -18.0 60 47 2.929 Sig (0.05) 86.5 91.8 5.3 104 73 -1.096 0.767 21.1 percent de varones que cree que las hijas y los hijos deben estudiar hasta el mismo nivel 95.7 93.5 -2.2 47 31 0.429 84.5 87.5 3.0 58 48 -0.441 0.944 RI 2: Mujeres con mayor capacidad para acceder a los servicios de Salud Reproductiva 24. percent de mujeres que ha acudido a un establecimiento de salud para una consulta sobre salud reproductiva o para atención en planificación familiar 20.8 32.8 12.0 186 146 -2.472 Sig (0.05) 28.8 32.2 3.4 258 212 -0.798 1.410 RI 2.2: AUMENTO DE LA VALORACIÓN POSITIVA DE LA SALUD DE LA MUJER DENTRO DE LOS HOGARES Y LA COMUNIDAD 33. percent de mujeres que realizó algún gasto en su salud en los últimos 12 meses 39.2 57.9 18.7 186 146 -3.387 Sig (0.05) 46.8 44.8 -2.0 258 212 0.433 1.543 RI 2.6: MUJERES INCREMENTAN EL CONOCIMIENTO SOBRE SUS NECESIDADES DE SALUD REPRODUCTIVA 38. percent de mujeres unidas que sabe cómo funciona al menos un método moderno 6.1 55.0 48.9 186 146 -9.893 Sig (0.05) 25.8 57.3 31.5 258 212 -6.936 Sig (0.05) 4.060 39. percent de usuarias de ritmo que conocen los días fértiles del ciclo 31.4 34.9 3.5 40 31 -0.311 32.4 37.4 5.0 43 39 -0.475 0.963 40. percent de mujeres unidas que conoce algún malestar del embarazo y del post parto que indican situación de riesgo 16.1 45.5 29.4 118 88 -4.616 Sig (0.05) 30.1 56.9 26.8 153 116 -4.416 Sig (0.05) 1.495 43a. percent porcentaje de mujeres que ha escuchado sobre Papanicolau o el examen de mamas 64.7 73.7 9.0 186 146 -1.755 Sig (0.05) 77.6 87.5 9.9 258 212 -2.784 Sig (0.05) 1.010 43b. percent porcentaje de mujeres que piensa que mientras una mujer está dando de lactar es difícil que quede embarazada 39.0 57.2 18.2 132 100 -2.751 Sig (0.05) 39.2 60.7 21.5 184 150 -3.910 Sig (0.05) 0.947 MIDTERM IMPACT EVALUATION OF THE REPROSALUD PROJECT 104 Tabla2: Porcentaje de cambio neto para cada indicador del Marco de Resultados y estimaciones de los valores de Odds Ratio. Comunidades con bancos comunales y comunidades con IE&C Indicadores Bancos IE&C Odds LB EI Cambio relativo LB EI Cambio relativo Ratio OE: Mujeres incrementan la utilización de intervenciones en salud reproductiva 1a. % de mujeres que ha tenido 4 o más controles con personal de salud durante el último 41.7 67.6 1.621 49.1 73.2 1.491 1.087 embarazo ocurrido dos años antes de la encuesta 2. % de mujeres que atendió su último parto, ocurrido dos años de la encuesta, con personal de salud 15.6 45.7 2.929 30.6 46.7 1.526 1.920 3. % de mujeres unidas que usa algún método anticonceptivo 62.7 68.2 1.088 67.5 71.6 1.061 1.025 4. % de mujeres unidas con necesidad insatisfecha de planificación familiar 48.3 42.0 0.870 42.6 42.1 0.988 0.880 (incluye a usuarias de ritmo que no conocen su período fértil) 4.1 % de mujeres unidas con necesidad insatisfecha de planificación familiar 23.5 15.9 0.677 21.0 20.5 0.976 0.693 RI 1: Mujeres tienen relaciones de género más equitativas con sus parejas y sus familias 5. % de mujeres unidas cuya pareja ayuda cuando los niños se enferman 16.7 47.4 2.838 30.3 32.7 1.079 2.630 6. % de mujeres que decide en qué se gasta el dinero ue gana ella 36.2 35.9 0.992 41.5 35.0 0.843 1.176 7. % de mujeres que decide conjuntamente con su pareja sobre relaciones 22.2 30.6 1.378 28.2 38.7 1.372 1.004 sexuales, métodos anticonceptivos y número de hijos 9. % de mujeres que hace una o más tareas en su casa conjuntamente con su pareja 26.3 38.6 1.468 33.1 67.2 2.030 0.723 11. % de mujeres que decide conjuntamente con su pareja hasta qué nivel 75.2 72.7 0.967 55.4 51.9 0.937 1.032 de educación deben estudiar hijas e hijos 11.a % de mujeres unidas que ha hablado con su pareja sobre el número de hijos que desean tener 75.8 78.3 1.033 73.5 83.9 1.141 0.905 RI 1.1: Mujeres fortalecen sus habilidades para conseguir cambios en sus relaciones de género 12. % de mujeres que está de acuerdo que la mujer decida cuidarse aún 51.7 55.7 1.077 76.4 65.5 0.857 1.257 cuando la pareja se oponga 12.1 % de mujeres que no tiene relaciones sexuales obligada o convencida 63.4 68.6 1.082 41.3 60.7 1.470 0.736 14. % de mujeres que acudiría a pedir ayuda a la policía-autoridades si su marido la golpease 19.5 43.2 2.215 18.5 35.7 1.930 1.148 16.a % de mujeres que sabe a dónde acudir a pedir ayuda o consejo en caso de ser golpeada 65.6 71.8 1.095 43.6 73.4 1.683 0.650 RI 1.2: Aumento de actitudes positivas de los varones en su relación con las mujeres y la familia 17. % de varones que cree que a la mujer no se le puede golpear en ningún caso 75.5 73.3 0.971 53.1 79.7 1.501 0.647 18. % de varones que cree que el hombre no tiene derecho a exigir relaciones 76.8 93.0 1.211 76.0 92.4 1.216 0.996 sexuales cuando la mujer no quiere RI 1.3: Mujeres incrementan el conocimiento sobre la situación de género 21. % de mujeres que cree que las hijas y los hijos deben estudiar hasta el mismo nivel 96.7 78.7 0.814 86.5 91.8 1.061 0.767 21.1 % de varones que cree que las hijas y los hijos deben estudiar hasta el mismo nivel 95.7 93.5 0.977 84.5 87.5 1.036 0.944 RI 2: Mujeres con mayor capacidad para acceder a los servicios de Salud Reproductiva 24. % de mujeres que ha acudido a un establecimiento de salud para una consulta 20.8 32.8 1.577 28.8 32.2 1.118 1.410 sobre salud reproductiva o para atención en planificación familiar RI 2.2: Aumento de la valoración positiva de la salud de la mujer dentro de los hogares y la comunidad 33. % de mujeres que realizó algún gasto en su salud en los últimos 12 meses 39.2 57.9 1.477 46.8 44.8 0.957 1.543 RI 2.6: Mujeres incrementan el conocimiento sobre sus necesidades de salud reproductiva 38. % de mujeres unidas que sabe cómo funciona al menos un método moderno 6.1 55.0 9.016 25.8 57.3 2.221 4.060 39. % de usuarias de ritmo que conocen los días fértiles del ciclo 31.4 34.9 1.111 32.4 37.4 1.154 0.963 40. % de mujeres unidas que conoce algún malestar del embarazo y del post parto 16.1 45.5 2.826 30.1 56.9 1.890 1.495 que indican situación de riesgo 43.a % porcentaje de mujeres que ha escuchado sobre Papanicolau o el exámen de mamas 64.7 73.7 1.139 77.6 87.5 1.128 1.010 43 b % porcentaje de mujeres que piensa que mientras una mujer está dando de lactar 39.0 57.2 1.467 39.2 60.7 1.548 0.947 es difícil que quede embarazada ANNEXES 105 Table 3: Results Framework Indicators Ranked According to Greatest Project Impact Relative to the Odds Ratio Value Cambio neto en puntos porcentuales entre la línea de base y la evaluación intermedia Indicadores Bancos IE&C Odds Ratio Odds Ratio de 1.999 a + 38. percent de mujeres unidas que sabe cómo funciona al menos un método moderno 48.9 31.5 4.060 5. percent de mujeres unidas cuya pareja ayuda cuando los niños se enferman 30.7 2.4 2.630 Odds Ratio de 1.999 a 1.199 2. percent de mujeres que atendió su último parto, ocurrido dos años antes de la encuesta, con personal de salud 30.1 16.1 1.920 33. percent de mujeres que realizó algún gasto en su salud en los últimos 12 meses 18.7 -2 1.543 40. percent de mujeres unidas que conoce algún malestar del embarazo y del post parto que indican situación de riesgo 29.4 26.8 1.495 24. percent de mujeres que ha acudido a un establecimiento de salud para una consulta sobre salud reproductiva o para atención en planificación familiar 12.0 3.4 1.410 12. percent de mujeres que está de acuerdo en que la mujer decida cuidarse aún cuando la pareja se oponga 4.0 -10.9 1.257 Odds Ratio de 1.199 a 1.100 6. percent de mujeres que decide en qué se gasta el dinero que gana ella. -0.3 -6.5 1.176 14. percent de mujeres que acudiría a pedir ayuda a la policía -autoridades si su marido la golpease 23.7 17.2 1.148 Odds Ratio de 1.099 a 1.050 1a. percent de mujeres que ha tenido 4 o más controles con personal de salud durante el embarazo ocurrido dos años antes de la encuesta 25.9 24.1 1.087 Odds Ratio de 1.049 a 1 11. percent de mujeres que decide conjuntamente con su pareja hasta qué nivel de educación deben estudiar hijas e hijos -2.5 -3.5 1.032 3. percent de mujeres unidas que usa algún método anticonceptivo 5.5 4.1 1.025 43a percent de mujeres que ha escuchado sobre Papanicolau o el examen de mamas 9.0 9.9 1.010 7. percent de mujeres que decide conjuntamente con su pareja sobre relaciones sexuales, métodos anticonceptivos y número de hijos 8.4 10.5 1.004 Odds Ratio menor que 1 18. percent de varones que cree que el hombre no tiene derecho a exigir relaciones sexuales cuando la mujer no quiere 16.2 16.4 0.996 39. percent de usuarias de ritmo que conocen los días fértiles del ciclo 3.5 5 0.963 43 b. percent de mujeres que piensa que mientras una mujer está dando de lactar es difícil que quede embarazada 18.2 21.5 0.947 21.1 percent de varones que cree que las hijas y los hijos deben estudiar hasta el mismo nivel -2.2 3 0.944 11a. percent de muje res unidas que ha hablado con su pareja sobre el número de hijos que desean tener 2.5 10.4 0.905 4. percent de mujeres unidas con necesidad insatisfecha de planificación familiar (incluye a usuarias de ritmo que no conocen su período fértil) -6.3 -0.5 0.880 21. percent de mujeres que cree que las hijas y los hijos deben estudiar hasta el mismo nivel -18.0 5.3 0.767 12.1 percent de mujeres que no tiene relaciones sexuales obligada o convencida 5.2 19.4 0.736 9. percent de mujeres que hace una o más tareas en su casa conjuntamente con su pareja 12.3 34.1 0.723 4.1 percent de mujeres unidas con necesidad insatisfecha de planificación familiar -7.6 -0.5 0.693 16.a percent de mujeres que sabe a dónde acudir a pedir ayuda o consejo en caso de ser golpeadas 6.2 29.8 0.650 17. percent de varones que cree que a la mujer no se le puede golpear en ningún caso -2.2 26.6 0.647