INTERMEDIATE EVALUATION OF THE SUSTAINABLE FOOD SECURITY NETWORKING PROGRAM CARE PERU EXECUTIVE SUMMARY 1. INTRODUCTION The Program "Sustainable Food Security Networking," REDESA, implemented in the framework of the USAID Title II Program, has as its purpose to improve the quality of life of 322,136 people -- representing 58,570 poor families in rural areas of the departments of Cajamarca, Ancash, Ayacucho and Puno -- by increasing their food security. The fulfillment of this purpose is evaluated through the Reduction of the Rate of chronic malnutrition of children under 3 years old by 11 percentage points by 2006. In order to achieve its purpose, the program has defined two Specific Objectives: Objective 1. Sustainable Increase in the incomes of poor families. The result sought for the year 2006 is a 25% increase in family incomes. Objective 2: Sustainable improvement in the conditions of health and nutrition of the poor families, especially of the children under 3 years old. The result sought for the year 2006 is the reduction of the incidence of acute diarrhea by one-third (from 29% to 20% by 2006). 2. COMPONENTS OF THE INTERMEDIATE EVALUATION In order to verify the levels of impact, effect and product obtained by the program during fiscal years 2002-2004, through an analysis of their main programmatic indicators and their management components, CARE has contracted the services of the consulting company SASE to undertake the Intermediate Evaluation. This evaluation will analyze the following key subjects: • Development Components: Pertinence, Viability, Efficiency, Effectiveness, Impact and Sustainability. • Development Factors: the country’s Institutional Policy guidelines and priorities at national, regional and local levels with respect to Food Security; Environmental Factors, Monitoring and Evaluation and Decentralization. • Programmatic Strategies: Construction of a Partnership Network, Programmatic Strategies, Communication for behavioral change, Land Development and Political Advocacy. The methodology for the evaluation is based on quantitative and qualitative methods. The former have as their purpose the quantification of the performance and impacts, and are sustained in the review of the operational plans, budgets and progress reports and in the application of a Household Survey in the target areas. The qualitative methods are applied fundamentally to understand the processes, conduct and conditions that influence the development of the Program, and how they are perceived by the stakeholders and beneficiaries, with the information having been collected in participatory processes. 3. RESULTS With respect to the Intervention strategy • The program is implementing an action strategy that has as its pillars the development of capabilities and the participation of all the stakeholders in this problem. By virtue of this Strategy, the program has changed the vision of CARE Peru from an isolated project implementer to an organization that impels concerted processes of social development. The beneficiary population and social actors in the program’s target zone recognize the importance of the actions that are being executed for the sustainable improvement of their standards of living. The challenge assumed by REDESA to work the issue of competitiveness in areas of poverty constitutes a contribution worthy of emphasis, as are the important achievements in the Partnership Network Strategy. • With respect to the Combined Management model, beginning with the incorporation of Civil Society into the Management of the program, significant contributions can be seen in the construction of the Social Capital and local institutionality that the development of the country requires. Nevertheless, it is important to consider that in spite of the advances and achievements obtained at the district and provincial levels, the beginning of actions of advocacy that will allow the sensitization of the civil employees of the regional governments remains pending. On the other hand, the recent promulgation of the National Plan to Overcome Poverty and the National Strategy of Food Security open up to REDESA a new perspective around advocacy in State Policy matters. • REDESA is contributing to the process of decentralization in the target zones by means of technical assistance in the strengthening of the capabilities of the civil employees of the local governments, the local population in general, and local actors to take part in the spaces for concerted action. Its contribution to this process can be seen through the active participation in the Policy Assemblies, the Agreements of Governance and in the Public Hearings to debate the legal framework of Decentralization. Its political advocacy of the formation and implementation of social policies in the subject of infant health and nutrition, mainly at the district level and, in some cases, at the provincial level, can also be mentioned. In addition, it is promoting citizen participation as a means of exercising the social monitoring of this process. • One of the principal achievements of REDESA is related to the construction of "sustainable networks," which is expressed in the formation of new leaders who advocate making the importance of the subject of Food Security more visible. This has permitted many authorities to become aware of the problem of malnutrition, and their having included actions and resources destined to support initiatives of food security within their strategic development plans and participatory budgets. • Among the successes of the Strategy of Communication for Behavioral Change (CBC), the following can be cited: a) The use of the training spaces for the diffusion of the messages of the CBC strategy; b) The constitution of the Facilitators’ Network; c) The revaluation of the role of the Community Health Agent as an educator and communicator in his/her community for the behavioral changes and the adoption of healthy practices; d) The design and implementation of studies of diagnosis for CBC; the elaboration and implementation of participatory CBC Plans; the identification of key messages and media adapted to the spaces and the social and cultural characteristics of the object population. • Among the limitations are: a) Having been placed within the beneficiaries of the Health and Nutrition Objective; b) The scarce use of the instrument of the CBC process in the field; c) The lack of widespread use of the key messages, including the diffusion of both sub-programmatic objectives, through radio spots in Quechua; d) The limitations of the Ministry of Health in its role as facilitator in the matter of personnel for extramural promotional preventive actions, and its deficiency of material resources; e) The lack of reinforcement of the messages about complementary feeding (consistency and quality of the nutrients). With respect to the indicators of the End Result, Strategic Objectives and results of REDESA (Impact). • The indicator that visualizes the effects at the level of the End Result is chronic malnutrition among children under 3 years old. The rate of malnutrition among the infant population of REDESA has been reduced from 34% to 29.9% between 2002 and 2004 (a reduction of 4.1 percentage points), a result indicating that the achievement of the End Result is on course. This reduction is important considering the tendency observed in this indicator at the national level, if the results of ENDES 2000 are compared with those of MONIN-CENAN 2002. • The indicators of the Program Strategic Objectives (PSO) permit the appreciation of important achievements. With respect to the Income Generation Objective, the indicators Gross Value of Production (GVP) and Family Income Level have increased by 151% and 87%, respectively, in relation to the Baseline (comparison in Peruvian currency - new sols). • These indicators are supported by the advances achieved in the Intermediate Results of this Strategic Objective, which can be seen in the indicators: Value of Sales from the Family’s Production (which is 82.7% superior to the Baseline); Use of the Technical Assistance (in 2004, 48.3% of the beneficiary families made use of technical assistance from all types of suppliers versus 5% in 2002); Participation in Producers’ Associations (12.9% of the beneficiary families in the program participate in producers’ groups); and Access to Credit, which is still very limited, as only 8.5% of the beneficiary families obtained credit at least once and 2.8%, two or more times. • The information of the survey is also eloquent in terms of achieving greater expenses in foodstuffs as a result of the improvement in income. The results indicate that, at the global level, the per capita expense in food in 2004 (S/.71) is 28.5% superior to the Baseline (S/.55). • With respect to the Health, Nutrition, Water and Sanitation Objective, whose key indicator is the prevalence of the diarrhea, the information shows evidence of a positive evolution in its decline from 35% in 2002 to 17% in 2004. • Intermediate results also demonstrate that some of the activities conducted show positive changes in health and nutrition practices, such as Only Maternal Lactation, which increased from 25% in 2002 to 65% in 2004, and the Feeding of children during episodes of diarrhea, which increased from 4% to 7.1% in that period. With respect to the promotion of the integral attention to mothers, whose effects are measured through three conditions (prenatal control, tetanus vaccination and attention at childbirth),,the information indicates that 20.9% of the women beneficiaries fulfill all three conditions. With respect to the result related to the adoption of good practices in hygiene and sanitation, interesting advances are verified only in the washing of hands after going to the bathroom, changing diapers and before preparing foods. On the other hand, the indicators on water and sanitation tell us that the beneficiaries are in a favorable situation with respect to access to these services compared to what was found in the Baseline, especially with respect to sanitation. • A clear achievement of the program is the participation of grassroots organizations in spaces for concerted action, with the objective of promoting and coordinating actions in health, nutrition, water and sanitation (Intermediate Result 3). The percentage of beneficiary families indicating that, in their locality, the JASS and Health Committees participate in Policy Assemblies confirm this affirmation (JASS: 27.1%; Health Committees: 20.6%). With respect to Efficiency • The levels of budgetary execution show that in the period under consideration, $11,823,610 dollars have been invested, representing 55% of the Title II resources assigned the Program and 42.8% of the total of resources for the period. • The unit costs for the production of the activities show substantive differences among the corridors, an aspect that denotes that in each corridor, the technical teams faced a different context in terms of prices and of answers to the Partnership Strategy. These differences take on a special dimension in the case of the activities that respond to the intermediate results of the Income Generation Objective. • The normalization of the unit costs in an index that permits placing the corridors (regions) in order from the most cost efficient to the least cost efficient shows that Puno is the corridor that displays the highest levels of Efficiency at the global level as well as in the sub-programmatic objectives. Ancash, Cajamarca and Ayacucho follow, in that order. With respect to Effectiveness • Taking the goals up to 2006 as a reference, we can observe that most of the activities have advanced beyond the goals programmed for the entire period of the program, some surpassing the goals by 100%, whereas there are activities whose physical advance is far behind in relation to the period of implementation. In the case of the Income Generation Objective, among the activities that have substantially surpassed the goals is the indicator “Number of Validated Pilot Projects” (with the execution of 290% of the programmed goal), whereas among those in which the goal has not been met is the activity “Number of Producers who obtain credit,” which advanced 17%. • With respect to the execution of the activities in the Health, Nutrition, Water and Sanitation Objective, the activity that surpasses the predicted goal is the Number of approved and implemented Projects (139%), whereas those that are in lowest levels are the Number of JASS that operate and maintain Water and Sanitation systems (30%) and the Number of Families benefited by potable water systems (39%). • The degree of fulfillment of the annual physical goals of each corridor shows that, although the goals are distributed among the four corridors in equitable way that is mutually agreed upon, the conditions and particularities of each corridor can be either advantageous or limiting for purposes of their being fulfilled. This, combined with the management that each one makes, determines the degree of attainment of the global goals. The Synthetic Index of the levels of fulfillment reflects these considerations, with Puno being in first place in the case of the Income Generation Objective, followed of Ancash; whereas in the case of the Health, Nutrition, Water and Sanitation Objective, the levels are similar. • The positive balance obtained in the qualification of the beneficiaries with respect to the implementation of the activities can be considered as a reference point for the implementing agents, since it expresses the perception that they have about the expected benefits. The lowest qualifications corresponded to credit access and the installations of latrines, which bear a relation to their physical advance. With respect to sustainability • The strategies developed by REDESA in all the corridors open up a range of opportunities in the target zones for those strategies to be adopted by the State. In this task, the local governments carry out the preponderant role. In the implementation process, they use resources for diverse activities, and participate and/or actively support the actions being promoted, implemented and strengthened by REDESA. • REDESA has fulfilled a role as facilitator at the local level, achieving a result that diverse institutions promote and carry out actions in the areas of health, nutrition, water and sanitation. It has also sponsored the strengthening of the institutional capability of the local stakeholders, who are increasingly orienting themselves to the incorporation of the strategies of the program, thus assuring the sustainability of the intervention. However, the achievement of sustainability implies overcoming limitations such as the high rate of turnover in personnel in the local governments and the state entities, especially in the Health sector. As a result of the program, producers’ associations and federations have been formed, which in turn have constituted support for the sustainability of the Program, especially in the economic sense. • The producers recognize that conditions are being generated to improve the commercialization, in the sense that they have improved the quality of their products, which results in their obtaining better prices. They also emphasize the contribution of CARE as the organizer of the supply and the search for markets for certain groups of producers. • The participation of the private sector can be seen fundamentally in the purchase of products and the technical assistance for the fulfillment of quality standards. With respect to Monitoring and Evaluation • The System of Monitoring and Evaluation of REDESA is in a stage of development; in order that it constitute an instrument of efficient management, it will be necessary to improve the levels of validation and feedback. Applying the tools that are used, information is collected that is important for the evaluation. It is necessary to make concepts and methodologies more uniform, however, especially those that refer to the elaboration of each indicator and the establishment of units of measurement. 4. RECOMMENDATIONS With respect to design and management • The analysis of the vertical logic enables us to see that some objectives of the Intermediate Results and of the Activities must be reframed. Similarly, the analysis of the horizontal logic places several of the indicators into question; as a result, the Reconstruction of Logical Framework is necessary. Although the program is characterized by its flexibility, it is recommended that instruments be elaborated that pass minimum guidelines to the implementing agents with respect to the Action Strategies and the development of the activities. With respect to the programming • The planning of the activities must take into account not only the economic resources available, but also the human resources, the target population, the complexity of the actions and the surrounding conditions, both advantages and limitations. The restructuring of the POA is recommended so that the use of resources for each goal to be reached is clearly quantified. With respect to the implementation of activities • To the extent possible, budgetary changes and budgetary reassignments among headings must be avoided. If they are inevitable, it will be necessary to undertake the corresponding reprogramming, and to make it official, in order to avoid the existence of differentiated achievements (activities with over-execution or scarce levels of fulfillment) at the end of the fiscal year. • The practice of taking permanent measurements of the degrees of satisfaction of the beneficiaries with respect to the Program would be very useful for the feedback and adjustment of program actions. With respect to Efficiency • The National Advisers would have to program ("handle") jointly with the Administration, the budgetary allocations and, in a coordinated fashion, analyze the REAL unit costs of the activities, fixing margins that are based on the real prices. This analysis would constitute a basic input of the POAs and the official documents of the Program. • The General Coordination of the program must incorporate an indicator of effectiveness that permits the periodic analysis, at the level of the different activities of the Intermediate Results, of the relation between the resources assigned and those executed. With respect to sustainability • In order that the income generation activities in poor families are reflected in an improvement of their nutritional levels, they must be accompanied by training activities on food consumption, nutrition and health for both groups of beneficiaries. Also, the work of sensitization - which has had important achievements with the results of the Monitoring of the Client – must be extended to all the local and provincial governments of the program target areas, as well as to the regional governments, with a view toward achieving their participation in the definition of a Strategy of Food Security and Infant Nutrition at the regional level. • Due to the achievements obtained in the strengthening of the participation spaces of the population, such as Policy Assemblies and the CCLs - in which the OCBs carry out an important decision-making role in the identification of their priorities - advocacy of the sensitizing and training of the promoters and community leaders as a basis of sustainability is recommended. • Advocacy of the development of the capabilities of the small producers, with a view toward their association and organization in order to be inserted in productive chains. • Emphasis on activities that permit the generation of standards required to institutionalize the activities of the program. Associated with it is the necessity to develop a set of operational guidelines for the development of the Strategies of the Program and to systematize the successful experiences of all the Corridors, with a view toward their replicability. • Establish a graduation system according to the levels of capabilities achieved by the grassroots organizations that are being strengthened by the activities of the program. A system of graduation established with criteria of continuous ascent not only guarantees the sustainability of the achievements reached, but fundamentally stimulates the population and its authorities to continue a development process by passing though stages. With respect to the System of Monitoring and Evaluation • The System of Monitoring and Evaluation must be fed by the POA, which in turn must respond to the cycle of the project. This will permit the evaluation of the state of each one of the activities, such as “being programmed,” “in execution,” “executed,” and “presented.” This way it will be possible to detect the level of the process that present bottlenecks, and solve them in a timely fashion. • The reports must offer monthly information, with dates of completion to be able to gradually evaluate the advances and, therefore, the efficiency. The information on all the levels must have the same characteristics, with a view to its consolidation and the subsequent analysis of the effectiveness in terms of results. With respect to Political Advocacy • Give priority to the actions of advocacy that are oriented to the sensitization of the political decision makers and authorities of the regional and local governments with respect to the advances obtained in the subject of food security and infant nutrition. The achievements of the Program must be diffused and be exposed in a special way to high level functionaries of the government, especially to those who have direct responsibility over the National Plan to Overcome Poverty and the National Strategy of Food Security. • In the measure that REDESA is contributing to the generation of Social Capital with an ample vision of development in which Food Security is given priority, it becomes essential to systematize the experience and to establish a methodology that permits the gathering of the meaning of the process of development at local and regional levels. 5. LESSONS LEARNED • The direct assumption of the causes that generate food insecurity becomes an important instrument for the definition of Strategies for the Struggle against Poverty and Food Security in the measure that they are made operational in a framework of integration, association and participation of the most representative social actors of the society on which they are going to act. • The actions that in the long run obtain a greater empowerment of the population and the participation of a greater number of social actors are those that are oriented toward the strengthening of capabilities, overcoming the “asistencialismo” (assistance that perpetuates dependence on outside assistance) that characterizes most of the Food Security programs. • The experience of the present evaluation demonstrates that it is not possible to measure the effectiveness of an integrated program if the Operational Plans are not developed in a manner that is coordinated and consensual among the implementing agents at all the levels and regions. • In order that the implementing agents can verify their achievements, a constant monitoring of the established goals for the period is required. • The qualification (“grading”) by the beneficiaries of the activities must be considered as a reference point for the implementing agents, since it expresses the perception on the benefits the beneficiaries expect to receive. • The advocacy activities that are relevant in the framework of a Program are those that have as their major objective, the well-being during childhood in the framework of a sustainable strategy. • The strengthening of the capabilities of the major local and regional stakeholders is what sustains the entire process of Decentralization, with sights on assuring that the authorities, as well as the communities, have the necessary and sufficient technical tools to confront this challenge. • A lesson learned in this period of execution of REDESA is the need to sponsor the joining income generating activities with health and nutrition activities in an effort to achieve an integrated vision of the program. • The development of a CBC strategy shows better results in the implementation of processes that are standardized and validated and that permit the establishment of a common system of indicators and processes of evaluation, as well as the adjustment of these processes to the particularities in each region. • The success of a CBC intervention is in the combination of mass media, especially radio, using the language and vocabulary according to the social and cultural profile of the population, and with messages adapted to each type of audience as part of the marketing of the adoption of suitable practices. • The sustainability of an intervention model that is sustained in the construction of a Partnership Network - for political advocacy - requires a process to change the organizational mentality and culture from vertical and centralized to horizontal and decentralized. Thus, all the actors will be able to commit themselves to the constant support of diverse activities, the leveraging of resources and especially, to the making of decisions. • A Development Plan that has as its objective the reduction of poverty and malnutrition must be constituted as an instrument that brings together populations in extreme poverty – traditionally isolated from economic circuits – with the Regional Development Centers. In this framework, the formation of new leadership contributes to democratic renovation if accompanied by a more transparent and publicly accountable management.