CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 1 CRS Niger PASAM-TAI Programme d’Appui à la Sécurité Alimentaire des Ménages - Tanadin Abincin Iyali Mid-Term Evaluation Final Report Submitted to PASAM-TAI March 3, 2016 TANGO International This study is made possible by the generous support of the American people through the United States Agency for International Development (USAID). CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 2 Table of Contents List of Acronyms .........................................................................................................................................................4 Executive Summary.......................................................................................................................................................6 Objectives and Methodology.......................................................................................................................................13 Background .............................................................................................................................................................13 Methodology ...........................................................................................................................................................13 Structure of the Report ............................................................................................................................................16 Strategic Objective 1 ...................................................................................................................................................16 IR 1.1 16 Output 1.1.1 ........................................................................................................................................................16 Output 1.1.2 ........................................................................................................................................................20 Output 1.1.3 ........................................................................................................................................................24 Output 1.1.4 ........................................................................................................................................................28 …and 28 Output 1.1.5 ........................................................................................................................................................28 Output 1.1.6 ........................................................................................................................................................28 IR 1.2 31 Output 1.2.1 ........................................................................................................................................................31 Output 1.2.2 ........................................................................................................................................................32 Output 1.2.3 ........................................................................................................................................................35 Strategic Objective 2 ...................................................................................................................................................36 Implementing Partners........................................................................................................................................36 IR 2.1 37 Output 2.1.1 ........................................................................................................................................................37 Output 2.1.2 ........................................................................................................................................................41 IR 2.2 43 Output 2.2.1 ........................................................................................................................................................44 Output 2.2.2 ........................................................................................................................................................46 IR 2.3 47 Output 2.3.1 ........................................................................................................................................................47 Output 2.3.2 ........................................................................................................................................................48 IR 2.4 49 Output 2.4.1 ........................................................................................................................................................49 Output 2.4.2 ........................................................................................................................................................50 Output 2.4.3 ........................................................................................................................................................52 Strategic Objective 3 ...................................................................................................................................................54 IR 3.1 55 …and 55 IR 3.2 55 Crosscutting Objective 1 .............................................................................................................................................58 Crosscutting IR 1.1: .................................................................................................................................................58 Crosscutting Output 1.1.1 ...................................................................................................................................58 Crosscutting IR 2.1 ..................................................................................................................................................59 Crosscutting Output 2.1.1 ...................................................................................................................................59 …and 59 Crosscutting Output 2.1.2 ...................................................................................................................................59 Crosscutting IR 3.1 .................................................................................................................................................. 61 Crosscutting Output 3.1.1 ...................................................................................................................................61 Crosscutting Output 3.1.2 ...................................................................................................................................63 Volunteerism, Partnerships and Coordination .............................................................................................................64 Volunteers ...............................................................................................................................................................64 Government of Niger...............................................................................................................................................65 Collaboration with other NGOs...............................................................................................................................67 CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 3 LIST OF TABLES Table 1 – Disasters that have affected communities and their responses ................................................... 54 Acknowledgements I would like to thank the staff of PASAM-TAI and CRS for the time and effort they contributed during this mid-term evaluation to accommodate our activities within their busy schedules. Many thanks are due to Chief of Party James Quarshie who came back from vacation to find the MTE underway and provided needed information, guidance, and overall coordination, while CRS Acting Head of Programs Adamou Abdourhimou assisted with the initiation of the exercise and provided support throughout. The Monitoring and Evaluation Coordinator, Oumarou Ali Dan Tessaoua facilitated logistics and document access along the way and took valuable time during an incredibly busy time of the year. Our work was ably supported by Deputy Chief of Program Ousseini Sountalma, the cluster supervisors and administrators in Mayahi and Kantche departments, the technical team leaders, and the field agents. When malaria was threatening to interrupt my work, Mayahi administrator Ahmadou Issahak graciously ensured that I was quickly diagnosed and received treatment. The work of PASAM-TAI and, therefore of the evaluation team, is only possible because of the partnership with communities and with the various partners who work with us to create programs and who contribute to the development of Niger. We greatly appreciate the time taken by so many people to explain to us what they are doing, particularly the hard-working women and men of the villages that we visited. Every effort was made to ensure the accuracy and validity of the findings and conclusions contained in this report. Given the vast range of activities in PASAM-TAI, errors and gaps in the analysis are possible. I take responsibility for any omissions or inaccuracies remaining in the text. Darren Hedley—Team Leader Mouhamed Barro—Nutrition and Health Consultant Alio Assoumane, Hasmiyou Diop, Yaou Alla Aminatou, Mahamane Sani Yahaya Rachida—Interviewers TANGO International October 2015 CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 4 List of Acronyms ACF Action Contre la Faim, French NGO (allied with ACH) ACH Acción Contra el Hambre, Spanish NGO (allied with ACF) AT Agent de Terrain BDL Bio-reclamation of Degraded Lands CADEV Caritas Développement CG Care group CGLE Care Group Local Expert CHW Community Health Worker CLTS Community Led Total Sanitation CO Country Office COFO Commissions Foncières, Land Commissions CRS Catholic Relief Services CSB Corn soy blends CSB+ Corn soy blends Plus CSI Centre de santé intégré, integrated health center DFAP Development Food Aid Program, USAID DRM Disaster Risk Management ECD Early Childhood Development ECE Early Childhood Education ENA Essential Nutrition Actions ET Evaluation Team EWS Early Warning Systems FFP Food for Peace FFW Food for Work FGD Focus group discussion GoN Government of Niger HH Household ICRISAT International Crop Research Institute for the Semi-Arid Tropics INRAN L’Institut Nationale de Recherche Agronomique du Niger IPTT Indicators Performance Tracking Table IYCF Infant and Young Child Feeding M&E Monitoring and Evaluation MCU Mother and Child Units MLG Male Learning Groups MTE Mid-Term Evaluation MUAC Mid-Upper Arm Circumference NGO Non-governmental organization ODF Open defecation free OSV Vulnerability Monitoring Observatory PASAM-TAI Programme d’Appui à la Sécurité Alimentaire des Ménages- Tanadin Abincin Iyali PICS Purdue Improved Crop Storage PLW Pregnant and Lactating Women PSP Private Service Provider PVS Participatory Variety Selection RAV Réseaux des Arts Vivants, Living Arts Network RISE Resilience in the Sahel Enhanced SAREL Introduction to Sahel Resilience Learning Project CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 5 SBCC Social Behavior Change Strategy SCAP-RU Système Communautaire d’Alerte Précoce et de Réponse aux Urgences SD Secure Digital SILC Savings and Internal Lending Communities SMS Short Message Service SO Strategic Objective TANGO Technical Assistance to NGOs U5 Children under five years of age UNICEF United Nations International Children's Ecumenical Fund USAID United States Agency for International Development VDC Village Development Committee WASH WAU Water, sanitation, and hygiene Water User Association WFP World Food Programme WHO World Health Organization CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 6 EXECUTIVE SUMMARY The mid-term evaluation (MTE) was carried out in August–September 2015, through a review of documentation, meetings with staff and partners, and field visits to a random sample of 14 project villages in Mayahi and Kantche departments. In each village, the Evaluation Team (ET) held focus group discussions (FGDs) with the chief and a general community gathering then separate FGDs with women and men, as well as FGDs and in-depth interviews with a range of community groups and individuals. The Programme d’Appui à la Sécurité Alimentaire des Ménages — Tanadin Abincin Iyali (PASAM-TAI) is a very important program in a region with tremendous challenges, including crushing poverty, the deepening effects of climate change, migrating populations, and the containment of terrorist threats. This Catholic Relief Services (CRS) project is part of United States Agency for International Development’s (USAID) work through the Resilience in the Sahel Enhanced (RISE) program. RISE-affiliated projects are aligned to promote livelihood strategies adapted to the harsh environment, weaving these into appropriate value chains in partnership with private partners, and strengthening governance capacities to help integrate these efforts sustainably with local institutions. In itself, PASAM-TAI is a very large, complex, and challenging program, operating at a significant scale with many different activities taking place. The program is broad programmatically, even by the integrated standards of Food for Peace Title II programs. In addition, it is geographically very extensive, working in 909 villages and surrounding hamlets and even with a new focus on a shorter list of 643 principal villages1 , it is more geographically spread out than the other Development Food Aid Program (DFAP) programs in Niger or than the typical Title II program. As explained by the program, the overall approach consists of offering training, methods, tools and some resources and allowing the communities to take these resources and pursue their own development processes without as much intense accompaniment as often occurs in traditional programs. Many of the activities are only found operating in a smaller number of villages, while others (e.g., 1,000 days food distribution, care group, seed distribution) are found essentially everywhere. This variability in activity levels should be balanced with an appreciation of the challenge that PASAM-TAI is facing – with considerable success – to implement so much activity in so many villages. In discussions with villagers, PASAM-TAI is broadly perceived as a program which provides Corn Soy Blends (CSB) and oil, promotes the health and nutrition of women and infants, gives out seeds and livestock, and organizes women for the purposes of promoting savings. The key messages that the villagers have grasped are in fact largely the key concepts of the program: the importance of hand￾washing, household and village hygiene, the importance of perinatal clinic consultations, and using latrines and mosquito nets. Respondents reported that they were changing their behavior and implementing these key concepts, and this was in some cases corroborated through evidence of improved community hygiene and reports by promoters and lead mothers of changed child feeding practices. Thus, there is no doubt that the program is affecting villagers’ lives in a positive and instructive way. The ET observed that, in contrast, villagers did not generally retain or mention simple messages nor cite obvious changes in practice in the areas of agriculture, livelihoods, and disaster management. Areas where residents display greater awareness and understanding (e.g., health, nutrition, and sanitation) could help define the program’s core and where it can potentially focus greater efforts for the remainder of the program. The 1,000 days concept is partly understood, and project data suggests that almost 75 percent of targeted participants can cite some of its components, villagers interviewed did not express an understanding that the 1,000 days was a critical developmental window for infants that would mark them for life, and that this was the reason for the focus of PASAM-TAI on MCUs. 1 The original list includes 909 villages and surrounding hamlets, but the project has focused most activities at the level of 643 principal villages, which at the same time can serve the satellite hamlets. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 7 Under Strategic Objective (SO) 1, the core activities are in nutrition and health, in which PASAM-TAI has made significant progress; the annual program monitoring provides reassurance of this progress in terms of childcare practices (e.g., 80 percent of households understand the importance of health services). The backbone of the program’s outreach to villages is the provision of supplementary Corn Soy Blends Plus (CSB+)2 and oil to pregnant and lactating women (PLW). The food distribution is effectively combined with a care group approach to nutrition, health, and hygiene education (discussed below) although in sampled villages, the care groups were not always active. PASAM-TAI manages to distribute to beneficiaries in 909 villages and hamlets and to remote locations that other assistance programs have been unable to reach. A generally robust system was in place to register valid beneficiaries with the assistance of local, long-term volunteers, and the program generally manages to overlap with other agencies in terms of food assistance. Anecdotal evidence provided during the MTE indicate that PASAM￾TAI is much appreciated and that it is having a positive nutritional impact in conjunction with other inputs. However, a key issue that the ET learned about was a lack of consistency in the enrollment of new beneficiaries. The ET found that many women who apparently met the criteria and who had already been registered at the village level through 2013 and 2014 (prior to the November 2014 cutoff date) were not receiving rations. This was reported in all sampled villages and triangulated during FGDs with men, by women, and in interviews with promoters. The promoters are in a good position to comment, given their role of enrolling beneficiaries and then witnessing that not all enrolled beneficiaries are actually entered into the list and given a card and code to receive food rations. Reasons for this lack of consistency were unknown to community members, nor did the ET find a definite explanation for this. The ET also found that while Mother-Child Units (MCUs) are supposed to be automatically removed from lists when the infant reaches the age of two, they often remained on the list for considerable times afterwards. Other concerns arose over hygiene during the ET’s observation of food distribution, as beneficiaries are required to divide commodities into their own containers to compose the individual rations. Nonetheless, the program was in the process of resolving this problem by providing MCUs with appropriate smaller containers. The care groups—guided by promoters, volunteers and “lead mothers” — are doing an effective job where they are active, and the model of informal household advising and counselling shows much promise. Program data indicates that 680 villages have care groups functioning, comprising 75 percent of the total villages and hamlets and probably covering all of the main villages, which gives a sense of the scale of work involved. This is an area receiving a considerable investment by PASAM-TAI, with 19,141 community health workers trained in child health and nutrition in FY14, for an 82.15 percent achievement of the year’s target. Training of the lead mothers is conducted by promoters, and this works reasonably well in most cases, though activity levels were not consistent, which may be due to the challenge of providing support to all villages and the fact that some promoters have not yet been trained. In addition, the program cites the problem of incentives to health volunteers, provided by other organizations, which at times affects volunteers’ activity in PASAM-TAI. Adolescents are addressed in the program by being targeted for inclusion in the care group activities, and they are also targeted through communications, literacy, and school-based activities (and possibly in SILC groups). This is a key target group, and PASAM-TAI could make good headway by ensuring integration of these activities and making a conscious effort to reach adolescents. Promoters regularly carry out growth monitoring, and information is used in a variety of ways such as the innovative display of underweight status in the village on giant scoreboards. Anthropometric measurements are often a challenge for volunteers. The ET observed some promoters and their procedure for taking Mid-Upper Arm Circumference (MUAC) measurements and found that they were not taking the measurements in the correct location in the middle of the arm. Some of the lead mothers expressed to the ET they felt they could assist with other health promotion work currently done by promoters such as growth monitoring, which could be a valid suggestion to consider, although such work does extend 2 The enhanced CSB+ was provided, though it is often referred to in the generic sense as “CSB” CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 8 beyond their current intended functions. The Community-Led Total Sanitation (CLTS) approach has taken off quite quickly, in an initiative that mobilizes the community and provides a good focal point for both household work (e.g., improving hygiene) and volunteering for the community. There were some issues mentioned with the program following up once the triggering process has started, and these are brought to a head during the certification. Most participants in FGDs displayed knowledge of hand￾washing and affirmed that they practiced it, which is a commendable breakthrough. To accompany this, the promising tippy-tap model of hand-washing facilities now needs some focused training and technical support to make use of materials that have been delivered to some communities. Support to Government of Niger services in the area of water supply is advancing fairly well, in a good partnership that reinforces government capacity to direct development in a key technical area. Some communities are mobilizing to raise funds to co-invest in water supply infrastructure, apparently with some encouragement from the government’s hydraulic department, not directly instigated by PASAM-TAI. The ET considered this a risky arrangement for co-investment in water supplies without adequate controls and clear plans. Such cases may eventually affect PASAM-TAI, so it would be advantageous for the program to track and offer advice for such community financial management efforts, even where initiated and funded by others. Some aspects of social marketing have been quite effective. The radio club discussions and the Living Arts Network (RAV, Réseaux de Arts Vivants) drama and communications work is definitely opening people’s minds and making an impression, and the impact could be greater if these are extended and better-integrated with other program components. Aquatabs have been introduced; people have used them; and some villagers expressed appreciation for them. However, when Aquatabs have been introduced for selling, villagers were not willing to buy them. Work with early childhood development centers was at an initial stage of training project staff, partners and facilitators, based on the curriculum developed in coordination with health and preschool departments. A launching ceremony took place in July 2015 before further field level implementation, so results in the coming months will shed light as to how effective and sustainable the intervention will be. In SO2, most villagers recognized the value of the seeds and livestock provision and the Savings and Internal Lending Communities (SILC) and expressed interest in having more of these opportunities. It is a noteworthy accomplishment for the program to reach a high percentage of villages and significant numbers in each village with these interventions. Moreover, for the most part, significant benefits accrue without large and ongoing investments in these activities. Over 31,000 beneficiaries had received agriculture/food security training in FY14, representing 83 percent of the program’s target, while 95 percent had applied new technologies or management practices. Different SO2 interventions enhance the livelihoods of different groups of people, and though sometimes limited in scope (e.g., gardening), the numbers of beneficiaries can add up to something significant. For example, the ET observed some villages with multiple SILC groups of 25 people and other villages with 50 people participating in a Bio￾reclamation of Degraded Lands project. The SILC groups in particular were functioning well, with clear procedures and potential for sustainability, and the community-based technical assistance is a good model for minimizing costs and maximizing the potential outreach of savings services. Project data indicates 243 villages with SILC, but the ET encountered one or more SILC groups in most sampled villages. Some individuals who had not yet had the chance to join a SILC group expressed desire to do so, and the overlaps of membership with care groups were obvious in some communities. This quite possibly strengthens both the SILC and care group activity. The household gardens show promise for dietary diversification and promotion of micronutrients, and the keyhole gardens are a promising innovation, but the ET considered that achieving significant results and replicability might still require substantial ongoing inputs. The program may need to assess how much more to invest in expanding these activities. The community irrigated gardens are a good pilot, from the limited view the ET had of them, and ongoing collaboration with the Agriculture Services of the Government of Niger can enhance management and drawing of lessons from them. The Bio-reclamation CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 9 of Degraded Lands project activity demonstrates important conservation agriculture techniques and adapted plants, but respondents reported some challenges with fencing, as well as with ensuring land access in some villages. The sheep fattening and goats program with SILC groups has been well received and is a clear livelihood support to beneficiaries. An area for review in the remaining months is how to maximize the health and physical development of the livestock procured for beneficiaries to ensure that beneficiaries receive an adequate return. The Government of Niger livestock services expressed interest in actively participating in program monitoring. The work on staple crops benefits from good quality seeds that beneficiaries have been selecting according to their own criteria, and initial signs are that it will deliver early-yielding, high quality crops, possibly in larger amounts, and that it will provide more reliable yields even during low-precipitation years. There was some promising feedback in some villages, where respondents stated that they had benefited from and would like to receive more training in crop production and other SO2 aspects. Soil fertility and other aspects of conservation agriculture are addressed in some aspects of the program (e.g., Bio-reclamation of Degraded Lands, farmer field schools) but do not seem to have been understood or applied as much by villagers so far, despite access to top-notch research institutes like the International Crop Research Institute for the Semi-Arid Tropics. This may change with the Resilience and Economic Growth in the Sahel partnership. Villages reported animal and crop disease as regular disasters, with large impacts on livelihoods, which could cancel out some benefits from improved seeds and livestock schemes. This should be considered by CRS and the Government of Niger for future or parallel programs. The para-vet animal health intervention shows some promise but has yet to make an impact on the villages visited during the MTE. In SO3, the main focus for PASAM-TAI in much of its first half has been disaster preparedness in the form of early warning reporting to authorities, in addition to learning actions, which communities can take on their own, through the committees affiliated with the emergency monitoring systems (Système Communautaire d'Alerte Précoce et de Réponse aux Urgences, SCAP-RU). Some of these have been functioning, others are less constant, and there were some very good examples of self-help efforts by communities as well as enhanced advocacy with authorities. Meanwhile, a problem reported by communities is a lack of responsiveness, mainly by authorities; some villagers also expressed that they did not feel they could count on PASAM-TAI to assist them in organizing a response to emergencies. The coordination work at communal level seems weak, and national disaster preparedness coordination advances made early on by PASAM-TAI/CRS need to be rebuilt. In the Crosscutting Objectives, PASAM-TAI has numerous women’s empowerment interventions and has already been making reasonable efforts to mainstream gender and strengthen women’s participation in areas like SCAP-RU and Village Development Committees (VDCs). A key line of action has started with Male Learning Groups in SO1, which is still preliminary and needs rapid rollout but is promising. The literacy program has the potential to greatly increase women’s confidence and abilities, and though the sustainability may be questionable, the effort is worth it. VDCs are struggling with uneven participation and high turnover, but they are indispensable, and a number of positive examples came out regarding the contribution VDCs can make. As they gradually become clearer on their role, they will participate with and empower other institutions and will coordinate efforts with government agencies and other organizations. The program states that the government is the primary partner and is closely involved in areas such as quarterly departmental meetings and annual steering committee meetings, coordinated development of curriculum for ECD and literacy, and joint planning of infrastructure development. The ET deems this represents an admirable level of effort in an area often underemphasized by other programs. Still, most of the government officials interviewed by the ET relayed that communication and coordination was not up to their expectations, citing an emphasis on planning-oriented meetings rather than joint review of activities, reporting that does not provide sufficient details, and limited opportunities to visit implementation sites. This is a complex aspect of the program, and the perspective of PASAM-TAI staff CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 10 is that collaboration is sufficient, but also that they are constrained by the program’s lack of ability to provide resources or incentives to government departments. Nonetheless, the degree of communication and collaboration seems to range from very limited in some areas such as health, to quite good in others such as water supply, and the program can probably learn from its own different experiences to see how to make some improvements. PASAM-TAI includes numerous innovative and fascinating activities, which have been useful on a pilot basis but are very challenging to manage within the framework of one program. The ability for the different activities to contribute to the overall goals is limited when field agents have limited time to directly train villagers and do the essential follow-up work, and when community groups and volunteers have limited capacity as yet to carry the work forward on their own. The issue of volunteerism is never easy to manage, and broadly the ET supports CRS’ approach, though there still could be more done to clarify the policy and harmonize it with other agencies, and promote assistance to key volunteers through internal community payments or in-kind support. Training is crucial to PASAM-TAI, and the program has mobilized a massive effort to promote learning among residents of the targeted villages. This challenge is made more difficult by low literacy levels in the area. Good learning takes place in a variety of settings such as in homes with care groups or community seed fairs using a variety of formal and informal methods, and this flexibility can help provide a learner-centered process, which supports community development. The amount of learning taking place is significant, as evidenced by villagers who shared what they had learned and expressed desire to learn more. Moreover, PASAM-TAI draws on specialist staff and consultants to lend expertise and create these learning programs. The program could still enhance the quality and sustainability of learning by reviewing learning approaches and materials across all program areas, in terms of consistency in defining learning objectives, having simple guidelines, and identifying outcomes, which can be monitored. The program reports that large numbers of individuals are trained, but not all of the training is consistently perceived and reported by respondents in the villages visited by the ET, perhaps in part because some of the individuals being trained (e.g., promoters, para-vets) are not able to share the training as expected, and the program has limited capacity to monitor this. In addition, in some program areas, if there is not a selected number of key concepts and outcomes in terms of measurable practices, it can be more difficult to trace the effects of training. The word “training” is also used to describe a wide range of learning activities including formal and lengthy classroom sessions, field-based demonstrations, briefings, or the offering of advice. It would be helpful to recognize and clearly communicate these different approaches, which would increase transparency and allow more nuanced comparison of program inputs with the measured outcomes. It seems probable that the heavy workload and turnover of field agents meant that the quality of training which they provide may not always be high, and having straightforward training guides could help ensure more consistency. Some components of PASAM-TAI, particularly in health and nutrition, benefit from having established concepts and materials created by previous programs by various organizations, and it may take time for other components to develop these materials. When focus groups discussed health, nutrition, and hygiene, participants consistently reflected knowledge of the themes and explained how their behavior had changed in accordance with program objectives. This is probably partly due to the training materials and guidelines related to care groups, for example, which were quite clear and focused, including “aide memoires.” Health and literacy materials also showed a specific approach to fostering learning; in some cases materials were available that participants could keep and refer to, principally literacy materials. Very good materials were seen for SILC, which makes it easier for participants to know their roles and master basic concepts. Other activities of SO2 either did not have training materials or guidelines available to the ET (i.e., market gardens, keyhole gardens, livestock fattening), or materials were sometimes of high quality (i.e., those for BDL), but the training methodology was less specific (see further discussion on SO2 training, under output 2.2.1). Training materials for SO3 appeared quite comprehensive, including well-prepared graphics. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 11 Some aspects of PASAM-TAI are sideline components to a typical Title II program, such as literacy or early childhood development, but they may represent valuable innovations. However, it is difficult to have major breakthroughs in a new component of a major integrated program, if there are limited resources, and that component is a somewhat lower priority. It would thus make sense at this mid-point to take stock and consider de-emphasizing some components. This could be done by 1) not continuing to expand a component, 2) scaling it back, 3) focusing on selected villages/communes and developing good pilots, and 4) stopping support altogether. While it may be beyond the remit of the ET to advise specific components to de-emphasize, criteria to consider include de-emphasizing where: 1) considerable additional resources are still needed to be reasonably sure of an impact, 2) there is still little or no evidence of replicability, 3) there is limited uptake of implementation work by community groups, 4) there is little prospect for ongoing support of the intervention by government or in the framework of other likely projects, and 5) it is not feasible to provide meaningful input across the 643 principal target villages of PASAM-TAI. Geographic spread is also difficult to deal with if, as USAID has stated, the final evaluation will hold the program responsible for all 900+ villages from the original program document. It helps to focus work on slightly larger villages, in a way to include the smaller hamlets; however, working in 643 villages is still a daunting task. Set against this is the fact that the program works in only two departments. In such situations, to reach scale, more emphasis should be placed on involving and supporting government partners, who can, within the admitted limitations of their capacity, provide follow-up while also working to maximize the capacities of partners such as community organizations. Key Recommendations:  Lack of complete understanding of1,000 days approach: Promote more than just behavior adoption; seek also to increase villagers’ understanding of the 1,000 days approach, that this period is crucial for the development of the child for the rest of their lives, and that therefore they should make particular efforts during this time. Review the approach, drawing on lessons learned by CRS elsewhere.  Problems in registration and graduation of food beneficiaries: Review the registration system; develop clear guidelines, and determine the extent of non-inclusion of valid beneficiaries (as was experienced in villages sampled during the MTE). Ensure that all staff and actors agree and are clear on how the system should work. Consider one-time registration for women who rightfully should have been registered. Ensure adequate graduation procedures.  Care groups have mixed levels of activity: Ensure training for lead mothers; review their relationship with promoters, and have some occasional contact and monitoring by field agents. Consider wider roles, and attempt to harmonize policies on incentives with other organizations in the same villages or communes.  SILC is working well: Continue good work already done, but also monitor and analyze innovations and involve selected groups in pilots of integration with care groups.  Emergency Preparedness: Strengthen the functioning of communal-level Vulnerability Monitoring Observatory (OSVs), particularly to clarify and ensure that communal authorities respond in some way and support villages within the limits of their capacity.  VDC roles unclear: Gather examples of good practices and promote the active participation of VDCs in an empowering way with other community-based organizations promoted by the program (including farmers' associations).  Training programs do not all have clear methods, focused concepts and outcomes in terms of practices: Review training materials and approaches to ensure that methods are clear and to attempt – within limits of time still available – to share approaches among the various training areas, review materials and methods (and durations) of training, and make more explicit the expected behaviors that will show that participants have internalized the training. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 12  Lack of clarity on strategy for value chains and farmers association: Promising ideas are emerging, but a clear strategy is needed to define the value chain approach and its indicators. Ideally this could be done in conjunction with USAID and its relevant partners. Farmers’ Associations’ role in this strategy should be elaborated to define their current structure and capabilities, what support they will need, and what role they will play.  Broad program focus coupled with mixed results: Emphasize select program areas, following the prioritization criteria suggested above, in order to have good examples of interventions and widespread impact.  Concerns over volunteers’ workload: Broadly continue the policy, but review it to ensure the program avoids overtaxing any particular volunteers particularly women, attempt to harmonize approaches with other NGOs, and promote opportunities for communities to support key volunteers  Uneven partnership with government departments: Renew efforts to strengthen partnerships with government agencies at departmental and communal level and thereby build the capacity of government and community organizations. This can help increase the spread of the PASAM-TAI approach and innovations, offset some work required of CRS staff, and sustain and build on its activities. Strong partnership with government agencies is a key part of USAID’s strategy, particularly with its Resilience in the Sahel Enhanced integration program. Though some components of PASAM-TAI have done reasonably well, partnerships with government agencies should be done in a more balanced way. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 13 OBJECTIVES AND METHODOLOGY Background The five-year USAID-financed multi-year Development Food Aid Program (DFAP) named Programme d’Appui à la Sécurité Alimentaire des Ménages (PASAM-TAI) is led by Catholic Relief Services/Niger in partnership with the Government of Niger (GoN), International Crop Research Institute for the Semi-Arid Tropics (ICRISAT), GRET (Groupe de Recherche et d’ Échange Technologique), and CARITAS. The goal of the program is to reduce food insecurity and malnutrition in rural households in the Maradi and Zinder regions of Niger. Strategic objectives of the program are as follows:  Strategic Objective 1 (SO1): Households (HH) with pregnant and lactating women (PLW) and children under five years of age (U5) have reduced chronic malnutrition.  Strategic Objective 2 (SO2): Vulnerable HHs have increased the production and consumption of food for nutrition and income.  Strategic Objective 3 (SO3): Target communities have improved integrated disaster risk management.  Crosscutting Objective 1(CCO1): Women and men expand gender roles to enhance sustainable results.3 The overall objective of the DFAP midterm evaluation (MTE) was to assess achievements against targets, identify the program’s strengths and weaknesses from inception to mid-term, and provide recommendations for improving program implementation during the second half of the program. The MTE used the strategic objectives as drivers of research questions, including cross cutting themes, with a discussion provided for each of the intermediate results (IR). In addition, the MTE reviewed the quality and effectiveness of partnership between PASAM-TAI and GoN departmental government services and other development partners in the region. Methodology The fieldwork for the PASAM-TAI MTE was carried out in August–September 2015, and this report relates to the state of implementation as of this date. TANGO appointed an international team leader who commenced planning in July 2015. The MTE had a qualitative focus with an iterative process to design the exercise, wherein the methodology evolved as observations and insights were made. A draft evaluation action plan and tools were developed before the start of field work, which included a preliminary desk review of available program documents. The PASAM-TAI implementation team provided a presentation on the program aims and progress to date to help create a clear evaluation framework and help prepare a complete list of key documents and contacts with inputs from all relevant staff involved in PASAM-TAI implementation. The ET also consisted of a local consultant specialized in nutrition and health who had just worked with TANGO on the MTE of the Mercy Corps DFAP. This facilitated considerable harmonization of approaches between the exercises as requested by USAID and CRS. In addition, the role of agricultural specialist was fulfilled by the team leader in addition to their own responsibilities after unsuccessful attempts to identify a separate agricultural specialist. 3 Pathways of key strategic objectives are further outlined in the mid-term evaluation SoW found in Annex 2 CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 14 The MTE is a qualitative research exercise; as such the findings are meant to reflect the types of experience and issues faced by PASAM-TAI and its participants, not to provide a statistically representative indication of the extent to which those issues are found across the entire intervention area. Some of the relevant features of qualitative research4 , and how they apply in this MTE, include:  Credibility – through prolonged engagement in the field, persistent observations, triangulation, openness to criticism by disinterested reviewers, search for and acknowledgement of negative instances that challenge hypothesis  Transferability – provision of (or awareness of) a detailed description of the setting studied, so readers can judge the applicability of findings to other settings. In the case of a program MTE, it is assumed that readers already have a sufficient knowledge of this, without taking additional space in the report. Also, in the case of PASAM-TAI, there is considerable similarity in context within the two targeted departments, enough as to broadly conclude that what is found in one village may easily apply in other villages.  Dependability – researchers document data, methods and decisions made during research  Confirmability – similar research methodology applied elsewhere could yield similar conclusions In terms of the number of villages to visit, there was no statistical way of determining the sample number, given that it is a qualitative research exercise. However, considering the desire for harmonization of methods, as well as the large number of intervention villages for PASAM-TAI (643 intervention villages5 ), a decision was made to expand the number of sample villages from the original terms of reference. A larger number reduces the possibility that findings can be discounted due to factors such as lack of geographical representativeness. Also, a slightly larger number allows more chances to have greater “saturation”6 of the findings to have a more complete characterization of the properties (categories) and dimensions (values or options within those categories). In other words, it would more likely bring to light any issues or special situations which may be limited to only a few villages. Fourteen was chosen as a feasible number of villages to visit within the MTE timeframe, and was deemed large enough to allow a more accurate and informative reflection of how PASAM-TAI was advancing towards its goals. Thus, an expanded team was hired using four interviewers who had also participated in the Mercy Corps MTE. TANGO invited PASAM-TAI to assign two monitoring and evaluation (M&E) staff to participate in the MTE to strengthen the team’s understanding of PASAM-TAI and to balance the external and internal perspectives, although gradually it became clear that this would not be possible, as the M&E staff were busy with another exercise. An evaluation protocol with an overall topic guide for key informant and focus groups interviews along with a list of groups and individuals to be interviewed was drafted and then modified after the pilot village visit. The main methods used were focus group discussions (FGDs) and key informant interviews as well as physical observations and verifications of various program activities and tools. The protocol was shared with the CRS Country Office (CO) during multiple stages at the commencement of the exercise for validation purposes and to include any suggestions for modifications as required. Given the large number of PASAM-TAI program components, particular effort was made to meet with community groups who could provide insights on the various activities and their own activities. 4 Lincoln and Guba criteria for naturalistic or qualitative inquiry, quoted in Seale, C. (1999) The Quality of Qualitative Research, London: Sage Publications 5 643 is the number of main villages, while if surrounding hamlets are included the number exceeds 900. 6 Strauss, A. and J. Corbin (1998) Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, London: Sage. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 15 Given that the number of targeted villages and activities in PASAM-TAI is very large, many activities are implemented in a limited set of villages. Even when a village was provided training for a certain activity, there may not be community groups actively continuing with any given activity. As such, the ET was given a list showing which activities and community groups were expected to be functioning in each village, to help ensure that the ET would interview some of the various groups. In fact, the situation in the field often varied from what was suggested in the list: in some cases, there were additional activities found that the program list did not reflect, while in other cases, a group indicated on the list was not active. Thus in order to attempt to capture activities that were not in the original sampled list of 14 villages, the ET arranged to visit additional villages in order to meet with those groups. Using 14 as the sample size for villages in which to conduct qualitative research, the number of villages was divided, with five villages in the Mayahi region and nine villages in the Kantche region, based on the proportion of villages in the program as a whole (with 209 villages in Mayahi, and 434 in Kantche). The overall list of villages was used as a sample frame, and a random list of 40 potential villages was formulated for each department. The first 14 villages were selected randomly (five for Mayahi and nine for Kantche), with a few exceptions. The first village used for piloting the topic guide and protocol was selected because it was identified as a village with many activities in a somewhat advanced state. In a few cases, a village was passed over to facilitate fieldwork logistics. This randomized selection ensured that villages were more representative and guarded against the common problem of visiting only the most advanced villages so as to get a one-sided view of the program. The selected villages also turned out to be fairly geographically representative in terms of distribution around the two regions. The team spent one day in each village to allow time to meet the various interest groups. By spending significant time with villagers in their own setting and asking open-ended questions about the program, the MTE allowed respondents to be more comfortable expressing their opinions. The qualitative data was analysed throughout the collection period with regular meetings conducted among the team members to share and discuss findings and implications. In addition, the team compiled a report on each village to ensure clarity of the data base. A certain degree of flexibility was maintained with the topic guides and evaluation fieldwork plan in order to adapt as important new aspects arose requiring further investigation. This flexible process reflected an on-going learning process. In order to visit all 14 original villages and several additional ones and allow time for analysis and write-up, and in view of the considerably smaller￾than-expected number of activities found in some villages, the team was at times divided into two sub￾teams, each assigned to a different village. Document review was a method used to get a clearer idea of the focus of PASAM-TAI, understand how the program structured training, and obtain a different perspective to compare with community feedback about the program. The ET requested all key project documents, methodological guidelines, and training guides prior to the fieldwork and continued to request these from the program and from specific coordinators during interviews. Several key documents that provided recent information on project progress included the FY14 annual report (with its IPTT annex), the quarterly report from July 2015, and a presentation dated August 22, which was used as a source of the most up-to-date figures on the program. Meetings were held in Niamey and Maradi with all available key program managers, technical and implementation staff, with additional interviews as needed. Key informant interviews were carried out with senior officers of various GoN departments and with the main non-governmental organization (NGO) partners in PASAM-TAI. CRS arranged interviews with relevant program partners, in view of the limited time available. In hindsight and considering subsequent feedback by FFP, additional interviews should have been arranged with USAID and RISE program partners, and a meeting with the donor also should have been prioritized. A debriefing was held in Niamey at the mid-point in the fieldwork to provide progress reports and share initial findings. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 16 Structure of the Report This report is structured according to the results framework of PASAM-TAI, mirroring the strategic objectives (SO), intermediate results (IR), and outputs of the program. In several places in the report, the ET combines several outputs or IRs that are closely related, either because the pertinent observations integrate these areas, or because there is not sufficient information to warrant separate discussions. This is clarified in the text, where two outputs/ IRs are combined and the following text relates to both. Recommendations are generally given throughout the text at the level of outputs, though not for all outputs. One exception is indicated in the text, with recommendations provided under SO3 as being related to Disaster Risk Management, while overarching recommendations are provided in the Executive Summary. An additional section on volunteerism, partnerships, and coordination is included at the end, as this was highlighted in the TOR and by both MTE respondents and CRS/USAID; the ET deemed it crucial to treat these in an integrated way. STRATEGIC OBJECTIVE 1 HHs (especially PLW and children U5) have adopted appropriate health, hygiene and nutrition behaviors. IR 1.1 HHs (especially PLW and children U5) have improved their health, hygiene and nutrition behaviors. The first intermediate result includes a wide range of outputs dealing with communications, community mobilization, 1,000-days food assistance, hygiene, and several activities with schools. Output 1.1.1  Communications: HHs have increased awareness of key health and nutrition messages. The first 1,000 days from conception until the infant reaches the age of two years is a crucial period of child development. During this period, the physical development is so rapid and complex that it is imperative that the nutritional needs are satisfied in order to avoid deficiencies that will have irreversible repercussions in the life of that child. The PASAM-TAI program, across SO1, implements activities to improve nutrition and health of children during this period of 1,000 days. These actions range from distribution of Corn Soy Blends Plus (CSB+) and oil to pregnant women and children under two years to sensitization on the 1,000-days approach, which is done by the promoters, volunteers and lead mothers wherever they are active. PASAM-TAI choses an awareness theme chosen each month.. Thus, when the ET asked community members to identify the main targets of PASAM-TAI program, almost all answered “pregnant women and children under two years”. They generally know the desired health and nutritional behaviors for these groups, and there are numerous signs of the desired practices increasingly being shown. When questioning villagers why the program focuses on these groups, however, they usually responded, “Because they are the most vulnerable.” When community members were asked to elaborate why they believed these groups were vulnerable, they generally answered that “children under two years are not able to feed themselves, and pregnant women are often weak.” Some stated that PASAM-TAI pitied these subsectors of the population. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 17 However, during these discussions, no participant mentioned the concept of a critical period of child development during which nutritional deficiencies have irreversible consequences in the future even if food inputs later become adequate. Village respondents noted that PASAM-TAI did not explain their reasons for choosing these beneficiaries, and even in interviews with some of the PASAM-TAI staff, these reasons did not come out clearly, indicating perhaps a need for further training. It is important that community members realize the importance of this critical period, so that the 1,000-days concept is more than just an expression. Mothers should understand that while PASAM-TAI cannot meet the nutritional needs and health of the entire population, it focuses on pregnant women and children under two years to ensure adequate development of the child starting at conception and up to the age of two. In doing so, PASAM-TAI can focus on optimizing physical development to avoid deficiencies and safeguard the intellectual and future capacities of children, thereby contributing to the campaign against poverty. To the extent that the GoN and other stakeholders may be promoting the 1,000 days approach, having a clear concept from PASAM-TAI will help gradually to instill a better understanding across the whole population. A number of other key messages have been successfully conveyed through a variety of means, particularly related to hygiene, health, and nutrition. During FGDs in the villages, in order to reveal the main messages that have been circulating within the villages, the ET asked respondents what they thought about the purpose of PASAM-TAI. The most commonly mentioned messages related to hand-washing, the importance of mother and infant health, exclusive breastfeeding, clinic consultations, and sanitation. People’s perceptions of the program focus includes CSB+ and oil, hygiene and sanitation, agriculture, health of infants and mothers, and culinary demonstrations; some also mentioned women’s savings. Adolescents: PASAM-TAI has many activities, which are of potential interest to adolescents. This is particularly the case under Output 1.1.1 where the project document foresees work to reach youth and adolescents with messages about nutrition, health, early pregnancy and continuing education, as many of the maternal and child health challenges arise from early pregnancies.7 In terms of communications work foreseen in Output 1.1.1 in the project document, the ET could not visit schools to observe and interview, and no youth groups have been formed. The program reports that care groups would sometimes talk with them, though during FGDs with care groups and women, there was no mention of specific sessions being carried out with adolescents. Youth, or “jeunes,” are young adults up to age 30, comprising a group that is prominent among PASAM-TAI participants, according to some FGDs. Literacy training is very important for young women, of whom many participate in the program. For adolescents in their early teens, one of the key linkages would be with the school-based activities, although the MTE was unable to see these activities because schools were out of session during the MTE fieldwork. Another potentially valuable input for adolescents is the radio program developed by PASAM-TAI on forced marriage with a good accompanying study package. However, given that this program is isolated, it could be expanded for improved benefits. Social Marketing Activities: Aquatabs are discussed in this report in the context of radio clubs and Community Led Total Sanitation (CLTS). There was no mention in the villages of sanitary platforms being provided by any organization. CNA and RAV: The ET conducted an interview with the Living Arts Network (RAV, Réseaux des Arts Vivants) coordinator, while the CNA (Cinéma Numérique Ambulant) representative did not come for the scheduled interview. RAV explained that their organization was created by a number of NGOs with the aim of helping change behavior by enabling people to look at problems in new ways and therefore change their conscience. Along with presenting dramas, they also facilitate discussions of the issues presented, such as exclusive breastfeeding and complementary feeding, maternal nutrition, and use of mosquito nets. 7 PASAM-TAI quarterly report FY15 Q3, 21 July 2015. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 18 In one village, two performances lasting at least two hours each were presented in 2014. The first provided dramatic sketches on hygiene and sanitation, and the second presented the importance of clinic consultations. Community members stressed that they also learned about the use of vegetables (e.g., cowpea, zucchini) in infant feeding, the use of insecticide-treated mosquito nets, the importance of hygiene and sanitation, and the use of various seeds. In a second village visited outside the 14 sampled villages, the ET found similar experiences. Lessons learned included knowing the difference between clinic consultations and traditional healers. Community members stressed that they realized that their health improved after one or two consultations with health care providers, which had not been the case with traditional healers. After watching these sketches, community members were more confident in modern medicine and regarded traditional healers as “liars” and “crooks.” They said that they would pass on this learning to their children and that they were looking forward to the next presentations. While it is positive to develop more trust and relationship with the clinics, it would be best not to alienate traditional healers, some of whom may provide beneficial services in health as well as broader community development. RAV visited 100 villages in 2013 and 200 villages in 2014/15. The “theatre forum” intervention benefited an estimated 83,970 people,8 which is an effective means for the program to make contact with the majority of village residents. The coordination with and involvement of CRS staff seems very good, and they expressed satisfaction with the partnership. They conduct follow-up visits to assess their impact, although this may be difficult to determine in isolation from other inputs to community learning. There does not seem to be any specific or planned integration between these sketches and other PASAM-TAI interventions, which is unfortunate because community interest would probably be at a maximum after seeing a drama sketch and after participating in a discussion. Radio Listening Groups: PASAM-TAI developed an awareness-raising and self-guided learning intervention using 12 carefully produced radio dramas, listened to by groups on radios with Secure Digital (SD) card capabilities, and accompanying study materials meant to encourage group discussions. In FY13, 298 radios were procured for distribution to listening clubs, and in FY14 104 facilitators were trained from 95 listening clubs. 9 During field visits, the ET encountered this activity in two of the 14 villages. In one village, the group noted that they listened to and discussed the programs they heard, but when their radio was damaged, the group discontinued. In a second village, a listening club facilitator was designated and trained, and a group met every Monday and Thursday afternoon for an hour, listening to each program twice to ensure that messages were understood. The facilitator conducted games and question-and-answer sessions, followed by further explanation as required. After two months, the group had listened to all 12 of the sketches, and since PASAM-TAI staff had told them to do what they wanted with the radio afterwards, the facilitator decided to pass the radio on to a nearby village in order for that village to benefit from the radio. Themes of the 12 programs included:  clinic consultations for general illness treatment and pre- and post-natal consultations  Aquatabs  the difference between a doctor and healer  hand-washing  water quality  birth spacing  taking children to the hospital when they are sick rather than using traditional remedies (emphasizing here the role of the husband) 8 PASAM-TAI quarterly report FY15 Q3, 21 July 2015. 9 According to FY13 and 14 annual reports. More detailed monitoring information on the radios and the listening club was not available. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 19  forced marriage and early pregnancy  giving birth at the health clinic, and  the role of fathers in family health. While both women and men expressed having gained knowledge from these sessions, the listening club facilitator discussed difficulties getting the community members to participate, particularly in the case of men. In addition, the ET found that some participants had great difficulty understanding the messages. When the listening club was formed, the head of the club received 60 tablets of Aquatabs, which they could sell (10 CFA Franc BCEAO, or XOF10) and use the proceeds to restock, but during the past year, the ET found that they had not yet liquidated their stock. Some women stated that many people do not drink water containing Aquatabs because they find the taste bitter. Aquatabs are being promoted by other organizations in Niger, and there may be benefits in comparing experiences more widely. The ET reviewed the radio programs and study guides, which seem very well made, and it is clear that these programs could stimulate quite sophisticated thinking about themes of central importance to PASAM-TAI. Some of the gender issues raised, such as agreement between a husband and wife on issues relating to health matters, are quite advanced. This activity seems to be isolated from the community mobilization activities with neighborhood care groups or the community hygiene activities. The ET found that more could be done with follow-ups, especially if some radio topics are not completely understood, as had been suggested by participants. At the time of the MTE, it is not clear if there will be a more in￾depth review of the results, which could help refine the materials or incorporate lessons-learned from the set of 12 programs. This could prove a valuable input that could easily be incorporated into other programs. While this radio program emphasized group listening and discussion, which has obvious advantages, there is potentially great benefit in sharing such materials through mass media, including dissemination through local radio stations. Currently, the programs are only shared through SD cards to the listening clubs. During FGDs, some villagers actually mentioned things they had learned from the radio about nutrition, health and agriculture, and even about Aquatabs. Recommendations (Output 1.1.1)  Reinforce training on the full meaning of the 1,000-days approach, in synergy with the GoN and other agencies promoting this and drawing on lessons learned by CRS elsewhere.  Follow up after presentation of RAV or CAN presentations with various community mobilization efforts to take advantage of the heightened interest and awareness.  Promote greater integration between the radio club and community mobilization activities.  Review the Aquatab intervention in light of this MTE. Compare with other experiences in Niger, and decide whether and how to continue.  Continue the radio club intervention, but provide clearer reporting on the results of training and provision of radios.  Make use of radio materials in community radio, which could be combined with call-in programs.  Promote more adolescent involvement in program activities such as care groups and greater integration of efforts to reach this key group, such as outreach communications and literacy. 10 At the time of the first draft report of the MTE, the CFA Franc BCEAO (XOF) equaled 0.0017 of one United States dollar. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 20 Output 1.1.2  Promoter and care groups: HHs are mobilized to promote key health, nutrition, and water, sanitation, and hygiene (WASH) practices. PASAM-TAI works with promoters, care groups, and other volunteers to convey the World Health Organization (WHO) health, nutrition, and hygiene recommendations to women, and specifically to pregnant women and mothers of infants up to two years. The aim of such work is to assure optimal growth and development of infants during the first 1,000 days from conception to two years of age. In the PASAM-TAI program document, Output 1.1.2 is centered around the care group (CG), composed of 8 to 12 lead mothers (or “mamans lumières”), who meet every two weeks to receive training and support from a CG local expert (CGLE). The lead mothers of a village are carefully selected to optimize their chances of success, and lead mothers explained to the ET that they were chosen by virtue of being mothers who are patient, do not get upset, and have an interest in serving the community. These criteria were confirmed separately by the program’s nutrition team leader and are in line with common standards and roles for care groups. To date, 5,563 lead mothers have been identified and incorporated into PASAM-TAI’s care group activity, 11 which is 71.3 percent of the life-of-activity final program goal of 7,803 lead mothers. Lead mothers are found in 634 care groups, which are spread across the 643 targeted villages. Establishing a system for ongoing training of mothers over such a large area is indeed an accomplishment. The CGLE role is played by the promoter or nutritional “relais” (referred to in the program document as Community health workers [CHW]), who receive significant training in a number of areas through monthly training from the PASAM-TAI Care Group Field Agents. These areas include the 1,000-days approach, the registration of beneficiaries for food assistance (discussed further in Output 1.1.3), and growth monitoring (discussed further in IR 1.2, Output 1.2.1). The promoter provides leadership in community awareness raising and sensitization, and they function as the main trainers for the lead mothers. Including promoters and others, 1,602 volunteers have been identified, which is 95 percent of the life-of-activity goal. Thus the program has done well to get these key individuals in place, though with only 932 being trained, it shows that there is still work to be done to complete their training.12 In almost all villages, a promoter who had received training was available. In one case, the village chief was working as a promoter, and he stated that he had not yet received training from PASAM-TAI. Yet the ET found that he was using a PASAM-TAI booklet to train lead mothers in his village, indicating that these materials seem to be quite useful as a means of support for the community volunteers. In many cases, the promoter had previously received training and worked with past projects in various capacities. Promoters receive a modest stipend from PASAM-TAI. Each lead mother is meant to work with a group of a maximum of 15 women from their neighborhood, and this group is denoted a “home CG”. The lead mothers hold bimonthly training sessions for their home CG and others on a theme determined by PASAM-TAI. In some villages, this system of training was working well and constituted a major factor explaining villagers’ high awareness of nutrition, health, and hygiene topics. In an atypical example, however, the ET examined the record of these sessions for one CG and found that almost 100 people were attending these sessions, including many children. For the PASAM-TAI methodology, 100 participants is above the prescribed limit for optimal learning,13 but it is 11 PASAM-TAI PowerPoint presentation, August 22 2015. 12 PASAM-TAI PowerPoint presentation, August 22 2015. 13 Having large meetings is different from the care group approach Also given that other NGOs are sometimes operating in the same villages, the volunteers may mix up what they do with PASAM-TAI and with another NGO. but it is noteworthy that in at least one or more communities, there was a demand for regular large sessions and they may also be effective. The ET suggests that if a activity such as this does emerge in a community which is different from the standard practice, it may be valuable for the program to review whether the non-conforming methodology points to something that might have some positive aspects to learn from, for example that other volunteers might include in their work. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 21 not clear whether this is a widespread practice. The lead mothers also visit women’s homes to sensitize or orient them and ensure they are applying what they learn. The term “sensibiliser” is used in this context to denote a process that involves sensitization, communication, or awareness raising for social behavior change. Lead mothers are given training on the monthly theme or CG lesson by volunteers or promoters, after which they transmit the BCC messages to members of their home CG. Some lead mothers are also trained as facilitators in early childhood development (ECD) and encouraged to participate in the activities of the ECD centers, although the ET found these centers to be non-operational at the time of the MTE in the sampled villages. The position taken by PASAM-TAI on monetary incentives for lead mothers is that giving such incentives would limit their sustainability, although the program provides other types of incentives such as providing appropriate materials like t-shirts and caps and ensuring adequate appreciation to volunteers by the community.14 Some lead mothers expressed to the ET that they sometimes found it difficult to maintain their work without any payment, and PASAM-TAI is well aware of this issue. The program’s approach is similar to other NGOs working in Niger and indeed in many other countries, and the ET deems this an appropriate position, especially for those who spend only a few hours a week. As for other categories of volunteers who spend more time and have greater responsibilities than lead mothers, it was reported that other organizations/programs at times provide more attractive conditions, which may have the effect of reducing volunteers’ enthusiasm for working with PASAM-TAI. These problems of differing policies on volunteer incentives are extremely common in developing countries and are often worth discussing with government and other agencies. The promoters are trained by PASAM-TAI in hygiene, the 1,000-days approach, and methods of registering pregnant women. Some of the promoters are trained in growth monitoring or acute malnutrition screening using Mid-Upper Arm Circumference (MUAC) followed by the monitoring of weight for age. Promoters also organize awareness-raising sessions for villagers in general, for example taking advantage of occasions where villagers are gathered (e.g., weddings) to convey program-related messages and raise awareness. In some villages visited, the promoters organize meetings where villagers are summoned to one place specifically for training sessions, although it was not possible for the ET to see any such sensitization taking place. As a good example of the care group approach in practice, three lead mothers in one village explained their history and work to the ET. In 2012, their promoter organized a meeting, and women of the village chose lead mothers, who then received training and began sensitizing other mothers right away using a training book with photos called a “flipchart”15 that depicted desirable and undesirable behaviors. The flipchart included topics like the importance of perinatal clinic consultations, giving birth in clinics, exclusive breastfeeding until six months of age, complementary feeding, and hand-washing. Each lead mother adopted seven households with which to conduct ongoing sensitization, for a total of about 15 women. Each Monday they went door-to-door to conduct a home visit session with the women of that household to follow the practice of the taught behaviors. Apparently, the mothers accepted the content of the sensitizations and did not have objections. In certain cases where the husbands of the women did not agree with the topic, such as birth spacing, the lead mothers attempted to sensitize the husbands as well. This low-key education approach may be important in talking through barriers people may have. In addition, meetings were held with all of the women of their adopted households together, and other women may also join such meetings. The lead mothers stated that they themselves receive trainings each month and supervision each week from the promoters or volunteers and that PASAM-TAI field agent 14 This is a complex issue which would require more space to go into adequate detail, and may raise as many questions as it answers, so the issue is mentioned here briefly without extensive details and explanations. 15 Not to be confused with the more common meaning in English of flipcharts being large papers on easels used in workshops and larger group meetings and training. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 22 visits every two weeks to meet with them. Some of the women are part of their community gardening activity, and two of the lead mothers are also part of the literacy training program and take notes when there is a group meeting. In this particular village, a very important part of the program was a meeting of all women in the village, which occurs every two months to check on whether the women are applying what they are learning and to publicly monitor the progress of each mother and her infant. Since all of the other mothers are there, everyone must be truthful about their condition and the condition of their children, and the meeting draws on public peer pressure (and to some extent, perhaps, shaming) to increase the encouragement for caring for children. In this way, the promoter, PASAM-TAI field agent, lead mothers and others (including clinic staff) are able to receive an update on the entire community and identify any infants who need assistance or any kind of intervention, including referrals to the clinic. The regular participation of the PASAM-TAI nutrition field agent was specifically mentioned in this village, and this could be a crucial factor in carrying out this activity, which was not seen in other villages. The care group approach appears to be an effective means of cascading messages and systematically engaging mothers in a learning process over time, including following-up, reinforcing learning, and working out problems as the women try to apply new behaviors. The home visits allow the lead mothers to observe conditions and promote learning in a natural setting, and there is an easy dissemination of the learning with a wider group in the community. The methodology is well standardized across PASAM￾TAI in terms of what is offered to the villages, and at least three of the 14 sampled villages represented this idealized model. Still, the way the methodology was implemented in some villages was not consistent with expectations, in that some of lead mothers interviewed by the ET did not seem as clear about their role nor were they very active, and they were not carrying out all the activities mentioned in the ideal example above. A key challenge that the ET observed was that the lead mothers did not normally receive any training from or have contact directly with PASAM-TAI staff. This could be exacerbated by the fact that in some villages, monitoring support for the promoters and lead mothers was interrupted by the resignation of the PASAM-TAI field agent. The modality of the cascade training involves the promoters and other nutrition volunteers being trained by program staff and then passing this on to the lead mothers. The relationship of lead mothers to promoters/volunteers is not always strong or conducive to consistent training, and PASAM-TAI staff could help monitor this training and do spot checks of lead mothers’ understanding and level of activity. The lead mothers are trained on a monthly basis by the promoters/volunteers, but some of them expressed a feeling of being left out of other sensitization campaigns and activities that the promoter or other volunteers are carrying out. It may be the case that some of these activities pertain to other projects operating in the same villages as PASAM-TAI, as community members may not always distinguish between the programs. Moreover, promoters have a key position in the community, including liaising with clinics. Often the promoters are males, which could be a factor in shaping a relationship with lead mothers in which the promoter naturally takes a lead and at times does not consult with lead mothers, a feature that is ironic since much of the work is centered on mothers and their issues. Certain insufficiencies were seen in lead mothers’ knowledge and clarity on their roles. The flipcharts are key aspects in the training of lead mothers and their sensitization work with mothers, and these materials appeared to be effective while promoters are using the developed CG modules. However, literacy levels are very low in Niger, and no doubt, this plays a role in hindering villagers’ understanding. Currently, some lead mothers are taking literacy training and are exposed to some materials in the local language, Hausa, although the photo books are not yet available in Hausa. Another issue noted was that at times, the lead mothers were not selected by women of the village as a whole but were nominated by other lead mothers. This could lead to a lack of accountability to the other villagers and by extension reduce the CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 23 level of commitment of those lead mothers. The program has recognized this and attempted to rebuild CGs based on friendship groups. Part of the strategy under Output 1.1.2 is to promote male participation in health care and nutrition. The model, which CRS has developed, draws on the popular Schools for Husbands approach but makes an adaptation, which CRS calls groupe d’apprentisage des hommes, or Male Learning Groups (MLGs). This will promote men’s involvement in nutrition; maternal, child, and general health; and training male volunteers to be peer leaders. The concept is very promising, and given that the care groups already are engaging husbands to support their wives, it should be likely to yield some interesting results. The ET did not encounter MLGs in the sampled villages in Mayahi district, though in some of the villages, the promoters said they work with men as well. After interviewing the PASAM-TAI Gender Coordinator, the ET sought to identify villages in Zinder to visit MLGs, but the activity had only reached the stage of staff training and had not yet begun implementation in the field. Given that the project is now approximately half completed, it is important for this exploratory initiative to advance quickly, and for the program to learn from it. One of the aspects of the care group approach, which is not very apparent in current practice, relates to the following statement in the PASAM-TAI program document: “As part of these (care group) visits, women will receive messages on family planning options as described in the National Reproductive Health Program.” Given that health education processes are already underway, it should not take much effort for the program to incorporate this and include it in program reporting. Recommendations (Output 1.1.2)  Ensure refresher training of lead mothers on their roles, the 1,000-days approach, and hygiene, including reference to other CRS projects with relevant experiences (PASAM-TAI reports that this is planned for November 2015). Reinforce understanding that if a child is malnourished during the first 1,000 days, it is very difficult to reverse these effects. Consider including use of MUAC and behavior change techniques to be taken as primary actors in PASAM-TAI’s theory of behavior change.  Review the work of lead mothers to ensure that orientation sessions do not have too many participants to be effective, and learn from distinctive practices that might have valuable aspects (e.g. larger orientation sessions).  Ensure that lead mothers are more consistently and centrally involved in awareness raising when appropriate.  Work with promoters, lead mothers, and other volunteers to ensure alignment and synergy so as to maximize the impact of these different volunteers  Provide some amount of direct support for lead mothers and monitor their knowledge and activities to ensure quality in the cascade training and support via promoters.  Make photo books/flipcharts (pagi volts) available in Hausa to support sensitization sessions.  Continue the discussion of appropriate incentives or motivation for the volunteers and lead mothers, while also discussing and coordinating with the GoN and other programs to harmonize approaches (see final section with discussion on Volunteers).  Develop a stronger linkage with clinics, integrated health centers (CSI), and district health offices in the activities of sensitization in the 1,000-days approach and hygiene.  Continue to roll out Male Learning Group activities throughout Zinder and Mayahi, with careful monitoring and discussion of the results.  Incorporate messages on family planning in care group activities. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 24 Output 1.1.3  Food commodities: Mother and Child Units (MCUs) are consuming quality Title II and locally￾made complementary foods. PASAM-TAI distributes CSB+ and oil to PLW with their infants up to the age of 24 months, in accordance with the 1,000-days approach. The benefit to the child of this long-term food supplement is incalculable during this crucial window of time, yielding long-term benefits for their development. The CSB+ and oil constitute a supplementary food, which provides a large part of the energy and micronutrient needs of the infant. The ration permits the embryonic or young infant to cope with the instability of food resources, particularly during the annual lean period. The ET found that the CSB+ and oil were quite popular among beneficiaries and even among others, with a general feedback that people appreciated the food quality. During FGDs, respondents frequently mentioned that the food (and particularly CSB+) was helping the development of children, and together with the ongoing nutrition, health, and hygiene education work, they felt that their infants were doing well. Some commented, for example, that better nutrition was resulting in healthier and livelier children. Of course, the ration was not enough for constant feeding, and some stated that the ration would last only 10 days, after they had waited two months to receive it. PASAM-TAI staff cite a challenge that many villagers utilize the ration in sub-optimal ways and against program indications, generally by sharing the food among siblings. A generally robust system was in place to register valid beneficiaries with the assistance of local, long￾term volunteers along with beneficiary cards used to crosscheck against the beneficiary list. The ET noted that the CSB+ and oil distribution is very much appreciated by the community and that it provides an effective accompaniment for a range of other education and community-development efforts, such as the hygiene and nutrition learning carried out by the CG and the promoters. PASAM-TAI manages to distribute to beneficiaries in over 900 villages and hamlets and to remote locations where other assistance program have never been able to reach. There were very few signs of overlap with other NGOs or other agencies in terms of food assistance. One example of complementarity was with Save the Children, which was working in the some of the same villages as PASAM-TAI but providing CSB+ and oil to severely and moderately acute malnourished children who has been referred to health centers after the monthly MUAC screening. In comparison, CRS is focusing on the prevention during the first 1,000 days of life. Receiving food assistance is not strictly conditional on beneficiaries’ participation in other activities like CG or literacy activities. However, the program explained that they encourage beneficiaries to take advantage of these other opportunities, such as going to the clinic for perinatal checkups and participating in neighborhood education through care groups. There was no written document available to the ET to lay out the methodology process of how this food distribution is managed, but the ET spoke with various groups in all sampled villages, along with nutrition team leaders, cluster supervisors, food commodity managers, and field agents to understand the system. Logistics Process The ET visited the CSB and oil warehouses in Maradi and observed community distribution to ensure that good standard practices were being followed. The CSB and oil are maintained in separate Maradi warehouses with generally good infrastructure. The ET noted hygienic conditions with logical and complete registries of shipments received and sent out. Boxes with damaged food received from the CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 25 United States are separated and any salvageable food is kept to one side. Security is good with 24-hour guarding and keys kept by the warehouse manager. Stacks are maintained neatly with good ventilation and–though temperatures were relatively mild during the MTE–the facilities seem capable of regulating temperature and humidity adequately. There is a good clearance of the food stacks from ceilings and walls all around with reasonable procedures for ensuring that supplies closer to expiration date are sent out first. The only problem observed was that the CSB+ workshop had some overdue maintenance issues such as loose netting material over the ventilation openings. The warehouses were full, given that the program was preparing to send out supplies in volumes determined by the registry of beneficiaries. Food commodities are sent in large trucks from Maradi to Mayahi and Kantche and are then sent out in smaller four-wheel drive trucks to distribution points located in clusters of villages with distribution supervised by field agents. CRS’ local partner CARITAS was responsible for distribution earlier in the program but was not active at the time of the MTE. The ET saw one of the distribution points, a fairly secure room in the home of a chief, where any extra supplies could be temporarily stored. Some FGD respondents mentioned that the distribution points were distant from their village, and this represented a hardship for women beneficiaries, so they requested delivery in their own villages. The distribution takes place every two months, as it was discovered that the initial effort to distribute monthly to so many locations was logistically very difficult to manage. Some FGD respondents said that the distribution could be delayed at times to about three months. To receive assistance, beneficiaries present their cards; the field agent calls the names of beneficiaries; and, with the assistance of the promoter, verifies the cards of each person against the beneficiary list. In exceptional cases when the mother does not present a card but her name is on the list, the promoter may vouch for her, and she may still receive food commodities. Beneficiary Registration and Distribution List In 2012, after PASAM-TAI had officially begun, and while waiting for the baseline survey to be done, the program undertook a comprehensive census of all inhabitants of the villages in Mayahi and Kantche departments. A complete database of the population was compiled, and this is likely to have a number of benefits for the program such as guarding against double counting of beneficiaries. During this in-depth process, it was easy to identify PLW with their infants of up to 24 months, and thus the first cohort of food beneficiaries was established. The hope was that as infants reached the age of 24 months, the database would automatically remove them from the list with the added intention that as additional women became pregnant and registered, each potential beneficiary would be registered by the volunteer community nutrition promoter who would issue them a number. Community members in the MTE village sample were clear about the selection criteria, and PLW were expecting to receive this assistance. Despite this, in all villages some potential beneficiaries were not receiving commodities. Various staff told the ET that new registrations took place, either on a monthly basis or at least on a regular basis, so that there should be a circulation of new beneficiaries coming into the program. It was also explained that potential beneficiaries may be identified through CG and health clinics, but it seems that the volunteer nutrition promoter is a key person for registering or submitting names of new beneficiaries to be considered for the program. There was a certain degree of uncertainty on this point when talking to different individuals, and it could be due to the different levels of configurations of activity from people available in each village. However, the names of potential new beneficiaries would be sent by the promoter to the nutrition team at cluster level (Mayahi and Matameye), who then officially enter these names into a distribution list, which they send to the M&E staff in Maradi to prepare a card for each woman, allowing them to receive the CSB and oil. However, the ET found a number of women in each of the sampled villages who met the criteria but who had never received a card or been placed on the distribution list, and this included many who were CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 26 registered as meeting the criteria well before the November 2014 cutoff period (after which new beneficiaries were not admitted). In fact, some village members said that only the initial cohort had been registered, and no others had made it onto the list. The numbers involved ranged from approximately 10 to much larger—in one village, the ET was told that about 100 women had been registered following the first cohort, but only three of these had received tickets. The ET was not able to find a clear reason for why this might have happened; some promoters suggested that those women who were registered after the first cohort were few in number and somehow it was not worthwhile to include them. The PASAM￾TAI nutrition team leader felt that names sent from the villages were somehow not being entered into the database, from which codes and cards would be issued, and names would be entered onto official lists of beneficiaries. The nutrition supervisor in Matameye explained that issuing of beneficiary cards was done manually and not automated. The process was not, for example, linked to the community census (because new pregnancies would not be reflected in the census done at the beginning of the program). This process of identifying all new beneficiaries in all villages could potentially take a very long time. In addition, since CARITAS was responsible for distribution at certain points in PASAM-TAI, it is unclear if they played a role; however, this seems unlikely unless they were actually formulating or adjusting the distribution list. Again, the MTE carried out its work principally in 14 randomly selected villages, through a qualitative exercise not considered statistically representative of all villages. The ET deems the risk as quite high, however, that numerous other villages will display similar problems (see methodology discussion). This non-inclusion of valid beneficiaries was perhaps the biggest single source of dissatisfaction with PASAM-TAI heard during the MTE, and because it was an issue raised beginning with the first village visited, the ET took considerable care to verify that the problem was in fact similar in every village. The villagers who reported this included women, men, some of whom were themselves potential beneficiaries but many others were not, and they certainly did not appear to have come to the interviews with an agenda to push this particular point but rather mentioned it among other comments, which they provided. Unless there is some explanation that the ET was not privy to, this represents a significant variation from the expected procedure of having all women and infants within the 1,000-days window registered for supplemental food rations, at least during the first few years of PASAM-TAI until the new-registrant cutoff of November 2014. This is regrettable for those women and infants who have not been able to benefit from the supplemental food that was intended for their benefit. Moreover, in villages where this type of non-inclusion of MCUs had occurred, it could reduce the nutritional impact of PASAM-TAI on the target villages. It also raises the question of whether and how the actual verifiable beneficiaries are cross-checked against the distribution list and food inventories, and if post-distribution surveys are occasionally done. A field agent also stated that he did not know why these women did not receive tickets after having been registered. Presumably, the community members would have raised their concern with the field agent, who should have explained or else obtained information to be able to answer their concern. This in itself suggests that there is not a clear process in place in PASAM-TAI for villagers to raise issues, have discussions, and obtain clarification of the rules. A related issue is that of graduation or removal of two-year old infants from the beneficiary list. One explanation provided to the ET was that given that all beneficiaries (and villagers) are maintained in the database, including infants, it would be an automated process for the database to mark the names of those who reach the age of 24 months so that they would be removed from the distribution list. In practice what happens is that the age of infant beneficiaries is regularly asked during distribution, so removal of infants from the list is actually based on what mothers declare as the child’s age, and a mother may state that the age of their infant is younger than their actual age to continue to benefit from the ration. Thus, in some villages it was reported that some beneficiaries were still receiving food after the age of two years. For example, in one village, the promoter reportedly sent a list of those who had reached 24 months but CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 27 continued to receive a ticket and food supplies. Meanwhile some new registrations had been done, which resulted in more beneficiaries currently than at the beginning of the program. In other villages, many had been removed from the list (e.g., from the original cohort in one village, 80 names had been removed), but new ones were not added. It was not possible during the MTE to quantify whether the trend should have been an overall reduction or not, nor was it possible to compare this against the volumes of food being shipped. However, this type of analysis will be necessary. This raises a final issue about the exit strategy: how PASAM-TAI is preparing beneficiaries for when they will stop receiving food rations. Although the program makes it clear that the ration is only available until the child is aged 24 months, it would be advisable to remind participants and encourage them to take part in other livelihood activities and prepare themselves for graduation. Food Delivery The ration that beneficiaries receive for the two-month period is 10 kg of Corn-Soya Blend Plus (CSB+) and 1.6 L of vegetable oil enriched with vitamin A (the monthly ration is 5 kg of CSB+ etc.). In practice, this is given to a group of five women in the form of two sacks of 25 kg of CSB+ and two 4 L cans of oil, and the women divide the CSB+ and oil near the distribution center. The ET was told by field staff that the nutrition team sometimes presents culinary demonstrations during distribution, which some FGD participants also mentioned. The conditions for sharing the CSB+ and oil are not optimal, as there is a lack of hygiene during the process of opening the sacks and oil and transferring them to other containers. The women do not take the time to wash their hands before dividing the supplies amongst themselves, and cooking would not necessarily remove all possible contaminants. The ET actually observed some children eating these foods without cooking them, which is a concern in itself and highlights the importance of hygiene. In addition, the containers used are not necessarily clean, and PASAM-TAI staff had also noticed this. The program arranged to obtain clean containers, and after having received some, these clean containers have been shared with beneficiaries in some distribution sites. In certain cases recounted to the ET, women have received poor quality CSB+ infested with insects. They were able to verify this because they generally sift the product before consuming it. In addition, on one occasion in one distribution center, a number of sacks of CSB+ were unfit for use due to rain damage incurred in transit. After this, the program reports that it was careful to ensure adequate rain proofing, and given that it was only one distribution center where the problem was seen, the ET deems that there is not a high risk of damage to food commodities in this way, although it should be followed up on. Recommendations (Output 1.1.3)  Design a protocol for identifying and registering beneficiaries as well as for monitoring and spot￾checking the distribution process.  Investigate the ET’s finding that women who may meet the conditions as beneficiaries and who have been registered at the village have not received food rations, comparing numbers of shipped rations with actual village distribution lists. This could be supplemented with verification interviews with samples from those lists.  Investigate the problems in the system, which have led to beneficiaries continuing to receive rations after infants have reached 24 months of age.  Ensure that measures have been taken to maintain the safe transport of commodities. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 28  Ensure the universal application of measures currently being implemented to promote hygienic conditions at village-level distribution points by providing appropriate smaller containers for dividing CSB and oil among beneficiaries.  Create a mechanism to remind participants when they are nearing the end of their period of receiving rations, and encourage them to take part in other livelihood activities Output 1.1.4  Primary school teachers have been trained in improved teaching methods. …and Output 1.1.5  Primary schools have improved sanitary facilities. It was not possible for the MTE to cover these two related components, as schools were out of session, and most of the people involved were not accessible, despite efforts to arrange this with cluster supervisors. A school director in one village mentioned that they were slated to do this activity, but it had not yet started. This output appears to be more of a government service than a community-based awareness raising process, and might be better placed under IR 1.2 (see below). One observation is that it can be difficult to really engage and get the attention of a government service unless a project is fully involved and really brings something to bear. Output 1.1.6  CLTS: HHs have improved sanitation systems. Hygiene is an important element in the fight against malnutrition, particularly in limiting infections that cause diseases. In addition to hand washing at critical times, the child's living environment should be adequately clean, and therefore the method of CLTS is appropriate to ensure an adequate environment and hygienic practices. A number of villages highlighted CLTS as part of the initial entry of PASAM-TAI into the community, which seems a very good means for identifying leaders based on actual work carried out based on community self-help. CLTS has advanced quickly and many villages have been certified as open defecation free (ODF). The ET met with CLTS committees in almost all villages sampled; only two villages in Maradi did not have CLTS activities, and in one village in Kantche, CLTS was done by another NGO. CLTS is triggered or started in villages that are placed on a list to receive state water services and which PASAM-TAI may potentially prioritize for a water supply intervention. The program begins by gathering background field data. Afterwards, a three-hour inception ceremony is held to initiate awareness on hygiene and methods for implementing ODF areas. Communities then make their action plan and appoint two people, one male and one female, as natural leader as part of a CLTS committee along with the village chief, the community prayer leader (or the Iman), and masons. The committee members receive a two-day training before proceeding to sensitize and mobilize the community, with supervision by PASAM-TAI field agents. As explained by CLTS participants during MTE fieldwork, the CLTS committee sensitizes the community on hand-washing at critical times. This was one of the messages that was repeated by almost CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 29 everyone during FGDs. Villagers were able to provide reasons for the importance of hygiene and hand￾washing with soap at critical times, such as hand-washing before meals, hand-washing after coming in from the fields, or after defecating or urinating. In terms of whether this is translating to the actual practice of hand-washing,16 the ET could not verify this in a systematic way. In one village, the ET asked to see the soap being used, and the respondents did not bring it, but this would need a more focused monitoring before drawing conclusions. Moreover, PASAM-TAI advises the use of ash in case the household cannot afford soap. In any case, in terms of increased knowledge of villagers, it is evident that the combination of CLTS, care groups, and more general awareness-raising is yielding results. One of the key activities of CLTS is the promotion of hand-washing facilities, using the tippy-tap model for hand-washing in the absence of taps. A recent evaluation of USAID RISE partners did find these functioning effectively in some PASAM-TAI villages, and project information shows that the strategy is advancing quickly. In the villages visited during this MTE, however, none of these were present and functioning. In some villages, PASAM-TAI had left some materials but had not trained community members on how to install and use them. In a few other villages, respondents said they were promised assistance but had not received it. The ET deems that the technology seems very simple, and that it should not be complicated to train and help residents to implement it. The ET found a similar experience with latrines among the villages, though these are more expensive and demanding to build. Most villages report having at least one or two latrines, while some report as many as many as eight or ten latrines. One village had well-made latrines; in another the latrines were found to be collapsing. Additionally, traditional latrines did not appear to be resistant to repeated rains. The principal focus of PASAM-TAI is to promote the construction of latrines and their effective use and to train masons to build latrines. In several villages, community members have dug pits and await the delivery of cement slabs by PASAM-TAI, even though it seemed fairly clear that PASAM-TAI does not finance the construction of latrines. Knowledge of latrine construction by masons was not universal, and it was not clear how active they were in the process. A prominent activity is the community cleaning campaigns, which are done in many of the villages, usually on a weekly basis. Each household sweeps outside their own door where the women generally sweep, and men pick up trash using wheelbarrows, a practice which turns out to be a very effective mobilization of practically the entire community. Unifying the village around an issue like this can be invaluable for other purposes, and it is a great way for the VDC and other village institutions to consolidate their capacities. CLTS is a fairly recent innovation in which the GoN Ministry of Hydraulics and the Environment and the Planning Commission have played central roles in implementing with several other partners. It was confirmed during interviews that they are involved in the design and implementation of CLTS activities and are broadly satisfied with them. One of the comments to the ET was that the program should limit the number of certified villages, as it is difficult to work with so many villages. For certification, the certification committee checks that each house has a latrine that they use and that street sweeping occurs. The fact that so many villages under PASAM-TAI have been certified as ODF is positive, but there is a much higher number where CLTS has been triggered. Reports were heard also that after certification, some villages lapsed in some of the qualifying criteria. It may be worth reflection on whether there is sufficient value of maintaining a wide CLTS geographic focus such that some villages are triggered without certification or some do not sustain all of the certification conditions. An alternative would be for the program to focus on a smaller number of villages. In either scenario (or combination thereof), the 16 See indicator CI4.13, which presumably will be included in the annual data gathering exercise. It may be foreseen that the indicator will be measured mainly in terms of use of tippy-tap facilities, but the promotion of the use of soap should not be limited to that. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 30 active engagement of government partners is essential. The program had done a review of the CLTS strategy with other stakeholders, which was found to be a very laudable effort at coordination. With regard to drinking water, there is a wide disparity in the quality of water in the villages—some have treated tap water while other villages mainly use stagnant pond water. In some villages, the partner NGO ANIMA SUTURA has been promoting and selling Aquatab tablets for purifying drinking water. In other villages, the communities reported having used Aquatabs to purify water, although such use is generally limited to the free distribution period. Very few villagers pay for Aquatabs. Some say they do not enjoy the taste of purified water after it has been treated with Aquatabs, while others say they feel better about using treated water. Some of the villages mention using a sieve filter method, which seems to have been well accepted, and one village denoted the Goulbi NGO as having promoted this method. The MTE did not have time to explore further the reasons why villagers use this method and have not yet adopted Aquatabs on a sustainable basis; it may simply be a matter of cost. If sieves were homemade or distributed for free, it is understandable that villagers would use them. A cloth filter can also be better than not using any filtration method, for example in removing a vector for guinea worm,17 and if people at least develop the habit of using the filter, it is a step towards using another treatment. In some villages, there are other NGOs performing the CLTS activities, even when PASAM-TAI triggered CLTS in these villages. Therefore, there are cases of overlap in the villages in question. Nonetheless, it seems that the program has reached an agreement with government stakeholders and with other NGOs to avoid having other NGOs function in areas once PASAM-TAI has begun their work. One of the challenging issues, which they have also been working on, is the best approach regarding the provision of latrine platforms—while some NGOs tend to give them away, PASAM-TAI is one of the organizations that does not offer this free service. Another issue encountered by the ET is that field agents seem to be overwhelmed with work, and there were reports of field agents frequently resigning. Indeed, given the number of villages where CLTS was triggered by PASAM-TAI and the requirement in terms of monitoring CLTS activities, agents find themselves with a number of villages to follow, making it difficult to carry out the certification process. After triggering, a four-month period is generally sufficient to achieve certification; however, the ET found that the villages triggered by PASAM-TAI might take twice as long to become certified due to a lack of monitoring. Recommendations  Improve communication on what, if any, assistance PASAM-TAI will provide for latrines, and emphasize self-help construction with the aid of masons.  Review and confirm if masons are in a position to provide technical/building services.  Ensure timely follow-up by field agents in the triggered villages to complete the process and ensure adequate construction of latrines.  Provide training on the construction and supply of materials necessary for the tippy-tap hand￾washing model, and ensure they are sustainable in the long term.  Review human resources policy to reduce staff turnover.  Investigate the sieves used for filtering water; discuss the program’s position on sieves, and see if sieves can be a bridge towards alternative methods such as Aquatabs usage. 17 Muller, R. 1979. Guinea worm disease: epidemiology, control, and treatment. Bulletin of World Health Organization. 57(5): 684-689. Accessed on 7 October, 2015 at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2395878/ CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 31 IR 1.2 Services for MCUs have accessed quality community- and facility-based health, WASH, and nutrition services. Output 1.2.1  Health providers have incorporated the 1,000-days approach into existing services. According to the program document, PASAM-TAI has a number of activities with the Ministry of Health, which provide training to skilled healthcare providers on the 1,000-days approach, training on the Integrated Management of Childhood Illnesses, and on ensuring that health centers have access to iron supplements. The latest annual report refers to training done for volunteers only, so it is not clear if the program intends to carry out the originally planned activities. Although these were not themes that would come up during the fieldwork, the ET held interviews with key health officials from the commune to the regional level. The chief physician for the Maradi region indicated that the relationship with PASAM-TAI was not very satisfactory, noting that his PASAM-TAI counterpart was always busy and communication was not adequate. He said that they need to have more contact with the districts, and that they had wanted to include CRS activities in their action plan in 2013 and 2014, but PASAM-TAI did not communicate or cooperate to do so. PASAM-TAI staff mention that they are aware of the need to improve collaboration with the departmental health departments and have recently done so. One of the challenges they face is that at times there are scheduling conflicts with other programs (e.g., multiple NGO activities taking place). The ET also held interviews with health directors of CSI and a health home, and they confirmed that they were aware of the good work being done by PASAM-TAI but did not themselves have any relationship with PASAM-TAI. PASAM-TAI state that it is difficult to establish more satisfactory relations with the CSIs as long as the question of resource transfer to them is not addressed. In the interest of focusing the MTE on the key areas among the many aspects of the program, the ET did not pursue this issue of partnership with the health department further, although the ET advises that the program attend to this. There could conceivably be areas of PASAM-TAI in which government services could be playing a stronger support role, such as with promoters and lead mothers. For example, in the village that had the best demonstration of the potential of these two groups of volunteers, clinic staff participated in the bi￾monthly meeting with mothers, which encouraged the meetings to take place and helped create an immediate linkage with other health services needed. In another village, the nurse was a tremendously capable and willing resource person who helped facilitate the work of the ET, but her knowledge of and engagement with PASAM-TAI was limited. Growth Monitoring and Promotion (GMP) As explained to the ET, promoters are trained in taking MUAC measurements and also in determining the nutritional state of infants in terms of the weight-for-age index. As for the latter, they take the weight (measured with a scale) and age of the child, and determine if the child falls within the ‘yellow’/moderate or ‘red’/severe underweight status. The MUAC measurement allows screening for acute malnutrition so that malnourished children can be referred to the nearest nutritional rehabilitation centers. Almost all children (particularly those up to five years old) in the village have monthly anthropometric measurements taken. The promoter also records in a notebook the number of children screened each month as well as the number of children who are moderately to severely malnourished. These data are CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 32 taken by the nutrition field agent for inclusion in the database. This constant screening allows malnourished children to be referred early enough for optimal nutritional recovery. This was the system explained to the ET during numerous interviews with promoters, although unfortunately, it was not possible to attend a regular session where children are weighed or screened. Also, it should be noted that not all sampled villages followed these procedures; it depended on whether the program had provided a scale or whether the promoter was able to borrow one from another NGO. At the entrance to certain villages, the ET found giant score boards mounted on cement foundations that provide information on the nutritional status of children (in terms of underweight status), which stated the percentage of mothers practicing exclusive breastfeeding. This is a good initiative to permit the sharing of information that is easy to understand among a wider group of individuals. This likely increases both the sense of community-shared responsibility and perhaps also peer pressure for young children to be optimally cared for. In this way, it also uses a similar approach of public monitoring as the large village meetings mentioned under Output 1.1.2. The challenge that PASAM-TAI faces with this system is that according to observations made by the ET, growth monitoring is not practiced in this way by all promoters, as observed in demonstrations given in some of the sampled villages. The methodology for taking MUAC is often not adequate, in that some promoters do not take sufficient care to find the correct spot in the arm halfway up in which to measure the MUAC. Some promoters do not have a complete mastery of the forms to fill out, tending to enter data in incorrect columns. This was observed in a number of villages, but often there is a high population in these villages, and the promoters’ sloppy measurement habits might have evolved partly due to the pressure they regularly face in handling a high number of patients. Some villages also do not have the necessary materials for weighing. Recommendations (Outcome 1.2.1)  Reinforce training for MUAC measurements, and ensure that promoters pass through testing mechanisms to ensure that they meet the necessary standards.  Consider including lead mothers in training for anthropometric measurements, on a pilot basis, for those who wish to do so. Output 1.2.2  HHs have improved water supply. The ET reviewed the approach to water supply, and finds that it is broadly being elaborated in an effective way and in accordance with the program plan, but there are a few issues around institution building and sustainability, which are discussed in this section. As mentioned elsewhere, PASAM-TAI is addressing one of the priority needs of its target communities by developing or improving water supplies. During FGDs, it was common for respondents to rank this very high on their list of priorities in areas in which they request assistance. The linkage with Output 1.1.6 is crucial, as water development should go hand in hand with sanitation and hygiene education. There is a strong engagement with the department of hydraulic engineering, in which PASAM-TAI is aiming to construct water networks or components in villages prioritized by the hydraulic department. The ET acknowledges that water systems are expensive, so PASAM-TAI would necessarily have to prioritize which villages receive its investment, and this prioritization is done in consultation with the hydraulic department. The PASAM-TAI program document states that only about 3-4 percent of actual physical water needs of the region will be addressed. Therefore, it is crucial that the program see its water supply work as a pilot demonstration and capacity￾building exercise for the benefit of the entire region (and indeed country), rather than a simple technical exercise to install pumps and taps. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 33 The ET observed a large pump house being installed in one village, which was connected to a water reticulation with six water points. Given that the MTE did not include engineers or technicians, nor were technical issues part of the TOR, these were not looked at in detail. A number of positive measures had been taken to safeguard this project such as hiring a manager and private technicians before the water started flowing, and a water user association (WAU) was overseeing the system. The association chairman stated that there was a financial and operating management plan in place, but it was not possible to obtain further details, and there were no written guidelines or documents covering these details provided to the ET. The issue of financial and operational management should be emphasized at this stage in the program to safeguard the investments being made and plan how funds are going to be managed. Some programs accept community-generated plans, while others have a standardized plan. Ideally PASAM-TAI could use the best of both approaches. In this particular village, the VDC chairman expressed a lack of understanding as to the role that the VDC should play in development in general, and with water supply in particular, so there is an issue in this case about how villages are going to manage and oversee these water supply systems. The types of challenges entailed in water supply systems were exemplified in a number of villages, where the community was collecting funds to co-invest in water and yet facing potential problems. For instance, one village had collected and paid the mayor’s office 150,000 XOF to obtain a pump, which at the time of the MTE had not yet been built. In another village, the community started raising funds to prepare to co￾invest, at the urging of the hydraulic department saying that PASAM-TAI would soon be coming to build, stating that the village’s contribution would be necessary. In this case, the community selected several individuals to collect and bank the funds for the rest of the community, where over one million XOF had been deposited in a bank account without clear oversight and no plan as to when the funds would be used. These are situations, which PASAM-TAI was not contractually responsible for, yet in which the program could provide some advice to help them fulfill their objectives, which would ultimately help meet the program’s objectives as well. If the program wishes to support overall capacity strengthening in the target villages, build on the numerous previous and concurrent projects, and provide a demonstration for broader learning, the program should take some interest in how the community manages these projects and bring them to the attention of the hydraulic department. By working with government to shore up financial management of village water supplies, PASAM-TAI can be reinforced. At the same time, it can leverage the impact of other funding and help add missing components to government programs. Specifically, in this type of situation, to safeguard contributions and maintain the trust of the villagers, the ET advises that a strong local institution is needed, with the VDC being the organization with the overall responsibility for community development and could lead, follow up, and ensure that the project is moving forward. These observations brought to light some key questions about local institutional capacity building and the sustainability of the water supply, which bear mentioning partly because of the potential importance of water supply for overall institution building in the villages. Water supply is almost always a prime project for building community institutional capacities for numerous reasons: (1) the whole community tends to support it, (2) it has a blanket impact on everyone, (3) it involves many resources (assets and cash flow) which both the community and outside agencies have invested, (4) it requires some institutional maturity of local organizations, which are managing the system, and (5) it quickly builds the esteem in which the community will hold those organizations.18 Water supply is also critically important to women and should create opportunities for their greater participation in community affairs, although the ET did not see much evidence of that in the few examples of WAUs. It is common for water supply projects to set up separate local institutions just for water, without a well thought-out plan of how those institutions will integrate 18 The management of water supply projects requires institutional maturity and tends to command the respect of the community. The ET notes that it is good practice to build water supplies where there is strong community motivation and organization to ensure that there is a strong foundation for sustainability. Subsequently, it could then develop the project in such a way that it builds and consolidates institutional capacity. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 34 with and reinforce other local institutions. The program is now working with villages to develop local institutions to manage the water systems, and the ET considers it as important that the optimal balance is reached in doing so. At the time of the MTE, PASAM-TAI is tending to prioritize investing in water supplies where the CLTS process is well advanced. This makes sense, and it would be particularly useful if—within this process—water supply development is explicitly made conditional on having a VDC and possibly a WAU, which meet some indicators of institutional maturity. For example, priority should be given to villages that have active VDC/WAUs in place; have implemented other, smaller projects; and have raised funds and met criteria for safeguarding those funds. In this way, the program could reward the efforts of communities which are more motivated (which seems to already be part of the plan with the CLTS) and ensure effectiveness of the water system. Additionally, an important question is the type of structure that will be used to ensure village engagement and oversight of water supplies. The PASAM-TAI program document refers in general to water committees, and the ET heard references to Water User Associations (AUE, Associations des Usagers de Service Publique d’Eau) and Water Point Management Committee (CPGE, Comite Gestion de Points d’Eau). These structures seem to have been recognized by the government in the Water Code, though their legal status may not yet be resolved. 19 At the same time, there are gaps and inconsistencies between the Water Code and the Rural Code, 20 and the program should carefully consider how to orient its efforts relative to emerging national policy. The ET questioned what the role of the VDC is in this scenario, if in fact these water committees are separate from the VDC, since a large water reticulation is often the biggest development project in any village. Without a clear and leading role for VDCs, at this stage in their development, it is unlikely that the VDC can be considered the overarching institution or any kind of high-level local authority. It is quite possible that the policy environment allows for flexibility in determining how citizen participation is institutionalized in each village, and what the name or particular configuration will be. If there are typical or standardized functions and guidelines for a WAU (and if they are adequate), the same could be applied to the VDC itself, and then one or more sub-committees could be responsible for maintenance, hygiene, and other tasks. If there are weaknesses that some may have identified in having VDCs manage water systems, the same could probably be said for a WAU, and the need would be to deal with the specific concerns. The VDC ultimately will be the body to spearhead development overall, and it seems short-sighted to set up separate water committees which will ultimately overshadow the VDCs. At the very least, if the main citizen representation is through a WAU, there is need to consider a specific form of coordination with the VDC. Recommendations (Output 1.2.2)  Consider making explicit criteria regarding where investments in water systems will be prioritized, according to local initiative and capacity.  Consider how to involve designating VDCs in water supply, given that they are the central entity for overall development, if necessary with some form of shared responsibility with a water sub￾committee.  Take an active interest in advising and building capacity on existing water projects or new plans in order to pave the way for the work performed by PASAM-TAI in water and in other areas, and help draw out the demonstration value of the program’s pilot interventions.  Review financial management and operating and maintenance plans; ensure that all stakeholders (particularly government) participate; and ensure that adequate funds and measures are in place to replace minor and major assets of the system. 19 Water Supply and Sanitation in Niger: Turning Finance into Services for 2015 and Beyond. 2011. Water and Sanitation Program. https://wsp.org/sites/wsp.org/files/publications/CSO-Niger.pdf. Accessed 3 December 2015. 20 Cotula, L. (2006) Land and Water Rights in the Sahel: Tenure Challenges of Improving Access to Water for Agriculture. 2006. London: IIED. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 35 Output 1.2.3  Early Childhood Development centers are operational. The 2014 annual report stated that almost 200 ECD centers were in place, but the program clarifies that by December 2014 the centers were created with children registered. The July quarterly reports indicate that work is at a more beginning stage with the completion of a curriculum guide (used to train a number of PASAM-TAI staff and government partners) after which they will train ECD facilitators. More recent information provided to the ET indicates that 201 centers are now operational after the training. The ET team was unfortunately not able to see any ECD activities at the time of the MTE, nor was it able to identify other sites outside of the sampled villages. There is some link with CGs, and while there is significant potential for relieving mothers’ workloads and for stimulating the development of infants, this activity does not seem very advanced at this mid-term point. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 36 STRATEGIC OBJECTIVE 2 Agriculture: By 2011, vulnerable households in targeted districts have diversified and/or increased their agricultural livelihoods in a sustainable manner. PASAM-TAI’s work in SO2 promotes a wide range of activities from horticulture to staple crop enhancement, ruminant livestock production, household savings, and environmental sustainability. The main intermediaries with the community are PASAM-TAI field agents, ICRISAT, GoN Ministries of Agriculture and of Livestock, and the SILC local field agent. Many of the strategies included were proven in previous projects of CRS and of USAID and its partners, including ICRISAT. Together they cover many of the issues of sustainable smallholder agricultural development in this band of predominantly rain-fed agriculture in southern Niger with areas of potential for irrigation. Implementing Partners The relationship with the GoN Agriculture Services is central to SO2. The director of this service in Mayahi opined that the collaboration with PASAM-TAI was excellent and that the program provides considerable accountability for all activities. There has been good communication with the service throughout the process, which has helped develop the Participatory Variety Selection (PVS) of staple crops and the monitoring of experimental plots with ICRISAT. They have also had some involvement in monitoring the irrigation development. At the same time they feel that accountability to and engagement with the Agriculture Services could be improved, for example by holding trimestral meetings that are more review-focused rather than plan-focused and by including frank discussion of the issues being faced. The Livestock Services work closely with PASAM-TAI in organizing markets for sheep and goats and in ensuring animal health. The livestock officer interviewed also expressed satisfaction with the partnership and the impact the program was having. They felt, however, that there could be better collegial exchange and monitoring in accordance with the program’s terms of reference. The Land Commissions (Commissions Foncières, COFO) also appreciated the support that PASAM-TAI had lent to the process of promoting the rural code, strengthening local institutional capacities, and establishing the community gardens and BDL sites with agreements over land use. Interestingly, COFO mentioned that they preferred a fluid relationship with PASAM-TAI with informal drop-in meetings and requests for assistance, rather than a more formal arrangement of making appointments, an arrangement which has become the norm recently. ICRISAT is a key partner for PASAM-TAI, in terms of technology development, training, and monitoring. A senior scientist associated with the program expressed broad satisfaction with the program and how the various ICRISAT components had evolved. These changes included the piloting of improved seed varieties, field school trials of varieties and soil fertility for farmers, and the BDL. They felt that the engagement and periodic review meetings were helpful, though more involvement at the design phase could have been beneficial. One issue mentioned to the ET was the timing of seed provision in relation to the rainy season, a provision that has been somewhat late in the past. There is also a challenge in coordinating ICRISAT’s inputs when field agents tend to be busy with other activities. PASAM-TAI coordinates with L’Institut Nationale de Recherche Agronomique du Niger (INRAN) as well, but the MTE did not allow time to meet with them. Community Associations that liaise with PASAM-TAI are numerous, as SO2activities appeal to a range of individuals and groups in participating villages, from VDCs to CGs and other groups more focused on livelihoods. When the ET asked PASAM-TAI staff which community groups would be able to explain SO2 activities, the key partners mentioned were farmer associations (organisations paysannes). Staff CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 37 mentioned that there were 315 farmer associations working with PASAM-TAI in product transformation, gardening, and savings. It was also mentioned that the program worked with existing institutions whenever they were present. In each sampled village, the ET asked which activities and community groups were associated with PASAM-TAI. While there were no farmer associations encountered in those villages, one farmer association was found in a neighboring village. SO2 groups tend to be beneficiaries of interventions that are quite specific, such as BDL, gardening committees, or SILC savings associations. In addition, some of these could be construed as farmer associations even though participants do not identify them as such and the functioning of each group is quite distinct. Unlike in SO1, where the CGs and promoters often have fairly wide-ranging knowledge of program activities, there is usually not a natural oversight group for the various activities of SO2, except perhaps for VDCs, which have an oversight of all activities and not just SO2. The ET interviewed a farmer association; a discussion of this particular farmer association is useful given that it has had broad impact in its village and provides an example of the potential of farmer associations. This association, formed by its own initiative four years ago without the assistance of any other organization, has recently begun to receive assistance from PASAM-TAI (e.g., seeds and a fertility management test plot). During the time of the MTE, the organization comprised 28 members and elected officers. The association met weekly, during which time they contributed 25 XOF. Funds have been used to buy fertilizer, lend to members for sheep fattening, and purchase a hectare of land on which members cultivate millet, sorghum, and cowpeas. During their first year, members had produced 100 kg of millet for 18,000 XOF, 48 kg of cowpeas for 12,800 XOF, and 15 kg of sorghum for 7,500 XOF, with proceeds shared among members. At the time of the MTE, the funds amounted to 315,000 XOF, and the association’s strategy was to revolve this fund as loans, rather than try to maintain it separately. One of the important roles is to ensure that members can access seeds each rainy season and in some cases are able to access funds or a loan for fertilizer. IR 2.1 Diversification: Households have increased and diversified the production of more nutritious foods for consumption and for income. PASAM-TAI promotes horticulture methods and the cultivation of climate-appropriate plants, as well as livestock rearing, recognizing the current importance of these activities in the region. The program also sees the need to better develop these activities to help villagers prepare for an uncertain future of climate change compounded with the pressure of a growing population in an already-stressed environment. The horticulture and small livestock activities under IR 2.1 are mostly opportunities that benefit women but they also entail investments of time that women must make, which may or may not have payoffs for them and their families. At times, children, the elderly, and the disabled can also take part in these activities, and this is an aspect, which merits further exploration. Output 2.1.1  Horticulture: HHs have increased horticulture production for consumption and for income Homestead Keyhole gardens: The Homestead Keyhole garden technique encourages households to allocate space to produce food throughout the year with a focus on vegetables for household consumption. PASAM-TAI’s intervention is intended to work through volunteers and form an interest group. The ET obtained information from some FGD participants who mentioned that nutrition promoters provided training. It was also mentioned that field agents also invested significant time working directly with this intervention. An effective demonstration site was observed in one village with a garden bed elevated 1.2 m above the ground with a CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 38 growing surface of approximately 6 m², which was built by a local mason trained by field agents. According to the standard keyhole design, it had a central well for adding water (including dishwater) and distributing nutrients; layers of ash to limit insect attacks; sand and compost; and a plastic film over the planting area to manage moisture. ICRISAT was providing training to the field agents about the keyhole garden technique, the selection of rapid-yielding plants, and the use of locally available materials for fertilizing and for pest control. Field agents then shared that information with villagers. This village had chosen a mix of okra, tomatoes, and carrots for the demonstration. In another village visited outside of the 14 sampled villages, the ET observed a demonstration keyhole garden developed by one beneficiary and directed by the field agents, which included the participation of other villagers. The plants in this garden included moringa, tomato, lettuce, okra, carrots, and cabbage. The moringa had already produced four times and did so every two weeks. The ET found that the produce was being used for household consumption; marketing was not foreseen for the immediate future due to the limited amounts involved. The field agent visited the village every two weeks to supervise the project. The beneficiaries were pleased with the results and recommended further training and more seed varieties. There is considerable potential with this technique, but it remains to be verified if households would go to the expense and trouble to adopt it themselves. Other villages had mixed experiences with household gardening. For example, one village had begun a simple form of backyard gardening without a raised bed and received seeds from PASAM-TAI but did not receive training or accompaniment. The women beneficiaries felt they did not have the capacity to make it work on their own, and eventually, the plants ended up dying. Several villages reported that they had received training by the nutrition promoter or by others, but they did not pursue the activity because of lack of water. A third village had a demonstration garden in one household that had been chosen because it had a water tap, but it turned out that a rat ate all the plants. A final village had a test garden in one home, but according to community members, the results were not conclusive, and it was not replicated. Keyhole gardens are a promising technique, and there was interest in the idea of household gardening in the sample of villages visited by the ET. However, despite the fact that the program is at the halfway point, household gardening seems to be at an initial stage without a sense that it can easily continue and expand without significant ongoing support from PASAM-TAI. There were no cases observed of household gardening spreading beyond the demonstration stage. The costs may vary from simply planting vegetables in the ground to built-up keyhole structures, but it seems much work is needed to inspire interest in the communities. Otherwise, even the modest costs of seeds may limit the sustainability of this intervention. Irrigated Horticulture Plots: Irrigated community gardens are possible in certain regions that have groundwater high enough for feasible pumping. For PASAM-TAI, this intervention was focused in the Zinder region. Nonetheless, one village in the Mayahi region had an irrigated gardening activity, which pre-dated PASAM-TAI and utilized drawn water from a well using an ox provided by the United Nations International Children's Ecumenical Fund (UNICEF). The women who participated appreciated the seeds provided by PASAM￾TAI. The program offers these through seed fairs and provides options to participants. These participants mentioned in particular that they had used seeds for carrots, lettuce, cabbage, onion, and tomatoes (although in the current year, the seeds arrived late). The products are consumed over a three-month period with participants drying and conserving some of the products. The carrots did not produce a large quantity, and the ET noted that people did not like them very much. On the other hand, tomatoes and cabbage were appreciated by villagers, including children, which were used for household consumption only and not for selling. The group members clearly saw the benefits and said that the activity was open CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 39 to everyone in the village, noting that women who did not participate were lazy. While it is encouraging that they are enthusiastic, the attitudes of early-adopters can at times be a barrier to others. For this reason, it is advisable to include discussion of the dynamics of extension and innovation diffusion and to help villagers learn the most effective ways to share their learning and encourage others to participate. The ET visited another irrigated horticulture garden in Zinder established prior to the program. The arrival of PASAM-TAI saw the formation of a committee with 12 women and 8 men. PASAM-TAI developed an irrigation system, which included 18 wells and provided three days training in plant spacing methods. While in season, participants planted carrots, tomatoes, corn, potatoes, onion, cabbage, zucchini, lettuce, peppers, Sahel apples, watermelon, moringa, and sugar cane. During the ET’s visit, participants were also growing maize and millet. The management of this gardening activity is very important, and linking it with SILC accountability systems would be a priority to ensure sustainability. The director of Agriculture Services stated that they see rain-fed cultivation as having inherent limitations, and thus encourage irrigation and market gardening, noting with approval the use of the California system, which is simple, minimizes losses and is appropriate for small-scale applications. He did have comments on challenges they see with the PASAM-TAI approach. For example, the Director said that preparatory studies were not done, that the program had selected sub-optimal sandy sites for water wells, that water drilling depth needed to be carefully managed in dry seasons, and that better training and communications with communities were needed. In the Director’s opinion, human-powered (treadle) pumps (which PASAM-TAI had procured and distributed) were not adequate; he recommended, instead, fuel-powered pumps, which would be more profitable, and he asserted that farmers were capable of paying for fuel. While there may not be complete agreement on all points, maintaining the strong support of Agriculture Services for complex water systems will be crucial to their sustainability. PASAM￾TAI staff responded that they did feel that there had been consultation and suggested that the individual interviewed was not involved in the program, though the interview was arranged by PASAM-TAI. Southern Niger has considerable underutilized irrigation potential overall, and Title II DFAP programs have the opportunity to build community capacity for managing small-scale projects and providing direct livelihood benefits, thereby demonstrating an approach for wider learning. PASAM-TAI has a limited amount of resources for these systems, so they should be maximized for nutritional and income benefits, with good engagement from partners and from documentation of lessons learned. There seems to be interest by the government and by donors in future irrigation development, and CRS may be in a good position to contribute. According to FFP, the development of financing for these activities may be possible through public/private partnership with the PMERSA/TMZ (Projet de Mobilisation des Eaux pour le renforcement de la sécurité Alimentaire – Tahoua, Maradi et Zinder), a project co-funded by the African Development Bank, the Spanish Cooperation Ministry and the Global Agriculture and Food Security Program (GAFSP). USAID advanced discussions with PMERSA and may provide an umbrella for PASAM-TAI to become incorporated with PMERSA activities. The COFO representative spoke to the ET of how the COFO works with PASAM-TAI on land allocations and noted that most sites have signed documents, which protect the sites. Working on these issues is very important given the potential for conflict in marginal Sahelian environments over access to land. The fact that PASAM-TAI empowers women in particular to access land and practice rehabilitative agriculture makes this an important process to continue, though there is need for ongoing support to ensure, for example, that land use agreements are reached and landowners honor them. BDL with Food for Work (FFW): CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 40 The BDL activity aims to restore degraded soils by preparing zaï holes, half-moons, and trench micro￾catchments, which are used to collect rainwater and to grow micronutrient-rich plant species. Land is set aside on long-term leases of 10 to 20 years with participants initially taking part on a FFW basis, receiving a 50 kg bag of soy-fortified bulgur wheat for a period of work ranging from one-and-a-half to three months; though the intention is that participants would sustain it over time. This is an application of established techniques developed by ICRISAT and should have immense positive impact for participants while demonstrating sustainable practices that can potentially be replicated by others. This type of intervention should be prioritized in the current context of desertification and population growth. Women’s participation is particularly emphasized, and this is an important opportunity for them to enhance their livelihoods and gain valuable knowledge. The ET reviewed a technical brochure from ICRISAT on BDL, and it seemed fairly complete in terms of agriculture, silviculture, and soil management aspects, while not overly complicated, and no doubt, it would serve as a good reference for field agents. It was not clear though how trainers would use this to train villagers, nor for example, how it would guide the process of developing the BDL sites and deal with issues such as access to land. Very little is said about basic organizational and social aspects of BDL, such as how responsibilities would be assigned (other than the common listing of officers such as chairman, treasurer etc.), how decisions are made, or how produce would be shared. An ICRISAT officer stated in an interview that they provided scientific input but not the means of putting this into practice. The ET encountered PASAM-TAI BDL groups in four villages, all in Mayahi district, and also heard mention of BDL activities being implemented by several NGOs including Action Contre le Faim (ACF). For example, in one village, the BDL workgroup/committee comprised 47 women and 3 men, which included one representative from each household. They received on-site orientation, a set of tools, and seeds for jujube, moringa, and okra plants. The land that they worked on belongs to the chief, who had assigned it to them for a 20-year period, and the activity was functioning without major issues. Another BDL site showed promise where the women use the plants for consumption and convert the land to productive use. Women were carrying water out to the sites, however, which was said to pose a burden for them. Participants also raised concerns about a lack of adequate fencing, which left the site open to theft and damage by animals. The COFO officer also cited fencing of BDL sites as an issue needing resolution, with a specific recommendation given that would employ the traditional method of using millet stalks. The issue with the lack of fencing has been recorded before in program reports, which note that measures have been taken with the promotion of living fences as well as enacting vigilance committees. In a third village visited by the ET, the experience was not as positive. The group found that only the jujube plant had grown well while other plants, including sorrel, moringa, sena, and okra had dried out, leading the group to question if an adequate study of the environmental conditions had been done. The chief had signed an agreement giving the group access to over one hectare over a 10-year period. However, during the current year, neither the BDL group nor the PASAM-TAI field agents were active, so the chief decided to plant millet and cowpea and to take advantage of the rainy season. After having completed their FFW work assignment, the field agent had not returned, so the participants were unsure of what to do, especially since most of their plants had not survived. In addition, although PASAM-TAI had promised to provide fencing, they had not followed through on this commitment. In a fourth village, the natural fencing material utilized was not adequate enough to keep animals from entering and damaging the semi-moon formations in the soil, and a lack of interest by the third year had led to the landowner planting on the land again. This experience has been mixed, with challenges with fencing and plant survival rates, and the lack of interest of participants and sustainability is a key symptom. However, the concept of putting marginal CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 41 land into the hands of a group of people for farming is often practiced, particularly for women’s groups (the RISE evaluation confirmed that this occurs across the region, even prior to RISE interventions). Thus, there is reason to suspect that this approach can be replicated and could be made to work more consistently. The FFW is itself a fairly popular activity in such a food insecure area, but it might be worth asking if it actually helps the BDL activity or rather minimizes the sense of ownership of the participants. The fact that women are the main participants in BDL raises questions about whether this leaves the plots more vulnerable to theft or to landowners backing out of land-use agreements. In cases where there is specific reason to doubt whether women will continue to have access to land, the BDL activity should be carefully reconsidered. This may be one of the areas where PASAM-TAI could scale back its commitments and reduce the complexity of its operations. Nonetheless, the BDL activity is one of the factors pushing forward the process of women gaining legal recognition for land use, so its value should also be weighed in this light. Recommendations (Output 2.1.1)  Promote the sharing of experiences in household gardening, including identifying positive practices.  Discontinue expansion of household gardening and focus on helping existing sites to yield good results; give priority where groups or individuals show an interest in investing their time.  Ensure good community management and documentation of irrigated horticulture and seek future program opportunities (i.e., outside of PASAM-TAI) to continue support for community-based irrigation.  Discuss with gardening committees and others how to share knowledge gained by some and how to encourage wider participation.  Review the irrigation solutions with the director and other relevant staff within Agriculture Services to ensure ongoing support and learning.  Prioritize partnership with the PMERSA as a public-private partnership for financing water supply for agriculture.  Follow up with existing BDL sites to maximize functionality and solve problems (e.g., low plant survival, lack of fencing and security).  Confer with ACF and other partners to review methods in BDL to ensure coordination and cross￾fertilization.  Consider reducing the number of communities with BDL to ensure adequate support, and prioritize those communities with better prospects for stable land use and proven commitments by participants.  Discuss among staff and participants the gender dynamics involved in gardening and BDL to ensure support for women so that experiences are empowering. Output 2.1.2  Livestock: HHs have improved livestock management and improved production and marketing skills. Livestock are key assets and buffers against shocks, and animal diseases are some of the most serious disasters regularly facing the communities. When asked about their main constraints and suggestions for development assistance programs, many villages mentioned that they would like to have sheep and goats. They also say that they have not seen any improvements in this area given that only the SILC groups benefit. All the residents of one village either own livestock or fatten others’ animals, and they asked for assistance to learn about plants for grazing and pasture, as they were concerned this food source was CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 42 becoming harder to find. One village, whose residents are principally from the Peulh ethnicity and who are mainly herders, requested more support with livestock. There are several activities in PASAM-TAI related to livestock management, production, and marketing including the provision of sheep and goats, beneficiary training, and training of para-veterinarians. The ET did not observe training and activities relating to para-veterinary during fieldwork. The major focus for PASAM-TAI is the intervention supporting SILC groups in which sheep are provided on loan for fattening (emboucher) and the traditional system of lending female goats that are passed on after providing goat offspring (Habbanaye). Villagers confirmed that they received a coupon worth 30,000 XOF to buy sheep generally at large fora, which the government livestock services helped organize (the program reports that eight fora have been held). For the transactions, business people presented their animals, and according to the livestock officer interviewed, the beneficiaries could buy a sheep with a 25,000 XOF coupon and use any remainder for animal feed. It should be noted that the 25,000 XOF figure differed from figures given by the villagers. The government service then verified the animals’ health, and selected animals were dewormed and vaccinated. All of this appears to work well, but the agent explained that 25,000 XOF is not enough, and that 40,000 XOF would be required for a mature sheep in the current market. Only after the sheep is fully grown can it become fattened within a three-month period. The ET met with several SILC groups that had participated in these activities. Members from one village reported that the scheme was working well and that they were able to sell a fattened sheep in six months for approximately 60,000 XOF, noting that they had passed into their third cycle already. Others stated that the sheep they had received were small and that six months was not enough to ensure that the animal was fattened, making the profit minimal. Members from one village stated that the goats provided by PASAM-TAI were small, mentioning that some had even died, for which PASAM-TAI was expecting a repayment. They suggested that these small animals took more time than was allowed before they were ready to breed. Some also complained that they had received tickets from PASAM-TAI but had not yet received the animals at the time of the MTE. It may be that the program budget for the individual animals is limited, but certainly, it will be important to monitor whether beneficiaries are actually earning a profit; otherwise, there is not likely a value in continuing the intervention in its current configuration. In one village, the ET interviewed a livestock committee that has been operating for 10 years, though it is not currently collaborating with PASAM-TAI. This group promoted the Habbanaye system with the assistance of ACF in 2012, although many of the animals have since died, and the remainder of the animals has been sold to pay for other expenses. Members of this village call on a veterinarian for vaccinations, a service that is often provided free of charge during times of epidemics. The group notes that the main limitations are a lack of pastureland and water, as well as frequent epidemics. This experience could provide valuable lessons, since it raises questions about the likely sustainability of community livestock efforts. While many livestock have been distributed to beneficiaries, careful monitoring is needed to strengthen sustainability of the activity, and the ET notes that livestock services expressed willingness to be more involved in monitoring. The ET also found in reviews of similar projects in other countries similar challenges regarding sustainability and the importance of monitoring. The disease management aspect is a particular challenge. Accompanying the livestock-lending scheme is a process of training to beneficiary groups including training on animal nutrition, diseases prevention, and treatment. Communities did not bring up this livestock management training during the ET’s fieldwork despite questions about assistance and training received from the program. It may be that with so many training topics and groups and the beneficiaries’ CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 43 limited time, training may often be limited to a few technical assistance comments. The program did not provide any livestock-related training materials for the ET to review. Most villages mention animal disease as one of the most significant problems that they face. They sometimes use traditional plant medicines, which they stated worked well. It may be worthwhile to further explore these claims in some way, perhaps in a separate project, as it has been established in other scenarios that indigenous people often have knowledge of plants that, after being tested with scientific procedures, are confirmed as having the claimed properties. In some cases, the villages mentioned that they do not do anything. None of them mentioned that they could consult with PASAM-TAI or government officials. In this regard, the PASAM-TAI project document speaks of both training para-veterinaries and para-vet assistants and linking farmers to existing services, but there was no mention of these during village visits, the interview with the government livestock officer, or in program reports. The program states that 73 para-veterinaries have been trained, have received first aid kits, and are providing services to targeted communities. The project document actually had a lower number of 53 para-vets, which the program has exceeded, and the intention was to provide services to approximately 63 percent of targeted households. This intervention may be in its early stages and require considerable time before it gains wide community recognition. Recommendations (Output 2.1.2)  Emphasize the control of animal disease, particularly establishing linkages with para￾veterinarians, and consider whether the idea of para-vet assistants can help expand the service coverage area.  Explore the potential validity of traditional knowledge of medicine or other plant uses other than simple consultations; this may need to be in the context of a separate program.  Ensure that suitable animals are being selected and purchased if participants are making an investment of time and money in this activity.  Provide research and training on fodder and pasture.  Review coupon values in line with market realities to ensure that beneficiaries can procure fully￾grown sheep. If the budget is insufficient then consider reducing beneficiary numbers so that sheep will at least provide a profit after fattening.  Set up a monitoring system with the appropriate livestock services, to review whether adequate profits are resulting.  Study similar experiences such as ACF’s Habbanaye project. IR 2.2 Staples: HHs have adopted improved varieties of staple crops for consumption and for income. The staple crops relate directly to program objectives of reducing malnutrition and augmenting income, and PASAM-TAI works to enhance these with training and demonstrations and provision of improved seeds for rain-fed and low-input cultivation. The related interventions of pest control under IR 2.3 are extremely important to mitigate the effects of disease. One issue which the MTE did not have adequate time to focus on was the role of women in staple crop production and the extent to which the program was adequately involving women in this area. During some FGDs, participants reported that husbands allowed their wives to cultivate staples in a small area, and it appeared that the program could find ways to increase opportunities for women. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 44 Output 2.2.1  Knowledge: HHs have increased knowledge of staple crops. One of the strengths of the program is that learning is promoted in a natural setting through practical demonstrations and through experimentation, a feature which recognizes the capacity of farmers to learn and experiment on their own and which can strengthen that capacity. The ET observed several demonstration plots during the fieldwork including outside the 14 sampled villages. One village reported that it had a plot the previous year (presumably for PVS) but that it had not continued to work with the plot during the present year. Another village had an active farmer association that had participated in field research on soil fertility and on varieties. The ET also observed a plot of 24 m2 located on the farmer association secretary’s farm. The plot was divided so that half the plot received an optimal mix of fertilizer and the other half did not. The same plot was also divided transversally with half the plot planted with a 55-437 seed variety and half with a RRB seed variety. In addition, there were two sections of the plot where compost had been used and where the most advanced plants could be found. While this was interesting, it seemed to complicate the interpretation of the results since the demonstration mainly focused on seed varieties and fertilizer. Neither the farmer association secretary nor the field agent seemed very sure of how to explain the results or suggest the best course of action to take, even in a preliminary sense (as the plants had not yet reached maturity). The association, however, regularly visited the plot and discussed progress along with the support of the field agent, a dynamic that is crucial for learning. The partnership with ICRISAT provides access to premium technical guidance, though to maintain this level of guidance, it is crucial for field agents to be diligent and to have an adequate technical level themselves. The government Agriculture Services officer provided feedback noting that they were neither uniformly satisfied with the field agents nor their management of plot-based trials, an issue for which the cluster management is partly responsible for in their opinion. There was some indication that turnover of field agents was disrupting follow-up visits to villages; field agents interviewed also expressed concern about the pressure of work. For ICRISAT, a program like PASAM-TAI can allow them to test their methods and ensure they are practical and relevant to villagers. This, however, requires a systematic learning process for them to get comprehensive feedback from village trials, and for the MTE, it was not clear if this mechanism was in place. Three other points came up during a farm visit. First, it was observed that the farmer’s plants were not the spaced according to a several-day training session farmers had received. While this was not ideal, it might be possible that it may take time for this learning to be applied. It was made apparent to the ET that no learning materials or memory aids had been provided to the farmers, making it more difficult for farmers to retain and practice what they are exposed to during training sessions or field orientations. Second, the ET noticed that farmers were trying out different plants and cultivation arrangements that went beyond the program’s focus. The program should be responsive to farmers’ ideas and initiatives and to helping identify potentially good practices and positive deviants. It should also accompany and support farmers in their natural curiosity and inclination to search for improvements. Otherwise, it becomes overly top￾down, which is not conducive to either sustainability or to capacity building. Third, this raises the question of whether or not the program’s focus is too narrow in relationship to the knowledge content. During the MTE, it was clear that good work is being done on improved varieties and some aspects of soil fertility. Yet, there was no mention of topics such as minimum tillage, mixed cropping, agroforestry (despite the presence of many trees in cultivated fields), strategies to mitigate crop disease risks, wind and other forms of erosion, and strategies to maximize the complementarity of men’s and women’s work. The ET acknowledges that the work with the REGIS project does address other areas of conservation agriculture, although this was not visible at the field level in the sampled villages or in program CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 45 documentation. The number of villages expected to benefit from the REGIS collaboration was not known, but given the importance of the themes it will address, it would be ideal to try to scale it up as much as possible. Several training materials under output 2.2.1 were available for review by the MTE, including guidelines for organizing demonstration parcels and holding a field day, which were helpfully detailed on these key aspects of promoting improved seeds. A PowerPoint presentation labelled “working with women” suggests that women can play a key role in production of improved seeds, provides guidelines for working with schools, and provides some pointers on obtaining seeds from suppliers. There are certainly other aspects of women’s participation in agriculture in general (and crop production in particular), but there was no sign of these issues being addressed, at least judging from the training guidelines available to the ET. Overall, numerous topics should be addressed in the area of staple crop production, as well as other aspects of SO2, which were not dealt with in the training materials given to the ET. Thus the ET considered the training curriculum to be incomplete and in need of further work. Program reports mention numerous trainings, but the MTE found no obvious or specific indicators that describe what participants are doing because of this training. If there are not materials, which spell out a selected number of key concepts and outcomes in terms of measurable practices, it can be more difficult to trace the effects of training.21 Further, the ET frequently heard about challenges of heavy workloads and high turnover of field agents, and this can disrupt the consistency of training, so having straightforward training guides could help ensure more of that consistency. Thus, while SO2 training materials are not very complete, it should be acknowledged that while health and nutrition aspects of PASAM-TAI benefit from having extensive and relevant materials created by other programs, the work in SO2 is somewhat more distinctive and has required new guidelines and training materials. What is important is that the program recognizes this and attempts to assemble the best possible training curriculum in the time remaining in the program. FGD respondents during the MTE tended to focus mostly on the improved seeds and did not highlight the trainings, although several respondents mentioned participating in farmer field schools. It may be worth asking which aspects of the training are new, which aspects are valuable for participants, and which aspects are simply repeating what the farmers already know. While there was no specific feedback to confirm this, the ET considered that there was a risk that the program might not make optimal use of resources and participants’ time by covering topics that have been included in training by some of the many other development programs taking place in the area currently or in previous years. No doubt, the ET was not provided a formal training needs assessment for PASAM-TAI, but this could be helpful for future. When asked what they would like to see from the program, for instance, village members often replied that they would like to receive more seeds, fertilizer, and animals, comments that are not uncommon in development assistance programs. Interestingly, though, in a few villages, members stated that they would like more input and supervision from field agents, so it is clear there is some learning taking place. Recommendations (Output 2.2.1)  Develop a written and systematic learning plan that sets out objectives and methodologies in the areas of rain-fed staple crop production.  Consider whether the REGIS training can help define a key set of messages and tangible practices to give concrete expression to the full scope of staples crop improvements. 21 Perhaps stating what is obvious to some: optimal learning arrangements would entail certain objectives, concepts, and capabilities that participants would learn. It would also entail incorporating a clear methodology that ensures optimal learning and retention of new knowledge and a way of following up to ensure the application of what is learned is systematically being built upon. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 46  Consolidate and/or develop learning materials and memory aids to help farmers learn new material.  Allow some flexibility and capacity to respond to issues of interest to farmers in terms of training and accompaniment of field trials.  Develop an explicit gender strategy in agriculture that highlights women’s role in the cultivation of staples. Output 2.2.2  Seeds: Improved seed management and supply chain established. PASAM-TAI’s focus areas include obtaining improved seeds for core crops (e.g., millet, cowpea, sorghum and peanuts), helping women and men farmers select varieties according to certain criteria (e.g., drought-resistant, quality, yield, early maturing seeds), and obtaining sustainable access to these seeds. Seed fairs are organized together with the Agriculture Services, with the latter expressing satisfaction with this collaboration. This is a simple and low-cost intervention which reaches a high percentage of households and which results in a positive impact. During FGDs in a number of villages, improved seeds were the only agricultural improvement people mentioned, and in in some cases, it was the only agricultural and livelihood activity of which they were aware. The seed fairs are a simple and sustainable institution, which efficiently combine learning and distribution of materials. However, members in one village reported that not all people were informed of the fair and therefore were not able to receive any seeds, and while the ET cannot determine how widespread this might have been, it is a valuable caution to ensure that these valuable events be as inclusive as possible. Most villagers expressed satisfaction with the improved seeds, though generally they commented that the quantity is very small even for demonstrations and especially for multiplication purposes. For example, one village noted that they received a coupon that provided for approximately 200 grams of millet, cowpeas, and sorghum. The members of this village also mentioned that they did not have money to pay for fertilizer, but that they still find improvements. Members of several other villages reported that the quality and yield from improved seed varieties are better than those from their traditional seeds and that farmers who have not been able to benefit from the improved seed variety are asking to obtain improved seeds. Another four mentioned that they had only recently received the seeds, so they were not able to comment at that time. Members from another village reported that because of using the improved seeds, they were able to harvest millet after only 70 days and with minimal precipitation, compared with previous harvest times of 90 to 120 days. While they found that the quality of the new seeds was not necessarily higher than their habitual seeds and that the quantity produced was relatively smaller, the early maturation was deemed very beneficial. Others commented that while the cowpea seeds they had received were producing yields, they were not convinced that they would continue using them. Many also said that fertilizer should be provided along with the seeds. Some villagers reported that they would save seeds for use during the following season and that this could be sustained. This would seem to be a critical point for the program to monitor, given that ICRISAT noted that they often see farmers eating their seeds during the lean period before they can plant them, which may be problematic. The ET was told that farmer associations were doing seed multiplication, but in the sampled villages, respondents did not mention this activity to the ET. PASAM-TAI staff state that their value chain approach (see also Output 2.4.3.) will help farmers obtain seeds from companies like SIMA and ASUSA in the future, though in general, there was limited information in project reports on value chain activities, and villages had no information and therefore were not able to comment on these plans. A reassuring step would have been for CRS to have arranged for the ET to interview commercial partners during the MTE, but this was not done. Hence the CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 47 recommendation is for the program to clarify its approach and strategy, define some realistic targets and then review these jointly with FFP and perhaps in conjunction with other RISE partners. These efforts – though tentative and requiring more clear elaboration – are important to the success of the program. The fact that seeds are chosen in a participatory manner according to villagers’ criteria is very positive. The ET found evidence that preferences of both women and men were taken into account where, for instance, women expressed a particular concern about quality. The interaction between a research institute and farmers in this process is excellent. However, other factors that should be a major factor affecting household livelihoods, such as disease resistance, were not mentioned. In addition, while tolerance of low precipitation and droughts is crucial, other climatic factors affecting crops in some locations include strong winds and flooding. Recommendations  Ensure monitoring of the recycling and multiplication of improved seeds.  Develop the strategy for sustainable access to improved seeds, in conjunction with the value chain approach (see also Output 2.4.3). IR 2.3 Environment: HHs have managed environmentally responsible integrated crop production systems. Output 2.3.1  Soil: HHs have improved soil fertility management techniques. Respondents in several villages highlighted that one of the key constraints they faced was a lack of soil fertility and given the rise in population growth and the context of climate change, it is expected that the pressures on land will increase. Thus, program efforts to enhance soil fertility are extremely important for current productivity as well as future sustainability of the region. The ET found it beneficial that the program is providing demonstration trials of soil fertility that will benefit members of participating villages. The ET encountered a PVS site in one village, discussed above under output 2.2.1. The farmer field schools and extension system should help to extend these benefits to the maximum number of beneficiaries. The types of soil conservation methods included in BDL, such as semi-moons, zai, and soil fertility-enhancing plants are key inputs of training in the villages, using proven methodologies and backed by the technical capacity of ICRISAT. There was little mention of these methods outside the context of BDL, however, and there is great potential for these having an impact on households’ management of their own land. Soil fertility messages are also shared through household gardening, which tends to be shared with a large number of people. The majority of beneficiaries of these activities are women, and their number probably includes at least a handful in most communities, so this could be an important core group for sharing these methods more widely. It would be valuable to monitor whether the methods learned by women will be eventually adopted by men in other applications, such as in staple crop production. It also would be valuable to know if women beneficiaries disseminate these methods with other women. In FY14, project data indicated that the program had only achieved 8 percent of its goal for land to be restored through soil conservation, which suggests that this program component will need a greater emphasis to meet its own targets, and a number of indications reinforced this during the ET. Members of CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 48 one village mentioned that soil fertility was one of the issues that they wished they could receive assistance from PASAM-TAI or other organizations. It was mentioned that PASAM-TAI did not provide assistance with fertilizer, training, or technical assistance for composting. Members from another village mentioned that they had received compost training from a government official, and they expressed interest in learning more about this topic and about organizing themselves to recover land. Despite having a very clear benefit in the PVS trial mentioned above, there was very little mention of compost during discussions with villagers. A good potential for strengthening composting in PASAM-TAI exists through collaboration with the REGIS conservation farming program, and CRS is already pursuing discussions with them. The ET also found scarce discussions of mixed cropping methods and of the use of live fences that fix nitrogen and enhance soil structure quality. While these are topics that might be mentioned during training sessions with one group or another, issues like composting should have messages that are understood by those involved and widely disseminated in the villages. If the REGIS training partnership addresses some of these issues, this should be made explicit and preferably translated into measurable indicators. Recommendations (Output 2.3.1)  Expand the work around soil fertility, including defining feasible and measurable indicators of the application of good practices such as the possibility of more widespread use of Apron Star.  Place greater emphasis on composting activity, promoting wider use of improved technologies and plant varieties for land restoration.  Continue to develop partnership with REGIS conservation farming to use and apply proven methods for wider application of composting.  Monitor (to the extent feasible) in particular the diffusion and application of improved methods in rain-fed agriculture and households’ own parcels of land, and to other non-beneficiary women. Output 2.3.2  Pests: HHs have improved integrated pest management techniques. Communities frequently mention that crop diseases are a frequent and major disaster they face (see IR 3.1), which may have an overriding effect on efforts at improving agriculture production and livelihoods. There was no mention of support or training from PASAM-TAI on this issue during the ET’s fieldwork. In one village, FGD respondents stated that they had bought an insecticide to treat a crop disease. When asked by the ET if they consulted PASAM-TAI or the mayor, they stated that since they had not received a response following the reporting of a flood in the past, they saw no point in doing so (more on this problem under IR 3.1). While the project document mentions some promising areas of work with regard to more pest-resistant varieties and practices, the ET found no discussion of these topics in the annual or quarterly reports. The ET did not find any mention of Indicators Performance Tracking Table (IPTT) indicators. It may be that more work is being done than is visible, and it simply needs to be incorporated and highlighted into monitoring systems. CRS has incorporated the use of Apron Star —a seed fungicide and insecticide that is effective for millet and sorghum when used correctly with fertilizer micro-dosing— in the Farmer Field Schools. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 49 While the program is already overstretched in terms of programming breadth and geographic spread, given how many villages mentioned crop diseases and infestations, it would be worthwhile to attempt to identify some intervention or partnership linkage to help address this topic and increase the reach. For example, one village mentioned that they had received support from INRAN to use natural predators and biological pest control, efforts that could prove valuable to build upon. ICRISAT also seems to do some work on this area, though their partnership with PASAM-TAI seems to focus mainly on improved species and fertility. PASAM-TAI could seek to determine if a relatively “light” intervention with few resources was possible. Recommendations (Output 2.3.2)  Enhance the visibility of existing work on pest management control and assist villagers in connecting with potential sources of assistance.  Determine with partners, such as INRAN and ICRISAT, whether some intervention in pest management is feasible. IR 2.4 Income: HHs have increased sources of revenue. Output 2.4.1  Post-harvest: HHs have improved post-harvest storage and processing techniques. The program promotes the Purdue Improved Crop Storage (PICS) bag as a means to prevent insects from spoiling post-harvest yields. PICS bags are made up of a 50 kg storage bag with additional bags placed in a concentric circle within the larger bag for three layers of storage protection. The purpose of such a design is meant to prevent insects from eating crops within the bag and to eventually starve them. The ET deems this a simple but brilliant innovation which are provided free of charge, and they should prove quite effective in greatly reducing losses that can be so devastating for farming households that have extremely narrow margins of survival and viability. The MTE found that members of one village were employing a PICS bag and that these users found it worked well. The ET noticed that other villages were not aware of this storage solution although it is presumed that PICS bags are in a stage where they are currently being introduced. Some village respondents specifically expressed a wish for assistance in conservation of cowpeas, and stated that they were not aware of the PICS bags. This highlights a general lack of information, at least in villages visited during the MTE, and a possible need for wider communication about PICS, which may help create a demand for obtaining them. Regarding the processing of grains, some of the villages expressed that the lack of a mill posed a constraint for their village. It was also mentioned that other projects have worked on mills, including one that had actually broken down. This highlights one of the downfalls of such approaches, that often they are not designed in a very sustainable way. Still, this approach of food processing is certainly in line with the general direction of the value chain approach, and though the program should be cautious about overextending itself in this area as with others, it is worth considering this as an area for the program to engage in or for CRS to pursue in other related projects. Further, given that other NGOs have carried out interventions with grinding mills in the program area, it may be possible to find partners to provide this component. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 50 Recommendations (Output 2.4.3)  Accelerate the rollout of the PICS bags in conjunction with seed fairs and implementation of the value chain approach, and monitor their usage with farmers.  Consider possible communication routes to increase awareness of and demand for PICS bags, in conjunction with the value chain approach.  Given the demand for assistance with processing grains, investigate whether PASAM-TAI can promote access to grinding mills or other related equipment through the value chain approach in partnership with other NGOs or in future projects by CRS. Output 2.4.2  Savings: SILC groups manage and provide financial services to their members. During the MTE, the ET met with six SILC groups, and one tontine (traditional rotating savings) group that was on track to form a SILC group. Numerous villages had several savings groups, thus a village of several thousand could easily have four or more groups of 25 members, resulting in a fairly widespread impact across the village. Generally, these groups worked quite well since the clarity and simplicity of the system have demonstrated to be manageable and since the participants, who are all women, knew how the groups functioned and what their respective roles entailed. The ET noticed a good example of a SILC that met every Wednesday that the ET deems was well organized. This group was comprised of 25 women and included a president, a treasurer, a secretary, a bookkeeper, and three women each of whom maintains one of the keys for the safety box (although the ET was unable to view the contents of the box given that one of the three key-holders was not present during the meeting). The members remarked that they had previous savings and loans groups before the implementation of PASAM-TAI, although these were not as well structured. While they reported that they had not received training by the PASAM-TAI field agents, they did receive periodic visits from their Private Service Provider (PSP) who provided encouragement. Each member of the group made a weekly contribution of 50 XOF to a social fund, from which they could access grants of 3,000 to 5,000 XOF. The funds were accessible to assist members in events such as a death in the family, illness, or natural disaster. This social fund made a difference to these women, and it was made evident to the ET that it proved a valuable buffer against shocks. The same group also maintains a larger loan fund through weekly subscription fee of 500 XOF, from which they could borrow up to 5,000 XOF, to be repaid within four weeks with an interest rate of 10 percent. In case of late repayment, a fine of 100 XOF is applied, and no further loans are given without repaying this sum. The group is in its third cycle, and loans have been used for fertilizer, to fatten livestock, for small retail businesses, peanut oil production, and social events and ceremonies. After nine months, members share out the balance, of up to 25,000 XOF. The PSP visits once a month to check the register and repayments and the SILC group reimburses his cost for his motorcycle’s gasoline for a sum of 1,000 XOF. Considering that the group is self-managing with non-binding technical advice provided by the PSP, this group seems to be working well and according to expected procedures. Members of the SILC mentioned several concerns. For instance, the treasurer maintains the fund, which may in fact be a large amount of money (in the range of USD 1,000), in a safe box at home, at considerable risk to herself and her family. This also limits her ability to move away from home for any length of time. Another challenge that the ET observed, which numerous other villages face, revolves around late repayments, a factor which reduces others’ loan possibilities. It was also mentioned to the ET that groups might also find it difficult to find people who are capable and trustworthy. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 51 A number of groups formed prior to PASAM-TAI, either with the assistance of an NGO or on their own, and all said that PASAM-TAI had helped them to become better structured and work more effectively. One SILC group was in their third cycle after having evolved from a savings association with 15 members without strong organization. This SILC shared out 26,200 XOF from the total fund of 525,000 XOF in the last cycle and now allow loans as high as 20,000 XOF. The grants from the social fund are non-repayable, a feature that members note is enacted in order to strengthen group cohesion. One of the issues discussed by the group is that the PSP was not consistently available to support them despite having been offered 5,000 XOF to the PSP as motivation, an offer the PSM did not accept. Now the group pays the secretary 1,250 XOF for her services of recording transactions. Although this is a creative adaptation, not having a trained PSP to guide groups may prove risky. Nonetheless, the SILC group had previously functioned as an association so the group may be fine after all. Generally, the group felt that decisions about loans worked well and that loans were repaid quickly. The group also noted that subscriptions/contributions were provided in uniform amounts, a feature deemed advantageous. One village had formed a tontine committee with eight lead mothers who met biweekly to contribute 100 XOF and to share out the balance to one of the members who was chosen by random selection. The group had grown to 17 members and the contribution had risen to 200 XOF. The group meetings include discussion about their work as lead mothers as well as the group savings. The PASAM-TAI field agent had advised them that they were ready to form a SILC group, and the group members felt confident of their sustainability. This approach of organically forming a SILC group seems to work well, is low-risk, and develops habits and capacities for more advanced savings and fund management. In addition, the ability to support the livelihoods of lead mothers makes a lot of sense, and this arrangement could be worth replicating. The linkages between SILC and literacy are also numerous, as literacy tutors often served as SILC secretaries. In one village that did not have a SILC group, the literacy participants expressed desire to have one formed so that they could take a logical next step in their personal development. In another village, the ET found a SILC group operating and also a pre-SILC group consisting of 15 members. FGD respondents noted that the pre-SILC group had apparently been promised a safe box by PASAM-TAI, although perhaps it was a misconception that they would get a safe box before officially reaching SILC status. This village mentioned that they faced a challenge when some members approached the keepers of the safe box and directly asking for a loan rather than following established procedures of asking for a loan with the entire group present. There were several other variations in the functioning of SILC groups. One group had increased their weekly subscription to 1,000 XOF, with some flexibility for those unable to pay that amount. These members were then allowed to share the 1,000 XOF subscription, a feature the ET found interesting. This group also made arrangements whereby the loans interest was retained by the person who paid it, rather than going into the general fund to be shared by all. Another group mentioned that they have weekly payments ranging from 300 XOF to 600 XOF and that instead of holding three members responsible for the key to the safe box, only one person held that responsibility. A common complaint revolved around members who did not repay their loans. Another challenge is the amount of money that individuals are responsible for, especially treasurers: the ET found one example where group members put pressure on treasurers to provide loan funds. When asked what they had gained from PASAM-TAI, members mentioned that they gained an understanding of the loan system (i.e., borrowing and repaying loans), as well as how to organize their group. They all appreciated the system of savings, loans, and repayments while some mentioned that they felt the interest rate was appropriate and important for growing the group savings. There was no mention of incompatible credit conditions or methods with other organizations, only that PASAM-TAI had helped existing groups to improve their CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 52 operations. Almost all the respondents appreciated the work of their PSP. Some stated that it had improved their living conditions; the full impact is something to be explored and measured during the final evaluation of PASAM-TAI. Recommendations  Discuss with groups any challenges they may be facing, such as the risks that treasurers face due to holding large funds and pressure to provide loans outside of normal meetings.  Ensure PSP support and record any instance of inconsistency.  In case of creative arrangements (e.g., a secretary doing record keeping or attributing loan interest paid to the payee), the ET recommends monitoring that particular arrangement. In this manner, lessons learned can be noted, and if problems arise, standard procedures can be reinforced.  Monitor and potentially scale up the concept of lead mothers forming SILC groups.  Work with mature SILC groups to gradually develop more complex institutions such as farmer associations. Output 2.4.3  Value chain: HHs have increased engagement in value chain activities. Since value chains were not something that villagers discussed or were able to comment on during the fieldwork, the ET is unable to corroborate information provided by the program, much less comment on the implementation to date. The approach remains relatively unclear and lacks visibility, and given that this approach is an important one for FFP and USAID with its partners under the RISE program, by the time of PASAM-TAI mid-term, the strategy should arguably be somewhat more advanced. As an example, value chain activities are not highlighted in the PASAM-TAI list of activities, so it is difficult to identify a village, which reflects the new approach. One of the program indicators is the number of farmers participating in value chains, which CRS takes to refer to the beneficiaries of BDL and improved seeds. This could be questioned, however, as to whether this can be counted towards the value chain objective, because these are all simply recipients of project resources (seeds and food-for-work) who receive some training. It not clear how they can be considered participants in a distinctive activity referred to as “value chains”, unless the activity is explicitly articulated in the context of a wider strategy. Usually when programs have a value chain component, they will emphasize relationships between private sector, other stakeholders, and the entire system, in order to encourage all to be more sustainable, competitive and responsive to markets, so on first consideration this indicator does not seem very appropriate. Nonetheless, the good news is that the strategy is emerging. PASAM-TAI’s value chain intervention focuses on the key products that farmers in Mayahi and Kantche can develop and sell at marketplaces. This intervention will helps farmers optimize input linkages while also developing more dynamism and linkages on the output side. One of the focus areas is to support the emergence of farmer associations with the program stating that 350 groups have been formed, including 35 irrigated gardening committees. As stated earlier, it is not clear which groups these may be, though presumably SILC groups are included, but the definition of and strategy for the Farmer Associations should be articulated in tandem with the value chain approach The program states that partnerships or agreements have been arranged with seed companies like SIMA and ASUSA, to provide credit, presumably through warrantage or inventory credit, for new activities such as sheep fattening, access to seeds, and sacks. Seed companies FESA and Mousa Dan Ladi created 26 retail locations in Kantche and Mayahi to provide good quality and reliable seeds at convenient locations. In these locations, a large quantity of seeds (worth several million XOF) have been purchased there, including what are presumed to be mostly PASAM-TAI beneficiaries given that the program covers those districts completely. Another innovation is to develop a cell phone IMAC information system on CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 53 agriculture markets which would include community agents to monitor markets and share information to a wide audience. An initial contact has been made with SINAGRO regarding collaboration on potential agricultural transformations which will require a participatory assessment and planning. Thus while this component is still being formulated, there appears to be considerable potential in ensuring access to seeds and sacks and ensuring markets are better understood. The livestock market is dynamic in this area, particularly given the proximity to the huge markets in Nigeria, and PASAM-TAI may be in a good position to accelerate discussions with stakeholders and gain a better foothold in these markets. This could, for example, attract additional energy and resources to tackle problems regarding animal disease prevention and treatment among other areas. Given that many aspects of the value chain strategy appear to be at an early stage, there is a risk that PASAM-TAI will not be able to put plans into action and properly monitor them. This will require careful discussion vis-à-vis finalizing the work and possibly incorporating it into other upcoming initiatives. This makes sense for value chain initiatives which require a network-wide view and approach. Recommendations (Output 2.4.3)  Clarify the term ‘farmer association’ (organisations paysannes) and how it is being used, including explaining which existing groups fall into this category; explain what is distinct about the value chain aspect of their functioning.  Reinforce efforts to strengthen farmer associations in line with this definition, as discussed previously, possibly in conjunction with an overall review of the role of VDCs and their interrelation with other community institutions.  Review and clearly define the value chain strategy, and present this for discussion with USAID/FFP.  Coordinate discussions on value chain plans with USAID and other DFAP and RISE partners, to set out actions that can be carried forward to a satisfactory stage within PASAM-TAI and then beyond, if necessary. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 54 STRATEGIC OBJECTIVE 3 Resilience and community capacity building: Targeted communities have improved integrated disaster risk management (DRM) In an effort to understand natural disasters in this area and how program interventions relate to this dynamic environment, the ET asked communities about disasters that have affected them in the past five years and what their responses have been. Their responses have been compiled in the following composite picture across the 14 sampled villages (See Table 1). Table 1 – Disasters that have affected communities and their responses Disaster Response Flooding (2010, 2013, 2014)—effects include houses collapsing, damage to seedlings, and unsafe stagnant water The GoN provided food assistance. One village took shelter in classrooms, built makeshift huts, and prepared list of those affected in order to receive assistance Drought (2011, 2012) and related food security crisis Young men migrate; sell land (on limited scale to wealthy individuals from Mayahi); sell animals; eat wild plants; diversify income generation (e.g., practice handicrafts) Strong winds (2013) cause houses to collapse No response Birds destroy millet and sorghum to consume their seeds Community burned the birds’ nests Crop pests (leafminer, or la mineuse) attacked millet (2011, 2012). Several villages mentioned that it caused a famine INRAN helped one community with biological pest management; migration for work Locust invasion (2013) affecting 50 percent of fields Appeal to God for assistance Other insects damaged cowpeas (2013); damage to both cowpeas and peanuts (2015) No response Caterpillars attack plants Purchase and use of powdered pesticide Aphids attack beans and peanuts Those who can afford it use fertilizers, others appeal to God for assistance Meningitis epidemic; respiratory illness; whooping cough Seek treatment in CSI clinic in neighboring village Chickenpox epidemic Villagers place pieces of canaries into boiling water for a medicinal drink Malaria epidemic that overwhelmed capacity of CSI Use of traditional medicine (e.g., use drink and vapor of kinkeliba plant) Measles epidemic (2015) Go for treatment to CSI; traditional medicines Cattle illness Use of traditional plant medicines which were apparently effective Poultry illness (all poultry in village died) No response It is important to take into account the specific crises that village members must face. For example, all villages face significant crop and animal loss due to insect infestations and diseases, crises they generally respond to on their own. Drought and flooding are common problems, and in several cases, the GoN was CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 55 able to provide food assistance for those affected by flooding. In several cases, there was no response, or the response was on a spiritual level, and traditional medicines are often used. For the responses obtained by the ET (see Table 1), there were no instances in which PASAM-TAI played a significant role, though some of the capacities and systems put in place should help. According to the program document, PASAM-TAI followed USAID guidance on Niger to prepare a disaster response plan to: “immediately work with affected communities to assess needs that can be met with local resources… work with communities, the national emergency response framework and external donors to identify unmet needs and mobilize resources… request to reprogram all available Title II food used for land restoration in SO2 to the crisis, either through FFW or as direct distribution of rations, depending on the severity of the situation.” It may be that some of the BDL activities were in fact part of a disaster response, given that disasters are chronic in many of the villages.With the hindsight of several years of experience, the program should be able to fine-tune its vision of how it may be relevant to disaster preparedness and response. Recommendations (Regarding DRM in general)  Look at the specific crises and ways in which villages respond to crises to review the various scenarios involving community responses and how PASAM-TAI might provide support.  Consider an increased focus on combating plant disease and pests, including through discussions and advocacy with the GoN to provide greater support and to channel some emergency response efforts to these challenges. Such efforts may well need to be in conjunction with other development actors, yet it is important to highlight this issue. IR 3.1 Community-Based Early Warning Systems (EWS) and Responses: Community-based EWS function as an integral part of the national EWS mechanism. …and IR 3.2 Targeted communities have managed disaster responses. The Système Communautaire d'Alerte Précoce et de Réponse aux Urgences (SCAP-RU) is a crucial system used for emergency monitoring and a way for communities to connect quickly with authorities to coordinate response. The strategy is similar to work being undertaken by other organizations (such as REGIS with its natural resource management project), and as it concerns local institutions these different interventions should be well-integrated. In terms of establishing functioning SCAP-RU committees, the ET located SCAP-RU systems in nine of the fourteen villages visited, although the MTE noted that many of them were struggling. This was divided as follows: in Mayahi, all five villages had SCAP-RU systems while in Kantche, four of the nine villages had SCAP-RU systems. Some said that they were meeting on a monthly basis while others said that they met only occasionally to send out information. Consistency of participation is an issue in most villages with the numbers of active members gradually declining. They generally started with approximately 12 members, usually at least a few women, and in some cases only a few received training on their roles. For example, some referred to a five-day training with themes including the role of SCAP-RU, social cohesion, and environmental problems. Some of the villages had members who were part of a SCAP-RU, which represented different villages. At times, this created challenges on the attendance of meetings and on maintaining themselves informed of meetings. Despite such complications, some committees persisted. In one village, a man and a woman CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 56 were found to be members in a SCAP-RU for five villages that met every week. Another committee meets once a month and prepares a report on all the problems (with one copy sent to the commune and one kept in group’s archive). When asked concretely what issues they report on, this group mentioned that they discuss food insecurity, early warning of insect infestation, vigilance with brushfires, monitoring for deforestation, and identifying child malnutrition. As these examples suggest, the villagers generally understand that their main role is to gather information on any disaster or problematic situation that the village experiences and to then pass it on to the competent authorities, such as the mayor’s office. This is essentially what PASAM-TAI expects, as well. In one village, the SCAP-R, formed through PASAM-TAI, seemed to work reasonably well. The secretary general received training on how to submit information via Short Message Service (SMS) to PASAM-TAI as well as how to fill out a form to send to the mayor’s office. This SCAP-RU committee was involved in other activities such as nutritional surveillance, market food price monitoring, and identifying potential risks relating to the infestation of crops. In one emergency, a house collapsed and killed a child, and the SCAP-RU sent a report to the mayor's office (the SMS system to communicate with PASAM-TAI was not yet in place at the time). The mayor’s office immediately responded with food, cash, and other assistance. This village credited PASAM-TAI for training them in the importance and manner of working on disaster preparedness. One village discussed how the SCAP-RU system increased their sense of social cohesion. A critical problem that a number of the SCAP-RUs systems faced was that when reports are generated, there is no response or feedback. This is a serious challenge because if they understand their main role is to report disasters knowing that someone will read the report or that someone will take action, it will be demotivating to continue to do the same thing if no response is provided. Three villages in Kantche and one in Mayahi cited this problem, noting that even when they diligently prepare reports every month, the villages receive no feedback, creating the impression that the reports are not that useful. In reality emphasis should be on transmitting the essential information at the times when it is needed, not on creating busywork. One village submitted a report on flooding to the mayor’s office, but there was no feedback from the commune or from PASAM-TAI, and the village admitted that they did not bother to later raise it directly with PASAM-TAI. In subsequent situations, they did not bother to report to PASAM-TAI or the mayor’s office. When respondents in one village were asked which department they expected would manage their issue (e.g., animal health), they did not have any awareness on the appropriate department for their issue. There seems to be a lack of basic understanding of which government entities might be able to help them with specific problems, aside from the mayor. Another issue the ET found relates to strengthening linkages at higher levels. While PASAM-TAI made early good progress in that respect with national coordination, staff changes interrupted the work, which is apparently now being restarted. Strengthening the capacities of commune-level institutions is crucial. While PASAM-TAI works with the Vulnerability Monitoring Observatory (OSV) structure, the ET did not hear about them during the fieldwork and did not have the opportunity to meet with them. Communities generally identify with the mayor and not with any other structure. Some of the SCAP-RUs were functioning at a multi-village level, and this may overlap somewhat with the intent of OSV. The RISE evaluation did find one good example of a SCAP-RU operating and finding good responsiveness from the commune authorities, which is a hopeful sign, though it may be that really good examples are not very numerous. Of course, the villages need to understand that the mayor’s resources may be limited, and that in terms of what they ask for, they may need to be realistic and even strategic. There should be some kind of response or some kind of request for assistance that is more feasible for government entities to handle and therefore more likely to attract a favorable reaction. One village discussed how they would like a health clinic in their village, and given that it may be very difficult for government to invest in a new clinic on an CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 57 impromptu basis, this sort of demand might detract from getting more immediate assistance in disaster responses, the latter response being one that governments are usually prepared to do. This is where having a clear focus on their main role is important and that it is established that SCAP-RU systems are for emergencies while VDCs should be employed for overall development issues. In one village that did not have a VDC, there were perhaps some signs of a mandate drift where the SCAP-RU committee members saw themselves as resolving conflicts between people or communities. While it is true that they should see problems in a holistic way and learn to work together with others, but being clear about their role, and if necessary help other community institutions to form and do their part. At the same time, the strict focus on transmitting information upwards may be counterproductive, especially until such a point where communal authorities and others have built the capacity to respond in a timely way. SCAP-RU groups in several villages had a somewhat wider sense of their role that included actively monitoring early warning signs of problems like child malnutrition, brush fires, insect infestation, and deforestation. If they are in contact with different people in the community and discussing these problems, it is natural that they may pass on any information they have that might be helpful, or they may also make linkages between community members who are faced with similar problems etc. One village mentioned that when they monitor an imminent flood, they search for houses that may be at risk from the flood and they educate people on developing some contingency planning. Disaster preparedness planning follows naturally from the identification and monitoring of risks and it may be feasible for more established SCAP-RUs, perhaps, to assist with this in conjunction with VDCs. If this could be done in a way that provides more substantial and meaningful results, without a significant additional time investment, it could encourage SCAP-RUs and help them to be more active. A final issue, which a number of people brought up, is the possibility of paying incentives to SCAP-RU members. This is a common request in many countries and a feature found in many types of community volunteer work. In addition, PASAM-TAI seems to have a policy of about this that takes into account the amount of time that people work as well as the direction that government entities and other NGOs are taking. Recommendations (IR 3.1 and 3.2)  Review the level of functioning of SCAP-RU committees and encourage meaningful reporting on a manageable basis rather than frequent report submissions.  Enable discussions with SCAP-RUs to learn what, if any, work they are doing aside from sending information upwards, to see if this could yield useful lessons.  Work more with commune-level officials (including OSVs where applicable) to develop their responsiveness in the medium- and long-term.  Consider working with selected SCAP-RUs (with VDCs and commune authorities) on contingency planning or other simple but useful roles.  Ensure integration of efforts with REGIS NRM and other programs operating in the same areas or on similar issues. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 58 CROSSCUTTING OBJECTIVE 1 Women and men expand gender roles to enhance sustainable results. Crosscutting IR 1.1: Target communities have improved gender equity Crosscutting Output 1.1.1 Women and adolescent girls have improved participation and decision making. Throughout the MTE, the ET interviewed more women than men, taking into consideration the types of groups that participate with PASAM-TAI, many of which are intended for women. SO1 mostly involves women, though efforts are underway to increase the role of men in family health. SO2 involves both men and women in accordance with the types of agriculture and livelihood activities they practice, though one of the focus areas for PASAM-TAI is to promote women’s savings and loans through SILC and through women’s involvement in horticulture and animal husbandry. SO3 tends to be led by men, though the program promotes women’s participation. One measure intended to promote a gender balance was found in cases when a village is part of a multi-village SCAP-RU where the village selects one woman and one man to represent them. In terms of the crosscutting activities, the VDCs are similar to SCAP-RU in that men tend to have roles that are more dominant. During fieldwork, the ET did not find much evidence of women’s active participation in SCAP-RU and VDCs. On the other hand, in the literacy groups there is strong women’s participation, with groups reported to be 60 percent comprised of women. Indeed, in some cases (particularly in Mayahi), it was found that women were more dedicated to learning, so most or all of the men were replaced by women. Gender issues are discussed throughout the report, with corresponding recommendations provided. Clearly one of the important areas where PASAM-TAI intends to work on this is through the MLG, which is just getting underway. The vision of the PASAM-TAI gender coordinator is to use this space to explore gender roles and relationships more broadly with the expectation that this should receive the full support of the whole program. One of the issues highlighted is that lead mothers tend to be left out, without much training or much of a role, while the promoters (who are often men) tend to do things on their own without involving the lead mothers. The MTE recommends balancing this and giving lead mothers a greater opportunity to participate in activities like growth monitoring. The BDL work is fairly burdensome for participants (who are mainly women), and when they appear to be less active, the sites t risk being repossessed by (male) landowners, making it difficult not to think that a gender dimension or that a gender imbalance makes this more likely to happen. Many of the activities give responsibilities and work to women, and to a certain extent women, along with their infants, enjoy the benefits that come with to the limited extent the MTE could discern. However, the program needs to be vigilant to guard against situations in which men gain control over resources in such a way as to limit benefits for the family as a whole. For example, there is often a problem with women’s savings and loans programs where women perform the difficult work and when they get a loan, their husband (or brother, etc.) may feel they have a right to use those funds. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 59 Crosscutting IR 2.1 Women and men have increased basic literacy and numeracy skills. Crosscutting Output 2.1.1  Crosscutting: Literacy and numeracy centers are established, equipped, and staffed. …and Crosscutting Output 2.1.2  Crosscutting: Beneficiaries are functionally literate. With literacy rates that are among the very lowest in the world, there is a strong case for including literacy and numeracy in PASAM-TAI, despite the fact that Food for Peace (FFP) Title II programs do not commonly include them. The approach that PASAM-TAI uses is a sound one, with high-quality materials developed to deepen and reinforce learning on core program themes of nutrition, hygiene, agricultural production and others. The training appears to be adequate, and often overlaps and/or builds on other literacy efforts of UNICEF and other organizations. An innovative system has been set up with the assistance of Tufts University to stimulate literacy practices using SMS messages on phones, on topics that are often of practical use to villagers. The ET encountered functioning literacy and numeracy centers in six villages. All five villages in Mayahi had literacy programs while in Kantche only one out of the nine villages visited by the ET had one. While the small sample size precludes drawing statistical conclusions, the reasons for this contrast should be reflected upon. The ET spoke with some young women who had graduated, who felt very positive about their achievement, and who discussed the importance of escaping ignorance. When asked by the ET to discuss what they had learned and how it would help them, these women respondents mentioned skills on nutrition and hygiene more than the ability to read and write. Other FGD participants mentioned reading and writing in Hausa, basic calculations, hygiene (e.g., hand-washing), keyhole gardening, and community cleaning as key learning areas. Those women (and men) who were literate were often seen as playing leadership roles in CGs or SILC groups. The literacy tutors were trained for three weeks with PASAM-TAI, and prior to that had been trained for four weeks, it was not clear if the training was complementary or repetitive. The tutors receive a stipend of 45,000 XOF per month. The groups have around 25 members per class with generally two tutors, but after observing the greater commitment of women, men were removed from some groups and they were reconstituted with all women participants. While literacy training for women may be a priority, it seems unfortunate if males do not also take up this opportunity. In another village, the male tutor had all male learners. The commitment made by learners is meritorious. The groups study for a period of three to four hours a day during five days a week. In one village, a single tutor worked with 30 women from nine in the morning to Noon and then with 20 men from two in the afternoon to five in the afternoon. The courses continue steadily for four months between December and April, and this is repeated the following year (the ET found that several villages initially had tried meeting during the April to July period but switched to the aforementioned months). The ET posed itself a relevant question regarding the possibilities for people who graduate who have now gained basic literacy in Hausa and what it is they can read. One PASAM-TAI literacy coordinator CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 60 emphasized that participants would be able to read more of the literacy materials on various program themes, which are practical and well made. In addition, there is a plan for community libraries, and although this idea seems to be tentative, there are some positive precedents for revolving book schemes that could provide ideas for PASAM-TAI or for related programs22. Others emphasized the arrangement of sending text messages with information each day. Some related feedback received from one group was group members did not have enough materials for the learners. This echoed the feedback from an officer from the Department of Education who commented on delays in delivering materials. FGD respondents also felt that some learners had difficulty understanding the lessons, which created a lack of equilibrium in the group. They suggested creating a second center or group, to have two levels within each to ensure more uniformity in their learning. A fifth village had a lapsed literacy program where the tutor, who was not from the village, had apparently been very inconsistent and had eventually stopped coming around five months earlier. This village mentioned that in addition to a reliable tutor, they needed a physical structure. While the 90 percent (or more) attendance rate speaks to the commitment of the learners, the quality of education bears some consideration. There is not a high percentage of those who complete the literacy program who can pass the government’s literacy exam, though the ET acknowledges that a closer look of this exam would be necessary to be able to draw conclusions about it. An impact evaluation by Tufts University23 notes significant differences in reading and mathematical abilities when comparing participants to non-participants from control groups. This is encouraging, though perhaps obvious: one would of course expect the literacy and numeracy training to have made some difference for learners. There were no significant differences in knowledge levels of participants regarding topics in health and nutrition, agriculture, NRM, and gender. Given the in-depth learning materials exploring these topics, one would expect more of an impact in this regard, although in fairness the program does not necessarily make firm commitment about what is expected here. The literacy classes have slightly improved participants’ self-confidence and willingness to participate in development. These are not exactly breakthrough results, at least at the time of this MTE. The sustainability of literacy programs can often be a challenge, and PASAM-TAI may need to review this issue carefully. One village clearly said that they could not pay for a tutor and they hoped that another NGO would pay for it. Still, as something of a sideline component to this Title II program, the effort in literacy and numeracy training is laudable. It is somewhat more difficult to be very innovative and have major breakthroughs in a component of a major integrated program, especially in which that component is somewhat of a lower priority. It appears to be moving in the right direction, and perhaps needs consistent effort to improve and to sustain the quality. Given that it is probably unlikely that PASAM-TAI could put much more time and resources into enhancing the quality of the intervention, further improvements might be possible through partnerships with government entities and with other actors such as UNICEF, and others that villages mentioned including AQUADEV, GIGAWA24. After all, at least some of the literacy groups were previously formed through other projects, and there is some likelihood that they or parts of the groups will work with others in the future. Recommendations (Outputs 2.1.1 and 2.1.2)  Consider ways to accommodate and encourage men who wish to participate in literacy and numeracy learning, even while giving priority to women. 22 Community libraries are common in Latin America including a successful project from Save the Children while observed by a member of the ET while working in Nicaragua. While conditions are different, important lessons may be learned from such projects. 23 The report was not yet available at the time of the MTE. 24 Further details are not available on these NGOs. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 61  Facilitate knowledge exchanges between literacy tutors and students to promote enhanced methods, seek improved learning outcomes, and share ideas about encouraging life-long learning.  Partnership with government is critical, and coordination with them should be prioritized. Crosscutting IR 3.1 Governance of targeted communities and national structures strengthened. An important part of promoting self-help community development is to cultivate institutions that encourage community participation that includes coordinating among different interest groups and individuals within the community, government entities, and with the traditional chiefs who are the natural leaders. The VDC approach of PASAM-TAI is well established and most countries have incorporated these into policy and national planning, thus CRS is helping to foster these somewhat embryonic institutions that should gradually mature and assume increasingly important roles. Crosscutting Output 3.1.1  Crosscutting: Community structures are strengthened. Some of the VDCs seemed reasonably active, showing the potential of such an institution, even with the relatively light touch of PASAM-TAI. There is considerable lack of clarity, and at times, confusion about the role of the VDCs. Role and Functioning The ET took an open-ended approach to discussing with VDCs their performance and their strengths and weaknesses of their work with PASAM-TAI. As the ET proceeded, certain questions were explored, such as:  Is there actually an identifiable structure?  How do they see their role and relationship with other organizations and individuals?  How do they consolidate their own capability, and what sort of sustainability do they have?  How do they support and empower others? In five of the villages visited during the MTE, there were VDCs operating in three out of the five villages in Mayahi, and two of the nine villages in Matameye. While not statistically significant, this difference is worth exploring. In addition to these five villages with functioning VDCs, one village in each of the two districts had a VDC operating in name, with specific individuals interviewed by the ET who admitted that they were not functioning. One of these individuals could not state what their role as VDC was, and the other could not remember the themes of the training they had received. The VDCs that were functioning were able to identify a chairperson, a secretary, a treasurer etc., and the VDC was formed with the support of PASAM-TAI. Some of the VDCs had started to collect funds for various purposes, and they were using various modalities such as collecting contributions or water fees. Some of the VDCs explained that they had been elected at a large community meeting facilitated by PASAM-TAI. In one outlier example, the VDC was a group of young men (ranging in age from 20 to 40 years of age) who had formed on their own initiative to promote development with other community members apparently accepting them. Though training was earlier provided to all VDCs, some of those interviewed were requesting training so that they could understand their role. This could be caused by high turnover, which the program staff said was a common problem. Where they existed, VDCs functioned, to at least some extent, and played a key role in the development process, although every story was different. One example was a village where the VDC members said they were functioning but CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 62 some members did not participate regularly except when the PASAM-TAI field agents were present. Their request was that PASAM-TAI field agents should be more present, because without them, various activities, from BDL to CLTS, were likewise not very active. As the ET met with leaders and various participants of PASAM-TAI activities in the villages, the general view was that the VDC was a broad-based committee that had oversight of all development activities. Most people understood that the VDC should have some sort of relationship with other community groups, to coordinate, monitor, and support their work. A program staff member expressed that the VDC was like the representative or replacement for the PASAM-TAI team in each village acting through a permanent presence and a source of leadership. Usually the VDC members stated that they worked closely with the chief. VDCs generally had at least some participation of women, who took part in FGDs, although their level of activity is unsurprisingly still subdued related to that of men. In most villages visited there was not much explicit engagement of youth, with one exception being where the VDC was actually a self-appointed and quite active group of young men. Yet there seemed to be some confusion of what a VDC would do in concrete terms and how specifically they should relate to other groups. VDCs should begin to play a governance role that has demonstrable results in such areas as nutrition and health, agriculture, environmental protection, and disaster preparedness. While villagers readily identify these activities of PASAM-TAI, for example, often it was not clear what value the VDC added. In this situation, there was some slight mandate drift observed, in which the VDC saw itself directly doing the work of other community groups. This is generally not a problem, but needs to be steered correctly. For example, one VDC said that part of their job was to go to the Mayor’s office to report a disaster, although that would mainly be the function of the SCAP-RU committee. It is the VDCs’ responsibility to broadly maintain relations with the communal authorities, and it is understandable that they might participate in disaster-related communications without actually disempowering the SCAP-RU. This may require some refinement of what their specific role may be to ensure optimal use of human resources. Of course, one of the factors is that some individuals may be members of both committees, so part of the process of institutional maturation is understanding how the institution is distinct from the individuals that compose it. Other examples like this point to some emerging roles for VDCs. One VDC was responsible for water points, charging 10 XOF per container, and managing the funds from this. With regard to the CLTS process, one VDC said that they monitor if community members are following up on the agreed actions, and ensure that the CLTS group continues with its sensitization work. Another VDC spoke a lot about BDL, how they had built 200 semi-moons and planted trees. Another VDC stated that their role was to organize sensitization meetings on exclusive breastfeeding, hand-washing and birth spacing. Again, these examples showed a lack of understanding of the roles that different community organizations could play, but for this fairly early stage of the evolution of these institutions, that is to be expected. What is important is that committed people take action, learn, and hopefully build a foundation for future sustainability. Water supply was an interesting example, which is also discussed under Output 1.2.3. The VDC in one village was actively involved in water supply management, which makes sense for a project of vital importance involving large resources. This could function for the initial period until there are too many other things for the VDC to do at which point a water sub-committee can be established. In one village where the VDC was very weak, a separate water committee was managing water points and collecting funds and was now receiving new infrastructure from PASAM-TAI. It is evident that the VDC in this CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 63 situation would become marginal with an uncertain role and that community members would not have great respect for it. Another important area for community institutions and PASAM-TAI’s potential contribution is the development of Farmer’s Associations, as discussed elsewhere in this report. Since these are envisaged as dealing with potentially a number of agriculture issues for one or more groups of farmers, they may assume quite a wide-ranging importance for communities, and the strategy for them should therefore be considered in conjunction with the vision for VDCs. One important development step for VDCs is to make and carry out plans, and while PASAM-TAI staff note that VDCs have done this, it was not possible to verify. The ET asked about this in some villages, and most said they did not have plans. Another important step is to develop financial management skills and some VDCs said that they were raising funds in order to do more activities. Whether raised by water fees, individual voluntary contributions, or other means, managing funds is always a very high-risk area. While the potential loss of some money is itself a serious problem, it is equally important to avoid the loss of unity, confidence, and momentum that might result. One VDC noted that they did have some misunderstandings about how to manage funds and this type of challenge is likely to grow. The issue of incentives comes up repeatedly, as discussed under SO3, and it requires a more extensive discussion to address it properly. CRS should have a clear policy, ideally harmonized with that of the GoN and of other NGOs. At times, it is impossible to provide an incentive, but other forms of encouragement and appreciation may help. It is interesting, for example, that in one village, it was mentioned that they would like more training for SCAP-RU while the VDC mentioned that they needed more involvement from the PASAM-TAI field agents. One village even said that the fact that the ET was there talking to them and showing interest was an important form of motivation. Recommendations (Output 3.1.1)  Review why Kantche seems to have a lower level of activity for VDCs.  Look for good practices of VDCs that show how they are contributing to work and how they are working effectively with other groups in areas such as disaster preparedness, hygiene, CLTS, and combating malnutrition.  Provide refresher training that helps VDCs establish the role they can play while focusing on a few simple activities.  Review financial management, as this is a potential point of sensitivity.  Water point management is of vital interest to the whole community and should be managed by VDCs at the initial stage.  Prioritize the development of Farmer Associations and consider how they interrelate with VDCs. Crosscutting Output 3.1.2  Crosscutting: Government structures are strengthened. This aspect is treated through a section on partnership below, because the ET interpreted that the main form of strengthening of structures is taking place through the partnership between PASAM-TAI and the Government of Niger. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 64 VOLUNTEERISM, PARTNERSHIPS AND COORDINATION Three major factors have a significant effect on all aspects of PASAM-TAI: the volunteerism approach, the PASAM-TAI partnership with the GoN, and its coordination with other NGOs. Thus, these issues are discussed separately in this section. Volunteers As discussed above in terms of some activities such as lead mothers, the FFW land reclamation, training for latrine construction, and keyhole gardening, PASAM-TAI staff at times do not provide adequate follow up and support to communities to ensure thorough and timely implementation and sustainability of program advances. One clear example is that materials for latrines and tippy-taps were distributed but not installed, which the communities explained was because they were not provided the necessary technical assistance. In some other projects, this might call for a review of how staff spend their time and whether this can be optimized (something PASAM-TAI did not request the MTE to do). However, in view of the MTE conclusion that PASAM-TAI is overstretched in terms of the breadth of the program, this should certainly be considered as a key factor, which is spreading human resources too thinly. Thus, as discussed in the executive summary and elsewhere, it would be desirable to reduce the range of activities and/or the geographical spread. An alternate interpretation to this challenge has been suggested by some, namely to expect volunteers to do more work by providing them with some form of payment. To pursue this would entail going against the CRS policy of having community volunteers working without personal financial incentives from the project. The consultant would not recommend fundamentally changing this policy, however, and particularly not as a way of mobilizing more human resources to implement project activities. Providing stipends to community members may supplant the intrinsic motivation of development and community service with a dependence on external assistance. Wherever possible, the project should try to encourage support for volunteers through internal and sustainable means, rather than using external funds. For example, lead mothers could be supported by the families that benefit from their training, and if this is awkward for them to ask for, PASAM-TAI field agents could encourage and help negotiate this. The funds associated with water supplies could be a source of support for a few key community “staff” such as health/nutrition promoters and water managers. Such arrangements are more likely to reflect overall community development priorities if the Water User Associations are seen as being under the authority of Village Development Committees. Depending on how the Farmer’s Associations operate, they also are likely to be important candidates. The best arrangement is combining resources such that external funds pay for training, support mechanisms, and some materials, while internal contributions (from individuals, community, and government) are made to carry out the activity. It would be a mistake to stop promoting the investment of time by a wide range of community volunteers. To balance out CRS’ currently strict – and in our view, correct – policy on volunteerism, much thought needs to be given regarding how to avoid undue burdens on volunteers. This is particularly the case for women, as it is often assumed that they can and should volunteer their time more than men do, and yet in reality their time may already be oversubscribed. In the context of PASAM-TAI implementation, the amount and pace of volunteers’ contribution should be based on capacity rather than the project goals (which are effectively an external requirement), so if volunteers are currently overtaxed, then their activities should be dialed back to a manageable level. The program should be sensitive and proactive in ensuring that it does not overtax volunteers, but rather it should carefully cultivate and protect what is an invaluable and sustainable local resource for development. To give adequate attention to this matter, a CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 65 specific review exercise on the volunteerism approach is in order, both to understand current commitments and assess if these are reasonable, and also to develop guidelines for the future. Some might question whether a food supplement such as that provided in PASAM-TAI would constitute an incentive and therefore would be unsustainable. This is quite a different matter, however, as the food ration is simply meant to provide for the nutritional needs for the development of villagers during a key developmental window (from conception to 24 months). This is not really intended to have an effect beyond the life of the project other than strengthening resilience and capacities, and it should not detract from beneficiaries participating in accompanying activities to promote self-reliance and sustainable livelihoods, though their participation in the latter should never be linked with their receipt of food supplements. Such a nutritional supplement, for a very clear and defensible target group, should not discourage specific, skilled individuals with community roles—like lead mothers, VDC or SCAP-RU officers—from contributing their time to promote the development of their children. Recommendations (Volunteerism)  Broadly continue the current policy – and avoid indiscriminate provision of program resources to provide incentives for community volunteers, which is not sustainable and can discourage the wider investment by villagers in the development of their own communities.  Explore opportunities to generate internal financing for key volunteer positions, particularly those requiring significant time demands and critical responsibilities such as managing large funds, and with special concern for women  Carry out a review exercise on the PASAM-TAI volunteerism approach to assess if current practices and expectations of volunteers go beyond reasonable levels; then establish guidelines and see if adjustments are needed in the workload expected of volunteers.  Pursue wider discussions in Niger with government and other stakeholders about harmonizing volunteer approaches. CRS should coordinate these efforts with USAID and its other partners. Government of Niger The program staff state that the GoN is the primary partner and is closely involved in the management of the program. Through this partnership, a considerable degree of the capacity-building objective is taking place (crosscutting output 3.1.2) in this section. One of the areas where this cooperation and partnership is displayed is through participation in the coordination meetings between the GoN and all NGOs, particularly at regional level and also nationally. In this form of coordination, CRS often plays a leading role, and this is valuable in that it is supporting the GoN in its own agenda and therefore contributing to governance capacity building. In general, and despite marked differences among sectors (see below), most government partners interviewed conveyed to the ET that while there were many efforts being made, they still were not completely satisfied with the partnership with PASAM-TAI at the time of the MTE, and they expressed this particularly in terms of communication and accountability. They felt that communication was limited to planning-oriented meetings rather than joint review of activities. They also mentioned that reports with insufficient details were provided and there were limited opportunities to visit implementation sites to understand what is going on. There seems to be very little coordination with the Ministry of Health and Nutrition, and the Department of Women’s Promotion and Child Protection, while there is somewhat more with the Ministries of Agriculture and Education, and there is relatively good coordination with COFO, the Ministry of Hydraulics and the Environments, and the Department of Animal Husbandry. Some of the good processes of coordination mentioned include joint planning of seed and livestock fairs, planning of new water supplies, and working out land use agreements. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 66 These are the opinions of the departmental and regional officials met with although there may be other aspects that should be taken into account. For example, it is common that different individuals from these agencies may be involved at different times, and systems for sharing information within government departments may not be optimal. Nevertheless, these are the senior partners involved in PASAM-TAI and the program should seek to gain and maintain their satisfaction, approval, and support. When the partnership works well, government officials and institutions will tend to learn and internalize positive procedures, which can influence policies, which can help to sustain the good work of PASAM-TAI and development in general. PASAM-TAI staff argue that they feel they do have good coordination, citing the example of the agriculture services – but if the director of the services states that he is not in agreement with everything and feels a need for enhanced coordination, this is a sign that something needs to be done. In terms of some specifics, as mentioned above, PASAM-TAI does hold periodic organized meetings, including quarterly review meetings at departmental level and annual national steering committee meeting. To have done this consistently is an accomplishment of the program. The annual steering committee allows all program partners, including ICRISAT, to participate, and this was something they appreciated. Most departmental government offices mentioned that they attend quarterly coordination meetings at department level, though, for example, the Department of Women’s Promotion and Child Protection was not included in this.25 Almost all government partners said that they wished to have better involvement with PASAM-TAI outside of this quarterly meeting to be able to monitor and ensure that techniques applied are adequate. This applies even for those who were more broadly satisfied with the coordination. The reports that they receive do not have enough detail to understand what is happening on the ground, and they do not have an opportunity to do site visits. The quarterly meetings tend to be focused on plans rather than an analysis of activities carried out, review of challenges, and review of solutions to problems, as some said. According to them, PASAM-TAI is accountable to government agencies, which are responsible for ensuring the good quality of development work performed by NGOs. In addition, even COFO, which were broadly satisfied with the partnership, felt that it was better for PASAM-TAI to maintain regular contact with members dropping by informally and resolving problems as they arose rather than relying on scheduled meetings which may not be adequate to deal with all the issues. It is important for CRS to clarify with partners how they see the partnership, how it should be organized for optimal effectiveness, and to design procedures (and perhaps compromises) that are feasible for each institution. CRS is accountable to USAID, which implies meeting demanding project timelines and a constant pressure of work, and it is important that the GoN agencies appreciate this aspect. However, both CRS and USAID are also accountable to the GoN and need the contribution of its agencies in order to make programs like PASAM-TAI work towards the long-term development of Niger. The government agencies bring to the program technical capacities, knowledge of policy and institutional realities, and knowledge of local conditions. By comprehensively practicing and putting those to effect, it contributes to the capacity-building of the government agencies and staff. Another important aspect of the government capacity building at all levels are the numerous trainings across the range of PASAM-TAI activities, and these are crucial especially in exposing field staff to concepts and approaches which are new (e.g., the 1,000-day approach, CLTS, and SCAP-RU) until they can effectively manage and utilize them. CLTS is a good example of an innovation introduced by UNICEF which is now so thoroughly internalized by government agencies that they effectively have ownership of the method and provide certification in it. PASAM-TAI has been good at providing numerous training opportunities to government staff and engaging staff as trainers. 25 The ET asked PASAM-TAI staff to assemble all partners for the meetings with the ET. It may be that certain departments of the GoN are not expected to be as involved in PASAM-TAI. CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 67 Recommendations (Government Relations)  Discuss with partners how they see the partnership and what they feel can be improved, as well as whether they feel they are learning and reinforcing capacities and sustainability. Collaboration with other NGOs Having some parallels with government coordination, PASAM-TAI has made significant efforts to coordinate with other NGOs and with multilateral agencies such as Acción Contra el Hambre (ACH) and UNICEF. The coordination among DFAP and RISE partners with the assistance of USAID provides a critical foundation for coordination among implementation agencies. Since the RISE program deals with so many thematic issues, this can provide a template for better understanding of the issues and potentially for coordination with other NGOs. Numerous coordination meetings take place at regional or even departmental level with the main NGOs, and these are probably very time-consuming, never adequate, but essential, nonetheless. These types of meetings are also essential moments for government officials to be appraised and to have an opportunity to coordinate. However, undoubtedly there are too many issues than can be accommodated within such broad meetings, and many of the actors are probably not present when things are being discussed that involve them. What was remarkable during the MTE was to observe how many NGOs are operating in the same department and villages as PASAM-TAI in many of the same sectors or thematic areas. The MTE could not devote much time to mapping these NGOs or investigating details on them, but the following is a general overview of the work of other NGOs/agencies as reported in the sampled villages:  ACH (Acción Contra el Hambre, Spain) and ACF (Action Contre le Faim, France) – combating malnutrition, CLTS, mosquito nets, medicine (e.g., anti-parasitic medicine and eye treatments), goats, money for women to reduce malnutrition, BDL  Aide et Action – school latrine construction, hygiene and sanitation, education reform, school scholarships  Al Ouma – water pump, training for exclusive breastfeeding and complementary feeding, hygiene and sanitation, nutrition training for men  Anima Sutra – credit for women, hygiene education, family planning, Aquatabs, awareness raising on sexually-transmitted diseases  Aquadev – literacy center  CARE – women’s credit (at the time of the MTE), CSB and oil for PLW and infants  GIGAWA – literacy  Goulbi – infant malnutrition prevention, soap for hand-washing, Aquatabs, CLTS, hygiene and sanitation  GSC – assistance to producers  HELP – combating malnutrition  IRD – water pump  MISICO AGRA – savings and loan association, cowpea conservation bags  PADEL – cereal bank (reselling grain during lean season), Habbanaye, well construction  PALEM – trade school  PASADEM – recuperation of degraded land  Red Cross – assistance during flooding  Sahel Bio – biological pest control and agriculture development  Introduction to Sahel Resilience Learning Project (SAREL) – entered a village five months prior to the MTE and sampled men for survey on nutrition and child care practices, weighing mothers and infants CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 68  Save the Children – food distribution and raising awareness about malnutrition, 1,000-day CSB and mosquito net distribution through CSIs (one village specified that this was currently taking place)  SNV – hygiene education, soap production, training for borehole repair, training water committees  UNICEF – primary school furniture and books, cereal bank, distribution of goats for women, cattle for water well traction, rice and CSB for pregnant and breastfeeding women, CLTS  World Food Programme (WFP) – cereal bank, subsidized sale, particularly during lean season, cash transfer to women (criteria unknown), cattle feeding In one village, 12 NGOs were currently working alongside PASAM-TAI. In four of the 14 villages, there were no other NGOs, but when it comes to NGOs and their collateral effects, working in the same commune can have significant influence. These organizations work in PASAM-TAI intervention areas, sometimes with overlapping mandates, sometimes with complementary approaches and intents, and occasionally with contradictory methods. Apparently, approaches on latrine promotion have been a contentious issue, and though the issue has been clearly identified, there still may not be complete agreements. Even though it may be impossible to have extensive and frequent discussions with all of the organizations on all of the issues, it would make sense for PASAM-TAI to have some strategy. The strategy would include identifying areas requiring coordination, reducing potential friction or problems, and optimizing opportunities and capacity building which each NGO represents. For example, Goulbi NGO works in one of the sampled villages doing awareness-raising on health and nutrition for children and pregnant women where an agent comes once a month to do training on hand-washing, diarrhea treatment with rehydration solutions, the importance of taking children to the health center, and household hygiene. PASAM-TAI should be exploring how to work with Goulbi to avoid bringing in confusing messages, reduce duplications and manage everyone’s resources optimally. As a minimum measure of coordination, in selected cases like this, PASAM-TAI could reduce the time that its health and nutrition staff would spend on those issues in those villages, if another NGO were doing the same work. To capture these types of coordination issues in all of the villages of PASAM-TAI would be extremely time-consuming, so it is likely that much of the coordination needs to be done by the villages themselves, and particularly by the VDCs while also involving the pertinent groups and individuals in each case. When different organizations work in a village or commune on literacy or women’s credit, the local institutions should be empowered to bring this to the attention of NGOs and government agencies and provide their opinion of how these efforts could be coordinated. Arguably, it may be important for local NGOs to play a stronger role here to give voice to local communities and to be able to coordinate development assistance. Admittedly, many NGOs are independent and this work would not be easy—it is often the donors behind the NGOs who provide pressure that is not conducive to coordination. Still, large agencies like CRS and USAID, and all of the partners co-implementing the RISE program, could set the tone and provide a good example of collaboration with the GoN. If NGOs only set their sights on the worst examples regarding the lack of coordination, such as when different organizations are claiming ownership to CLTS in the same village, they really do not do justice to their mandate and it is the children of Niger who suffer as a result. The DFAP partners and other International NGOs undoubtedly have coordination meetings already, and they should provide leadership on this matter in a way that builds the capacity of Nigerien NGOs. Community participation in NGO coordination becomes crucial when it comes to capacity building. Some type of training need assessments that should be done periodically, perhaps in shared exercises, and CRS PASAM-TAI Mid-Term Evaluation – FINAL REPORT, March 3, 2016 page 69 currently this does not seem to be taking place. It is potentially a huge waste of resources if each NGO comes along and repeats training on hygiene or conservation agriculture, for instance, without first determining whether the learners already know the content matter. Undoubtedly, it can be useful to have refreshers, but this can be done in a way that reinforces but also extends learning. Schools do not allow teachers to teach whatever they feel like, rather there is a systematic approach in which each teacher builds on the work from previous years while continuing to progress. To some extent, even if fairly limited, this should be done with NGOs. If there are topics that are difficult for villagers to understand, as is the case when PASAM-TAI discovered through its radio programs, then it should be possible to determine which other activities or NGOs can contribute. In terms of sustainability, it is important for NGOs to agree broadly on policies on key issues such as volunteerism and incentives, or financial management frameworks for local institutions. The water reticulation in one village does not just concern the organization that installed it, it also behooves PASAM-TAI to reinforce the community to sustain that water supply, as part of its promotion of hygiene and health which is the focus of PASAM-TAI. As PASAM-TAI begins to prepare for phasing out of its target villages, these issues become increasingly pertinent. Recommendations (Collaboration with other NGOs)  Build on the existing collaboration among DFAP and other RISE partners to ensure optimal coordination, and use that enhance collaboration with other NGOs/agencies  Maintain a conversation about what projects have been implemented in the past in the implementation areas and how to build on them in terms of capacity-building and reinforcing development gains.  Discuss sustainability, phase-out plans, and how to coordinate with other organizations.  Continue working with key government partners to promote optimal coordination and discuss cases of NGOs intervening in similar areas (such as those identified during the MTE).  Seek opportunities to build the capacity of local NGOs.